Face-mounted electronic device and method for transmitting visual object corresponding to user of face-mounted electronic device

Abstract

A face-mounted electronic device according to an embodiment may comprise a communication circuit. The electronic device may comprise a first camera for acquiring an image including at least a portion of a user's face. The electronic device may comprise a second camera for acquiring an image of the surrounding environment of the electronic device. The electronic device may comprise a memory for storing at least one computer program including instructions. The electronic device may comprise at least one processor. The instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to start a communication connection with an external electronic device by using a communication circuit in order to transmit a voice and / or an image. The instructions may cause an avatar corresponding to the user to be generated on the basis of an image including at least a portion of the user's face and / or a surrounding environment image that changes according to the movement of the electronic device. The instructions may cause the avatar to be transmitted to the external electronic device. The instructions may cause the amount of movement of the electronic device to be identified on the basis of a first image, which is a surrounding environment image at the time of starting the communication connection, and a surrounding environment image changed according to the movement of the electronic device. The instructions may cause whether the amount of movement of the electronic device exceeds a designated first value to be identified. The instructions may cause the transmission of the avatar to be stopped in correspondence to the movement amount of the electronic device exceeding the designated first value.

Description

A method for transmitting a visual object corresponding to a face-mounted electronic device and a user of the face-mounted electronic device.

[0001] The present disclosure relates to a face-mounted electronic device and a method of operating the face-mounted electronic device, and to a technique for transmitting a visual object corresponding to a user of the face-mounted electronic device and the face-mounted electronic device.

[0002] A face-mounted electronic device in the form of a Head Mounted Display (HDM) is worn on the user's head and includes a display that provides images to the user. The face-mounted electronic device can provide images in the form of Augmented Reality (AR), Virtual Reality (VR), or Mixed Reality (MR).

[0003] Face-mounted electronic devices can provide video calling capabilities to users by transmitting or receiving video based on a communication connection with an external electronic device. During video calls, the face-mounted electronic device can utilize a camera-captured image of the user or an avatar, a virtual graphic object representing the user.

[0004] An electronic device can establish a communication connection with an external electronic device using a communication circuit and can perform a video call. When performing a video call, the electronic device can generate and / or display an avatar corresponding to the user of the electronic device. The electronic device can determine the characteristics of the avatar display (e.g., the size and position of the avatar) based on the movement of the electronic device corresponding to the movement of the user wearing the electronic device.

[0005] A user of an electronic device may move while wearing the electronic device during a video call. The electronic device may transmit an avatar to an external electronic device even when the user leaves the location where the communication connection was established, and may determine the characteristics (e.g., avatar size, avatar position, avatar angle) of the avatar displayed based on the location of the electronic device at the time of the communication connection. If the location of the electronic device at the time of the communication connection is maintained as the criterion for determining the characteristics of the avatar displayed, an unintended avatar may be displayed on the external electronic device when the user leaves.

[0006] The external electronic device can display the received avatar on its display. The external electronic device can adjust the avatar so that it continues to be displayed on the display of the external electronic device even if the avatar moves according to the movement of the user of the electronic device. Since the electronic device or the external electronic device cannot determine whether the user does not want the avatar to be displayed, the avatar can continue to be displayed on the display of the external electronic device even in situations where the user of the electronic device does not want the avatar to be displayed.

[0007] The technical problems to be achieved in this document are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0008] According to one embodiment, a face-mounted electronic device includes a communication circuit. The electronic device includes a first camera for acquiring an image including at least a portion of a user's face. The electronic device includes a second camera for acquiring an image of the surroundings of the electronic device. The electronic device includes a memory storing at least one computer program including instructions. The electronic device includes at least one processor. The instructions, when individually or collectively executed by the at least one processor, cause the electronic device to initiate (or perform) a communication connection for transmitting voice and / or video with an external electronic device using the communication circuit. The instructions cause the electronic device to generate an avatar corresponding to the user based on an image including at least a portion of the user's face and / or an image of the surroundings. The instructions cause the avatar to be transmitted to the external electronic device. The instructions cause the electronic device to determine a movement amount of the electronic device based on a first image, which is an image of the surroundings at the time of initiating (or performing) the communication connection, and an image of the surroundings that has changed according to the movement of the electronic device. The instructions cause the electronic device to determine whether the movement amount of the electronic device exceeds a specified first value. The instructions cause transmission of the avatar to be stopped in response to the movement of the electronic device exceeding the first value specified above.

[0009] A method of operating a face-mounted electronic device according to one embodiment includes an operation of initiating (or performing) a communication connection for transmitting voice and / or image with an external electronic device using the communication circuit. The method of operating the face-mounted electronic device includes an operation of generating an avatar corresponding to the user based on an image including at least a portion of the user's face and / or an image of the surrounding environment. The method of operating the face-mounted electronic device includes an operation of transmitting the avatar to the external electronic device. The method of operating the face-mounted electronic device includes an operation of determining a movement amount of the electronic device based on a first image, which is an image of the surrounding environment at the time of initiating (or performing) the communication connection, and an image of the surrounding environment changed according to movement of the electronic device. The method of operating the face-mounted electronic device includes an operation of determining whether the movement amount of the electronic device exceeds a specified first value. The method of operating the face-mounted electronic device includes an operation of stopping the transmission of the avatar in response to the movement amount of the electronic device exceeding the specified first value.

[0010] In one embodiment, a face-mounted electronic device determines the movement of the electronic device based on an image of the surrounding environment that changes according to the movement of the electronic device, and stops transmitting an avatar based on the movement of the electronic device. The electronic device can retransmit the avatar according to specified conditions, and in the retransmission situation, re-determines the image of the surrounding environment to determine the characteristics (position of the avatar, size of the avatar, angle of the avatar) at which the avatar is displayed.

[0011] According to one embodiment, a face-mounted electronic device can prevent an avatar with unintended characteristics (e.g., size of the avatar, position of the avatar, angle of the avatar) from being displayed on an external electronic device even if the electronic device moves in response to the user's movement, because the avatar transmission is stopped based on the amount of movement of the electronic device. The electronic device can prevent an avatar from being continuously displayed on the display of the external electronic device in a situation where the user of the electronic device does not want the avatar to be displayed.

[0012] In one embodiment, a face-mounted electronic device may transmit an avatar based on an image of the surrounding environment at the time of retransmitting the avatar, rather than an image of the surrounding environment acquired at the time of establishing communication, in a situation where the electronic device retransmits the avatar. The electronic device may generate and / or display an avatar corresponding to the situation of retransmitting the avatar.

[0013] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs from the description below.

[0014] FIG. 1 is a block diagram of an exemplary electronic device capable of performing the operations described herein.

[0015] Figure 2 is a perspective view schematically showing the configuration of an electronic device according to one embodiment.

[0016] FIG. 3A is a drawing showing the front of a head mounted display device (300) according to one embodiment of the present disclosure.

[0017] FIG. 3b is a drawing showing the back of a head mounted display device (300) according to one embodiment of the present disclosure.

[0018] FIG. 4A is a diagram illustrating an electronic device according to an example. FIG. 4B is a diagram illustrating an avatar generated by an electronic device according to an example. FIG. 4C is a diagram illustrating an avatar generated by an electronic device according to an example.

[0019] Figure 5 is a block diagram of an electronic device according to one embodiment of the present invention.

[0020] FIG. 6A, FIG. 6B, and FIG. 6C are drawings for explaining an avatar generated by an electronic device according to one embodiment of the present invention.

[0021] FIG. 7A is a diagram illustrating the amount of movement of an electronic device according to one embodiment of the present invention. FIG. 7B is a diagram illustrating the amount of rotation of an electronic device according to one embodiment of the present invention.

[0022] FIG. 8 is a diagram illustrating an avatar displayed on an external electronic device according to various embodiments of the present invention.

[0023] FIG. 9 is a diagram illustrating conditions under which an electronic device (600) transmits an avatar according to one embodiment of the present invention.

[0024] FIG. 10 is a diagram illustrating conditions under which an electronic device (600) transmits an avatar according to one embodiment of the present invention.

[0025] FIG. 11 is a drawing illustrating a screen displayed by an electronic device (600) according to one embodiment of the present invention.

[0026] Figure 12 is a flowchart of the operation of an electronic device according to one embodiment of the present invention.

[0027] FIG. 13a is a flowchart of the operation of an electronic device according to one embodiment of the present invention. FIG. 13b is a flowchart of the operation of an electronic device according to one embodiment of the present invention.

[0028] FIG. 14 is a drawing illustrating a screen displayed by an electronic device according to one embodiment of the present invention.

[0029] FIG. 1 is a block diagram of an exemplary electronic device capable of performing the operations described herein.

[0030] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0031] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit (CPU) or an application processor (AP)) or an auxiliary processor (123) (e.g., a graphic processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0032] The auxiliary processor (123) may control at least a part of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0033] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0034] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (OS) (142), middleware (144), or an application (146).

[0035] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0036] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0037] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0038] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0039] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0040] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0041] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0042] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0043] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0044] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0045] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0046] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0047] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). NR access technology can support high-speed transmission of high-capacity data (eMBB, enhanced mobile broadband), minimizing terminal power and connecting multiple terminals (mMTC, massive machine type communications), or high reliability and low latency communications (URLLC, ultra-reliable and low-latency communications). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0048] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0049] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0050] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0051] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0052] FIG. 2 is a perspective view schematically illustrating the configuration of an electronic device according to one embodiment.

[0053] The wearable electronic device (200) illustrated in FIG. 2 (e.g., the electronic device (101) of FIG. 1) may be at least partially similar to the electronic device (101) of FIG. 1 or may further include other embodiments of the electronic device (101). The wearable electronic device (200) may include augmented reality (AR) glasses in the form of glasses, smart glasses, and / or a video see-through (VST) device.

[0054] Referring to FIG. 2, a wearable electronic device (200) may include a bridge (bridge, 201), a first rim (rim, 210), a second rim (220), a first end piece (end piece, 230), a second end piece (240), a first temple (temple, 250), and / or a second temple (260).

[0055] According to one embodiment, the bridge (201) can connect the first rim (210) and the second rim (220). The bridge (201) can be positioned over the user's nose when the user wears the wearable electronic device (200). The bridge (201) can separate the first rim (210) and the second rim (220) based on the user's nose.

[0056] According to one embodiment, the bridge (201) may include a camera module (203), a first gaze tracking camera (205), a second gaze tracking camera (207), and / or an audio module (209).

[0057] According to one embodiment, the camera module (203) (e.g., the camera module (180) of FIG. 1) can capture a front view (e.g., in the -y-axis direction) of a user (e.g., a user of a wearable electronic device (200)) and obtain image data. The camera module (203) can capture an image corresponding to the user's field of view (FoV) or measure a distance to a subject (e.g., an object). The camera module (203) may include an RGB camera, a high resolution (HR) camera, and / or a photo video (PV) camera. The camera module (203) may include a color camera that performs an auto focus (AF) function and an optical image stabilization (OIS) function to obtain a high-quality image.

[0058] According to one embodiment, the first gaze tracking camera (205) and the second gaze tracking camera (207) can track the gaze of the user. The first gaze tracking camera (205) and the second gaze tracking camera (207) can capture the pupil of the user's eyes looking in a direction opposite to the capturing direction of the camera module (203). For example, the first gaze tracking camera (205) can partially capture the left eye of the user, and the second gaze tracking camera (207) can partially capture the right eye of the user. The first gaze tracking camera (205) and the second gaze tracking camera (207) can detect the pupil of the user (e.g., the left eye and the right eye) and track the gaze direction corresponding to the movement of the detected pupil. The tracked gaze direction can be utilized to move the center of a virtual image including a virtual object corresponding to the gaze direction. The first gaze tracking camera (205) and / or the second gaze tracking camera (207) can track the user's gaze using, for example, at least one of an EOG sensor (electro-oculography or electrooculogram), a coil system, a dual Purkinje system, bright pupil systems, or dark pupil systems.

[0059] According to one embodiment, the audio module (209) (e.g., the audio module (170) of FIG. 1) may be positioned between the first gaze tracking camera (205) and the second gaze tracking camera (207). The audio module (209) may convert a user's voice into an electrical signal, or convert an electrical signal into sound (e.g., sound source data, audio signal). The audio module (209) may include a microphone.

[0060] According to one embodiment, the first rim (210) and the second rim (220) may form a frame (e.g., a glasses frame) of a wearable electronic device (200) (e.g., AR glasses). The first rim (210) may be disposed in a first direction (e.g., an x-axis direction) of the bridge (201). The first rim (210) may be disposed at a position corresponding to the user's left eye. The second rim (220) may be disposed in a second direction (e.g., a -x-axis direction) of the bridge (201) opposite to the first direction (e.g., an x-axis direction). The second rim (220) may be disposed at a position corresponding to the user's right eye. The first rim (210) and the second rim (220) may be formed of a metal material and / or a non-conductive material (e.g., a polymer).

[0061] According to one embodiment, the first rim (210) may surround and support at least a portion of the first glass (215) (e.g., the first display) disposed on the inner surface. The first glass (215) may be positioned in front of the user's left eye. The second rim (220) may surround and support at least a portion of the second glass (225) (e.g., the second display) disposed on the inner surface. The second glass (225) may be positioned in front of the user's right eye. The user of the wearable electronic device (200) may view a foreground (e.g., a real image) of an external object (e.g., a subject) through the first glass (215) and the second glass (225). The wearable electronic device (200) may implement augmented reality by superimposing a virtual image on the foreground (e.g., the real image) of the external object and displaying it. According to one embodiment, the wearable electronic device (200) may implement virtual reality and may express virtual reality through the first glass (215) and the second glass (225).

[0062] According to one embodiment, the first glass (215) and the second glass (225) may include a projection type transparent display. The first glass (215) and the second glass (225) may each form a reflective surface as a transparent plate (or transparent screen), and an image generated from the wearable electronic device (200) may be reflected (e.g., total internal reflection) through the reflective surface and incident on the left and right eyes of the user. In one embodiment, the first glass (215) may include an optical waveguide that transmits light generated from a light source of the wearable electronic device (200) to the left eye of the user. For example, the optical waveguide may be formed of glass, plastic, or a polymer material, and may include a nano-pattern (e.g., a grating structure or mesh structure having a polygonal or curved shape) formed on the inside or surface of the first glass (215). The optical waveguide may include at least one diffractive element (e.g., a Diffractive Optical Element (DOE), a Holographic Optical Element (HOE)) or at least one reflective element (e.g., a reflective mirror). The optical waveguide may guide display light emitted from a light source toward a user's eye using at least one diffractive element or reflective element included in the optical waveguide. In one embodiment, the diffractive element may include an input / output optical element, and the reflective element may include a total internal reflection (TIR) ​​element. For example, light emitted from a light source may be guided along a path to the optical waveguide through the input optical element, and light traveling inside the optical waveguide may be guided toward the user's eye through the output optical element. The second glass (225) may be implemented in substantially the same manner as the first glass (215).

[0063] According to one embodiment, the first glass (215) and the second glass (225) may include, for example, a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED). Although not shown, when the first glass (215) and the second glass (225) are formed of one of a liquid crystal display, a digital mirror display, or a silicon liquid crystal display, the wearable electronic device (200) may include a light source that irradiates light to the screen output areas of the first glass (215) and the second glass (225). In another embodiment, if the first glass (215) and the second glass (225) are capable of generating light on their own, for example, if they are made of either organic light emitting diodes or micro LEDs, the wearable electronic device (200) can provide a good quality virtual image to the user even without including a separate light source.

[0064] According to one embodiment, the first rim (210) may include a first microphone (211), a first recognition camera (213), a first light-emitting device (217), and / or a first display module (219). The second rim (220) may include a second microphone (221), a second recognition camera (223), a second light-emitting device (227), and / or a second display module (229).

[0065] In one embodiment, the first light-emitting device (217) and the first display module (219) may be included in a first end piece (230), and the second light-emitting device (227) and the second display module (229) may be included in a second end piece (240).

[0066] According to one embodiment, the first microphone (211) and / or the second microphone (221) can receive the voice (e.g., spoken voice, spoken sound source data) of a user wearing the wearable electronic device (200) on the head, and convert the received user's voice into an electric signal.

[0067] According to one embodiment, the first recognition camera (213) and / or the second recognition camera (223) can recognize the surrounding space of the wearable electronic device (200). The first recognition camera (213) and / or the second recognition camera (223) can detect a user's gesture within a certain distance (e.g., a certain space) of the wearable electronic device (200). The first recognition camera (213) and / or the second recognition camera (223) can include a GS (global shutter) camera capable of reducing the RS (rolling shutter) phenomenon in order to detect and track the user's rapid hand movements and / or fine movements of the fingers. The wearable electronic device (200) can detect an eye corresponding to the dominant eye and / or the auxiliary eye among the user's left and / or right eyes using the first gaze tracking camera (205), the second gaze tracking camera (207), the first recognition camera (213), and / or the second recognition camera (223). For example, the wearable electronic device (200) can detect an eye corresponding to the dominant eye and / or the auxiliary eye based on the user's gaze direction toward an external object or a virtual object.

[0068] According to one embodiment, the first light emitting device (217) and / or the second light emitting device (227) may emit light to increase the accuracy of the camera module (203), the first gaze tracking camera (205), the second gaze tracking camera (207), the first recognition camera (213) and / or the second recognition camera (223). The first light emitting device (217) and / or the second light emitting device (227) may be used as an auxiliary means to increase the accuracy when capturing the user's pupils using the first gaze tracking camera (205) and / or the second gaze tracking camera (207). The first light-emitting device (217) and / or the second light-emitting device (227) may be used as an auxiliary means when the user's gesture is captured using the first recognition camera (213) and / or the second recognition camera (223), and when it is not easy to detect an object to be captured (e.g., a subject) due to a dark environment or mixing and reflected light from multiple light sources. The first light-emitting device (217) and / or the second light-emitting device (227) may include, for example, at least one of an LED, an IR LED, and / or a xenon lamp.

[0069] According to one embodiment, the first display module (219) and / or the second display module (229) can emit light and transmit the light to the left eye and / or the right eye of the user using the first glass (215) and / or the second glass (225). The first glass (215) and / or the second glass (225) can display various image information using the light emitted through the first display module (219) and / or the second display module (229). The first display module (219) and / or the second display module (229) can include the display module (160) of FIG. 1. The wearable electronic device (200) can display a foreground of an external object (e.g., an actual external environment) and an image (e.g., a virtual object, a 3D (3-dimension) object) emitted through the first display module (219) and / or the second display module (229) by overlapping them through the first glass (215) and / or the second glass (225).

[0070] In one embodiment, the first end piece (230) may be coupled to a portion (e.g., in the x-axis direction) of the first rim (210). The second end piece (240) may be coupled to a portion (e.g., in the -x-axis direction) of the second rim (220). In one embodiment, the first light-emitting device (217) and the first display module (219) may be included in the first end piece (230). The second light-emitting device (227) and the second display module (229) may be included in the second end piece (240).

[0071] According to one embodiment, the first end piece (230) can connect the first rim (210) and the first temple (250). The second end piece (240) can connect the second rim (220) and the second temple (260).

[0072] According to one embodiment, the first temple (250) can be operatively connected to the first end piece (230) using a first hinge portion (255). The first hinge portion (255) can be configured to be rotatable so that the first temple (250) can be folded or unfolded with respect to the first rim (210). The first temple (250) can extend, for example, along the left side of the user's head. A distal portion (e.g., in the y-axis direction) of the first temple (250) can be configured in a curved shape so as to be supported, for example, on the user's left ear when the user wears the wearable electronic device (200). The second temple (260) can be operatively connected to the second end piece (240) using a second hinge portion (265). The second hinge portion (265) may be configured to be rotatable so that the second temple (260) can be folded or unfolded relative to the second rim (220). The second temple (260) may extend, for example, along the right side of the user's head. The distal portion (e.g., in the y-axis direction) of the second temple (260) may be configured in a curved shape so as to be supported, for example, on the user's right ear when the user wears the wearable electronic device (200).

[0073] According to one embodiment, the first temple (250) may include a first printed circuit board (251), a first audio output module (253) (e.g., audio output module (155) of FIG. 1) and / or a first battery (257) (e.g., battery (189) of FIG. 1). The second temple (260) may include a second printed circuit board (261), a second audio output module (263) (e.g., audio output module (155) of FIG. 1) and / or a second battery (267) (e.g., battery (189) of FIG. 1).

[0074] According to one embodiment, various electronic components (e.g., at least some of the components included in the electronic device (101) of FIG. 1), such as the processor (120), the memory (130), the interface (177), and / or the wireless communication module (192) disclosed in FIG. 1, may be disposed on the first printed circuit board (251) and / or the second printed circuit board (261). The processor may include, for example, one or more of a central processing unit, an application processor, a graphic processing unit, an image signal processor, a sensor hub processor, or a communication processor. The first printed circuit board (251) and / or the second printed circuit board (261) may include, for example, a printed circuit board (PCB), a flexible PCB (FPCB), or a rigid-flexible PCB (RFPCB). In some embodiments, the first printed circuit board (251) and / or the second printed circuit board (261) may include a Main PCB, a slave PCB positioned to partially overlap the Main PCB, and / or an interposer substrate between the Main PCB and the slave PCB. The first printed circuit board (251) and / or the second printed circuit board (261) may be electrically or operatively connected to other components (e.g., a camera module (203), a first gaze tracking camera (205), a second gaze tracking camera (207), an audio module (209), a first microphone (211), a first recognition camera (213), a first light-emitting device (217), a first display module (219), a second microphone (221), a second recognition camera (223), a second light-emitting device (227), a second display module (229), a first audio output module (253) and / or a second audio output module (263)) using electrical paths such as FPCBs and / or cables.For example, the FPCB and / or cable may be positioned on at least a portion of the first limb (210), the bridge (201), and / or the second limb (220). In some embodiments, the wearable electronic device (200) may include only one of the first printed circuit board (251) or the second printed circuit board (261).

[0075] In one embodiment, the first audio output module (253) and / or the second audio output module (263) may transmit audio signals to the user's left and / or right ears. The first audio output module (253) and / or the second audio output module (263) may include, for example, a piezo speaker (e.g., a bone conduction speaker) that transmits audio signals without a speaker hole. In some embodiments, the wearable electronic device (200) may include only one of the first audio output module (253) or the second audio output module (263).

[0076] In one embodiment, the first battery (257) and / or the second battery (267) may supply power to the first printed circuit board (251) and / or the second printed circuit board (261) using a power management module (e.g., the power management module (188) of FIG. 1 ). The first battery (257) and / or the second battery (267) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell. In some embodiments, the wearable electronic device (200) may include only one of the first battery (257) or the second battery (267).

[0077] According to one embodiment, the wearable electronic device (200) may include a sensor module (e.g., the sensor module (176) of FIG. 1). The sensor module may generate an electrical signal or data value corresponding to an internal operating state of the wearable electronic device (200) or an external environmental state. The sensor module may further include, for example, at least one of a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a color sensor, an infrared (IR) sensor, a biometric sensor (e.g., an HRM sensor), a temperature sensor, a humidity sensor, or an illuminance sensor. In some embodiments, the sensor module may recognize biometric information of a user by using various biometric sensors (or biometric recognition sensors), such as an e-nose sensor, an electromyography sensor, an electroencephalogram sensor, an ECG sensor, or an iris sensor.

[0078] According to one embodiment, the wearable electronic device (200) is described as a device that displays augmented reality using the first glass (215) and the second glass (225) in the above description, but is not limited thereto and may be a device that displays virtual reality (VR).

[0079] FIG. 3A is a drawing showing the front of a head mounted display device (300) according to one embodiment of the present disclosure.

[0080] FIG. 3b is a drawing showing the back of a head mounted display device (300) according to one embodiment of the present disclosure.

[0081] Referring to FIGS. 3A and 3B, a head mounted display device (300) can be worn on a part of a user's body to provide a user interface.

[0082] In one embodiment, the head-mounted display device (300) may output images and / or videos to the user. Alternatively, the head-mounted display device (300) may provide images related to augmented reality services and / or virtual reality services. For example, the head-mounted display device (300) may provide the user with experiences of augmented reality, virtual reality, mixed reality, and / or extended reality.

[0083] For example, a head-mounted display device (300) can provide augmented reality to a user. The head-mounted display device (300) can transmit a virtual object image output from the display (160) toward the user's eyes, and the virtual object image can utilize data regarding images of the real world captured through a plurality of cameras (330a, 330b, 330c).

[0084] In one embodiment, the head mounted display device (300) may be, for example, a head mounted display (HMD), a face mounted display (FMD), or may be, but is not limited to, smart glasses or a headset that provides extended reality such as augmented reality (AR), virtual reality glass (VR), or mixed reality.

[0085] In one embodiment, a head mounted display device (300) may include at least a portion of a housing (310), a plurality of cameras (330a, 330b, 330c), and a display (160).

[0086] In one embodiment, a head mounted display device (300) may include a housing (310). The housing (310) may be configured to accommodate at least one component. The housing (310) may include a first side (311a) (e.g., a front side), a second side (311b) opposite the first side (311a) (e.g., a back or wearing side), and a third side (311c) (e.g., a side side) between the first side (311a) and the second side (311b).

[0087] In one embodiment, the housing (310) may include a bridge (314). The bridge (314) may be configured to face a portion of the user's body, such as the nose. For example, the bridge (314) may be supported by the user's nose.

[0088] In one embodiment, the housing (310) may correspond to the main body of the head mounted display device (300). The housing (310) may be identical to the main body of the head mounted display device (300).

[0089] In one embodiment, the housing (310) may be mounted on the user's head by a wearable structure such as a template or strap.

[0090] In one embodiment, the head mounted display device (300) may include a lens structure (330). The lens structure (330) may include a plurality of lenses configured to adjust the focus of an image provided to a user. For example, the plurality of lenses may be configured to adjust the focus of an image output by the display (160). The plurality of lenses may be positioned at positions corresponding to the position of the display (160). The plurality of lenses may include, for example, a Fresnel lens, a pancake lens, a multi-channel lens, and / or any other suitable lens.

[0091] In one embodiment, the display (160) may be positioned corresponding to the lens structure (330).

[0092] In one embodiment, the head mounted display device (300) may include a display (160). The display (160) may be configured to provide an image (e.g., a virtual image) to a user. For example, the display (160) may include a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), and / or a micro light emitting diode (micro LED).

[0093] In one embodiment, when the display (160) includes at least one of a liquid crystal display (LCD), a digital mirror display, or a silicon liquid crystal display, the head mounted display device (300) may include a light source that irradiates light onto a screen output area of ​​the display (160).

[0094] In one embodiment, if the display (160) can generate light on its own, for example, if the display (160) includes at least one of an organic light emitting diode or a micro LED, the head mounted display device (300) can provide a good quality virtual image to the user even without including a separate light source.

[0095] In one embodiment, if the display (160) includes an organic light emitting diode or micro LED, a light source is unnecessary, and thus the head mounted display device (300) can be made lightweight. The head mounted display device (300) can include the display (160) and at least one transparent member. A user can use the head mounted display device (300) while wearing it on their face. The at least one transparent member can be formed of a glass plate, a plastic plate, or a polymer, and can be manufactured to be transparent or translucent.

[0096] In one embodiment, at least one transparent member may be positioned to face the user's right or left eye.

[0097] In one embodiment, if the display (160) is transparent, it may be positioned so as to face the user's eyes and form a screen display unit. In one embodiment, the display (160) may include a light source (not shown) configured to transmit a light signal to an area where an image is output.

[0098] In one embodiment, the display (160) can provide images to the user by generating optical signals on its own.

[0099] In one embodiment, the display (160) may be positioned on the second side (311b) of the housing (310). For example, one side of a pair of lenses of the display (160) may be positioned so that it is exposed to the outside through the second side (311b).

[0100] In one embodiment, the display (160) may be composed of organic light emitting diodes (OLEDs). For example, OLEDs can express red (R, red), green (G, green), and blue (B, blue) through self-luminescence of organic materials. However, the present invention is not limited thereto, and one pixel may include R, G, and B, and one chip may be implemented with multiple pixels including R, G, and B.

[0101] In one embodiment, the display (160) can display various images. Here, the image is a concept including still images and moving images, and the display (160) can display various images such as broadcast content, multimedia content, etc. In addition, the display (160) can also display a user interface (UI) and icons.

[0102] In one embodiment, the display (160) includes a separate IC chip, and the IC chip can display an image based on an image signal received from the processor (120). In one embodiment, the IC chip can generate driving signals for a plurality of light-emitting elements based on the image signal received from the processor (120), and display an image by controlling the light emission of a plurality of pixels included in the display panel based on the driving signals.

[0103] In one embodiment, the display (160) may include a plurality of pixels for displaying a virtual image. The display (160) may further include infrared pixels that emit infrared light.

[0104] In one embodiment, the display (160) may further include a photo-receiving pixel (e.g., a photo sensor pixel) disposed between the pixels and configured to receive light reflected from a user's eyes, convert it into electrical energy, and output it. The photo-receiving pixel may be referred to as a "gaze tracking sensor." The gaze tracking sensor may detect infrared light reflected by the user's eyes from light emitted by infrared pixels included in the display (160).

[0105] In one embodiment, the head mounted display device (300) can detect the user's gaze direction (e.g., eye movement) through light-receiving pixels.

[0106] In one embodiment, the head mounted display device (300) may determine the location of the center of the virtual image based on the gaze directions of the user's left and right eyes (e.g., the direction in which the pupils of the user's left and right eyes are looking) detected through one or more light-receiving pixels.

[0107] In one embodiment, the head mounted display device (300) may include at least one display. The head mounted display device (300) may include the display (160) as a main display and a 3D display as a secondary display or an external display.

[0108] In one embodiment, the display (160) may include a condenser lens and / or a transparent waveguide. For example, the transparent waveguide may be at least partially positioned on a portion of the glass.

[0109] In one embodiment, light emitted from the display (160) may be incident on one end of the glass, and the incident light may be transmitted to the user through a waveguide and / or waveguide (e.g., waveguide) formed within the glass. The waveguide may be made of glass, plastic, or polymer, and may include a nano-pattern formed on one inner or outer surface, for example, a grating structure having a polygonal or curved shape.

[0110] In one embodiment, the incident light can be propagated or reflected within the waveguide by the nano-pattern and provided to the user.

[0111] In one embodiment, the waveguide may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) or at least one reflective element (e.g., a reflective mirror).

[0112] In one embodiment, the waveguide can guide display light emitted from a light source to a user's eye using at least one diffractive element or reflective element.

[0113] In one embodiment, the waveguide serves to transmit light generated by the display to the user's eyes.

[0114] In one embodiment, the waveguide may be made of glass, plastic, or polymer and may include nano-patterns formed on some of its internal or external surfaces, for example, a grating structure having a polygonal or curved shape.

[0115] In one embodiment, light incident on one end of the waveguide can be propagated inside the display (160) optical waveguide by the nano-pattern and provided to the user. In addition, the optical waveguide composed of a free-form prism can provide the incident light to the user through a reflective mirror. The optical waveguide can include at least one diffractive element (e.g., a Diffractive Optical Element (DOE), a Holographic Optical Element (HOE)) or at least one reflective element (e.g., a reflective mirror). The optical waveguide can guide display light emitted from a light source unit to the user's eyes by using at least one diffractive element or reflective element included in the optical waveguide.

[0116] In one embodiment, the diffractive element may include an input optical member / output optical member (not shown). For example, the input optical member may mean an input grating area, and the output optical member (not shown) may mean an output grating area. The input grating area may serve as an input terminal that diffracts (or reflects) light output from (e.g., a Micro LED) to transmit the light to a transparent member (e.g., a first transparent member, a second transparent member) of a screen display unit. The output grating area may serve as an outlet that diffracts (or reflects) light transmitted to a transparent member (e.g., a first transparent member, a second transparent member) of a waveguide to a user's eye.

[0117] In one embodiment, the reflective element may comprise a total internal reflection (TIR) ​​optical element or waveguide for total internal reflection. For example, total internal reflection may refer to a method of guiding light such that light (e.g., a virtual image) entering through the input grating region is 100% reflected from one surface (e.g., a specific surface) of the waveguide, thereby transmitting 100% of the light to the output grating region.

[0118] In one embodiment, light emitted from the display (160) may be guided along an optical path through an input optical element into a waveguide. Light traveling within the waveguide may be guided toward the user's eyes through an output optical element. The screen display may be determined based on the light emitted toward the user's eyes.

[0119] In one embodiment, the head mounted display device (300) may include a sensor (176). The sensor (176) may be configured to detect the depth of a subject. The sensor (176) may be configured to transmit a signal toward the subject and / or receive a signal from the subject. For example, the transmitted signal may include near infrared, ultrasound, and / or laser. The sensor (176) may be configured to measure the time of flight (ToF) of the signal to measure the distance between the head mounted display device (300) and the subject. The sensor (176) may be disposed on a first surface (311a) of the housing (310).

[0120] In one embodiment, the sensor (176) may include a depth sensor. The depth sensor may be used to determine the distance to an object. The depth sensor (e.g., the depth sensor (335) of FIG. 3A) may include ToF (Time of Flight) technology. ToF technology may include technology that measures the distance to an object using signals (such as near-infrared, ultrasound, or laser). ToF technology may measure the flight time of a signal by transmitting a signal and measuring the signal at a receiving unit.

[0121] In one embodiment, the camera (180) of FIG. 1 may include multiple cameras (330a, 330b, 330c).

[0122] In one embodiment, the plurality of cameras (330a, 330b, 330c) may include at least some of the first camera (330a), the second camera (330b), or the third camera (330c). The plurality of cameras (330a, 330b, 330c) may capture images of the exterior of the housing (310), for example, a user and / or other subjects. For example, the plurality of cameras (330a, 330b, 330c) may convert optical signals into input data and provide the data to the processor (120). In one embodiment, the processor (120) may receive the received input data and transmit output data to the display (160). The processor (120) may combine data received from each of the plurality of cameras (330a, 330b, 330c) and process the combined data to control the display (160).

[0123] In one embodiment, a first camera (330a) including at least one camera for capturing and a second camera (330b) including at least one camera for recognition are spaced apart from a first surface (311a) of the housing (310) so as to capture a direction in which the first surface (311a) of the housing (310) faces.

[0124] In one embodiment, the camera (180) of FIG. 1 may include at least some of a first camera (330a), a second camera (330b), or a third camera (330c).

[0125] In one embodiment, the first camera (330a) and the second camera (330b) may be spaced apart from each other on the first side (311a) of the housing (310). The first camera (330a) and the second camera (330b) may be positioned to face different directions to capture various directions, such as the first side (311a) or the third side (311c).

[0126] In one embodiment, the first camera (330a) may be configured to acquire an image from a subject. The first camera (330a) may be formed in plurality, and one of the plurality of first cameras (330a) may be disposed in a portion of the first surface (311a) of the housing (310), and another first camera (330a) may be disposed in a portion of the first surface (311a) of the housing (310) and another portion of the housing (310).

[0127] In one embodiment, a plurality of first cameras (330a) may be positioned on each side of the depth sensor (335). The plurality of first cameras (330a) may include an image stabilizer actuator (not shown) and / or an autofocus actuator (not shown). For example, the plurality of first cameras (330a) may include at least one of a camera configured to acquire a color image, a global shutter camera, a rolling shutter camera, or a combination thereof.

[0128] In one embodiment, the second camera (330b) may be configured to recognize a subject. The second cameras (330b) may be formed in multiple numbers, and the multiple second cameras (330b) may be configured to detect and / or track an object (e.g., a human head or hand) or space with three or six degrees of freedom. For example, the multiple second cameras (330b) may include global shutter cameras. The multiple second cameras (330b) may be configured to perform simultaneous localization and mapping (SLAM) using depth information of the subject. The multiple second cameras (330b) may be configured to recognize gestures of the subject.

[0129] In one embodiment, a plurality of second cameras (330b) may be arranged on the first side (311a) of the housing (310).

[0130] In one embodiment, the first camera (330a) and the second camera (330b) may be cameras for taking pictures, and may be referred to as high resolution (HR) or photo video (PV) cameras, and may include high-resolution cameras. The first camera (330a) and the second camera (330b) may include a color camera equipped with functions for obtaining high-quality images, such as an auto focus (AF) function and an optical image stabilizer (OIS). However, the present invention is not limited thereto, and the first camera (330a) and the second camera (330b) may include a global shutter (GS) camera or a rolling shutter (RS) camera.

[0131] In one embodiment, the head-mounted display device (300) may include a plurality of third cameras (330c). The plurality of third cameras (330c) may be configured to recognize a user's face. For example, the plurality of third cameras (330c) may be configured to detect and track a user's facial expressions.

[0132] In one embodiment, the third camera (330c) may include at least one camera for facial recognition or at least one camera for eye tracking.

[0133] In one embodiment, the head-mounted display device (300) may further include an eye tracking camera in at least some of the plurality of third cameras (330c). The eye tracking camera may be used to detect and track eye movements.

[0134] In one embodiment, the third camera (330c) can detect and track the pupil. The third camera (330c) can include multiple cameras corresponding to the left and right eyes.

[0135] In one embodiment, at least one of the plurality of cameras (330a, 330b, 330c) may include a camera used for 3 degrees of freedom (DoF), 6 degrees of freedom (DoF) head tracking, hand detection and tracking, gesture and / or spatial recognition.

[0136] In one embodiment, at least one of the plurality of cameras (330a, 330b, 330c) may include a global shutter (GS) camera to detect and track head and hand movements. For example, a stereo camera may be used for head tracking and spatial recognition, so two global shutter (GS) cameras of the same specification and performance may be used, and a rolling shutter (RS) camera may be used to detect and track fine movements such as rapid hand and finger movements.

[0137] In one embodiment, at least one of the plurality of cameras (330a, 330b, 330c) may be primarily, but not necessarily limited to, a global shutter (GS) camera having superior camera performance (e.g., image drag) compared to other cameras, and for example, a rolling shutter (RS) camera may be used. At least one of the plurality of cameras (330a, 330b, 330c) may perform spatial recognition for six degrees of freedom (DoF) and simultaneous localization and mapping (SLAM) functions through depth capturing. At least one of the plurality of cameras (330a, 330b, 330c) may also perform a user gesture recognition function.

[0138] In one embodiment, the head-mounted display device (300) may include an inertial measurement unit (IMU) sensor. The IMU sensor may include at least one of an acceleration sensor, a gyroscope, or a magnetometer. The head-mounted display device (300) may detect the user's movements based on the IMU sensor.

[0139] In one embodiment, although not shown in the drawing, the head mounted display device (300) may include at least some of a sensor (not shown), a lighting unit (not shown), a plurality of microphones (not shown), a plurality of speakers (not shown), a battery (not shown), and a printed circuit board (not shown).

[0140] In one embodiment, the sensors (not shown) may be present in one or more configurations for various purposes (e.g., a gyro sensor, an acceleration sensor, a geomagnetic sensor, and / or a gesture sensor), and for example, the sensors (not shown) may perform at least one of head tracking for six degrees of freedom (DoF), pose estimation & prediction, gesture and / or spatial recognition, and / or slam functionality with depth imaging.

[0141] In one embodiment, the lighting unit (not shown) may have various uses depending on the location to which it is attached. For example, the lighting unit (not shown) may be attached around the second side (311b) of the head-mounted display device (300). The lighting unit (not shown) may be used as an auxiliary means to facilitate eye gaze detection when the gaze tracking camera (not shown) captures the pupil. The lighting unit (not shown) may use an infrared light emitting device (IR LED) of a visible light wavelength or an infrared wavelength.

[0142] For example, a lighting unit (not shown) may be attached to the front (311a) of the head-mounted display device (300) or its surroundings. The lighting unit (not shown) may be used as a means to supplement the surrounding brightness when multiple front cameras (330a, 330b) are shooting. The lighting unit (not shown) may be used particularly in dark environments or when it is difficult to detect a subject to be shot due to mixed or reflected light from multiple light sources.

[0143] In one embodiment, the lighting unit (not shown) may be omitted. The lighting unit (not shown) may be replaced by infrared pixels included in the display (160). The lighting unit (not shown) may also be included in the head mounted display device (300) to assist the infrared pixels included in the display (160).

[0144] In one embodiment, multiple microphones (not shown) can process external acoustic signals into electrical voice data. The processed voice data can be utilized in various ways depending on the function (or application) being performed by the head-mounted display device (300).

[0145] In one embodiment, a plurality of speakers (not shown) can output audio data received from the communication circuit or stored in the memory (120).

[0146] In one embodiment, one or more batteries (not shown) may be included in the head mounted display device (300) and may supply power to components that make up the head mounted display device (300).

[0147] In one embodiment, a printed circuit board (not shown) can transmit electrical signals to each circuit (e.g., camera, display, audio, or sensor) and other printed circuit boards through a flexible printed circuit board (FPCB).

[0148] In one embodiment, a printed circuit board (not shown) may have control circuitry (not shown) positioned thereon that controls components that make up the head mounted display device (300).

[0149] FIG. 4A is a diagram illustrating an electronic device according to an example. FIG. 4B is a diagram illustrating an avatar generated by an electronic device according to an example. FIG. 4C is a diagram illustrating an avatar generated by an electronic device according to an example.

[0150] The electronic device (400) can establish a communication connection with an external electronic device (410) using a communication circuit, and can perform a video call based on the communication connection. When performing a video call, the electronic device (400) can generate an avatar (430) corresponding to the user (402) of the electronic device (400). While performing the video call, the electronic device (400) can transmit the generated avatar (430) to the external electronic device (410), and the external electronic device (410) can display the avatar (430) received from the electronic device (400) on the display of the external electronic device (410). The external electronic device of the present disclosure may be a variety of devices. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices. An external electronic device (411) according to an example may be in the form of an HMD (Head Mounted Device). The external electronic device (411) in the form of an HMD can display an avatar (431) received from the electronic device on a display.

[0151] According to one embodiment, the electronic device (400) can move in response to the movement of the user (402) wearing the electronic device (400). The electronic device (400) can determine the characteristics (e.g., size of the avatar, position of the avatar, angle of the avatar) of the avatar (430) displayed based on the movement of the electronic device (400). For example, the electronic device (400) can change the size of the avatar based on the movement of the electronic device (400) in the forward-backward direction (+x, -x direction), or change the position displayed on the display of the external electronic device (410) based on the movement of the electronic device (400) in the left-right direction (+y, -y direction). However, since the size and / or location of the avatar (430) is determined based on the surrounding environment (401) and the location of the electronic device (400) at the time of the communication connection, an unintended avatar (440) may be displayed on the external electronic device (410) in a situation where the user (402) leaves. For example, in a situation where the user (402) leaves forward, the avatar may be displayed abnormally large.

[0152] According to one embodiment, the external electronic device (410) can display the received avatar (430) on the display. The external electronic device (410) can adjust the avatar so that it is continuously displayed on the display of the external electronic device (410) even if the avatar (430) moves according to the movement of the user (402) of the electronic device (400). For example, the external electronic device (410) can display the avatar (450) adjusted so that it does not disappear from the display even if the avatar moves. When the movement of the avatar according to the movement of the user (402) of the electronic device (400) is completed, the external electronic device (410) can adjust the avatar (450) to a specific location on the display screen of the display (e.g., a substantial central portion of the display screen of the display). However, the external electronic device (410) can adjust the display of the avatar (450) to a specific location (e.g., a substantial central portion of the display screen) of the display of the external electronic device (410) even in a situation where the user (402) wants to leave. Since the electronic device (400) cannot determine whether the user (402) does not want the avatar to be displayed, the avatar (450) can be continuously transmitted to the external electronic device (410). Accordingly, the avatar (450) can be displayed on the display of the external electronic device (410). For example, even in a situation where the user (402) leaves in the -y direction, the external electronic device (410) adjusts the avatar (450) so that it does not disappear from the display and adjusts it to a specific location (e.g., a substantial central portion of the display screen) of the display, so that the avatar (450) can be continuously displayed on the display.

[0153] According to one embodiment, in order to prevent abnormal transmission of an avatar (440) due to movement of an electronic device (400) or to prevent a situation in which an avatar (450) is displayed on an external electronic device (410) even when a user (402) leaves, the electronic device (400) needs a criterion for stopping avatar transmission. For example, if a movement amount or rotation amount exceeds a specified condition, the electronic device (400) needs to stop avatar transmission to the external electronic device (410). Electronic devices that stop or retransmit avatar transmission will be described below with reference to FIGS. 5 to 13.

[0154] Figure 5 is a block diagram of an electronic device according to one embodiment of the present invention.

[0155] According to one embodiment, an electronic device (500) (e.g., the electronic device (101) of FIG. 1, the electronic device (200) of FIG. 2, the head mounted display device (300) of FIG. 3, and the electronic device (600) of FIG. 6) may include a processor (510), a memory (520), a communication circuit (530), a first camera (540), and / or a second camera (550). Various embodiments of the present document may be implemented even if some of the illustrated configurations are omitted or replaced with other configurations. In addition to the illustrated configurations, the electronic device may further include at least some of the configurations and / or functions of the electronic device (101) of FIG. 1 and the electronic device (200) of FIG. 2. At least some of the respective components of the illustrated (or not illustrated) electronic devices may be operatively, functionally, and / or electrically connected to each other.

[0156] According to one embodiment, the processor (510) may include at least one processing circuitry, and the processor (510) may include at least one processor (510). The operations described below may be performed individually or collectively by at least one processor (510) included in the processor (510).

[0157] According to one embodiment, the memory (520) may store at least one computer program, and the at least one computer program may include instructions that may be executed by the processor (510). The operations of the processor (510) described below may be performed according to the execution of the instructions contained in the memory (520).

[0158] According to one embodiment, the communication circuit (530) may include various hardware and / or software configurations to support wireless communication with an external electronic device (104). The communication circuit (530) may support short-range wireless communication (e.g., Wi-Fi or Bluetooth) and / or cellular wireless communication (e.g., 4G LTE or 5G NR). The communication circuit (530) may include at least some of the configurations and / or functions of the communication module (190) of FIG. 1.

[0159] According to one embodiment, the first camera (540) may include at least one lens and at least one image sensor (image sensor). For example, the first camera (540) may acquire an image corresponding to a subject by using an image sensor that converts light emitted from or reflected by the subject and transmitted through at least one lens into an electrical signal. The first camera (540) may be positioned in a direction facing the user's face. The first camera (540) may capture at least a portion of the user's face. The first camera (540) may track changes on the user's face while the user wears or uses the electronic device (500).

[0160] According to one embodiment, the second camera (550) may include at least one lens and at least one image sensor. For example, the second camera (550) may obtain an image corresponding to a subject by using an image sensor that converts light emitted from the subject or reflected by the subject and transmitted through at least one lens into an electrical signal. The second camera (550) may be a camera distinct from the first camera (540). The second camera may be a camera that photographs the surrounding environment in which the electronic device (500) is located. The second camera may obtain an image of the surrounding environment that changes as the electronic device (500) moves.

[0161] According to one embodiment, the first camera (540) and the second camera (550) included in the electronic device (500) may include some of the gaze tracking cameras (e.g., the first gaze tracking camera (205) and the second gaze tracking camera (207) of FIG. 2) and the recognition cameras (e.g., the first recognition camera (213) and the second recognition camera (223) of FIG. 2), but are not limited to the above cameras. The first camera (540) may include at least one camera for acquiring an image including a user's face. The second camera (550) may include at least one camera for acquiring an image of the surrounding environment of the electronic device (500). The first camera (540) and the second camera (550) may transmit the acquired images to the processor (510).

[0162] According to one embodiment, the processor (510) may perform a communication connection with an external electronic device (104) using a communication circuit (530). The processor (510) may perform a voice call and / or a video call through the communication connection with the external electronic device (104).

[0163] According to one embodiment, when performing a communication connection with an external electronic device (104), the processor (510) may transmit at least one image to the external electronic device (104) using the communication circuit (530). For example, in a situation where a video call is performed with the external electronic device (104), the processor (510) may transmit an image of the user or an avatar representing the user to the external electronic device (104). The avatar described in the present disclosure may refer to a visual object corresponding to the user. For example, the avatar may be a visual object generated based on the characteristics of the user, or the avatar may be a visual object selected by the user. The present disclosure is not limited to avatars, and may be applied to displaying visual objects corresponding to the user as well as avatars.

[0164] According to one embodiment, the processor (510) may transmit information corresponding to an avatar to an external electronic device. The information corresponding to the avatar may include information extracted from images captured by the first camera (540) and the second camera (550). For example, the information corresponding to the avatar may include at least one of the avatar's face, the avatar's size, and the avatar's position. The processor may transmit the avatar to the external electronic device and transmit the information corresponding to the avatar to the external electronic device based on the images captured by the first camera (540) and the second camera (550). The external electronic device (104) may change the avatar displayed on the display of the external electronic device based on the information corresponding to the avatar.

[0165] According to one embodiment, the processor (510) may extract additional information based on images acquired through the first camera (540) and the second camera (550). The processor may transmit the extracted additional information to an external electronic device. The processor may extract additional information from images acquired through the first camera (540) and the second camera (550), even if the information does not correspond to an avatar. For example, the additional information may include information indicating the surrounding environment in which the user is located. If an object is included in the surrounding environment image acquired by the second camera (550), the processor may extract additional information related to the object and transmit the extracted information to the external electronic device. The external electronic device (104) may receive the additional information and display an object corresponding to the additional information on a display.

[0166] According to one embodiment, the first camera (540) can acquire an image including a portion of the user's face while the electronic device (500) and the external electronic device (104) are wearing the electronic device. The first camera (540) can capture a photograph of the user wearing the electronic device (500). The first camera (540) can include at least one camera for detecting movement of the user's face. The first camera (540) can capture a photograph of not only the user's face but also a portion of the user's body.

[0167] According to one embodiment, the first camera (540) can acquire an image including a portion of the user's face that changes according to the movement of the user's face. The first camera (540) can operate continuously as well as temporarily. The first camera (540) can include the first gaze tracking camera and the second gaze tracking camera of FIG. 2, and can track the user's gaze to identify the movement of the user's pupils. For example, the first camera (540) can identify a blinking motion of the user's eye of the electronic device (500). The first camera (540) can monitor not only the gaze but also other body parts (e.g., lips). The first camera (540) can transmit the acquired image including a portion of the user's face to the processor (510).

[0168] According to one embodiment, the second camera (550) can obtain an image of the surrounding environment of the electronic device (500) while the user wears the electronic device (500). The second camera (550) can include at least one camera for capturing an external image. The surrounding environment can be a space in which the electronic device (500) is located, and the surrounding environment image can be an image for rendering the space in which the electronic device (500) is located or a rendered image. The surrounding environment image can be an image of the space in which the user wearing the electronic device (500) is located, or an image that does not include the user. The surrounding environment image can be an image of the surrounding environment including objects in the space in which the electronic device (500) is located.

[0169] According to one embodiment, the second camera (550) can acquire an image of the surrounding environment that changes according to the movement of the electronic device (500). The second camera (550) can operate continuously as well as temporarily. The electronic device (500) can move according to the movement of the user wearing the electronic device (500), and the second camera (550) can capture the surrounding environment that changes according to the movement of the electronic device (500). The second camera (550) can transmit the acquired image of the surrounding environment to the processor (510).

[0170] According to one embodiment, the processor (510) may generate an avatar, which is a graphic object representing the user, based on an image including a portion of the user's face and / or an image of the surrounding environment that changes according to the movement of the electronic device (500). The processor (510) may generate an avatar at the time of performing a video call. The processor may reflect the face of the avatar corresponding to the user's face based on an image including at least a portion of the user's face acquired by the first camera to the generated avatar. The processor may reflect the characteristics (e.g., the size of the avatar, the position of the avatar, the angle of the avatar) for displaying the avatar based on the changing image of the surrounding environment acquired by the second camera to the generated avatar. The electronic device of the present disclosure generating an avatar may include reflecting the face of the avatar according to the image including at least a portion of the user's face acquired by the first camera to the generated avatar. The electronic device of the present disclosure generating an avatar may include reflecting the characteristics (e.g., the size of the avatar, the position of the avatar, the angle of the avatar) for displaying the avatar based on the changing image of the surrounding environment acquired by the second camera to the generated avatar.

[0171] According to one embodiment, the processor (510) may generate an avatar to be transmitted to an external electronic device using an image acquired using a second camera (550). The second camera (550) may acquire an image including a user's face. The processor (510) may extract information representing the user's face based on the image including the user's face acquired using the second camera (550). The processor (510) may identify an area corresponding to the user's face in the image acquired using the second camera (550) and identify a plurality of points in the area corresponding to the user's face. The plurality of points may refer to a plurality of points commonly included in a human face. The processor (510) may extract information representing the user's face based on the plurality of points included in the user's face. The information representing the user's face may include the position, shape, size, and / or color of a specific area on the user's face (e.g., an area corresponding to the eyes, nose, and / or mouth). Information representing a user's face may include information representing relationships (e.g., distances) between specific regions (e.g., regions corresponding to the eyes, nose, and / or mouth). The processor (510) may generate an avatar by reflecting the information representing the user's face. The processor may transmit the generated avatar to an external electronic device and change the characteristics of the avatar displayed on the display of the external electronic device based on an image acquired by the first camera (540).

[0172] According to one embodiment, the processor (510) may generate an avatar and transmit it to an external electronic device. The processor (510) may acquire an image including a portion of the user's face using the first camera (540) while the user wears the electronic device (500). The processor (510) may identify change points (e.g., eyes, nose, mouth) of the user's face in the image including a portion of the user's face. The change points may refer to at least one point on the user's face that may be set based on the user's face. The processor (510) may identify the location of the change points for each image acquired by the first camera (540) and generate information for updating the face of the avatar corresponding to the user's face. For example, the processor (510) may generate information that renders the user's face.

[0173] According to one embodiment, the processor (510) may transmit information reflecting a designated change point location for each image acquired by the first camera (540) to an external electronic device. For example, the processor (510) may transmit information for updating an avatar's face corresponding to the user's face for each image acquired by the first camera (540) to the external electronic device, and the external electronic device (104) may display an avatar according to the received change point location on a display. According to one example, the location of a change point corresponding to an eyelid on an image of the user's face acquired by the first camera may change in response to a blinking motion of the user. The processor (510) may transmit the location of the change point corresponding to the eyelid that changes as the user blinks to the external electronic device. The external electronic device (104) may update (or adjust) the location of the eyelid of the avatar's face in response to the location of the change point corresponding to the eyelid.

[0174] The processor (510) can create an avatar and change the display characteristics of the created avatar (e.g., the size, position, and angle of the avatar). The display characteristics of the avatar may include the display location and display size on the display of the external electronic device. The processor (510) can change the display characteristics of the avatar based on an image of the surrounding environment acquired using the second camera (550). Even after creating the avatar, the processor (510) can acquire an image including the user's face using the first camera (540) while the user is wearing the electronic device (500). The processor (510) can change the display characteristics of the avatar for each image of the surrounding environment acquired by the second camera and transmit the changed display characteristics to the external electronic device. The external electronic device (104) can display the avatar on the display according to the display characteristics of the avatar changed by the processor. The size and position of the avatar displayed on the display of the external electronic device can be changed based on information transmitted by the processor to the external electronic device. For example, the processor (510) may determine the size of the avatar based on the surrounding environment image acquired using the second camera (550). The surrounding environment image may change in response to the movement of the user wearing the electronic device (500). The processor may detect the movement of the electronic device based on the changed surrounding environment image. For example, if the electronic device (500) moves in response to the movement of the user, the surrounding environment image acquired by the second camera (550) may change in response to the movement of the electronic device. The processor (510) may change the position of the avatar displayed on the display of the external electronic device in response to the movement of the electronic device. The processor (510) may detect the direction and / or amount of movement of the electronic device based on the changed surrounding environment image.

[0175] According to one embodiment, the processor (510) can check the degree to which the surrounding environment image transmitted through the second camera (550) has changed, and determine the size of the avatar based on the changed surrounding environment image. The processor (510) can detect the movement of the electronic device (500) based on the changed surrounding environment image. The processor (510) can determine the size of the avatar displayed on the external electronic device (104) in response to the detected movement of the electronic device (500). For example, the processor (510) can detect a decrease in the size of an object located in front of the electronic device (500) in the changed surrounding environment image. In response to the decrease in the size of the object, the processor can detect that the electronic device is moving backward. When the processor (510) detects the movement of the electronic device moving backward, the processor can increase the size of the avatar displayed on the external electronic device (104). For example, the processor (510) may detect an increase in the size of an object located in front of the electronic device (500) in a changing surrounding environment image. In response to the increase in the size of the object, the processor (510) may detect that the electronic device moves forward. When the processor (510) detects the movement of the electronic device (500) moving forward, the processor (510) may reduce the size of an avatar displayed on the external electronic device (104).

[0176] According to one embodiment, the processor (510) may determine the position of the avatar displayed based on the changed surrounding environment image. The processor (510) may detect movement of the electronic device (500) based on the changed surrounding environment image. The processor (510) may determine the position of the avatar displayed on the external electronic device (104) in response to the detected movement of the electronic device (500). For example, the processor (510) may detect a decrease in the size of an object located on the left side of the electronic device (500) on the changed surrounding environment image. In response to the decrease in the size of the object, the processor may detect that the electronic device is moving to the right. When the processor (510) detects the movement of the electronic device moving to the right, the processor (510) may change the position of the avatar displayed on the external electronic device (104) to the right (or left). For example, the processor (510) may detect an increase in the size of an object located on the left side of the electronic device (500) on the changed surrounding environment image. The processor can detect that the electronic device moves to the left in response to an increase in the size of the object. When the processor (510) detects the movement of the electronic device moving to the left, the processor (510) can change the position of the avatar displayed on the external electronic device (104) to the left (or right). The processor (510) detecting the movement of the electronic device (500) by using the change in the size of an object located in front (or left) of the electronic device (500) is only an example, and the processor (510) may not necessarily detect the movement of the electronic device (500) based on the change in the size of the object in front (or left). The processor (510) can detect the movement of the electronic device (500) based on the size and / or relative position of an object located in one direction on a three-dimensional plane.According to one embodiment, the processor (510) may determine the location of the electronic device (500) based on the surrounding environment image, and may also detect movement of the electronic device (500) in response to the changing surrounding environment image.

[0177] According to one embodiment, the processor (510) may determine, at the time of connecting communication with the external electronic device (104), an image of the surrounding environment acquired through the second camera (550) as a first image for determining an avatar. The first image may be an image acquired by the second camera (550) before and after the time of connecting communication with the external electronic device (104) and transmitted to the processor (510). The processor (510) may temporarily store the first image received from the second camera (550) in the memory (520) and may compare the first image with an image of the surrounding environment changed from the first image. The processor (510) may use the result of the comparison to determine the characteristics of the avatar displayed (e.g., the size of the avatar, the position of the avatar, the angle of the avatar).

[0178] According to one embodiment, the processor (510) may transmit the generated avatar to the external electronic device (104) while connecting the communication. The transmitted avatar may include information related to the avatar, including the size of the avatar, the direction of movement of the avatar (or the position on the display of the external electronic device (104), the degree of movement of the avatar, the position of the electronic device (500), and / or the facial expression of the avatar. For example, the avatar or the characteristics of the avatar may vary depending on the images acquired by the first camera (540) and the second camera (550), and the processor (510) may continuously update the avatar using the images received from the first camera (540) and the second camera (550). The processor (510) may continuously transmit the updated avatar to the external electronic device (104).

[0179] According to one embodiment, the processor (510) can determine the amount of movement of the electronic device (500) based on the first image and the surrounding environment image changed according to the movement of the electronic device (500). The processor (510) can determine the location of the electronic device (500) based on the surrounding environment image, and can determine the amount of movement of the electronic device (500) based on the surrounding environment image changed according to the movement of the electronic device (500).

[0180] For example, the amount of movement of the electronic device (500) may refer to the degree of difference between the position of the electronic device (500) in the first image and the position of the electronic device (500) in the surrounding environment image that has changed according to the movement of the electronic device (500). The amount of movement used in the present disclosure may be a value that refers to the amount of change in the position between the initial position and the final position of the electronic device (500). The electronic device (500) may move in response to the movement of the user while connecting communication, and the surrounding environment image may change while connecting communication. As the surrounding environment image changes, the position of the electronic device (500) on the surrounding environment image may change, and the processor (510) may determine the degree to which the position of the electronic device (500) changes as the amount of movement of the electronic device (500). For example, the amount of movement of the electronic device (500) may be determined in response to the amount of movement of the user in a situation where the user is moving.

[0181] According to one embodiment, the processor (510) can determine the amount of movement of the electronic device using at least one sensor included in the electronic device (500).

[0182] According to one embodiment, the processor (510) may stop avatar transmission using at least one sensor. The electronic device (500) may include at least one sensor. The at least one sensor may detect whether the user is wearing the electronic device (500). For example, a sensor that tracks the user's gaze may detect that the user has taken off the electronic device (500) if the user's gaze cannot be tracked. The at least one sensor may transmit a result of detecting that the user has taken off the electronic device (500) to the processor (510). The processor (510) may stop avatar transmission in response to the user taking off the electronic device (500). The processor (510) may determine whether to stop avatar transmission based on the result detected by the at least one sensor as well as the user taking off the electronic device (500). For example, the sensor may detect a state preset by the user to stop avatar transmission, and the processor (510) may stop avatar transmission based on the result detected by the sensor.

[0183] According to one embodiment, the processor (510) may determine whether the movement amount of the electronic device (500) exceeds a specified first value based on an image of the surrounding environment that changes according to the movement of the electronic device (500). The processor (510) may stop avatar transmission in response to the movement amount of the electronic device (500) exceeding the specified first value. The processor (510) may transmit information to the external electronic device (104) instructing the external electronic device (104) not to move the avatar being displayed to a specified specific location on the display screen (e.g., a substantial central portion of the display screen) in response to the movement amount of the electronic device (500) exceeding the specified first value. Although only a situation in which avatar transmission is stopped is described below, the same may be applied to a situation in which the external electronic device (104) transmits information instructing the external electronic device (104) not to move the avatar being displayed to a specified specific location on the display screen (e.g., a substantial central portion of the display screen).

[0184] For example, the specified first value may be a value for determining whether the movement of the electronic device (500) is a movement that stops avatar transmission. The user may not want avatar transmission while performing a communication connection, and the specified first value may be a reference value for stopping avatar transmission when the movement of the electronic device (500) exceeds the specified first value.

[0185] The first value may refer to a value that can be set by the manufacturer of the electronic device (500). However, the first value may be changed by input from the user of the electronic device (500).

[0186] According to one embodiment, the processor (510) may receive a user input to determine whether to stop avatar transmission when the movement amount of the electronic device (500) exceeds a specified first value. The electronic device (500) may include at least one display and may display a pop-up on the display for selecting whether to stop avatar transmission. The electronic device (500) may include an interface unit (e.g., a touch input unit) for receiving a user input. The processor may determine whether to stop avatar transmission based on a user input (e.g., a voice input, a gesture input detectable by a camera) input to the interface unit. The processor (510) may stop avatar transmission in response to receiving a user input instructing to stop avatar transmission. The processor (510) may not stop avatar transmission in response to receiving a user input instructing not to stop avatar transmission even when the movement amount of the electronic device (500) exceeds the specified first value.

[0187] According to one embodiment, the processor (510) may stop the avatar transmission in response to receiving a user input instructing to stop the avatar transmission. The user input instructing to stop the avatar transmission may be received through a user input receiving area (e.g., a physical button, a touch input unit, or a voice input unit) included in the interface unit. The processor (510) may stop the avatar transmission in response to the user input instructing to stop the avatar transmission input through the interface unit even if the movement amount of the electronic device (500) does not exceed a specified first value.

[0188] According to one embodiment, the processor (510) may transmit information related to the interruption of avatar transmission to the external electronic device (104) while interrupting avatar transmission (or before or after interrupting avatar transmission). When interrupting avatar transmission, the electronic device (500) may transmit a signal to the external electronic device (104) instructing to interrupt avatar output. The signal instructing to interrupt avatar output may include information instructing not to output the avatar on the display of the external electronic device (104). The processor (510) may transmit, while interrupting avatar transmission, content to replace the avatar on the display of the external electronic device (104) and / or information instructing to output the content. The content to replace the avatar may include content output on the display of the external electronic device (e.g., a graphic object reflecting information generated based on an image including the user's face acquired by the first camera (540)) while the external electronic device (104) stops outputting the avatar. The processor (510) may transmit information to the external electronic device (104) instructing the avatar not to move to a specific location on the display screen of the designated display (e.g., a substantial central portion of the display screen of the display) even if the avatar displayed on the display of the external electronic device (104) leaves the screen due to movement of the electronic device (500).

[0189] According to one embodiment, even while not transmitting an avatar, the processor (510) can acquire images using the first camera (540) and the second camera (550), so that information generated based on the first camera (540) and the second camera (550) can be transmitted to an external electronic device (104).

[0190] For example, the processor (510) may transmit a graphic object that does not reflect the movement of the electronic device (500) according to the movement of the user to the external electronic device (104) while the avatar transmission is paused. For example, the processor (510) may transmit a graphic object that reflects information generated based on an image including the user's face acquired by the first camera (540) while the avatar transmission is paused.

[0191] According to one embodiment, when the processor (510) stops transmitting the avatar, it may store a second image, which is an image of the surrounding environment at the time when the transmission of the avatar is stopped, in the memory.

[0192] According to one embodiment, the processor (510) may determine whether the electronic device is at the location where the avatar transmission was stopped, based on the second image and the surrounding environment image changed according to the movement of the electronic device while the avatar transmission was stopped. The processor (510) may compare the surrounding environment image acquired by the second camera with the second image while the avatar transmission was stopped, to determine whether the electronic device is at the location where the avatar transmission was stopped. If at least a portion of an area included in the surrounding environment image acquired by the second camera while the avatar transmission was stopped matches at least a portion of an area of ​​the second image, the processor (510) may determine that the electronic device is at the location where the avatar transmission was stopped. According to one embodiment, the electronic device may determine that the electronic device is at the location where the avatar transmission was stopped, if the electronic device is within a certain distance from the location where the avatar transmission was stopped.

[0193] The processor (510) can transmit the avatar to an external electronic device when the electronic device is at a location where the avatar transmission has stopped.

[0194] According to one embodiment, the processor (510) may determine whether the movement amount of the electronic device, which is determined based on the first image, which is the image of the surrounding environment at the time of starting (or performing) the communication connection, and the image of the surrounding environment changed according to the movement of the electronic device, while the avatar transmission is stopped, is less than a specified second value. In response to the fact that the movement amount of the electronic device (500) is less than the specified second value, the processor (510) may transmit the avatar to the external electronic device (104). If the movement amount of the electronic device (500) becomes less than the specified second value and the avatar is retransmitted, the processor (510) may change the first image. For example, even if the electronic device (500) does not return to the location where the communication was connected, if the movement amount of the electronic device (500) becomes less than the second value and the avatar is retransmitted, the surrounding environment image acquired at the location where the avatar is retransmitted may be set as the first image.

[0195] For example, the specified second value may be a value for determining whether the movement of the electronic device (500) is a movement for retransmitting the avatar. The user may want to transmit the avatar while pausing the avatar transmission, and the specified second value may be a reference value for retransmitting the avatar when the movement amount of the electronic device (500) is less than the specified second value. For example, the second value may be determined to be a value less than or equal to the first value.

[0196] According to one embodiment, the second value may refer to a value that can be set by the manufacturer of the electronic device (500). However, the second value may be changed by an input from a user of the electronic device (500). The processor (510) may receive a user input instructing to transmit the avatar while the avatar transmission is paused. The user input instructing to transmit the avatar may be received through an area (e.g., a physical button, a touch input unit, or a voice input unit) that receives a user input included in the interface unit. The processor (510) may retransmit the avatar in response to the user input instructing to transmit the avatar inputted through the interface unit. When the processor (510) retransmits the avatar by receiving the user input instructing to transmit the avatar, the processor (510) may change the first image. For example, when the processor (510) retransmits the avatar, the surrounding environment image acquired at the retransmitting location may be set as the first image.

[0197] According to one embodiment, the processor (510) can learn a user's pattern of instructing to stop or resend avatar transmission based on an input instructing the user to stop transmitting the avatar and an input instructing the user to resend the avatar. The processor can generate a model that predicts the user's pattern based on the user's input. The processor can use the first camera, the second camera, and at least one sensor to determine a change in the user's state, and predict whether the user will instruct to stop or resend the avatar transmission based on the change in the user's state. The processor can stop transmitting the avatar in response to a predicted user input instructing to stop transmitting the avatar. The processor can resend the avatar in response to a predicted user input instructing to resend the avatar.

[0198] According to one embodiment, the external electronic device (104) can establish a communication connection with the electronic device (500) using at least one communication circuit (530). The external electronic device (104) can perform a voice call and / or a video call through the communication connection with the electronic device (500).

[0199] According to one embodiment, the external electronic device (104) can receive an avatar generated and transmitted by the electronic device (500). The external electronic device (104) can display the received avatar on the display of the external electronic device (104) while performing a video call with the electronic device (500).

[0200] According to one embodiment, the external electronic device (104) may receive an avatar generated by the electronic device (500) and display the avatar on the display of the external electronic device (104). The external electronic device (104) may receive the avatar continuously transmitted by the electronic device (500) and update the avatar output on the display. The external electronic device may receive information reflecting the change point location transmitted by the electronic device. For example, the external electronic device (104) may receive information for updating the face of an avatar corresponding to the user's face included in an image acquired by the first camera (540) of the electronic device. The external electronic device (104) may display an avatar according to the received change point location on the display. The avatar according to the received change point location may refer to an avatar newly generated by the external electronic device (104) by reflecting the change point location or an avatar that reflects the change point location in the avatar being displayed. In one example, the position of a change point corresponding to an eyelid on a user's face image acquired by a first camera may change in response to the user blinking. The electronic device may transmit the position of the change point corresponding to the eyelid that changes as the user blinks to an external electronic device (104). The external electronic device (104) may update (or adjust) the position of the eyelid on the avatar's face in response to the position of the change point corresponding to the eyelid.

[0201] The external electronic device (104) can adjust the avatar to a specific location on the screen (e.g., a location in the substantial center of the display) when the movement of the avatar according to the movement of the user of the electronic device (500) is completed.

[0202] According to one embodiment, the external electronic device (104) can change the avatar display according to information transmitted by the electronic device (500). For example, in a situation where the electronic device (500) stops transmitting the avatar, the external electronic device (104) can display information on the display of the external electronic device (104) indicating that the user of the electronic device (500) does not want to transmit the avatar. For example, in a situation where the electronic device (500) stops transmitting the avatar, the external electronic device (104) can display an avatar that does not change according to the movement of the electronic device (500) on the display of the external electronic device (104).

[0203] FIGS. 6A, 6B, and 6C are diagrams illustrating an avatar generated by an electronic device according to an embodiment of the present invention. For example, FIG. 6A is a diagram illustrating an electronic device (600) at a point in time when communication is established with an external electronic device. For example, FIGS. 6B and 6C are diagrams illustrating an avatar generated by an electronic device according to an embodiment of the present invention.

[0204] A user (602) of an electronic device (600) can perform a video call with a user of an external electronic device (410) (e.g., the external electronic device (104) of FIG. 5) while wearing the electronic device (600). The space where the electronic device (600) performs a video call with the external electronic device (410) is referred to as a surrounding environment (601).

[0205] The electronic device (600) may include a first camera (e.g., the first camera (540) of FIG. 5) and a second camera (e.g., the second camera (550) of FIG. 5). The first camera (e.g., the first camera (540) of FIG. 5) may include at least one camera for acquiring an image of a user's face (620). The second camera (e.g., the second camera (550) of FIG. 5) may include at least one camera for acquiring an image of the surrounding environment (601) of the electronic device (600).

[0206] Referring to FIG. 6A, the electronic device (600) can acquire an image of a user's face (620) while the user wears the electronic device. The image including the user's face may refer to an image captured by a first camera (e.g., the first camera (540) of FIG. 5) of the user's face (620). The electronic device (600) can acquire images of not only the user's face (620) but also a part of the user's body (602). The electronic device (600) can acquire an image of the user's face (620) that changes according to movement on the user's face (620). The first camera can operate continuously as well as temporarily. For example, the first camera can track the user's (602) gaze to determine the movement of the user's (602) pupils. The first camera (540) can monitor not only the gaze but also other body parts (e.g., lips). The first camera (540) can include multiple cameras. For example, the first camera (540) may include a camera that detects a part of the user's body and may detect movements on the user's body (e.g., face, arms, legs, torso, fingers) that control the avatar. The processor (510) may detect movements on the user's body based on the first camera (540) and control the avatar according to the movements on the user's body. For example, the processor (510) may stop transmitting the avatar or transmit the avatar in response to detecting a preset movement on the user's body. However, it should be understood that the first camera (540) is not limited to detecting the user's body. The first camera (540) may also capture (or detect) an area other than the user's body.

[0207] Referring to FIG. 6B, the electronic device (600) can acquire an image of the surrounding environment while the user wears the electronic device (600). While the electronic device (600) is performing a communication connection with an external electronic device (410), the electronic device (600) can determine the characteristics (location of the avatar, size of the avatar, angle of the avatar) on which the avatar is displayed based on the image of the surrounding environment (610, 615) of the electronic device (600). The surrounding environment (601) may be a space where the electronic device (600) is located. The image of the surrounding environment (610, 615) may be an image for rendering the space where the electronic device (600) is located or a rendered image. The image of the surrounding environment (610, 615) may be an image of the space where the user (602) wearing the electronic device (600) is located, or may be an image that does not include the user (602).

[0208] The electronic device (600) can generate an avatar, which is a graphic object representing the user (602), based on an image of the user's face (620) and images of the surrounding environment (610, 615) that change according to the movement of the electronic device (600). The electronic device (600) can generate an avatar when performing a video call. The video call can include all types of communication methods based on wired / wireless communication methods. The processor can reflect the face of the avatar corresponding to the user's face in the generated avatar based on an image including the user's face acquired by the first camera. The processor can reflect the characteristics of the avatar display (e.g., size of the avatar, position of the avatar, angle of the avatar) in the generated avatar based on an image of the surrounding environment that changes acquired by the second camera.

[0209] In one embodiment, the electronic device (600) may determine the face of an avatar based on an image of a user's face (620) acquired using a first camera. For example, the electronic device (600) may determine that the user (602) has performed an eye blinking action based on the image acquired by the first camera, and may determine that the avatar generated by the electronic device (600) blinks.

[0210] According to one embodiment, the electronic device (600) can determine the size of the avatar and the position at which the avatar is displayed on the display of the external electronic device (410) based on the surrounding environment image (610, 615) acquired using the second camera. The electronic device (600) can determine the surrounding environment image (610) acquired through the second camera as the first image (610) for determining the avatar while the user wears the electronic device (600). The electronic device (600) can compare the first image (610) and the surrounding environment image (615) changed from the first image according to the movement of the electronic device (600). The electronic device (600) can use the result of the comparison to determine the size of the avatar and the position at which the avatar is displayed on the display of the external electronic device (410).

[0211] For example, when the user (602) moves to the right (e.g., +y direction), the surrounding environment image (615) acquired by the second camera may change. The changed surrounding environment image (615) may be a changed surrounding environment image acquired at a location where the electronic device (600) has moved to the right (e.g., +y direction). The electronic device (600) may detect the movement of the electronic device based on the changed surrounding environment image (615). The electronic device (600) may be set to display an avatar (635) that has moved to the left on the display according to the detected rightward movement of the electronic device. The same may be applied to the leftward movement of the user (602), and thus, a related description will be omitted.

[0212] For example, when the user (602) moves backward (e.g., in the +x direction), the surrounding environment image (616) acquired by the second camera may change. The changed surrounding environment image (616) may be a changed surrounding environment image acquired at a location where the electronic device (600) has moved backward. The electronic device (600) may detect the movement of the electronic device based on the changed surrounding environment image (616). The electronic device (600) may be set to display an avatar (636) with a reduced size according to the detected backward movement of the electronic device. Since the same may be applied to the forward movement of the user (602), a related description will be omitted.

[0213] According to one embodiment, the electronic device (600) may determine the characteristics of the avatar display based on a user input indicating the size of the avatar and the location where the avatar is to be displayed on the display of the external electronic device (410). The user input may be received through an area (e.g., a physical button, a touch input unit, or a voice input unit) included in an interface unit of the electronic device (600) that receives user input. The user input may be received through a control device that controls the electronic device. For example, the control device may refer to an accessory and / or a controller that performs communication with the electronic device.

[0214] According to one embodiment, the electronic device (600) may transmit at least one avatar or data corresponding to the avatar among the generated avatars (630, 635, 636) to the external electronic device (410) while connecting a communication connection. The transmitted avatar may include information related to the avatar, including the size of the avatar, the direction of movement (or position on the display of the external electronic device (410)) of the avatar, the degree of movement of the avatar, the position of the electronic device (600), and / or the facial expression of the avatar. The external electronic device (410) may receive the avatar generated and transmitted by the electronic device (600). The external electronic device (410) may display the received avatar on the display of the external electronic device (410) while performing a video call with the electronic device (600).

[0215] FIG. 7A is a diagram illustrating the amount of movement of an electronic device according to one embodiment of the present invention.

[0216] In one embodiment, a user may move while wearing an electronic device (600) that performs a communication connection with an electronic device (104). For example, the user (602) may move to a different location within the surrounding space (601) to change the location where a video call is performed. When the user (602) moves, the electronic device (600) may determine the amount of movement (700) of the electronic device (600) according to the movement of the user (602).

[0217] According to one embodiment, the electronic device (600) can determine the amount of movement (700) of the electronic device (600) based on the first image (610) and the surrounding environment image (710) that has changed according to the movement of the electronic device (600). The electronic device (600) can determine the location (611, 711) of the electronic device (600) based on the surrounding environment images (610, 710), and can determine the amount of movement (700) of the electronic device (600) based on the location (711) of the electronic device (600) that has changed according to the movement of the electronic device (600). For example, the amount of movement (700) of the electronic device (600) may be a value representing the difference between the position (611) of the electronic device (600) in the first image (610) and the position (711) of the electronic device (600) in the surrounding environment image (710) that has changed according to the movement of the electronic device (600). Since the electronic device (600) may continuously move in response to the movement of the user while connecting communication, the surrounding environment image (710) may continuously change while connecting communication. As the surrounding environment image (710) changes, the position (711) of the electronic device (600) on the surrounding environment image (710) may change from the position (611) of the electronic device (600) on the basic image (610), and the electronic device (600) may determine the degree to which the position of the electronic device (600) changes as the amount of movement (700) of the electronic device (600).

[0218] According to one embodiment, the electronic device (600) may stop avatar transmission when the movement amount (700) of the electronic device (600) exceeds a specified first value. For example, the specified first value may be a value for determining whether the movement of the electronic device (600) is a movement for stopping avatar transmission. The specified first value may be a reference value for stopping avatar transmission when the movement of the electronic device (600) exceeds the specified first value.

[0219] According to one embodiment, the electronic device (600) may transmit information to the external electronic device (104) instructing the external electronic device (104) not to move the avatar being displayed to a specific location on the designated display (e.g., a substantially central portion of the display screen of the display) if the amount of movement (700) of the electronic device (600) exceeds a specified first value.

[0220] According to one embodiment, the electronic device (600) can prevent an unintended avatar from being displayed on the external electronic device (104) even if the electronic device (600) moves in response to the user's movement because the electronic device (600) stops transmitting the avatar based on the amount of movement (700) of the electronic device (600). The electronic device (600) can prevent a situation in which the avatar is continuously displayed on the display of the external electronic device (104) in a situation in which the user of the electronic device (600) does not want it.

[0221] FIG. 7b is a diagram illustrating the amount of rotation of an electronic device according to one embodiment of the present invention.

[0222] According to one embodiment, a user (602) may rotate while wearing an electronic device (600) that performs a communication connection with an external electronic device (104). For example, the user (602) may rotate while maintaining a position (e.g., the position (611) of the electronic device at the time of connection of communication in FIG. 7A) to change the angle at which a video call is performed. When the user (602) rotates, the electronic device (600) may check the rotation amount (721) of the electronic device (600) according to the rotation of the user (602).

[0223] According to one embodiment, the electronic device (600) can determine the rotation amount (721) of the electronic device (600) based on the first image and the surrounding environment image (720) changed according to the rotation of the electronic device (600). For example, the rotation amount (721) of the electronic device (600) can be a value representing the position of the electronic device (600) in the first image and the rotation angle of the electronic device (600) in the surrounding environment image (720) changed according to the rotation of the electronic device (600). Since the electronic device (600) can rotate in response to the rotation of the user (602) while connecting communication, the surrounding environment image (720) can change while connecting communication. As the surrounding environment image (720) changes, the electronic device (600) on the surrounding environment image (720) may be rotated compared to the electronic device (600) on the first image (610), and the electronic device (600) may determine the degree to which the electronic device (600) is rotated as the rotation amount (721) of the electronic device (600). According to one embodiment, the electronic device (600) may determine the degree to which at least one object included in the surrounding environment image is rotated, and determine the rotation amount (721) of the electronic device based on the degree to which the object is rotated.

[0224] According to one embodiment, the electronic device (600) can detect a rotation angle of the electronic device (600) using at least one sensor (e.g., a 6-axis sensor). The electronic device (600) can determine a rotation amount (721) of the electronic device based on the rotation angle of the electronic device (600).

[0225] According to one embodiment, the electronic device (600) may stop transmitting the avatar if the rotation amount (721) of the electronic device (600) exceeds a specified angle. The electronic device (600) may transmit information to the external electronic device (410) instructing the external electronic device (410) not to move the avatar being displayed to a specific location on the specified display (e.g., a substantial central portion of the display screen of the display) if the rotation amount (721) of the electronic device (600) exceeds the specified angle.

[0226] For example, the electronic device (600) may stop transmitting the avatar when both a condition in which the amount of movement of the electronic device (600) exceeds a specified first value and a condition in which the amount of rotation (721) of the electronic device (600) exceeds a specified angle are satisfied. Alternatively, the electronic device (600) may transmit information to the external electronic device (410) instructing the external electronic device (410) not to move the avatar being displayed to a specific location on the specified display (e.g., a substantial central portion of the display screen of the display) when both a condition in which the amount of movement of the electronic device (600) exceeds a specified first value and a condition in which the amount of rotation (721) of the electronic device (600) exceeds a specified angle are satisfied.

[0227] According to one embodiment, the electronic device (600) may determine the speed, acceleration, and / or angular velocity of the electronic device (600) based on the first image and / or the surrounding environment image changed according to the rotation (or movement) of the electronic device (600). The electronic device (600) may determine whether the speed, acceleration, and / or angular velocity of the electronic device (600) exceeds a specified value. The electronic device (600) may determine whether to transmit an avatar and whether to stop transmitting the avatar based on whether at least one value among the amount of movement of the electronic device (600), the amount of rotation of the electronic device (600), the speed of the electronic device (600), the acceleration of the electronic device (600), and the angular velocity of the electronic device (600) exceeds a specified value (e.g., a first value, a second value). The specified value may be set by the manufacturer separately from the first value and the second value, and may be changed by the user.

[0228] According to one embodiment, the electronic device (600) may include at least one sensor (e.g., an IMU sensor) (not shown). The electronic device (600) may detect the amount of movement of the electronic device (600), the amount of rotation of the electronic device (600), the speed of the electronic device (600), and the angular velocity of the electronic device (600) using the at least one sensor. The electronic device (600) may stop transmitting the avatar based on the result detected using the at least one sensor.

[0229] FIG. 8 is a diagram illustrating an avatar displayed on an external electronic device according to various embodiments of the present invention.

[0230] According to one embodiment, the electronic device (600) can change the position at which an avatar is displayed on the display of the external electronic device (410) based on an image of the surrounding environment (e.g., the image of the surrounding environment (710, 720) of FIG. 7) that changes according to the movement of the electronic device (600). The external electronic device (410) can display an avatar (805) positioned at the changed position of the display of the external electronic device (410) according to the movement of the electronic device (600) on the display.

[0231] According to one embodiment, when the movement of the avatar due to the movement of the electronic device (600) is completed, the external electronic device (410) may adjust the avatar to a specific location on the display screen (e.g., a substantially central portion of the display screen of the display). When the movement of the electronic device (600) is stopped, the external electronic device (410) may move the avatar displayed on the side of the display of the external electronic device (410) to a specific location on the display screen (e.g., a substantially central portion of the display screen of the display). The display screen of the display may be displayed in a portion of the display. It should be understood that the specific location at which the avatar is displayed refers to a specific location on the display screen, and not a specific location of the display. For example, even if the avatar is displayed in the center of the display screen, it may not be displayed in the central portion of the display.

[0232] According to one embodiment, the electronic device (600) may not transmit an avatar if the amount of movement of the electronic device (600) (e.g., the amount of movement of the electronic device (700) of FIG. 7) exceeds a specified first value. The external electronic device (410) cannot receive the avatar and therefore cannot display the avatar (805) on the display.

[0233] According to one embodiment, the electronic device (600) may transmit information to the external electronic device (410) instructing the external electronic device (410) not to move the avatar being displayed to a specific location on the display screen (e.g., a substantial central portion of the display screen) when the amount of movement of the electronic device (600) (e.g., the amount of movement (700) of the electronic device of FIG. 7) exceeds a specified first value. Since the external electronic device (410) does not move the avatar to the specific location on the display screen (e.g., a substantial central portion of the display screen), the avatar may disappear from the display as the electronic device (600) moves.

[0234] In one embodiment, the electronic device (600) may, in response to receiving a user input instructing to stop transmitting the avatar, transmit information to the external electronic device (410) instructing not to transmit the avatar or not to move the avatar being displayed by the external electronic device (410) to a specific location on the display screen of the display (e.g., a substantially central portion of the display screen of the display).

[0235] The external electronic device (410) may display information (810) on the display indicating that the avatar cannot be displayed in response to the avatar disappearing from the display.

[0236] An electronic device (600) can acquire a user's face image and an image of the surrounding environment using a first camera (e.g., a first camera (540) of FIG. 5) and a second camera (e.g., a second camera (550) of FIG. 5) while the user wears the electronic device (600). Since the electronic device (600) can acquire a user's face image and an image of the surrounding environment even when it does not transmit an avatar, it can transmit information generated based on the first camera (e.g., a first camera (540) of FIG. 5) and the second camera (e.g., a second camera (550) of FIG. 5) to an external electronic device (410). The electronic device (600) can stop avatar transmission when the movement amount (700) of the electronic device (600) exceeds a specified first value, and in response to stopping avatar transmission, can transmit information generated based on a first camera (e.g., a first camera (540) of FIG. 5) and a second camera (e.g., a second camera (550) of FIG. 5) to an external electronic device (410).

[0237] For example, while the electronic device (600) stops transmitting the avatar, the electronic device (600) may transmit information based on an image including the user's face acquired by the first camera (e.g., the first camera (540) of FIG. 5), and the external electronic device (410) may display a graphic object on the display according to the information based on the image including the user's face. According to an example, the graphic object may be a graphic object (820) that does not reflect the movement of the electronic device (600). The graphic object (820) displayed on the external electronic device (410) may change in response to a change in the user's face and may not change in response to the movement of the electronic device (600).

[0238] For example, the electronic device (600) can identify change points (e.g., eyes, nose, mouth) of the user's face in an image including the user's face acquired by the first camera (e.g., the first camera (540) of FIG. 5). The change points may refer to at least one point on the user's face that can be set based on the user's face. The electronic device (600) can identify the location of the change points for each image acquired by the first camera (e.g., the first camera (540) of FIG. 5) and generate information for updating the face of an avatar corresponding to the user's face. For example, the electronic device (600) can generate information that renders the user's face.

[0239] The electronic device (600) can transmit information reflecting a designated change point location for each image acquired by the first camera (e.g., the first camera (540) of FIG. 5) to an external electronic device. For example, the electronic device (600) can transmit information for updating a graphic object corresponding to the user's face for each image acquired by the first camera (e.g., the first camera (540) of FIG. 5) to the external electronic device, and the external electronic device can display a graphic object according to the received change point location on the display. According to one example, the location of a change point corresponding to an eyelid on an image of the user's face acquired by the first camera can change in response to a blinking motion of the user. The electronic device (600) can transmit the location of the change point corresponding to the eyelid that changes as the user blinks to the external electronic device. The external electronic device can update (or adjust) the location of the eyelid of the graphic object's face in response to the location of the change point corresponding to the eyelid.

[0240] FIG. 9 is a diagram illustrating conditions under which an electronic device (600) transmits an avatar according to one embodiment of the present invention.

[0241] Even while the avatar transmission is stopped, the electronic device (600) can obtain an image of the surrounding environment (910) using a second camera (e.g., the second camera (550) of FIG. 5). Even while the avatar transmission is stopped, the electronic device (600) can check (or determine) the amount of movement (900) of the electronic device (600) based on the image of the surrounding environment (910) that changes according to the movement of the electronic device (600). The amount of movement (900) of the electronic device (600) can be determined based on the first image (610) at the time of communication connection and the changed image of the surrounding environment (910), and the value may not be initialized even if the avatar transmission is stopped. The electronic device (600) can determine the location (611, 911) of the electronic device (600) based on the surrounding environment image (610, 910), and can determine the amount of movement (900) of the electronic device (600) based on the location (911) of the electronic device (600) that has changed according to the movement of the electronic device (600). For example, the amount of movement (900) of the electronic device (600) may be a value representing the difference between the location (611) of the electronic device (600) in the first image (610) and the location (911) of the electronic device (600) in the surrounding environment image (910) that has changed according to the movement of the electronic device (600).

[0242] According to one embodiment, the electronic device (600) may not retransmit the avatar even if the movement amount (900) of the electronic device (600) becomes less than the first value while stopping the avatar transmission. Since the condition for the electronic device (600) to stop the avatar transmission and the condition for retransmitting the avatar may not match, the electronic device (600) may determine whether to retransmit the avatar based on a separate condition (e.g., the condition that the movement amount (900) of the electronic device (600) becomes less than the second value).

[0243] According to one embodiment, the electronic device (600) may determine whether the movement amount (900) of the electronic device (600) is less than a specified second value while stopping the avatar transmission. The electronic device (600) may transmit the avatar to the external electronic device (410) in response to the movement amount (900) of the electronic device (600) being less than the specified second value. For example, the specified second value may be a value for determining whether the movement of the electronic device (600) is a movement for retransmitting the avatar. The specified second value may be a reference value for retransmitting the avatar when the movement amount (900) of the electronic device (600) is less than the specified second value.

[0244] According to one example, the electronic device (600) may change the first image (e.g., the first image (610) of FIG. 6) when the movement amount (900) of the electronic device (600) becomes less than a specified second value and the avatar is retransmitted. For example, even if the electronic device (600) does not return to the location where the communication was connected, the electronic device (600) may retransmit the avatar when the movement amount (900) of the electronic device (600) becomes less than the second value. The electronic device (600) may set the surrounding environment image (e.g., 910) acquired at the location where the avatar is retransmitted as the first image. As the first image is changed, the electronic device (600) may determine the characteristics (size of the avatar and the location at which the avatar is displayed on the display of the external electronic device (410)) at which the avatar is displayed based on the re-set reference image (e.g., reference image (910)). The size of the avatar and the location where the avatar is displayed on the display of the external electronic device (410) are described in FIG. 6, so the related content is omitted in FIG. 9.

[0245] According to one example, the electronic device (600) may not change the first image if the movement amount (900) of the electronic device (600) is less than a specified second value and retransmits the avatar. The avatar retransmitted by the electronic device (600) may be an avatar generated based on the first image at the time of the communication connection (e.g., the first image (610) of FIG. 6). For example, the electronic device (600) may display a pop-up on a display included in the electronic device (600) for receiving a user input regarding whether to change the first image. The electronic device (600) may determine whether to change the first image based on the user input.

[0246] According to one embodiment, the electronic device (600) may determine the amount of movement (not shown) of the electronic device (600) based on the location of the electronic device (600) that connected the video call as well as the location of the electronic device (600) that stopped transmitting the avatar. According to one example, the electronic device (600) may store the location where the avatar transmission was stopped and calculate the amount of movement (not shown) of the electronic device (600) from the location where the avatar transmission was stopped. If the amount of movement (not shown) of the electronic device (600) from the location where the avatar transmission was stopped is less than a specified value, the electronic device (600) may transmit the avatar to the external electronic device (410). The electronic device (600) calculating the amount of movement (not shown) of the electronic device (600) may include calculating it based on the location information of the electronic device (600) at the location where the avatar transmission was stopped, a SLAM map, and an image of the surrounding environment.

[0247] According to one embodiment, when the electronic device (600) stops transmitting the avatar in response to the movement amount of the electronic device (600) exceeding a first value, the electronic device (600) may store information related to the departure location of the electronic device (600). The information related to the departure location of the electronic device (600) may include an image of the surrounding environment of the electronic device (600), location information of the electronic device (600), and a Simultaneous Localization and Mapping (SLAM) map. The location information of the electronic device (600) may refer to information indicating the location where the electronic device (600) is located, and the electronic device (600) may obtain the location information using at least one sensor or camera (e.g., a second camera). The electronic device (600) may also receive the location information of the electronic device (600) from an external server. A SLAM map may refer to a map generated based on data (e.g., an image of the surrounding environment) acquired by an electronic device (600) using a camera (e.g., a second camera) or at least one sensor. The electronic device (600) may generate a SLAM map of the space in which the electronic device (600) is located, including surrounding objects (e.g., walls, obstacles).

[0248] In one embodiment, the electronic device (600) can only stop transmitting the avatar when the user wishes to leave, since the electronic device (600) retransmits the avatar based on the amount of movement (900) of the electronic device (600). The electronic device (600) can transmit the avatar as the user intends by retransmitting the avatar when the user approaches the location where the communication connection was made.

[0249] FIG. 10 is a diagram illustrating conditions under which an electronic device (600) transmits an avatar according to one embodiment of the present invention.

[0250] The user can transmit the avatar even if the movement amount of the electronic device (600) does not fall below the specified second value. For example, the user can make a video call using the avatar from a changed location.

[0251] The electronic device (600) may receive a user input instructing to transmit the avatar while the avatar transmission is paused. The user input instructing to transmit the avatar may be received through a user input receiving area (e.g., a physical button, a touch input unit, or a voice input unit) included in the interface unit. The electronic device (600) may retransmit the avatar in response to the user input instructing to transmit the avatar inputted through the interface unit.

[0252] According to one embodiment, the electronic device (600) can acquire an image of the surrounding environment using the second camera even while the transmission of the avatar is stopped. When the electronic device (600) retransmits the avatar by receiving a user input instructing to transmit the avatar, the electronic device (600) can change the first image (e.g., the first image (610) of FIG. 6). For example, when the electronic device (600) retransmits the avatar, the surrounding image (1010) acquired at the retransmission location can be set as the first image. As the first image is changed, the electronic device (600) can determine the characteristics of the avatar display (the size of the avatar and the position at which the avatar is displayed on the display of the external electronic device (104)) based on the re-set first image (1010). The size of the avatar and the position at which the avatar is displayed on the display of the external electronic device (104) have been described in FIG. 6, and therefore, related details are omitted in FIG. 10.

[0253] According to FIGS. 9 and 10, the electronic device (600) can change the first image (910, 1010) in a situation where an avatar is retransmitted. The electronic device (600) may not determine the characteristics for displaying the avatar based on the first image (610) acquired at the time of connecting communication. The electronic device (600) can transmit the avatar according to the characteristics for displaying the avatar corresponding to the situation where an avatar is retransmitted.

[0254] FIG. 11 is a drawing illustrating a screen displayed by an electronic device (600) according to one embodiment of the present invention.

[0255] The electronic device (600) may include at least one display. The electronic device (600) may display a screen for video calling with an external electronic device (104) on at least one display.

[0256] According to one example, the electronic device (600) may receive a user input for determining whether to stop avatar transmission when the amount of movement of the electronic device (600) (e.g., the amount of movement (700) of the electronic device of FIG. 7) exceeds a specified first value. The electronic device (600) may display a pop-up (or notification) (1120) for selecting whether to stop avatar transmission on at least one display. The electronic device (600) may include an interface unit (e.g., a touch input unit) for receiving a user input. The processor may determine whether to stop avatar transmission based on a user input input to the interface unit (e.g., a voice input, a gesture input detectable by a camera, a user movement acquired through a sensor). The electronic device (600) may stop avatar transmission in response to receiving a user input instructing to stop avatar transmission. In response to receiving a user input instructing not to stop avatar transmission, the electronic device (600) may not stop avatar transmission even if the movement amount of the electronic device (600) exceeds a specified first value.

[0257] According to one example, the electronic device (600) may display a pop-up (1130) on the display for selecting information to be transmitted to the external electronic device (104) when the amount of movement of the electronic device (600) exceeds a specified first value. For example, the electronic device (600) may display a pop-up on the display for transmitting a graphic object that does not reflect the movement of the electronic device (600) according to the movement of the user. The graphic object that does not reflect the movement of the electronic device (600) may be a graphic object based on an image including the user's face acquired by the first camera.

[0258] According to one example, the electronic device (600) may receive a user input to determine whether the movement is for changing the location for performing a video call when the amount of movement of the electronic device (600) exceeds a specified first value. The electronic device (600) may display a pop-up (or notification) (1140) for selecting whether to change the location for performing the video call. In response to receiving a user input for changing the location for performing the video call, the electronic device (600) may set an image of the surrounding environment acquired at the changed location as the first image when the change in the location of the electronic device (600) is completed.

[0259] Figure 12 is a flowchart of the operation of an electronic device according to one embodiment of the present invention.

[0260] The operations described through FIG. 12 can be implemented based on instructions that can be stored in a computer recording medium or memory (e.g., memory (520) of FIG. 5). The illustrated method (FIG. 12) can be executed by the electronic devices described through FIGS. 1 to 11 above (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2, head mounted display device (300) of FIG. 3, electronic device (600) of FIG. 6), and the technical features described above will be omitted below. The order of each operation of FIG. 12 can be changed, some operations can be omitted, and some operations can be performed simultaneously.

[0261] In operation 1210, the electronic device may initiate (or perform) a communication connection to transmit voice and / or video with an external electronic device using a communication circuit. The electronic device may perform a voice call and / or a video call through the communication connection with the external electronic device. When performing the communication connection with the external electronic device, the electronic device may transmit at least one image to the external electronic device using the communication circuit. For example, the electronic device may transmit an image of the user or an avatar representing the user to the external electronic device in a situation where a video call is performed with the external electronic device.

[0262] The electronic device may, in operation 1220, generate an avatar corresponding to the user based on an image including at least a portion of the user's face and an image of the surrounding environment that changes according to the movement of the electronic device.

[0263] According to one embodiment, the operation of the electronic device to create an avatar may include an operation of reflecting information related to the face and movement of the avatar to the avatar of the user that has been previously registered (or created). The electronic device may check information related to the face of the avatar corresponding to an image including at least a portion of the user's face and / or information related to the movement of the avatar corresponding to an image of the surrounding environment that changes according to the movement of the electronic device. According to one example, the electronic device may transmit the registered (or created) avatar to an external electronic device, and transmit information related to features on the face of the avatar (e.g., change points) that change in response to a change in the image including at least a portion of the user's face to the external electronic device. According to one example, the electronic device may transmit the registered (or created) avatar to the external electronic device, and transmit information related to at least one of the size, position, or angle of the avatar that changes in response to an image of the surrounding environment that changes when the electronic device moves to the external electronic device.

[0264] In one embodiment, the electronic device can determine the face of an avatar based on an image containing the user's face acquired using a first camera. For example, the electronic device can generate an avatar based on an image of the surrounding environment acquired using a second camera, and change (or determine) the face of the avatar based on the image containing the user's face.

[0265] The electronic device can determine at least one of the size, position, or angle of the avatar based on an image of the surrounding environment acquired using the second camera.

[0266] According to one embodiment, the electronic device may determine a first image, which is an image of the surrounding environment acquired through the second camera, as a reference image for determining an avatar when initiating (or performing) a communication connection with an external electronic device. The first image may be an image acquired by the second camera and transmitted to the electronic device before or after initiating (or performing) a connection with the external electronic device, and the electronic device may temporarily store the first image received from the second camera in a memory, and compare the first image with an image of the surrounding environment changed from the first image. The electronic device may use the result of the comparison to determine the characteristics (position of the avatar, size of the avatar, angle of the avatar) on which the avatar is displayed.

[0267] The electronic device may transmit the avatar to the external electronic device at operation 1230.

[0268] The transmitted avatar may include information related to the avatar, including the size of the avatar, the direction of movement of the avatar (or the position on the display of the external electronic device), the extent to which the avatar moves, the position of the electronic device, and / or the facial expression of the avatar. For example, the characteristics of the avatar being displayed (the position of the avatar, the size of the avatar, the angle of the avatar) may vary depending on the images acquired by the first camera and the second camera, and the electronic device may continuously update the characteristics of the avatar being displayed (the position of the avatar, the size of the avatar, the angle of the avatar) using the images received from the first camera and the second camera. The electronic device may continuously transmit the characteristics of the updated avatar being displayed (the position of the avatar, the size of the avatar) to the external electronic device. The electronic device may identify changes in the user's face (e.g., eyes, nose, mouth) in an image including the user's face. The changes may refer to at least one point on the user's face that may be set based on the user's face. The electronic device can transmit information reflecting the specified change point location for each image acquired by the first camera (540) to an external electronic device. For example, the electronic device can transmit information for updating the avatar's face for each image acquired by the first camera (540) to the external electronic device.

[0269] An electronic device can transmit information related to accessories that may be included in an avatar to an external electronic device. Accessories that may be included in an avatar may include visual objects representing the avatar. For example, accessories may refer to clothing, glasses, shoes, and / or hats that adorn the avatar.

[0270] The electronic device can transmit at least one text corresponding to the avatar to an external electronic device. The at least one text corresponding to the avatar may include text indicating the avatar's status. For example, the at least one text corresponding to the avatar may include text indicating the avatar's facial expression, location, and / or size.

[0271] The electronic device can extract additional information based on images acquired through the first camera (540) and the second camera (550). The electronic device can transmit the extracted additional information to an external electronic device. Even if the information does not correspond to an avatar, the electronic device can extract additional information from the images acquired through the first camera (540) and the second camera (550). For example, the additional information may include information indicating the surrounding environment in which the user is located. If an object is included in the surrounding environment image acquired by the second camera (550), the electronic device can extract additional information related to the object and transmit the extracted information to the external electronic device.

[0272] The electronic device can, in operation 1240, determine the amount of movement of the electronic device based on a first image, which is an image of the surrounding environment at the time of starting (or performing) the communication connection, and an image of the surrounding environment changed according to the movement of the electronic device.

[0273] The electronic device can determine the amount of movement of the electronic device based on the first image and the surrounding environment image that changes according to the movement of the electronic device. The electronic device can determine the location of the electronic device based on the surrounding environment image, and can determine the amount of movement of the electronic device based on the surrounding environment image that changes according to the movement of the electronic device. The electronic device can determine whether the amount of movement of the electronic device exceeds a specified first value based on the surrounding environment image that changes according to the movement of the electronic device.

[0274] For example, an electronic device can determine the initial location of the electronic device based on a first image, and determine the location after the electronic device has moved based on an image of the surrounding environment that changes as the electronic device moves. The electronic device can determine the amount of movement of the electronic device based on the distance between the initial location and the location after the movement.

[0275] According to one embodiment, the electronic device may determine the amount of movement of the electronic device based on at least one sensor included in the electronic device.

[0276] The electronic device may determine, at operation 1250, whether the movement of the electronic device exceeds a specified first value.

[0277] The electronic device may, at operation 1260, stop transmitting the avatar in response to the movement of the electronic device exceeding the specified first value.

[0278] The electronic device may stop transmitting the avatar in response to the movement of the electronic device exceeding a specified first value. The electronic device may stop transmitting the avatar in response to receiving a user input instructing the device to stop transmitting the avatar even if the movement of the electronic device does not exceed the specified first value. While stopping the avatar transmission, the electronic device may transmit information related to the stop of the avatar transmission to the external electronic device.

[0279] According to one embodiment, when the electronic device stops transmitting an avatar in response to the movement amount of the electronic device exceeding a first value, the electronic device may store information related to a departure location of the electronic device. The information related to the departure location of the electronic device may include an image of the surrounding environment of the electronic device, location information of the electronic device, and a Simultaneous Localization and Mapping (SLAM) map. The location information of the electronic device may refer to information indicating a location where the electronic device is located, and the electronic device may obtain the location information using at least one sensor or camera (e.g., a second camera). The electronic device may also receive the location information of the electronic device from an external server. The SLAM map may refer to a map generated based on data (e.g., an image of the surrounding environment) obtained by the electronic device using a camera (e.g., a second camera) or at least one sensor. The electronic device may generate a SLAM map of a space where the electronic device is located, including surrounding objects (e.g., a wall, an obstacle).

[0280] According to one embodiment, an electronic device can determine whether the electronic device is at a location where avatar transmission was interrupted using an image of the surrounding environment acquired using at least one sensor or a second camera included in the electronic device. The electronic device can determine that the electronic device is at a location where avatar transmission was interrupted even if the electronic device is at a location less than a predetermined distance from the location where avatar transmission was interrupted. The electronic device can determine whether the electronic device is at a location where avatar transmission was interrupted using not only the image of the surrounding environment but also information related to the location of the electronic device. The electronic device can update a SLAM map in response to the movement of the electronic device and determine whether the electronic device is at a location where avatar transmission was interrupted by checking the location of the electronic device on the updated SLAM map. If the electronic device is at a location where avatar transmission was interrupted, the electronic device can retransmit the avatar.

[0281] According to one embodiment, the electronic device may determine whether the movement amount of the electronic device, which is determined based on a first image, which is an image of the surrounding environment at the time of starting (or performing) the communication connection, and an image of the surrounding environment changed according to the movement of the electronic device, is less than a specified second value while the avatar transmission is stopped. The electronic device may transmit the avatar to an external electronic device in response to the movement amount of the electronic device being less than the specified second value. For example, the specified second value may be a value for determining whether the movement of the electronic device is a movement for retransmitting the avatar. The user may want to transmit the avatar while the avatar transmission is stopped, and the specified second value may be a reference value for retransmitting the avatar when the movement amount of the electronic device is less than the specified second value. When the electronic device retransmits the avatar because the movement amount of the electronic device is less than the specified second value, the electronic device may change the first image (e.g., the first image (610) of FIG. 6).

[0282] According to one embodiment, the electronic device can determine the movement amount of the electronic device based on the location of the electronic device connected to the video call or the location of the electronic device that stopped transmitting the avatar. According to one example, the electronic device can store the location where the avatar transmission stopped and calculate the movement amount of the electronic device from the location where the avatar transmission stopped. If the movement amount of the electronic device from the location where the avatar transmission stopped is less than a specified value, the electronic device can transmit the avatar to an external electronic device. The electronic device calculating the movement amount of the electronic device may include calculating it based on the location information of the electronic device at the location where the avatar transmission stopped, the SLAM map, and the surrounding environment image.

[0283] According to one embodiment, the electronic device (600) may determine the rotation amount of the electronic device (e.g., the rotation amount (721) of FIG. 7B), the speed, the acceleration, and / or the angular velocity of the electronic device (600) based on a first image (e.g., the first image (610) of FIG. 6) and an image of the surrounding environment changed according to the rotation (or movement) of the electronic device (600). The electronic device (600) may determine whether the speed, the acceleration, and / or the angular velocity of the electronic device (600) exceeds a specified value. The electronic device (600) may determine whether to transmit the avatar based on whether at least one value of the movement amount of the electronic device (600), the rotation amount of the electronic device (600), the speed of the electronic device (600), the acceleration of the electronic device (600), and the angular velocity of the electronic device (600) exceeds a specified value (e.g., a second value). The specified value may be set by the manufacturer separately from the second value and may be changed by the user.

[0284] According to one embodiment, an electronic device may receive a user input instructing to transmit an avatar. In response to receiving the user input, the electronic device may determine the characteristics (location of the avatar, size of the avatar, angle of the avatar) at which the avatar is displayed based on an image of the surrounding environment that changes according to the movement of the electronic device after receiving the user input. When retransmitting the avatar by receiving the user input instructing to transmit the avatar, the first image (e.g., the first image (610) of FIG. 6) may be changed. For example, when the electronic device retransmits the avatar, the image of the surrounding environment acquired at the retransmitting location may be set as the first image. As the first image is changed, the electronic device may determine the size of the avatar and the location at which the avatar is displayed on the display of the external electronic device based on the first image that has been re-set. The electronic device may transmit the determined avatar to the external electronic device.

[0285] Figure 13a is a flowchart of the operation of an electronic device according to one embodiment of the present invention.

[0286] The operations described through FIG. 13a can be implemented based on instructions that can be stored in a computer recording medium or memory (e.g., memory (520) of FIG. 5). The illustrated method (FIG. 13) can be executed by the electronic devices described through FIGS. 1 to 11 above (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2, head mounted display device (300) of FIG. 3, electronic device (600) of FIG. 6), and the technical features described above will be omitted below. The order of each operation of FIG. 13a can be changed, some operations can be omitted, and some operations can be performed simultaneously. The operation of FIG. 13a can be performed continuously with the operation of FIG. 12.

[0287] According to one embodiment, the electronic device can determine whether the electronic device is at a location where the avatar transmission was stopped, based on a second image, which is an image of the surroundings at the time the avatar transmission was stopped, and an image of the surroundings that has changed according to the movement of the electronic device while the avatar transmission was stopped, in operation 1301.

[0288] When the electronic device stops transmitting the avatar, it can store in its memory a second image, which is an image of the surrounding environment at the time the transmission of the avatar is stopped.

[0289] The electronic device can compare the surrounding environment image acquired by the second camera with the second image while the transmission of the avatar is stopped to determine whether the electronic device is at the location where the transmission of the avatar was stopped.

[0290] The electronic device may determine that the electronic device is at the location where the avatar transmission was interrupted if at least a portion of an area included in an image of the surrounding environment acquired by the second camera matches at least a portion of an area of ​​the second image while the avatar transmission was interrupted. In one embodiment, the electronic device may determine that the electronic device is at the location where the avatar transmission was interrupted if the electronic device is within a certain distance from the location where the avatar transmission was interrupted.

[0291] According to one embodiment, the electronic device may transmit the avatar to an external electronic device if, in operation 1302, the electronic device is at a location where it stopped transmitting the avatar.

[0292] Figure 13b is a flowchart of operations of an electronic device according to one embodiment of the present invention.

[0293] According to one embodiment, the electronic device may determine, at operation 1310, whether the movement amount of the electronic device, which is determined based on a first image of the surrounding environment at the time of starting (or performing) the communication connection while stopping the avatar transmission, and an image of the surrounding environment changed according to the movement of the electronic device, is less than a specified second value.

[0294] According to one embodiment, the electronic device can determine the distance between the location where the electronic device initiated (or performed) a communication connection and the current location of the electronic device as the amount of movement of the electronic device.

[0295] According to one embodiment, the electronic device may transmit the avatar to the external electronic device in response to the movement amount of the electronic device being less than the specified second value, at operation 1320.

[0296] For example, the specified second value may be a value for determining whether the movement of the electronic device is a movement for retransmitting the avatar. The user may want to transmit the avatar while pausing the avatar transmission, and the specified second value may be a reference value for retransmitting the avatar when the movement of the electronic device is less than the specified second value.

[0297] FIG. 14 is a drawing illustrating a screen displayed by an electronic device according to one embodiment of the present invention.

[0298] The electronic device (600) may include at least one display. The electronic device (600) may display a screen for video calling with an external electronic device (104) on at least one display.

[0299] According to one example, in response to the interruption of transmission of an avatar, the electronic device (600) may display a notification window (1440) on the display notifying the interruption of transmission of the avatar. The notification window (1440) may include an icon (1441) notifying the interruption of transmission of the avatar and / or text (1442) notifying the interruption of transmission of the avatar.

[0300] According to one example, the electronic device (600) can receive a user input to determine whether to transmit an avatar. The electronic device (600) can display a pop-up (or notification) (1450) on at least one display to select whether to transmit the avatar. The electronic device (600) can include an interface unit (e.g., a touch input unit) for receiving user input. The electronic device can determine whether to stop transmitting the avatar based on a user input (e.g., a voice input, a gesture input detectable by a camera, or a user movement acquired by a sensor) input to the interface unit.

[0301] According to one embodiment, a face-mounted electronic device may include communication circuitry. The electronic device may include a first camera for acquiring an image including at least a portion of a user's face. The electronic device may include a second camera for acquiring an image of an environment surrounding the electronic device. The electronic device may include a memory storing at least one computer program including instructions. The electronic device may include at least one processor. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to initiate (or perform) a communication connection for transmitting audio and / or video with an external electronic device using the communication circuitry. The instructions may cause the electronic device to generate an avatar corresponding to the user based on an image including at least a portion of the user's face and / or an image of the environment surrounding the user that changes according to movement of the electronic device. The instructions may cause the avatar to be transmitted to the external electronic device. The instructions may determine the amount of movement of the electronic device based on a first image, which is an image of the surrounding environment at the time of initiating (or performing) the communication connection, and an image of the surrounding environment changed according to the movement of the electronic device. The instructions may determine whether the amount of movement of the electronic device exceeds a specified first value. The instructions may cause the avatar transmission to be stopped in response to the amount of movement of the electronic device exceeding the specified first value.

[0302] In one embodiment, in a face-mounted electronic device, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to store, in the memory, a second image that is an image of the surrounding environment at the time when transmission of the avatar is stopped.

[0303] In one embodiment, in a face-mounted electronic device, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine whether the electronic device is at a location where it stopped transmitting the avatar based on an image of the surrounding environment changed in response to movement of the electronic device while the electronic device was stopped transmitting the second image and the avatar. The instructions may cause the electronic device to transmit the avatar to an external electronic device if the electronic device is at a location where it stopped transmitting the avatar.

[0304] In one embodiment, in a face-mounted electronic device, the instructions may cause the at least one processor, when individually or collectively executed, to determine whether a movement amount of the electronic device, determined based on a first image, which is an image of the surrounding environment at a time point when the communication connection is started (or performed), and an image of the surrounding environment changed according to movement of the electronic device, while the electronic device is stopping transmission of the avatar, is less than a specified second value. The instructions may cause the electronic device to transmit the avatar to an external electronic device in response to the movement amount being less than the specified second value.

[0305] In one embodiment, in a face-mounted electronic device, a characteristic to be displayed on a display may be determined based on the first image and an image of the surrounding environment that changes according to movement of the electronic device. In the face-mounted electronic device, the characteristic to be displayed on the display may be an avatar including at least one of a size, a position, or an angle of the avatar.

[0306] In one embodiment, in a face-mounted electronic device, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to receive a user input instructing the device to transmit an avatar while the avatar transmission is paused. The instructions may cause the device to transmit the avatar in response to receiving the user input instructing the device to transmit the avatar.

[0307] In one embodiment, in a face-mounted electronic device, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine a location of the electronic device based on the surrounding environment image acquired using the second camera while performing the communication connection.

[0308] In one embodiment, in a face-mounted electronic device, the instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to determine, as a movement amount of the electronic device, a distance between a location of the identified electronic device and a location at which the identified electronic device initiated (or performed) a communication connection based on the first image.

[0309] In one embodiment, in a face-mounted electronic device, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine whether a rotation amount of the electronic device exceeds a specified value based on a change in the image of the surrounding environment. The instructions may cause the electronic device to stop transmitting the avatar in response to the rotation amount of the electronic device exceeding the specified value.

[0310] In one embodiment, in a face-mounted electronic device, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate a graphical object corresponding to the user's face in response to a movement amount of the electronic device exceeding a specified value. The instructions may cause the graphical object to be displayed on a display of the external electronic device to be set to a predetermined value and / or a size. The instructions may cause the graphical object to be transmitted to the external electronic device.

[0311] A method of operating a face-mounted electronic device according to one embodiment may include an operation of initiating (or performing) a communication connection for transmitting voice and / or image with an external electronic device using the communication circuit. The method of operating the face-mounted electronic device may include an operation of generating an avatar corresponding to the user based on an image including at least a portion of the user's face and / or an image of the surrounding environment that changes according to movement of the electronic device. The method of operating the face-mounted electronic device may include an operation of transmitting the avatar to the external electronic device. The method of operating the face-mounted electronic device may include an operation of determining an amount of movement of the electronic device based on a first image, which is an image of the surrounding environment at the time of initiating (or performing) the communication connection, and an image of the surrounding environment that changes according to movement of the electronic device. The method of operating the face-mounted electronic device may include an operation of determining whether the amount of movement of the electronic device exceeds a specified first value. The method of operating the face-mounted electronic device may include an operation of stopping transmission of the avatar in response to the amount of movement of the electronic device exceeding the specified first value.

[0312] According to one embodiment, the method of operating the face-mounted electronic device may include an operation of storing, in the memory, a second image, which is an image of the surrounding environment at the time of stopping transmission of the avatar, when transmission of the avatar is stopped.

[0313] In one embodiment, the method for operating the face-mounted electronic device may include an operation of determining whether the electronic device is at a location where the avatar transmission was stopped, based on an image of the surrounding environment changed according to movement of the electronic device while the second image and the avatar transmission were stopped. The method for operating the face-mounted electronic device may include an operation of transmitting the avatar to an external electronic device, if the electronic device is at a location where the avatar transmission was stopped.

[0314] In one embodiment, the method for operating the face-mounted electronic device may include an operation for determining whether the movement amount of the electronic device, which is determined based on a first image, which is an image of the surrounding environment at the time of starting (or performing) the communication connection, and an image of the surrounding environment changed according to the movement of the electronic device, is less than a specified second value while the transmission of the avatar is stopped. The method for operating the face-mounted electronic device may include an operation for transmitting the avatar to an external electronic device in response to the fact that the movement amount of the electronic device is less than the specified second value.

[0315] In one embodiment, in the method of operating the face-mounted electronic device, a characteristic to be displayed on the display is determined based on the first image and an image of the surrounding environment that changes according to movement of the electronic device, and the characteristic to be displayed on the display may be an avatar including at least one of a size, a position, or an angle of the avatar.

[0316] In one embodiment, the method of operating the face-mounted electronic device may include an operation of receiving a user input instructing to transmit the avatar while the avatar transmission is paused. The method of operating the face-mounted electronic device may include an operation of transmitting the avatar in response to receiving the user input instructing to transmit the avatar.

[0317] According to one embodiment, the method of operating the face-mounted electronic device may include an operation of confirming the location of the electronic device based on the surrounding environment image acquired using the second camera while performing the communication connection.

[0318] According to one embodiment, the method for operating the face-mounted electronic device may include an operation of determining a distance between the identified location of the electronic device and the location at which the electronic device initiated (or performed) a communication connection based on the first image as the amount of movement of the electronic device.

[0319] In one embodiment, the method for operating the face-mounted electronic device may include an operation for determining whether the rotation amount of the electronic device exceeds a specified value based on a change in the surrounding environment image. The method for operating the face-mounted electronic device may include an operation for stopping the avatar transmission in response to the rotation amount of the electronic device exceeding the specified value.

[0320] According to one embodiment, the method for operating the face-mounted electronic device may include an operation of generating a graphic object corresponding to the face of the user in response to a movement amount of the electronic device exceeding a specified value. The method for operating the face-mounted electronic device may include an operation of setting a position and / or size at which the graphic object is displayed on a display of the external electronic device to a predetermined value. The method for operating the face-mounted electronic device may include an operation of transmitting the graphic object to the external electronic device.

[0321] Electronic devices according to various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this document are not limited to the aforementioned devices.

[0322] The various embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" each include all possible combinations of the items listed together in that phrase. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish the corresponding component from other corresponding components and do not limit the corresponding components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as being “coupled” or “connected” to another component (e.g., a second component), with or without the terms “functionally” or “communicatively,” it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0323] The term "module" as used in this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0324] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more commands stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one command among the one or more commands stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one command called. The one or more commands may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0325] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0326] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include a single or multiple entities. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In a face-mounted electronic device, communication circuit; A first camera for acquiring an image including at least a portion of a user's face; A second camera for acquiring an image of the surrounding environment of the electronic device; A memory storing at least one computer program including instructions; and Contains at least one processor, The above instructions, when executed individually or collectively by the at least one processor, cause the electronic device to: Initiate a communication connection to transmit voice and / or video to an external electronic device using the above communication circuit. Generating an avatar corresponding to the user based on an image including at least a portion of the user's face and / or an image of the surrounding environment that changes according to the movement of the electronic device, Transmitting the above avatar to the external electronic device, The movement amount of the electronic device is confirmed based on the first image, which is an image of the surrounding environment at the time of starting the communication connection, and the image of the surrounding environment changed according to the movement of the electronic device. Checking whether the movement of the electronic device exceeds a specified first value, and A face-mounted electronic device that stops transmitting the avatar in response to the movement of the electronic device exceeding the first value specified above.

2. In the first paragraph, the instructions, when the at least one processor individually or collectively executes, the electronic device, A face-mounted electronic device that stores a second image, which is an image of the surrounding environment at the time of stopping transmission of the avatar, in the memory when transmission of the avatar is stopped.

3. In the second paragraph, the instructions, when the at least one processor individually or collectively executes, the electronic device, Based on the image of the surrounding environment changed according to the movement of the electronic device while the second image and the avatar transmission are stopped, it is confirmed that the electronic device is at the location where the avatar transmission was stopped, and A face-mounted electronic device that transmits the avatar to an external electronic device when the electronic device is in a position where the avatar transmission has stopped.

4. In the first paragraph, the instructions, when the at least one processor individually or collectively executes, the electronic device, While the above avatar transmission is stopped, it is checked whether the movement amount of the electronic device is less than a specified second value based on the first image, which is an image of the surrounding environment at the time of starting the communication connection, and the image of the surrounding environment changed according to the movement of the electronic device, and A face-mounted electronic device that transmits the avatar to an external electronic device in response to the movement amount of the electronic device being less than the second value specified above.

5. In the first paragraph, the instructions, when the at least one processor individually or collectively executes, the electronic device, Determine the characteristics of the avatar displayed based on the first image and the surrounding environment image that changes according to the movement of the electronic device, A face-mounted electronic device wherein the avatar is a display characteristic that includes at least one of the size, position, or angle of the avatar.

6. In the first paragraph, the instructions, when the at least one processor individually or collectively executes, the electronic device, While the above avatar transmission is being suspended, a user input is received instructing the avatar to be transmitted, A face-mounted electronic device configured to transmit an avatar in response to receiving user input instructing the device to transmit the avatar.

7. In the first paragraph, the instructions, when the at least one processor individually or collectively executes, the electronic device, A face-mounted electronic device that determines the location of the electronic device based on the surrounding environment image acquired using the second camera while performing the communication connection.

8. In the 7th paragraph, the instructions, when the at least one processor individually or collectively executes, the electronic device, A face-mounted electronic device that determines the distance between the location of the electronic device confirmed above and the location where the electronic device initiated a communication connection based on the first image as the amount of movement of the electronic device.

9. In the first paragraph, the instructions, when the at least one processor individually or collectively executes, the electronic device, Based on the change in the above-mentioned surrounding environment image, it is determined whether the rotation amount of the electronic device exceeds a specified value, and A face-mounted electronic device that stops transmitting the avatar in response to the rotation of the electronic device exceeding a specified value.

10. In the first paragraph, the instructions, when the at least one processor individually or collectively executes, the electronic device, Generate a graphic object corresponding to the user's face in response to the movement of the electronic device exceeding a specified value, Setting the position and / or size at which the graphic object is displayed on the display of the external electronic device to a predetermined value, and A face-mounted electronic device that transmits the graphic object to the external electronic device.

11. In a method of operating a face-mounted electronic device, An action of initiating a communication connection to transmit voice and / or image to an external electronic device using the communication circuit of the face-mounted electronic device; An action of generating an avatar corresponding to the user based on an image including at least a portion of the user's face and / or an image of the surrounding environment that changes according to the movement of the electronic device; An action of transmitting the above avatar to the external electronic device; An operation of checking the amount of movement of the electronic device based on a first image, which is an image of the surrounding environment at the time of starting the communication connection, and an image of the surrounding environment changed according to the movement of the electronic device; An operation for determining whether the movement amount of the electronic device exceeds a specified first value; and A method of operating a face-mounted electronic device, comprising an action of stopping the avatar transmission in response to the movement amount of the electronic device exceeding the first value specified above.

12. In the 11th paragraph, the method of operating the face-mounted electronic device is as follows: A method of operating a face-mounted electronic device, comprising an operation of storing a second image, which is an image of the surrounding environment at the time of stopping transmission of the avatar, in the memory of the face-mounted electronic device when transmission of the avatar is stopped.

13. In the 12th paragraph, the method of operating the face-mounted electronic device is as follows: An operation of determining whether the electronic device is at a location where the avatar transmission was stopped based on an image of the surrounding environment changed according to movement of the electronic device while the second image and the avatar transmission were stopped; and A method of operating a face-mounted electronic device, comprising an action of transmitting the avatar to the external electronic device when the electronic device is at a location where the avatar transmission has stopped.

14. In the 11th paragraph, the method of operating the face-mounted electronic device is as follows: While the above avatar transmission is stopped, an operation of checking whether the movement amount of the electronic device is less than a specified second value based on a first image which is an image of the surrounding environment at the time of starting the communication connection and an image of the surrounding environment changed according to the movement of the electronic device; and A method of operating a face-mounted electronic device, comprising an action of transmitting the avatar to an external electronic device in response to the movement amount of the electronic device being less than the second value specified above.

15. In the 11th paragraph, the avatar, The characteristics displayed on the display are determined based on the first image and the surrounding environment image that changes according to the movement of the electronic device, A method of operating a face-mounted electronic device, wherein the characteristics displayed on the display are an avatar including at least one of the size, position or angle of the avatar.

Images