Electronic device, method, and storage medium for acquiring images for tracking

Abstract

A wearable device may comprise a memory for storing instructions, a camera, and at least one processor. The instructions, when executed individually or collectively by the at least one processor, can cause the wearable device to: in a first mode, acquire images including at least one first image having a first attribute for tracking a body part of a user and at least one second image having a second attribute for tracking an external electronic device, the second attribute being different from the first attribute, and acquire first feature values for tracking the body part from the at least one first image and second feature values for tracking the external electronic device from the at least one second image; change the mode of the wearable device from the first mode to a second mode on the basis of the first feature values and the second feature values; and acquire other images while in the second mode.

Description

Electronic device, method, and storage medium for acquiring images for tracking

[0001] The following descriptions relate to electronic devices, methods, and storage media for acquiring images for tracking.

[0002] To provide an enhanced user experience, electronic devices are being developed that provide extended reality services that display computer-generated information in conjunction with external objects in the real world or virtual objects in the virtual world. The electronic devices may include wearable devices that can be worn by a user. For example, the electronic devices may include user equipment, augmented reality (AR) glasses, virtual reality (VR) glasses, and / or a head-mounted device (HMD) (e.g., video see-through (VST) HMD, optical see-through (OST) HMD).

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0004] A wearable device may include a memory, a camera, and at least one processor. The instructions, when executed by the at least one processor, may cause the wearable device to acquire, through the camera, images, in a first mode, including at least one first image having a first attribute for tracking a body part of a user and at least one second image having a second attribute for tracking an external electronic device that is different from the first attribute. The number of the at least one first image may correspond to the number of the at least one second image. The instructions, when executed by the at least one processor, may cause the wearable device to acquire first feature values ​​for tracking the body part from the at least one first image and second feature values ​​for tracking the external electronic device from the at least one second image. The instructions, when executed by the at least one processor, may cause the wearable device to change a mode of the wearable device from the first mode to a second mode based on the first feature values ​​and the second feature values. The instructions, when executed by the at least one processor, may cause the wearable device to acquire, through the camera, other images in the second mode. The number of images having the first property of the other images may differ from the number of images having the second property of the other images.

[0005] A method performed by a wearable device may include, in a first mode, acquiring images, including at least one first image having a first attribute for tracking a body part of a user and at least one second image having a second attribute for tracking an external electronic device that is different from the first attribute. The number of the at least one first image may correspond to the number of the at least one second image. The method may include acquiring first feature values ​​for tracking the body part from the at least one first image and second feature values ​​for tracking the external electronic device from the at least one second image. The method may include changing a mode of the wearable device from the first mode to the second mode based on the first feature values ​​and the second feature values. The method may include acquiring other images through the camera in the second mode. The number of images having the first property of the other images may differ from the number of images having the second property of the other images.

[0006] A non-transitory computer-readable storage medium may store one or more programs comprising instructions that, when executed by at least one processor of a wearable device including a camera, cause the wearable device to, in a first mode, acquire, through the camera, images comprising at least one first image having a first attribute for tracking a body part of a user and at least one second image having a second attribute for tracking an external electronic device different from the first attribute. The number of the at least one first image may correspond to the number of the at least one second image. The non-transitory computer-readable storage medium may store one or more programs comprising instructions that, when executed by the at least one processor, cause the wearable device to acquire first feature values ​​for tracking the body part from the at least one first image and second feature values ​​for tracking the external electronic device from the at least one second image. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by the at least one processor, cause the wearable device to change a mode of the wearable device from the first mode to a second mode based on the first characteristic values ​​and the second characteristic values. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by the at least one processor, cause the wearable device to acquire other images through the camera in the second mode.The number of images having the first property of the other images may differ from the number of images having the second property of the other images.

[0007] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0008] FIG. 2A illustrates an example of a perspective view of a wearable device according to various embodiments.

[0009] FIG. 2b illustrates an example of one or more hardware elements arranged within a wearable device according to various embodiments.

[0010] FIGS. 3A and 3B illustrate examples of the appearance of a wearable device according to various embodiments.

[0011] Figure 4 illustrates an example of a method for acquiring images for tracking body parts and images for tracking external electronic devices.

[0012] Figure 5 illustrates an exemplary block diagram of a wearable device.

[0013] Figure 6 illustrates an example of an operation flow for a method of acquiring images according to a mode of a wearable device.

[0014] Figures 7a, 7b, and 7c illustrate examples of methods for acquiring images depending on the mode of a wearable device.

[0015] FIG. 8A illustrates an example of a method for changing the mode of a wearable device based on movement of an external electronic device.

[0016] FIG. 8b illustrates an example of a method for changing the mode of a wearable device based on user input.

[0017] FIG. 9 illustrates an example of an operational flow for a method of acquiring images including images for tracking body parts and images for tracking external electronic devices depending on the mode of the wearable device.

[0018] The terms used in this disclosure are used only to describe specific embodiments and may not be intended to limit the scope of other embodiments. The singular expression may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by those of ordinary skill in the art described in this disclosure. Terms defined in general dictionaries among the terms used in this disclosure may be interpreted as having the same or similar meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined in this disclosure. In some cases, even if a term is defined in this disclosure, it cannot be interpreted to exclude embodiments of the present disclosure.

[0019] The various embodiments of the present disclosure described below illustrate a hardware-based approach as an example. However, since the various embodiments of the present disclosure include techniques utilizing both hardware and software, the various embodiments of the present disclosure do not exclude a software-based approach.

[0020] In addition, in the present disclosure, expressions such as "more than" or "less than" may be used to determine whether a specific condition is satisfied or fulfilled. However, this is merely a description for expressing an example and does not exclude descriptions such as "more than" or "less than." Conditions described as "more than" may be replaced with "more than," conditions described as "less than," and conditions described as "more than and less than" may be replaced with "more than and less than." In addition, hereinafter, "A" to "B" mean at least one of the elements from A (including A) to B (including B).

[0021] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0022] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0023] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor)) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0024] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0025] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0026] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0027] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0028] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0029] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0030] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0031] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0032] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0033] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0034] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0035] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0036] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0037] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0038] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0039] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0040] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0041] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0042] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0043] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0044] FIG. 2A illustrates an example of a perspective view of a wearable device according to various embodiments. FIG. 2B illustrates an example of one or more hardware components arranged within a wearable device according to various embodiments.

[0045] According to one embodiment, the wearable device (103) may have the form of glasses that can be worn on a body part of the user (e.g., head). The wearable device (103) of FIGS. 2A and 2B may be an example of the electronic device (101) of FIG. 1. The wearable device (103) may include a head-mounted display (HMD). For example, the housing of the wearable device (103) may include a flexible material, such as rubber and / or silicone, that is configured to fit closely to a portion of the user's head (e.g., a portion of the face surrounding both eyes). For example, the housing of the wearable device (103) may include one or more straps capable of being twined around (e.g., wrapped, curved, or fitted) the user's head, and / or one or more temples attachable to an ear of the head.

[0046] Referring to FIG. 2A, according to one embodiment, a wearable device (103) may include at least one display (250) and a frame (200) supporting at least one display (250).

[0047] According to one embodiment, a wearable device (103) can be worn on a part of a user's body. The wearable device (103) can provide extended reality (XR) to a user wearing the wearable device (103). For example, the extended reality can provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality. For example, the wearable device (103) can display a virtual reality image provided from at least one optical device (282, 284) of FIG. 2B on at least one display (250) in response to a user's designated gesture acquired through the motion recognition cameras (260-2, 263) of FIG. 2B.

[0048] According to one embodiment, at least one display (250) may provide visual information to a user. For example, at least one display (250) may include a transparent or translucent lens. At least one display (250) may include a first display (250-1) and / or a second display (250-2) spaced apart from the first display (250-1). For example, the first display (250-1) and the second display (250-2) may be positioned at positions corresponding to the user's left and right eyes, respectively.

[0049] Referring to FIG. 2B, at least one display (250) can provide the user with visual information transmitted from external light and other visual information distinct from the visual information through a lens included in the at least one display (250). The lens can be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. For example, the at least one display (250) can include a first surface (231) and a second surface (232) opposite to the first surface (231). A display area can be formed on the second surface (232) of the at least one display (250). When the user wears the wearable device (103), external light can be transmitted to the user by being incident on the first surface (231) and transmitted through the second surface (232). As another example, at least one display (250) can display an augmented reality image combined with a virtual reality image provided from at least one optical device (282, 284) on a real screen transmitted through external light, in a display area formed on the second surface (232).

[0050] In one embodiment, at least one display (250) may include at least one waveguide (233, 234) that diffracts light emitted from at least one optical device (282, 284) and transmits the diffracted light to a user. The at least one waveguide (233, 234) may be formed based on at least one of glass, plastic, or polymer. A nano-pattern may be formed on at least a portion of the exterior or interior of the at least one waveguide (233, 234). The nano-pattern may be formed based on a grating structure having a polygonal and / or curved shape. Light incident on one end of the at least one waveguide (233, 234) may be propagated to the other end of the at least one waveguide (233, 234) by the nano-pattern. At least one waveguide (233, 234) may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) and at least one reflective element (e.g., a reflective mirror). For example, at least one waveguide (233, 234) may be arranged within the wearable device (103) to guide a screen displayed by at least one display (250) to the user's eyes. For example, the screen may be transmitted to the user's eyes based on total internal reflection (TIR) ​​occurring within the at least one waveguide (233, 234).

[0051] The wearable device (103) can analyze an object included in a real image collected through a shooting camera (260-4), combine a virtual object corresponding to an object to be provided with augmented reality among the analyzed objects, and display the virtual object on at least one display (250). The virtual object can include at least one of text and an image regarding various information related to the object included in the real image. The wearable device (103) can analyze the object based on a multi-camera such as a stereo camera. For the object analysis, the wearable device (103) can perform spatial recognition (e.g., simultaneous localization and mapping (SLAM)) using a multi-camera and / or time-of-flight (ToF). A user wearing the wearable device (103) can view an image displayed on at least one display (250).

[0052] According to one embodiment, the frame (200) may be configured as a physical structure that allows the wearable device (103) to be worn on the user's body. According to one embodiment, the frame (200) may be configured so that, when the user wears the wearable device (103), the first display (250-1) and the second display (250-2) can be positioned corresponding to the user's left and right eyes. The frame (200) may support at least one display (250). For example, the frame (200) may support the first display (250-1) and the second display (250-2) to be positioned corresponding to the user's left and right eyes.

[0053] Referring to FIG. 2A, the frame (200) may include a region (220) that at least partially contacts a part of the user's body when the user wears the wearable device (103). For example, the region (220) of the frame (200) that contacts a part of the user's body may include a region that contacts a part of the user's nose, a part of the user's ear, and a part of the side of the user's face that the wearable device (103) makes contact with. According to one embodiment, the frame (200) may include a nose pad (210) that contacts a part of the user's body. When the wearable device (103) is worn by the user, the nose pad (210) may contact a part of the user's nose. The frame (200) may include a first temple (204) and a second temple (205) that contact a part of the user's body that is distinct from the part of the user's body.

[0054] For example, the frame (200) may include a first rim (201) that surrounds at least a portion of the first display (250-1), a second rim (202) that surrounds at least a portion of the second display (250-2), a bridge (203) that is disposed between the first rim (201) and the second rim (202), a first pad (211) that is disposed along a portion of the edge of the first rim (201) from one end of the bridge (203), a second pad (212) that is disposed along a portion of the edge of the second rim (202) from the other end of the bridge (203), a first temple (204) that extends from the first rim (201) and is fixed to a portion of the wearer's ear, and a second temple (205) that extends from the second rim (202) and is fixed to a portion of the ear opposite the ear. The first pad (211) and the second pad (212) may be in contact with a portion of the user's nose, and the first temple (204) and the second temple (205) may be in contact with a portion of the user's face and a portion of the user's ear. The temples (204, 205) may be rotatably connected to the rim through the hinge units (206, 207) of FIG. 2B. The first temple (204) may be rotatably connected to the first rim (201) through the first hinge unit (206) disposed between the first rim (201) and the first temple (204). The second temple (205) may be rotatably connected to the second rim (202) through the second hinge unit (207) disposed between the second rim (202) and the second temple (205). According to one embodiment, the wearable device (103) can identify an external object (e.g., a user's fingertip) touching the frame (200) and / or a gesture performed by the external object by using a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of a surface of the frame (200).

[0055] According to one embodiment, the wearable device (103) may include hardwares that perform various functions (e.g., hardwares to be described later based on the block diagram of FIG. 5). For example, the hardwares may include a battery module (270), an antenna module (275), at least one optical device (282, 284), speakers (e.g., speakers 255-1, 255-2), a microphone (e.g., microphones 265-1, 265-2, 265-3), a light-emitting module (not shown), and / or a printed circuit board (PCB) (290) (e.g., a printed circuit board). The various hardwares may be arranged within the frame (200).

[0056] According to one embodiment, the microphones (e.g., microphones 265-1, 265-2, 265-3) of the wearable device (103) may be disposed on at least a portion of the frame (200) to acquire sound signals. A first microphone (265-1) disposed on the bridge (203), a second microphone (265-2) disposed on the second rim (202), and a third microphone (265-3) disposed on the first rim (201) are illustrated in FIG. 2B , but the number and arrangement of the microphones (265) are not limited to the embodiment of FIG. 2B . When the number of microphones (265) included in the wearable device (103) is two or more, the wearable device (103) may identify the direction of the sound signal by using a plurality of microphones disposed on different portions of the frame (200).

[0057] According to one embodiment, at least one optical device (282, 284) can project a virtual object onto at least one display (250) to provide various image information to a user. For example, at least one optical device (282, 284) can be a projector. At least one optical device (282, 284) can be disposed adjacent to at least one display (250) or can be included within at least one display (250) as a part of at least one display (250). According to one embodiment, the wearable device (103) can include a first optical device (282) corresponding to a first display (250-1) and a second optical device (284) corresponding to a second display (250-2). For example, at least one optical device (282, 284) may include a first optical device (282) disposed at an edge of a first display (250-1) and a second optical device (284) disposed at an edge of a second display (250-2). The first optical device (282) may transmit light to a first waveguide (233) disposed on the first display (250-1), and the second optical device (284) may transmit light to a second waveguide (234) disposed on the second display (250-2).

[0058] In one embodiment, the camera (260) may include a recording camera (260-4), an eye tracking camera (ET CAM) (260-1), and / or a motion recognition camera (260-2, 260-3). The recording camera (260-4), the eye tracking camera (260-1), and the motion recognition cameras (260-2, 260-3) may be positioned at different locations on the frame (200) and may perform different functions. The eye tracking camera (260-1) may output data indicating the position or gaze of the eyes of a user wearing the wearable device (103). For example, the wearable device (103) may detect the gaze from an image including the user's pupils obtained through the eye tracking camera (260-1). The wearable device (103) can perform gaze interaction with at least one object using the user's gaze acquired through the gaze tracking camera (260-1). The wearable device (103) can express a portion corresponding to the eye of an avatar representing the user in a virtual space using the user's gaze acquired through the gaze tracking camera (260-1). The wearable device (103) can render an image (or screen) displayed on at least one display (250) based on the position of the user's eyes. For example, the visual quality (e.g., resolution, brightness, saturation, grayscale, PPI) of a first region related to the gaze and a second region distinguished from the first region within the image may be different from each other. For example, if the wearable device (103) supports an iris recognition function, user authentication can be performed based on iris information acquired using the gaze tracking camera (260-1).An example in which the gaze tracking camera (260-1) is positioned toward the user's right eye is illustrated in FIG. 2B, but the embodiment is not limited thereto, and the gaze tracking camera (260-1) may be positioned solely toward the user's left eye, or may be positioned toward both eyes.

[0059] In one embodiment, the capturing camera (260-4) can capture an actual image or background to be aligned with a virtual image in order to implement augmented reality or mixed reality content. The capturing camera (260-4) can capture an image of a specific object existing at a location viewed by the user and provide the image to at least one display (250). The at least one display (250) can display a single image in which information about an actual image or background including an image of the specific object acquired using the capturing camera (260-4) is superimposed on a virtual image provided through at least one optical device (282, 284). The wearable device (103) can compensate for depth information (e.g., the distance between the wearable device (103) and an external object acquired using a depth sensor) using the image acquired using the capturing camera (260-4). The wearable device (103) can perform object recognition using the image acquired using the capturing camera (260-4). The wearable device (103) may perform a pass-through function to display an image acquired through a photographing camera (260-4) by overlaying it on at least a portion of a screen representing a virtual space on at least one display (250). In one embodiment, the photographing camera (260-4) may be positioned on a bridge (203) positioned between a first rim (201) and a second rim (202).

[0060] The gaze tracking camera (260-1) can implement more realistic augmented reality by tracking the gaze of a user wearing the wearable device (103) and thereby matching the user's gaze with visual information provided to at least one display (250). For example, when the wearable device (103) looks straight ahead, the wearable device (103) can naturally display environmental information related to the user's front at a location where the user is located on at least one display (250). The gaze tracking camera (260-1) can be configured to capture an image of the user's pupil to determine the user's gaze. For example, the gaze tracking camera (260-1) can receive gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the gaze tracking camera (260-1) can be positioned at positions corresponding to the user's left and right eyes. For example, the gaze tracking camera (260-1) may be positioned within the first rim (201) and / or the second rim (202) to face the direction in which the user wearing the wearable device (103) is positioned.

[0061] The motion recognition camera (260-2, 260-3) can provide a specific event on a screen provided on at least one display (250) by recognizing the movement of the user's entire body or a part of the user's body, such as the user's torso, hand, or face. The motion recognition camera (260-2, 260-3) can recognize the user's motion (gesture recognition), obtain a signal corresponding to the motion, and provide a display corresponding to the signal on at least one display (250). The wearable device (103) can identify the signal corresponding to the motion and perform a designated function based on the identification. The motion recognition camera (260-2, 260-3) can be used to perform a spatial recognition function using SLAM and / or a depth map for 6 degrees of freedom pose (6 DoF pose). The wearable device (103) can perform gesture recognition and / or object tracking functions using motion recognition cameras (260-2, 260-3). In one embodiment, the motion recognition cameras (260-2, 260-3) can be positioned on the first rim (201) and / or the second rim (202).

[0062] The camera (260) included in the wearable device (103) is not limited to the above-described gaze tracking camera (260-1) and motion recognition cameras (260-2, 260-3). For example, the wearable device (103) can identify an external object included in the user's field of view (FoV) using a camera positioned toward the user's FoV. The wearable device (103) can identify an external object based on a sensor for identifying the distance between the wearable device (103) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (260) positioned toward the FoV can support an autofocus function and / or an optical image stabilization (OIS) function. For example, the wearable device (103) may include a camera (260) (e.g., a face tracking (FT) camera) positioned toward the face to obtain an image including the face of a user wearing the wearable device (103).

[0063] Although not shown, in one embodiment, the wearable device (103) may further include a light source (e.g., an LED) that emits light toward a subject (e.g., a user's eyes, face, and / or an external object within the FoV) being photographed using the camera (260). The light source may include an infrared wavelength LED. The light source may be disposed on at least one of the frame (200) and the hinge units (206, 207).

[0064] According to one embodiment, the battery module (270) may supply power to electronic components of the wearable device (103). In one embodiment, the battery module (270) may be disposed within the first temple (204) and / or the second temple (205). For example, the battery module (270) may be a plurality of battery modules (270). The plurality of battery modules (270) may be disposed within each of the first temple (204) and the second temple (205). In one embodiment, the battery module (270) may be disposed at an end of the first temple (204) and / or the second temple (205).

[0065] The antenna module (275) can transmit signals or power to the outside of the wearable device (103), or receive signals or power from the outside. In one embodiment, the antenna module (275) can be positioned within the first temple (204) and / or the second temple (205). For example, the antenna module (275) can be positioned close to one surface of the first temple (204) and / or the second temple (205).

[0066] The speaker (255) can output an audio signal to the outside of the wearable device (103). The audio output module may be referred to as a speaker. In one embodiment, the speaker (255) may be positioned within the first temple (204) and / or the second temple (205) so as to be positioned adjacent to the ear of a user wearing the wearable device (103). For example, the speaker (255) may include a second speaker (255-2) positioned within the first temple (204) and thus positioned adjacent to the user's left ear, and a first speaker (255-1) positioned within the second temple (205) and thus positioned adjacent to the user's right ear.

[0067] The light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state or emit light with an action corresponding to a specific state, in order to visually provide information regarding a specific state of the wearable device (103) to the user. For example, when the wearable device (103) requires charging, it may emit red light at a regular cycle. In one embodiment, the light-emitting module may be disposed on the first rim (201) and / or the second rim (202).

[0068] Referring to FIG. 2B, according to one embodiment, a wearable device (103) may include a printed circuit board (PCB) (290). The PCB (290) may be included in at least one of the first temple (204) or the second temple (205). The PCB (290) may include an interposer disposed between at least two sub-PCBs. One or more hardwares included in the wearable device (103) (e.g., hardwares illustrated by different blocks in FIG. 5) may be disposed on the PCB (290). The wearable device (103) may include a flexible PCB (FPCB) for interconnecting the hardwares.

[0069] According to one embodiment, the wearable device (103) may include at least one of a gyro sensor, a gravity sensor, and / or an acceleration sensor for detecting a posture of the wearable device (103) and / or a posture of a body part (e.g., a head) of a user wearing the wearable device (103). Each of the gravity sensor and the acceleration sensor may measure gravitational acceleration and / or acceleration based on mutually perpendicular designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). The gyro sensor may measure an angular velocity of each of the designated three-dimensional axes (e.g., an x-axis, a y-axis, and a z-axis). At least one of the gravity sensor, the acceleration sensor, and the gyro sensor may be referred to as an inertial measurement unit (IMU). According to one embodiment, the wearable device (103) may identify a user's motion and / or gesture performed to execute or terminate a specific function of the wearable device (103) based on the IMU.

[0070] FIGS. 3A and 3B illustrate examples of the appearance of a wearable device according to various embodiments.

[0071] The wearable device (103) of FIGS. 3A and 3B may be an example of the electronic device (101) of FIG. 1. According to one embodiment, an example of the appearance of a first side (310) of a housing of the wearable device (103) may be illustrated in FIG. 3A, and an example of the appearance of a second side (320) opposite to the first side (310) may be illustrated in FIG. 3B.

[0072] Referring to FIG. 3A, a first surface (310) of a wearable device (103) according to one embodiment may have a shape that is attachable or conforms to a body part of a user (e.g., the face of the user). Although not shown, the wearable device (103) may further include a strap for being fixed to a body part of a user, and / or one or more temples (e.g., the first temple (204) and / or the second temple (205) of FIGS. 2A and 2B). A first display (250-1) for outputting an image to a left eye among the user's two eyes, and a second display (250-2) for outputting an image to a right eye among the user's two eyes may be disposed on the first surface (310). The wearable device (103) is formed on the first surface (310) and may further include a rubber or silicone packing to prevent interference by light (e.g., ambient light) different from the light emitted from the first display (250-1) and the second display (250-2).

[0073] According to one embodiment, the wearable device (103) may include cameras (260-1) for photographing and / or tracking both eyes of the user adjacent to each of the first display (250-1) and the second display (250-2). The cameras (260-1) may be referred to as the gaze tracking camera (260-1) of FIG. 2B. According to one embodiment, the wearable device (103) may include cameras (260-5, 260-6) for photographing and / or recognizing the face of the user. The cameras (260-5, 260-6) may be referred to as FT cameras. The wearable device (103) may control an avatar representing the user in a virtual space based on the motion of the user's face identified using the cameras (260-5, 260-6).

[0074] Referring to FIG. 3b, a camera (e.g., cameras (260-7, 260-8, 260-9, 260-10, 260-11, 260-12)) and / or a sensor (e.g., a depth sensor (330)) for obtaining information related to the external environment of the wearable device (103) may be disposed on a second surface (320) opposite to the first surface (310) of FIG. 3a. For example, the cameras (260-7, 260-8, 260-9, 260-10) may be disposed on the second surface (320) to recognize external objects. Cameras (260-7, 260-8, 260-9, 260-10) may be referenced to the motion recognition cameras (260-2, 260-3) of FIG. 2B.

[0075] For example, using cameras (260-11, 260-12), the wearable device (103) can obtain images and / or videos to be transmitted to each of the user's eyes. The camera (260-11) can be placed on the second face (320) of the wearable device (103) to obtain an image to be displayed through the second display (250-2) corresponding to the right eye among the two eyes. The camera (260-12) can be placed on the second face (320) of the wearable device (103) to obtain an image to be displayed through the first display (250-1) corresponding to the left eye among the two eyes. The cameras (260-11, 260-12) can be referred to as the shooting camera (260-4) of FIG. 2B.

[0076] According to one embodiment, the wearable device (103) may include a depth sensor (330) disposed on the second face (320) to identify a distance between the wearable device (103) and an external object. Using the depth sensor (330), the wearable device (103) may obtain spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the wearable device (103). Although not shown, a microphone may be disposed on the second face (320) of the wearable device (103) to obtain a sound output from an external object. The number of microphones may be one or more, depending on the embodiment.

[0077] Figure 4 illustrates an example of a method for acquiring images for tracking body parts and images for tracking external electronic devices.

[0078] The wearable device (103) of FIG. 4 may represent an example of the electronic device (101) of FIG. 1 and the wearable device (103) of FIGS. 2A to 3B. In the example (400) of FIG. 4, the wearable device (103) provides an augmented reality (AR) environment, but the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may provide an extended reality (XR) environment including a virtual reality (VR) environment.

[0079] FIG. 4 illustrates an example (400) of a method for a user (410) wearing a wearable device (103) to acquire images for tracking an external electronic device (420) and tracking a body part.

[0080] For example, tracking of the body part may be used to obtain information about the position or movement of the body part of the user (410). For example, the tracking of the body part may be referred to as a first tracking. For example, the body part may include at least one of a hand or a head of the user (410). For example, the tracking of the hand may be referred to as hand tracking. Similarly, for example, the tracking of the head may be referred to as head tracking.

[0081] For example, the tracking of the external electronic device (420) may be used to obtain information about the location, movement, and / or input of the external electronic device (420). For example, the tracking of the external electronic device (420) may be referred to as a second tracking. For example, the external electronic device (420) may be connected to the wearable device (103). For example, the wearable device (103) may establish a connection with the external electronic device (420) using a communication technique. For example, the communication technique may include a wired communication technique or a wireless communication technique. For example, the wearable device (103) may obtain the information from the external electronic device (420) based on the established connection. For example, the external electronic device (420) may be connected to the wearable device (103) and may be used to obtain an input of the user (410). For example, the external electronic device (420) may be referred to as a controller, control device, or input device of the wearable device (103).

[0082] According to one embodiment, the wearable device (103) can perform the second tracking through an image of a hand (e.g., a right hand) that is holding the external electronic device (420), and can perform the first tracking through an image of a hand (e.g., a left hand) that is not holding the external electronic device (420).

[0083] Referring to example (400), the wearable device (103) can be worn by the user (410). For example, the user (410) can wear the wearable device (103) on the head. Also, referring to example (400), the external electronic device (420) can be held by the user (410). For example, the user (410) can hold the external electronic device (420) with the user's (410) right hand. The example (400) of FIG. 4 is merely for convenience of explanation, and the embodiments of the present disclosure are not limited thereto.

[0084] Referring to example (400), the wearable device (103) can obtain a first image (450) to perform tracking of the body part (e.g., the head) of the user (410). For example, the wearable device (103) can obtain the first image (450) through a field of view (FoV) of a camera (e.g., the camera module (180) of FIG. 1) of the wearable device (103). For example, the first image (450) for tracking the body part can include an external object (430) and an external electronic device (420). The wearable device (103) can perform the tracking of the body part based on the amount of change in the boundary of the external object (430) and the external electronic device (420) of the first image (450). For example, the wearable device (103) may perform the tracking of the body part based on the positions of the boundary lines of the external object (430) and the external electronic device (420) in the first image (450) of the first timing and the positions of the boundary lines of the external object (430) and the external electronic device (420) in the first image (450) of the second timing different from the first timing. The boundary lines may include the location, outline, placement, etc. of the external object. In the above example, the boundary lines constituting the appearance of the external object (430) and / or the external electronic device (420) are described as examples, but the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may perform the tracking of the body part based on at least one point of the external object (430).

[0085] Referring to example (400), the wearable device (103) can acquire a second image (460) to track an external electronic device (420) of a user (410). For example, the wearable device (103) can acquire the second image (460) through a field of view (FoV) of a camera (e.g., a camera module (180) of FIG. 1) of the wearable device (103). For example, the second image (460) for tracking the external electronic device (420) can include light sources (421, 422, 423, 424) of the external electronic device (420). For example, the light sources (421, 422, 423, 424) can include light-emitting elements of the external electronic device (420). The wearable device (103) can perform the tracking of the external electronic device (420) based on the amount of change in the boundary of the light sources (421, 422, 423, 424) of the second image (460). For example, the wearable device (103) can perform the tracking of the external electronic device (420) based on the positions of the boundary of the light sources (421, 422, 423, 423) in the second image (460) at the first timing and the positions of the boundary of the light sources (421, 422, 423, 423) in the second image (460) at the second timing different from the first timing. In the above example, the boundary of the light sources (421, 422, 423, 423) is described as an example, but the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) can perform the tracking of the external electronic device (420) based on at least one point of the light sources (421, 422, 423, 423).

[0086] For example, the first image (450) and the second image (460) may have different attributes. For example, the first image (450) may have a first attribute, and the second image (460) may have a second attribute that is different from the first attribute. For example, the attributes may include brightness, color, and saturation of the image. For example, the first image (450) may have the first attribute, first brightness. In contrast, the second image (460) may have the second attribute, second brightness. The second brightness may be darker than the first brightness. For example, the attributes may be referred to as brightness, a characteristic, a parameter, or a characteristic parameter.

[0087] Referring to example (400), the wearable device (103) may use the first image (450) having the first brightness to perform the tracking of the body part. The first image (450) may have the first brightness so that the appearance of the external object (430) and the external electronic device (420) are clearly visible. At this time, in the first image (450) having the first brightness, the boundary line (or point) of the external electronic device (420) and the external object (430) may be clearly visible. Alternatively, the wearable device (103) may use the second image (460) having the second brightness to perform the tracking of the external electronic device (420). The second image (460) may have the second brightness so that the shape of the external object (430) is not visible or appears blurry. At this time, in the second image (460) having the second brightness, the light sources (421, 422, 423, 424) of the external electronic device (420) can be clearly seen. The light sources (421, 422, 423, 424) can be displayed more clearly in the second image (460) having the second brightness than in the first image (450) having the first brightness.

[0088] In example (400), an example of a wearable device (103) that acquires one first image (450) and one second image (460) is illustrated, but embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may acquire a plurality of first images (450) and a plurality of second images (460). At this time, the plurality of first images (450) and the plurality of second images (460) may be acquired based on the FPS (frames per second) for the camera of the wearable device (103) to acquire images (or videos). For example, when the FPS set to the camera is 60 FPS, the wearable device (103) may acquire 60 frames per second. The frame may be referred to as an image or a frame image. Each of the above frames may include at least one of a first image (450) or a second image (460). In one example, the wearable device (103) may alternately acquire the first image (450) and the second image (460). Specific details related thereto are described below in FIGS. 7A to 7C.

[0089] Embodiments of the present disclosure propose an electronic device and method for controlling acquisition of images (or videos) used for the first tracking and the second tracking in an XR environment. In the XR environment, a wearable device (103) can identify a position and direction of a user (410) (or a body part of the user (410)) by using information acquired through a camera (e.g., a camera module (180) of FIG. 1) or a sensor (e.g., a sensor module (176) of FIG. 1). A technology for recognizing the position and direction of the user (410) may be referred to as the first tracking or head tracking. In addition, the wearable device (103) can utilize an external electronic device (420) or the hand of the user (410) to interact with a virtual object. When using an external electronic device (420), the wearable device (103) can recognize the location and direction of the external electronic device (420) and interact with the virtual object. The technology for recognizing the location and direction of the external electronic device (420) may be referred to as the second tracking or controller tracking.

[0090] In the first tracking and the second tracking, images (or videos) of the camera may be used. In this case, the images (e.g., the first image (450)) used in the first tracking may be different from other images (e.g., the second image (460)) used in the second tracking. The images used in the first tracking may be acquired through a camera dedicated to the first tracking, and the other images used in the second tracking may be acquired through another camera dedicated to the second tracking for an external electronic device (420). In addition, the images and the other images may be acquired through the same camera. When the images and the other images are acquired through a plurality of cameras including the camera and the other camera, the power consumption of the wearable device (103) may increase, and the cost for producing the wearable device (103) may increase. Accordingly, it may be efficient to acquire the above images and the other images through one camera.

[0091] When the images and the other images are acquired through a single camera, the properties of the images may be different from the properties of the other images. For example, the brightness of the images may be brighter than the brightness of the other images. In the case of the images, a bright image may be used because feature values ​​including a boundary or point of an object within the images are used. In contrast, in the case of the other images, a dark image may be used because feature values ​​including light emitted by an external electronic device (420) within the other images (e.g., light sources (421, 422, 423, 424)) are used. The bright image may represent an image having the first property, and the dark image may represent an image having the second property. The wearable device (103) may acquire the images for the first tracking and the other images for the second tracking through the single camera in order to provide both the first tracking and the second tracking. Accordingly, the wearable device (103) can alternately use a first filming scheme (or shooting scheme) (hereinafter, “first scheme”) for acquiring the bright images for the first tracking and a second filming scheme (hereinafter, “second scheme”) for acquiring the other dark images for the second tracking. That is, the wearable device (103) can alternately repeat the operation of acquiring the images through the first scheme and acquiring the other images through the second scheme. The first scheme and the second scheme can be used within a shooting mode for acquiring images.

[0092] For example, the first technique for acquiring the images and the second technique for acquiring the other images can be identified based on the exposure algorithm for acquiring the images. For example, the technique can represent a case where an auto exposure algorithm is used. In contrast, the second technique can represent a case where a fixed exposure algorithm is used. For example, a case where the auto exposure algorithm is used can have a longer exposure time than a case where the fixed exposure algorithm is used. In this case, the wearable device (103) can calculate the exposure time used in the auto exposure algorithm. In other words, the wearable device (103) can acquire light through the camera for a longer time to acquire the images using the first technique compared to a case where the other images are acquired using the second technique, and can acquire the images based on the first technique.

[0093] Additionally and / or alternatively, for example, the first technique and the second technique may be identified based on a gain value for acquiring an image. For example, the first technique may represent a case where a relatively high gain value is used. Conversely, the second technique may represent a case where a relatively low gain value is used. The wearable device (103) may acquire image information about a real environment through the camera and perform processing on the image information based on the gain value. Based on the processing, the wearable device (103) may acquire the images or the other images. At this time, the higher the gain value, the higher the brightness of the acquired image. In other words, the wearable device (103) may acquire the images by using the high gain value through or by the first technique. Alternatively, the wearable device (103) can acquire the other images by using the low gain value through or by the second technique.

[0094] Additionally and / or alternatively, for example, the first technique and the second technique may be identified based on the brightness of a light source included in the wearable device (103). For example, the light source included in the wearable device (103) may include an infrared radiation (IR) illuminator. For example, the first technique may indicate a case where the brightness of the light source is relatively high. Conversely, the second technique may indicate a case where the brightness of the light source is relatively low. The light source may be used as a light used by the wearable device (103) to acquire images. In other words, the wearable device (103) may acquire the images by utilizing the high brightness of the light source through the first technique. Conversely, the wearable device (103) may acquire the other images by utilizing the low brightness of the light source through the second technique.

[0095] In general, the wearable device (103) may alternately use the first technique and the second technique to acquire relatively bright images and relatively dark other images through or using a single camera. In this case, the wearable device (103) may have difficulty supporting a case where the images are relatively more required than the other images (or the other images are the images). Accordingly, inefficiencies may occur in the first tracking and the second tracking of the wearable device (103). For example, if the external electronic device (420) is located outside the FoV of the camera, the second tracking cannot be performed, and thus, using the second technique and the first technique at the same frequency may cause inefficiency. In addition, for example, if it is necessary to support the accuracy and fast response speed of the external electronic device (420), more other images may be required to be used for the second tracking. However, since the wearable device (103) alternately acquires the images and the other images through the one camera, it may be inefficient because it cannot acquire relatively more of the other images.

[0096] Hereinafter, the electronic device and method according to embodiments of the present disclosure may use a variable sequence (e.g., alternately acquiring two bright images and one dark image) rather than a fixed sequence (e.g., alternately acquiring one bright image and one dark image). For example, the variable sequence may be operated depending on whether a specified condition is satisfied. For example, the specified condition may be related to the status (e.g., quality) of tracking using a tracking module within the wearable device (103), a user's input to the wearable device (103) (e.g., a gesture or input to a physical button), a specific software application or settings within the wearable device (103), and / or the movement of an external electronic device (420). The electronic device and method according to embodiments of the present disclosure may flexibly acquire images required for each situation when using the first tracking and the second tracking through a single camera. Electronic devices and methods according to embodiments of the present disclosure can utilize a shooting mode appropriate to the tracking module's performance or the user's usage scenario. Accordingly, electronic devices and methods according to embodiments of the present disclosure can provide an effective user experience by utilizing a shooting mode appropriate for each service.

[0097] Figure 5 illustrates an exemplary block diagram of a wearable device.

[0098] The wearable device (103) of FIG. 5 may be an example of the electronic device (101) of FIG. 1, the wearable device (103) of FIGS. 2A to 3B, or the wearable device (103) of FIG. 4. The external electronic device (420) of FIG. 5 may be an example of the external electronic device (420) of FIG. 4. Although FIG. 5 illustrates a wearable device (103) that can be worn by a user, the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may represent an example of the electronic device (101) of FIG. 1.

[0099] Referring to FIG. 5, an exemplary situation is illustrated in which a wearable device (103) and an external electronic device (420) are connected to each other based on a wired network and / or a wireless network. For example, the wired network may include a network such as the Internet, a local area network (LAN), a wide area network (WAN), or a combination thereof. For example, the wireless network may include a network such as long term evolution (LTE), 5g new radio (NR), wireless fidelity (WiFi), Zigbee, near field communication (NFC), Bluetooth, Bluetooth low-energy (BLE), or a combination thereof. Although the wearable device (103) and the external electronic device (420) are illustrated as being directly connected, the wearable device (103) and the external electronic device (420) may be indirectly connected via one or more routers and / or access points (APs).

[0100] Referring to FIG. 5, according to one embodiment, a wearable device (103) may include at least one of a processor (510), a camera (520), a sensor (530), a display (540), a communication circuit (550), and a memory (560). The camera (520), the sensor (530), the display (540), the communication circuit (550), and the memory (560) may be electrically and / or operably coupled with each other by a communication bus. Hereinafter, operably coupled hardware components may mean that a direct connection or an indirect connection is established between the hardware components, either wired or wireless, such that a second hardware component is controlled by a first hardware component among the hardware components. Although illustrated based on different blocks, the embodiment is not limited thereto, and some of the hardware components illustrated in FIG. 5 (e.g., at least a portion of the processor (510), memory (560), and communication circuit (550)) may be included in a single integrated circuit such as a system on a chip (SoC). The type and / or number of hardware components included in the wearable device (103) is not limited to those illustrated in FIG. 5. For example, the wearable device (103) may include only some of the hardware components illustrated in FIG. 5.

[0101] According to one embodiment, the processor (510) of the wearable device (103) may include a hardware component for processing data based on one or more instructions. The hardware component for processing data may include, for example, an arithmetic and logic unit (ALU), a floating point unit (FPU), and a field programmable gate array (FPGA). As an example, the hardware component for processing data may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP), and / or a neural processing unit (NPU). The number of processors (510) may be one or more. For example, the processor (510) may have a multi-core processor structure, such as a dual core, a quad core, or a hexa core. The processor (510) of FIG. 5 may include the processor (120) of FIG. 1.

[0102] According to one embodiment, the camera (520) of the wearable device (103) may include one or more optical sensors (e.g., a charged coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor) that generate electrical signals representing the color and / or brightness of light. The plurality of optical sensors included in the camera (520) may be arranged in the form of a two-dimensional array. The camera (520) may acquire the electrical signals of each of the plurality of optical sensors substantially simultaneously, and generate an image corresponding to light reaching the optical sensors of the two-dimensional grid, and including a plurality of pixels arranged two-dimensionally. For example, photographic data captured using the camera (520) may mean one image acquired from the camera (520). For example, video data captured using the camera (520) may mean a sequence of a plurality of images acquired from the camera (520) according to a specified frame rate. According to one embodiment, a wearable device (103) may further include a flash light that is positioned so that the camera (520) receives light and outputs light in the direction in which the camera (520) receives light. The number of cameras (520) included in the wearable device (103) may be one or more, as described above with reference to FIGS. 2A and 2B, and / or FIGS. 3A and 3B.

[0103] According to one embodiment, the sensor (530) of the wearable device (103) may include at least one sensor. For example, the sensor (530) may include at least a portion of the sensor module (176) of FIG. 1. For example, the sensor (530) may include an IMU (or IMU sensor). For example, the sensor (530) may include a gyro sensor, a gravity sensor, and / or an acceleration sensor.

[0104] According to one embodiment, a display (540) of a wearable device (103) may output visualized information to a user. The number of displays (540) included in the wearable device (103) may be one or more. For example, the display (540) may be controlled by a processor (510) and / or a graphic processing unit (GPU) (not shown) to output visualized information to a user. The display (540) may include a flat panel display (FPD) and / or electronic paper. The FPD may include a liquid crystal display (LCD), a plasma display panel (PDP), a digital mirror device (DMD), one or more light emitting diodes (LEDs), and / or micro LEDs. The LEDs may include organic LEDs (OLEDs). The display (540) of FIG. 5 may include the display module (160) of FIG. 1.

[0105] According to one embodiment, the communication circuit (550) of the wearable device (103) may include hardware for supporting transmission and / or reception of electrical signals between the wearable device (103) and an external electronic device (520). The communication circuit (550) may include, for example, at least one of a modem (modulator and demodulator (MODEM), an antenna, and an optical / electronic (O / E) converter. The communication circuit (550) may support transmission and / or reception of electrical signals based on various types of communication means, such as Ethernet, Bluetooth, Bluetooth low energy (BLE), ZigBee, long term evolution (LTE), and 5G new radio (NR). The communication circuit (550) of FIG. 5 may include the communication module (190) and / or the antenna module (197) of FIG. 1.

[0106] According to one embodiment, the memory (560) of the wearable device (103) may include a hardware component for storing data and / or instructions input to and / or output from the processor (510). The memory (560) may include, for example, a volatile memory such as a random-access memory (RAM), and / or a non-volatile memory such as a read-only memory (ROM). The volatile memory may include, for example, at least one of a dynamic RAM (DRAM), a static RAM (SRAM), a cache RAM, and a pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a flash memory, a hard disk, a compact disc, and an embedded multimedia card (eMMC). The memory (560) of FIG. 5 may include the memory (130) of FIG. 1.

[0107] Although not illustrated, in one embodiment, the wearable device (103) may include an output means for outputting information in a form other than a visualized form. For example, the wearable device (103) may include a speaker for outputting an acoustic signal. For example, the wearable device (103) may include a motor for providing haptic feedback based on vibration.

[0108] Referring to FIG. 5, according to one embodiment, one or more instructions (or commands) representing operations and / or actions to be performed on data by a processor (510) of the wearable device (103) may be stored in the memory (560) of the wearable device (103). A set of one or more instructions may be referred to as a program, firmware, an operating system, a process, a routine, a sub-routine, and / or an application. Hereinafter, when an application is installed in an electronic device (e.g., a wearable device (103)), it may mean that one or more instructions provided in the form of an application are stored in the memory (560), and that the one or more applications are stored in a format executable by the processor of the electronic device (e.g., a file having an extension designated by the operating system of the wearable device (103)). According to one embodiment, the wearable device (103) may perform the operations of FIGS. 6 and 9 by executing one or more instructions stored in the memory (560).

[0109] Referring to FIG. 5, according to one embodiment, programs installed in the wearable device (103) may be classified into any one of different layers, including an application layer (570), a framework layer (580), and / or a hardware abstraction layer (HAL) (590), based on the target. For example, within the hardware abstraction layer (590), programs (e.g., drivers) designed to target the hardware of the wearable device (103) (e.g., a camera (520), a display (540), and / or a communication circuit (550)) may be classified. For example, within the framework layer (580), programs (e.g., an image acquisition module (581), a tracking module (583), and / or a mode change module (585)) designed to target at least one of the hardware abstraction layer (590) and / or the application layer (570) may be classified. Programs classified as a framework layer (580) can provide an executable API (application programming interface) based on other programs.

[0110] Referring to FIG. 5, according to one embodiment, programs designed to target users controlling wearable devices (103) may be classified within the application layer (570). For example, a program classified within the application layer (570) may include at least one application that provides an XR environment. However, the embodiments of the present disclosure are not limited thereto. For example, a program classified within the application layer (570) may call an API to cause the execution of functions supported by programs classified within the framework layer (580).

[0111] Referring to FIG. 5, according to one embodiment, the wearable device (103) may acquire an image through the camera (520) based on the execution of the image acquisition module (581) within the framework layer (580). For example, the wearable device (103) may acquire an image through the image acquisition module (581) based on a set mode. For example, the mode may include a first mode, a second mode, and a third mode. For example, the first mode may represent a shooting mode that alternately acquires at least one first image for the first tracking and at least one second image for the second tracking at the same FPS (or number of images). In other words, the first mode may represent a shooting mode that alternately uses the first technique and the second technique at the same frequency. The first mode may be referred to as a basic mode. For example, the second mode may represent a shooting mode in which the at least one first image for the first tracking and the at least one second image for the second tracking are alternately acquired at different FPS (or numbers of images). In other words, the second mode may represent a shooting mode in which the first technique and the second technique are alternately used at different frequencies. For example, the third mode may represent a shooting mode in which one of the at least one first image for the first tracking and the at least one second image for the second tracking is acquired. In other words, the third mode may represent a shooting mode in which one of the first technique and the second technique is used. The wearable device (103) may determine, through the image acquisition module (581), from image information acquired from the camera (520) whether the image information is data for the first tracking or data for the second tracking.For example, the image information may indicate visual information and attributes of the image information to the external environment. For example, the attributes may include a bright image (or an image acquired through automatic exposure) or a dark image (or an image acquired through fixed exposure).

[0112] Referring to FIG. 5, according to one embodiment, the wearable device (103) may perform at least one tracking based on the execution of the tracking module (583) within the framework layer (580). For example, the at least one tracking may include head tracking, hand tracking, and / or controller tracking. However, the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may identify a quality for each of the at least one tracking based on the execution of the tracking module (583). For example, the quality may be identified based on feature values ​​within an image acquired for each of the at least one tracking. For example, the feature values ​​may include a boundary or point of an external object (or an external electronic device). For example, the feature values ​​may include a boundary or point of a light source included in the external electronic device. For example, the quality may be compared to at least one reference value set for each tracking, such as head tracking, hand tracking, and / or controller tracking. Based on the comparison of the quality and the at least one reference value, the mode may be selected.

[0113] Referring to FIG. 5, according to one embodiment, the wearable device (103) may change the mode based on the execution of the mode change module (585) within the framework layer (580). For example, the wearable device (103) may change the mode based on whether a specified condition is satisfied. For example, the specified condition may be related to the quality of each tracking of the tracking module (583), a user's input to the wearable device (103) (e.g., a gesture or input to a physical button), a specific software application or setting within the wearable device (103), or a movement of an external electronic device (420).

[0114] Referring to FIG. 5, according to one embodiment, an external electronic device (420) may be connected to a wearable device (103). After being connected to the wearable device (103), the external electronic device (420) may be used to provide a user with input to the wearable device (103). For example, the external electronic device (420) may include a button. For example, the button may be used to change the mode of the wearable device (103).

[0115] Figure 6 illustrates an example of an operation flow for a method of acquiring images according to a mode of a wearable device.

[0116] At least some of the methods of FIG. 6 may be performed by the wearable device (103) of FIG. 5. For example, at least some of the methods may be controlled by the processor (510) of the wearable device (103). In the embodiments described below, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0117] In operation (600), according to one embodiment, the wearable device (103) may perform booting. For example, the wearable device (103) may perform the booting based on an input. For example, the input may include an input from a user wearing the wearable device (103), or an input to a physical button of the wearable device (103) for booting.

[0118] In operation (605), according to an embodiment, the wearable device (103) may execute a basic mode. For example, the wearable device (103) may execute the basic mode based on the booting. For example, the basic mode may be referred to as a first mode. For example, the basic mode may represent a shooting mode that alternately acquires one image for tracking (or first tracking) a body part of the user and one image for tracking (or second tracking) an external electronic device (420) connected to the wearable device (103).

[0119] In operation (610), according to one embodiment, the wearable device (103) may acquire images in the basic mode. For example, the wearable device (103) may acquire the images set at a specified FPS. For example, the specified FPS may indicate the number of images (or frames) per hour set for the camera (520) of the wearable device (103). For example, the specified FPS may be a performance of the camera (520) or a value set for the wearable device (103).

[0120] For example, the images based on the specified FPS may include at least one first image(s) for the first tracking and at least one second image(s) for the second tracking. For example, the number of the at least one first image(s) in the images acquired in the basic mode may correspond to the number of the at least one second image(s) in the images. In other words, the FPS of the at least one first image(s) may be the same as the FPS of the at least one second image(s). As described above, the wearable device (103) may sequentially acquire, in the basic mode, an image for the first tracking, an image for the second tracking, an image for the first tracking, and an image for the first tracking, through or using the camera (520).

[0121] For example, the at least one first image may have a first attribute, and the at least one second image may have a second attribute that is different from the first attribute. For example, the first attribute and the second attribute may each include a first brightness and a second brightness. For example, the first brightness may be brighter than the second brightness. For example, the at least one first image having the first brightness may be used to obtain first feature values ​​for the first tracking. For example, the first feature values ​​may include a boundary line or point of an external object for the first tracking. In addition, for example, the at least one second image having the second brightness may be used to obtain second feature values ​​for the second tracking. For example, the second feature values ​​may include light sources of an external electronic device (420) for the second tracking.

[0122] In operation (615), according to an embodiment, the wearable device (103) may identify whether a specified condition is satisfied. For example, the wearable device (103) may identify whether the specified condition is satisfied to change the mode (or, shooting mode) of the wearable device (103). For example, the specified condition may be related to the quality of each tracking (e.g., first and second tracking), the user's input to the wearable device (103) (e.g., a gesture or an input to a physical button), a specific software application or setting within the wearable device (103), or the movement of an external electronic device (420). The shooting mode may include a first mode (or the basic mode), a second mode, and a third mode. For example, the first mode may represent a shooting mode in which at least one first image for the first tracking and at least one second image for the second tracking are alternately acquired at the same FPS (or number of images). The first mode may be referred to as a basic mode. For example, the second mode may represent a shooting mode in which the at least one first image for the first tracking and the at least one second image for the second tracking are alternately acquired at different FPS (or numbers of images). For example, the third mode may represent a shooting mode in which one of the at least one first image for the first tracking and the at least one second image for the second tracking is acquired.

[0123] In one embodiment, the wearable device (103) can identify a quality of each tracking based on the feature values. The quality may be referred to as pose quality. For example, the wearable device (103) can identify a first quality of the first tracking based on the first feature values. For example, the wearable device (103) can identify a second quality of the second tracking based on the second feature values.

[0124] According to one embodiment, the wearable device (103) may change the mode (or shooting mode) of the wearable device (103) based on the comparison result between the tracking-specific quality and the tracking-specific reference value. For example, the wearable device (103) may compare the first quality with a first reference value set for the first tracking. Additionally, for example, the wearable device (103) may compare the second quality with a second reference value set for the second tracking. The first reference value set may include at least one reference value. The second reference value set may include at least one reference value. For example, assume that the first reference value set includes a first reference value and the second reference value set includes a second reference value. For example, the wearable device (103) may execute the first mode when the first quality is equal to or greater than the first reference value and the second quality is equal to or greater than the second reference value. For example, the wearable device (103) may execute the first mode when the first quality is less than the first reference value and the second quality is less than the second reference value. For example, the wearable device (103) may execute the second mode when the first quality is equal to or greater than the first reference value and the second quality is less than the second reference value, or when the first quality is less than the first reference value and the second quality is equal to or greater than the second reference value. For example, in the second mode executed in response to the case where the first quality is equal to or greater than the first reference value and the second quality is less than the second reference value, the number of images (or FPS) for the second tracking may be greater than (or higher than) the number of images (or FPS) for the first tracking.Additionally and / or alternatively, in the second mode, which is executed in response to, for example, the first quality being less than the first reference value and the second quality being greater than or equal to the second reference value, the number of images (or FPS) for the first tracking may be greater than (or higher than) the number of images (or FPS) for the second tracking. In the above example, the case where each of the first reference value set and the second reference value set includes one reference value is described, but the embodiments of the present disclosure are not limited thereto. For example, each of the first reference value set and the second reference value set may include a plurality of reference values. Based on the plurality of reference values, the number of images (or FPS) for a specific tracking acquired through the camera (520) may be adjusted. For example, assume a case where the first reference value set includes a plurality of reference values ​​(e.g., a first reference value and a second reference value). When the first quality is less than the first reference value, the wearable device (103) can acquire images for the first tracking based on the first FPS. When the first quality is greater than or equal to the first reference value and less than the second reference value, the wearable device (103) can acquire images for the first tracking based on a second FPS that is less than the first FPS. In this case, each of the first FPS and the second FPS may be a higher value than the FPS of the images for the second tracking (i.e., the number of images for the first tracking is greater than the number of images for the second tracking).

[0125] In one embodiment, the wearable device (103) can change modes based on input. For example, the wearable device (103) can change modes based on gestures set within the wearable device (103). For example, the wearable device (103) can include a first gesture for the first mode, a second gesture for the second mode, and a third gesture for the third mode. Additionally and / or alternatively, for example, the wearable device (103) can change modes based on input to a physical button included in the wearable device (103). Specific details related thereto are described below in FIG. 8B.

[0126] According to one embodiment, the wearable device (103) can change the mode based on the executed software application or the settings within the wearable device (103). For example, the wearable device (103) can recognize the mode set within the software application in response to executing the software application. The wearable device (103) can change the photographing mode of the wearable device (103) to the set mode. In addition, for example, the wearable device (103) can recognize the set mode according to the settings within the wearable device (103). For example, the settings within the wearable device (103) can be changed based on at least a portion of a user's input. The wearable device (103) can change the photographing mode to the set mode. For example, when a software application (e.g., a game) requiring relatively large amounts of movement of the external electronic device (420) (or controller) is executed or the settings within the wearable device (103) are changed, the wearable device (103) may change the shooting mode from the first mode to the second mode. At this time, the second mode may indicate a shooting mode in which the number of images for the first tracking is less than the number of images for the second tracking. This may be to increase the accuracy or response speed of tracking the movement of the external electronic device (420). Specific details related thereto are described below in FIG. 8A.

[0127] According to one embodiment, the wearable device (103) may change the capturing mode based on the movement of the external electronic device (420) (or controller) connected to the wearable device (103). For example, the wearable device (103) may change the capturing mode from the first mode to the second mode based on identifying that the movement is less than a reference frequency. The second mode, which is executed in response to the movement being less than the reference frequency, may represent a capturing mode in which the number of images for the first tracking is greater than the number of images for the second tracking. This is because the second tracking of the external electronic device (420) is not required when the movement is less than the reference frequency. Specific details related thereto are described below in FIG. 8A.

[0128] According to one embodiment, the wearable device (103) can recognize the movement of the external electronic device (420) based on the sensor (530). For example, the wearable device (103) can obtain information about the movement of the external electronic device (420) through the sensor (530) (e.g., an IMU sensor). The wearable device (103) can change the shooting mode based on the movement identified based on the information.

[0129] As described above, the wearable device (103) can identify whether the specified condition for changing the shooting mode is satisfied. For example, the wearable device (103) can identify whether the specified condition is satisfied periodically (or aperiodically) from the timing of acquiring the images in operation (610). Additionally and / or alternatively, for example, the wearable device (103) can identify whether the specified condition is satisfied when an event occurs after acquiring the images. For example, the event may include the execution of a specific software application. Additionally and / or alternatively, for example, the wearable device (103) can identify whether the specified condition is satisfied when a signal requesting a change of mode is received from an external electronic device (420) (or controller) after acquiring the images.

[0130] In operation (615), if the above-mentioned condition is satisfied, the wearable device (103) may perform operation (620). Alternatively, in operation (615), if the above-mentioned condition is not satisfied, the wearable device (103) may perform operation (610). For example, the wearable device (103) may, in the current shooting mode (e.g., the basic mode), acquire images and perform tracking based on the images. The tracking may include the first tracking and the second tracking. In other words, by returning to operation (610), the wearable device (103) may acquire images without changing the basic mode (or while the basic mode is maintained).

[0131] In operation (620), the wearable device (103) according to one embodiment can change the mode. For example, the wearable device (103) can recognize the mode to be changed based on the specified condition. The wearable device (103) can change the shooting mode from the basic mode (or the first mode) to the recognized mode.

[0132] In operation (625), the wearable device (103) according to one embodiment may acquire other images in the changed mode. For example, the wearable device (103) may acquire the other images based on the FPS (or number of images) set in the changed mode.

[0133] For example, if the changed mode is the second mode, the other images may include at least one third image for the first tracking and at least one fourth image(s) for the second tracking. For example, the number (or FPS) of the at least one third image(s) may be different from the number (or FPS) of the at least one fourth image(s).

[0134] In addition, for example, when the changed mode is the third mode, the other images may include one of at least one third image for the first tracking and at least one fourth image for the second tracking. For example, when the first tracking is unnecessary (e.g., when the user's head is fixed), the other images may include at least one fourth image for the second tracking among the at least one third image for the first tracking and the at least one fourth image for the second tracking. Conversely, when the second tracking is unnecessary (e.g., when the user does not hold the external electronic device (420), the other images may include at least one third image for the first tracking and at least one fourth image for the second tracking.

[0135] Figures 7a to 7c illustrate examples of a method for acquiring images depending on the mode of a wearable device.

[0136] The modes of FIGS. 7A to 7C may represent the shooting modes of the wearable device (103). For example, the modes may include a first mode, a second mode, and a third mode. FIG. 7A illustrates an example (700) of a method for acquiring images based on the first mode. FIG. 7B illustrates examples (720, 740) of a method for acquiring images based on the second mode. FIG. 7C illustrates examples (760, 780) of a method for acquiring images based on the third mode.

[0137] Referring to example (700) of FIG. 7A, the wearable device (103) can acquire images in the first mode. For example, the wearable device (103) can acquire the images through the camera (520) in the first mode. For example, the images acquired in the first mode can include first images (701) for the first tracking and second images (703) for the second tracking. For example, the first images (701) can be frames for the first tracking (e.g., head tracking or hand tracking). For example, the second images (703) can be frames for the second tracking (e.g., controller tracking). For example, the first images (701) for the first tracking can have a first attribute. For example, the first attribute can include a first brightness. For example, the second images (703) for the second tracking may have a second attribute that is different from the first attribute. For example, the second attribute may include a second brightness that is darker than the first brightness. In the example (700) of FIG. 7A, an example is described in which the wearable device (103) alternately acquires the images for the first tracking and the images for the second tracking, but the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may alternately acquire two images for the first tracking and two images for the second tracking. In this case, the FPS of the first images (701) may be the same as the FPS of the second images (703). For convenience of explanation, FIG. 7A illustrates examples of first images (701) and second images (703) having the same length with respect to time, but the embodiments of the present disclosure are not limited thereto.For example, each of the first images (701) having the first property uses the first technique that requires relatively more exposure time compared to each of the second images (703) having the second property, so the length of time (or length of time) required to acquire each of the first images (701) may be formed to be longer than the length of time (or length of time) required to acquire each of the second images (703).

[0138] Referring to example (720) of FIG. 7B, the wearable device (103) can acquire images in the second mode. For example, the wearable device (103) can acquire the images through the camera (520) in the second mode. For example, the images acquired in the second mode can include first images (721) for the first tracking and second images (723) for the second tracking. For example, the first images (721) can be frames for the first tracking (e.g., head tracking or hand tracking). For example, the second images (723) can be frames for the second tracking (e.g., controller tracking). For example, the first images (721) for the first tracking can have a first attribute. For example, the first attribute can include a first brightness. For example, the second images (723) for the second tracking may have a second attribute that is different from the first attribute. For example, the second attribute may include a second brightness that is darker than the first brightness. In the second mode of example (720), the FPS of the first images (721) may be higher than the FPS of the second images (723). In other words, the wearable device (103) may acquire images that include a greater number of the first images (721) than the number of the second images (723) in the second mode of example (720). In example (720) of FIG. 7B, the case where the FPS of the first images (721) is twice the FPS of the second images (723) is illustrated, but the embodiments of the present disclosure are not limited thereto. For example, the FPS of the first images (721) may be 3 or 4 times the FPS of the second images (723). Additionally, for example, the FPS of the first images (721) may be 1.5 times the FPS of the second images (723).In this case, the wearable device (103) can alternately acquire three first images (721) and two second images (723). For convenience of explanation, FIG. 7B illustrates examples of first images (721) and second images (723) having the same length in time, but the embodiments of the present disclosure are not limited thereto. For example, since each of the first images (721) having the first property uses the first technique that requires a relatively longer exposure time compared to each of the second images (723) having the second property, the length of time (e.g., the length of time) required to acquire each of the first images (721) can be formed to be longer than the length of time (e.g., the length of time) required to acquire each of the second images (723).

[0139] Referring to example (740) of FIG. 7B, the wearable device (103) can acquire images in the second mode. For example, the wearable device (103) can acquire the images through the camera (520) in the second mode. For example, the images acquired in the second mode can include first images (741) for the first tracking and second images (743) for the second tracking. For example, the first images (741) can be frames for the first tracking (e.g., head tracking or hand tracking). For example, the second images (743) can be frames for the second tracking (e.g., controller tracking). For example, the first images (741) for the first tracking can have a first attribute. For example, the first attribute can include a first brightness. For example, the second images (743) for the second tracking may have a second attribute that is different from the first attribute. For example, the second attribute may include a second brightness that is darker than the first brightness. In the second mode of example (740), the FPS of the first images (741) may be lower than the FPS of the second images (743). In other words, the wearable device (103) may acquire images that include a smaller number of the first images (741) than the second images (743) in the second mode of example (740). In the example (740) of FIG. 7B, the case where the FPS of the second images (743) is twice the FPS of the first images (741) is illustrated, but the embodiments of the present disclosure are not limited thereto. For example, the FPS of the second images (743) may be three or four times the FPS of the first images (741). Additionally, for example, the FPS of the second images (743) may be 1.5 times the FPS of the first images (741).In this case, the wearable device (103) can alternately acquire three second images (743) and two first images (741). For convenience of explanation, FIG. 7B illustrates first images (741) and second images (743) having the same length in time as an example, but the embodiments of the present disclosure are not limited thereto. For example, since each of the first images (741) having the first property uses the first technique that requires a relatively longer exposure time compared to each of the second images (743) having the second property, the length of time (e.g., the length of time) required to acquire each of the first images (741) can be formed to be longer than the length of time (e.g., the length of time) required to acquire each of the second images (743).

[0140] Referring to FIG. 7C, the wearable device (103) can acquire images in the third mode. For example, the wearable device (103) can acquire the images through the camera (520) in the third mode. The images acquired in the third mode of example (760) may include first images (761) for the first tracking. The images acquired in the third mode of example (760) may not include second images (763) for the second tracking. For example, the first images (761) may be frames for the first tracking (e.g., head tracking or hand tracking). For example, although the frames for the second images (763) are not shown in example (760), the second images (763) may be frames for the second tracking (e.g., controller tracking). For example, the first images (761) for the first tracking may have a first property. For example, the first property may include a first brightness. Alternatively, the images acquired in the third mode of example (780) may include second images (783) for the second tracking. The images acquired in the third mode of example (780) may not include the first images (781) for the first tracking. For example, although the frame for the first images (781) in example (780) is not shown, the first images (781) may be frames for the first tracking (e.g., head tracking or hand tracking). For example, the second images (783) may be frames for the second tracking (e.g., controller tracking). For example, the second images (783) for the second tracking may have a second property different from the first property. For example, the second property may include a second brightness that is darker than the first brightness.For convenience of explanation, in FIG. 7C, first images (761 or 781) and second images (763 or 783) having the same length in time are shown as examples, but the embodiments of the present disclosure are not limited thereto. For example, each of the first images (761 or 781) having the first property uses the first technique that requires relatively more exposure time compared to each of the second images (763 or 783) having the second property, so the length of time (e.g., length of time) required to acquire each of the first images (761 or 781) may be formed to be longer than the length of time (e.g., length of time) required to acquire each of the second images (763 or 783).

[0141] As described above, the wearable device (103) can execute or change the mode of the wearable device (103) based on specified conditions. Examples of executing or changing the mode are as follows. However, the electronic device and method utilizing a variable sequence according to embodiments of the present disclosure are not limited to the following examples.

[0142] For example, the wearable device (103) may execute the first mode in response to booting of the wearable device (103). Additionally and / or alternatively, for example, the wearable device (103) may execute the first mode when the wearable device (103) is configured to utilize a fixed shooting sequence within the wearable device (103). Additionally and / or alternatively, for example, the wearable device (103) may execute the first mode when the first quality of the first tracking is greater than or equal to a first threshold value and the second quality of the second tracking is greater than or equal to a second threshold value. Additionally and / or alternatively, for example, the wearable device (103) may execute the first mode when the first quality is less than the first threshold value and the second quality is less than the second threshold value.

[0143] For example, the first quality may be identified based on first feature values ​​obtained from each of the first images for the first tracking (e.g., the first images (701) of FIG. 7A, the first images (721, 741) of FIG. 7B, and the first images (761, 781) of FIG. 7C). For example, the first feature values ​​may include a boundary line or a point of an external object included in each of the first images for the first tracking. For example, the wearable device (103) may identify the first quality based on the amount of change in the position of the boundary line (or point) between the first images. For example, the second quality may be identified based on second feature values ​​obtained from each of the second images for the second tracking (e.g., the second images (703) of FIG. 7A, the second images (723, 743) of FIG. 7B, and the second images (763, 783) of FIG. 7C). For example, the second feature values ​​may include light sources of the external electronic device (420) included in each of the second images for the second tracking. For example, the wearable device (103) may identify the second quality based on the amount of change in the positions of the light sources between the second images. As described above, when there are many external objects that can recognize the boundary line within the FoV of the camera (520) of the wearable device (103), the first quality of the first tracking may be increased. In contrast, in environments where it is difficult to recognize the boundary within the FoV (e.g., an area without any pattern, such as a white wall), the first quality may be lowered. Additionally and / or alternatively, in cases where it is difficult to recognize the external electronic device (420) due to the external electronic device (420) being located outside the FoV, the second quality of the second tracking may be lowered.

[0144] For example, assume that an external electronic device (420) is positioned within the FoV of the camera (520) of the wearable device (103) in an environment where it is difficult to recognize the boundary line. The wearable device (103) can recognize the first quality as low because it is difficult to recognize the boundary line, and the wearable device (103) can recognize the second quality as high because the external electronic device (420) is positioned within the FoV. When the second quality is higher than the reference value of the second tracking (e.g., the second reference value), the wearable device (103) can execute (or change) the second mode to increase the first quality. At this time, the executed second mode may have an FPS of an image for the first tracking that is higher than an FPS of an image for the second tracking.

[0145] For example, if the external electronic device (420) is not used for a specified period of time or there is no movement of the external electronic device (420), the wearable device (103) may reduce the number of images acquired for the second tracking and increase the number of images acquired for the first tracking. The wearable device (103) may execute the second mode. At this time, the executed second mode may have an FPS of an image for the first tracking higher than an FPS of an image for the second tracking. Thereafter, the wearable device (103) may execute the first mode from the second mode (e.g., next) when the wearable device (103) detects movement of the external electronic device (420) based on an input or sensor for the wearable device (103). In the above example, an example of changing from the second mode to the first mode is described, but the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may change from the second mode of example (740) to the second mode of example (720).

[0146] For example, the wearable device (103) may execute the second mode when an external electronic device (420) is positioned outside the FoV. At this time, the executed second mode may have an FPS of an image for the second tracking that is higher than the FPS of an image for the first tracking, as in example (740). This may be to recognize an external electronic device (420) positioned outside the FoV and perform the second tracking of the external electronic device (420).

[0147] Additionally and / or alternatively, for example, the wearable device (103) may execute the second mode to increase the accuracy or response speed of the second tracking for an external electronic device (420) required by the execution or service of a specific software application. In this case, the executed second mode may have an FPS of an image for the second tracking that is higher than the FPS of an image for the first tracking, as in example (740).

[0148] For example, the wearable device (103) may execute the third mode based on the settings of a specific software application or service. For example, the wearable device (103) may execute the third mode based on the setting that activates input using only the external electronic device (420). In this case, the executed third mode may indicate a shooting mode for acquiring images for the second tracking, such as in example (780). Additionally and / or alternatively, for example, based on the setting being deactivated, the wearable device (103) may change from the third mode to the first mode, which is the default mode. For example, the wearable device (103) may execute the third mode based on the setting indicating that the external electronic device (420) is not to be used. At this time, the third mode being executed may represent a shooting mode for acquiring images for the first tracking, such as in example (760).

[0149] FIG. 8A illustrates an example of a method for changing the mode of a wearable device based on movement of an external electronic device.

[0150] The external electronic device (420) of FIG. 8A may represent the external electronic device (420) of FIG. 4 and the external electronic device (420) of FIG. 5. The wearable device (103) of FIG. 8A may represent the electronic device (101) of FIG. 1, the wearable devices (103) of FIGS. 2A, 2B, 3A, and 3B, the wearable device (103) of FIG. 4, and the wearable device (103) of FIG. 5. For example, the wearable device (103) may be connected to the external electronic device (420). The external electronic device (420) may be used to provide input to the wearable device (103). For example, the input may be caused by a user (410).

[0151] FIG. 8A illustrates examples (800, 805) of changing the mode based on the movement of the external electronic device (420). The mode may represent a shooting mode in which the wearable device (103) acquires images for tracking through the camera (520).

[0152] Referring to example (800), a user (410) may wear a wearable device (103) and hold an external electronic device (420). For example, the user (410) may sit on a chair while wearing the wearable device (103). At this time, a body part (810) of the user (410) may be supported by an external object (820). For example, the body part (810) may include the elbow of the right hand on which the external electronic device (420) is held. For example, the external object (820) may include the armrest of the chair. When the body part (810) is supported by the external object (820), the wearable device (103) may expect that the movement of the external electronic device (420) will be small. For example, the wearable device (103) may anticipate that the movement will be small based on recognizing that the body part (810) of the user (410) is supported by an external object (820). Accordingly, the wearable device (103) may change the mode. For example, the wearable device (103) may execute the second mode based on identifying that the movement is less than a reference frequency while the first mode, which is the default mode, is executing. At this time, the executed second mode may have an FPS of an image for the first tracking that is higher than an FPS of an image for the second tracking, as in example (720).

[0153] Referring to example (805), the user (410) can wear the wearable device (103) and hold the external electronic device (420). For example, the user (410) can utilize the virtual environment (830) provided by the wearable device (103). For example, the virtual environment (830) can include a game that requires a lot of movement of the external electronic device (420) or a high level of accuracy or response speed of the second tracking for the movement. For example, when a software application providing the virtual environment (830) is executed, the wearable device (103) can change the mode. For example, the wearable device (103) can execute the second mode based on identifying the execution of the software application providing the virtual environment (830) while the first mode, which is the default mode, is being executed. At this time, the second mode being executed may have an FPS of an image for the second tracking higher than an FPS of an image for the first tracking, as in example (740).

[0154] FIG. 8b illustrates an example of a method for changing the mode of a wearable device based on user input.

[0155] The external electronic device (420) of FIG. 8B may represent the external electronic device (420) of FIG. 4 and the external electronic device (420) of FIG. 5. The wearable device (103) of FIG. 8B may represent the electronic device (101) of FIG. 1, the wearable devices (103) of FIGS. 2A, 2B, 3A, and 3B, the wearable device (103) of FIG. 4, and the wearable device (103) of FIG. 5. For example, the wearable device (103) may be connected to the external electronic device (420). The external electronic device (420) may be used to provide input to the wearable device (103). For example, the input may be caused or input by the user (410).

[0156] FIG. 8b illustrates examples (850, 855) of changing the mode based on input to the wearable device (103). The mode may represent a shooting mode in which the wearable device (103) acquires images for tracking through the camera (520).

[0157] Referring to example (850), the wearable device (103) may obtain an input (860) from a user (410). For example, the input (860) may include a designated gesture. For example, the designated gesture may be associated with a specific mode. In example (850), an “X”-shaped input (860) may be associated with the first mode. In example (850), while the wearable device (103) is executing a mode different from the first mode (e.g., a second mode or a third mode), in response to obtaining the input (860), the mode may be changed to the first mode. However, the embodiments of the present disclosure are not limited thereto. In addition, although example (850) illustrates an example in which the input (860) is performed through a right hand holding an external electronic device (420), the embodiments of the present disclosure are not limited thereto. For example, the wearable device (103) may recognize left-hand-based input (860) based on tracking of the left hand (i.e., hand tracking) and change the mode based on the input (860).

[0158] Referring to example (855), the wearable device (103) may include a physical button (870). The physical button (870) may be used to change the mode of the wearable device (103). The wearable device (103) may change the mode based on an input of the user (410) to the physical button (870). However, the embodiment of the present disclosure is not limited thereto. Although not illustrated, for example, the physical button (870) may also be included in an external electronic device (420). The wearable device (103) may recognize an input of the user (410) to the physical button (870) included in the external electronic device (420). For example, in response to obtaining an input for a physical button (870) of a user (410), an external electronic device (420) may provide (or transmit) information about the input to the wearable device (103). The wearable device (103) may change the mode based on the information.

[0159] FIG. 9 illustrates an example of an operational flow for a method of acquiring images including images for tracking body parts and images for tracking external electronic devices, depending on the mode of the wearable device.

[0160] At least some of the methods of FIG. 9 may be performed by the wearable device (103) of FIG. 5. For example, at least some of the methods may be controlled by the processor (510) of the wearable device (103). In the embodiments described below, the operations may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0161] In operation (900), according to one embodiment, the wearable device (103) may acquire images including at least one first image having a first attribute and at least one second image having a second attribute in a first mode. For example, the wearable device (103) may acquire the at least one first image and the at least one second image through the camera (520) in the first mode. For example, the first mode may represent a shooting mode that alternately acquires one image having the first attribute and one image having the second attribute. The first mode may be referred to as a basic mode of the wearable device (103).

[0162] For example, the at least one first image having the first property may be used for tracking (e.g., first tracking) a body part (e.g., head or hand) of a user wearing the wearable device (103). In addition, the at least one second image having the second property may be used for tracking (e.g., second tracking) an external electronic device (420) connected to the wearable device (103). For example, the number of the at least one first image(s) in the images acquired within the basic mode may correspond to the number of the at least one second image(s) in the images. In other words, the FPS of the at least one first image(s) may be the same as the FPS of the at least one second image(s).

[0163] In operation (910), according to one embodiment, the wearable device (103) may obtain first feature values ​​from the at least one first image(s) and second feature values ​​from the at least one second image(s). For example, the wearable device (103) may obtain the first feature values ​​or the second feature values ​​using each of the images.

[0164] For example, the first attribute and the second attribute may include a first brightness and a second brightness, respectively. For example, the first brightness may be brighter than the second brightness. For example, the at least one first image having the first brightness may be used to obtain first feature values ​​for the first tracking. For example, the first feature values ​​may include a boundary line or point of an external object for the first tracking. In addition, for example, the at least one second image having the second brightness may be used to obtain second feature values ​​for the second tracking. For example, the second feature values ​​may include light sources of an external electronic device (420) for the second tracking.

[0165] In operation (920), according to one embodiment, the wearable device (103) may change the mode of the wearable device (103) from the first mode to the second mode. For example, the wearable device (103) may change the mode from the first mode to the second mode based on the first feature values ​​and the second feature values. For example, the second mode may represent a shooting mode in which the at least one first image(s) for the first tracking and the at least one second image(s) for the second tracking are alternately acquired at different FPS (or numbers of images).

[0166] In one embodiment, the wearable device (103) can identify a quality of each tracking based on the feature values. The quality may be referred to as pose quality. For example, the wearable device (103) can identify a first quality of the first tracking based on the first feature values. For example, the wearable device (103) can identify a second quality of the second tracking based on the second feature values.

[0167] According to one embodiment, the wearable device (103) may change the mode (or shooting mode) of the wearable device (103) based on the comparison result between the tracking-specific quality and the tracking-specific reference value. For example, the wearable device (103) may compare the first quality with a first reference value set for the first tracking. Additionally, for example, the wearable device (103) may compare the second quality with a second reference value set for the second tracking. The first reference value set may include at least one reference value. The second reference value set may include at least one reference value. For example, assume that the first reference value set includes a first reference value and the second reference value set includes a second reference value. For example, the wearable device (103) may execute the first mode when the first quality is equal to or greater than the first reference value and the second quality is equal to or greater than the second reference value. For example, the wearable device (103) may execute the first mode when the first quality is less than the first reference value and the second quality is less than the second reference value. For example, the wearable device (103) may execute the second mode when the first quality is equal to or greater than the first reference value and the second quality is less than the second reference value, or when the first quality is less than the first reference value and the second quality is equal to or greater than the second reference value. For example, in the second mode executed in response to the case where the first quality is equal to or greater than the first reference value and the second quality is less than the second reference value, the number of images (or FPS) for the second tracking may be greater than (or higher than) the number of images (or FPS) for the first tracking.Additionally and / or alternatively, in the second mode, which is executed in response to, for example, the first quality being less than the first reference value and the second quality being greater than or equal to the second reference value, the number of images (or FPS) for the first tracking may be greater than (or higher than) the number of images (or FPS) for the second tracking. In the above example, the case where each of the first reference value set and the second reference value set includes one reference value is described, but the embodiments of the present disclosure are not limited thereto. For example, the first reference value set and the second reference value set may each include a plurality of reference values. Based on the plurality of reference values, the number of images (or FPS) for specific tracking (e.g., the first tracking and the second tracking) acquired via the camera (520) may be adjusted. For example, assume a case where the first reference value set includes a plurality of reference values ​​(e.g., a first reference value and a second reference value). When the first quality is less than the first reference value, the wearable device (103) can acquire images for the first tracking based on the first FPS. When the first quality is greater than or equal to the first reference value and less than the second reference value, the wearable device (103) can acquire images for the first tracking based on a second FPS that is less than the first FPS. In this case, each of the first FPS and the second FPS may be a higher value than the FPS of the images for the second tracking (i.e., the number of images for the first tracking is greater than the number of images for the second tracking).

[0168] Although not illustrated in FIG. 9, according to one embodiment, the wearable device (103) can change modes based on input. For example, the wearable device (103) can change modes based on gestures set within the wearable device (103). For example, the wearable device (103) can include a first gesture for the first mode, a second gesture for the second mode, and a third gesture for the third mode. In addition, for example, the wearable device (103) can change modes based on input to a physical button included in the wearable device (103). For specific details related thereto, reference may be made to the example of FIG. 8B below.

[0169] Although not illustrated in FIG. 9, according to one embodiment, the wearable device (103) may change the mode based on a software application being executed or a setting within the wearable device (103). For example, the wearable device (103) may recognize a mode set within the software application in response to executing the software application. The wearable device (103) may change the mode of the wearable device (103) to the set mode. Additionally and / or alternatively, for example, the wearable device (103) may recognize a set mode based on the setting within the wearable device (103). For example, the setting within the wearable device (103) may be changed based on at least a portion of a user's input. The wearable device (103) may change the mode to the set mode. For example, when a software application (e.g., a game) requiring a relatively large amount of movement (compared to a predetermined amount of movement) of the external electronic device (420) (or controller) is executed or the settings within the wearable device (103) are changed, the wearable device (103) may change the mode from the first mode to the second mode. At this time, the second mode may indicate a shooting mode in which the number of images for the first tracking is less than the number of images for the second tracking. This may be to increase the accuracy or response speed of tracking the movement of the external electronic device (420). For specific details related thereto, reference may be made to the example of FIG. 8A below.

[0170] Although not illustrated in FIG. 9, according to one embodiment, the wearable device (103) may change the mode based on the movement of the external electronic device (420) (or controller) connected to the wearable device (103). For example, the wearable device (103) may change the mode from the first mode to the second mode based on identifying that the movement is less than a reference frequency. The second mode, which is executed in response to the movement being less than the reference frequency, may represent a shooting mode in which the number of images for the first tracking is greater than the number of images for the second tracking. This is because the second tracking of the external electronic device (420) is not required when the movement is less than the reference frequency. For specific details related thereto, reference may be made to the example of FIG. 8A below.

[0171] Although not illustrated in FIG. 9, according to one embodiment, the wearable device (103) may recognize the movement of the external electronic device (420) based on the sensor (530). For example, the wearable device (103) may obtain information about the movement of the external electronic device (420) through the sensor (530) (e.g., an IMU sensor). The wearable device (103) may change the mode based on the movement identified based on the information.

[0172] Referring to the above, the wearable device (103) can identify whether a specified condition for changing the mode is satisfied. If the specified condition is satisfied, the wearable device (103) can change the mode. Conversely, if the specified condition is not satisfied, the wearable device (103) can acquire images in the current mode (e.g., the first mode) and perform tracking based on the images. The tracking can include the first tracking and the second tracking.

[0173] In operation (930), according to one embodiment, the wearable device (103) may acquire other images in the second mode. For example, when the changed mode is the second mode, the other images may include at least one third image(s) for the first tracking and at least one fourth image(s) for the second tracking. For example, the number (or FPS) of the at least one third image may be different from the number (or FPS) of the at least one fourth image. However, the embodiments of the present disclosure are not limited thereto. For example, when the changed mode is the third mode, the other images may include one of at least one third image for the first tracking and at least one fourth image for the second tracking. For example, when the first tracking is unnecessary (e.g., when the user's head is fixed), the other images may include at least one third image for the first tracking and at least one fourth image for the second tracking among the at least one fourth image for the second tracking. Conversely, when the second tracking is unnecessary (e.g., when the user does not hold the external electronic device (420), the other images may include at least one third image for the first tracking and at least one fourth image for the second tracking among the at least one third image for the first tracking and the at least one fourth image for the second tracking.

[0174] As described above, the wearable device (103) may include a memory (560) storing instructions and including one or more storage media. The wearable device (103) may include a camera (520). The wearable device (103) may include at least one processor (510) including a processing circuit. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to acquire, through the camera (520), images including at least one first image having a first attribute for tracking a body part of a user and at least one second image having a second attribute for tracking an external electronic device (420) different from the first attribute. The first number of the at least one first image may correspond to the second number of the at least one second image. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to obtain first feature values ​​for tracking the body part from the at least one first image and second feature values ​​for tracking the external electronic device (420) from the at least one second image. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to change a mode of the wearable device (103) from the first mode to the second mode based on the first feature values ​​and the second feature values.The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to acquire, in the second mode, another set of images through the camera (520). A third number of at least one third image having the first property of the other images may differ from a fourth number of at least one fourth image having the second property of the other images.

[0175] In one embodiment, the body part may include at least one of the user's head or the user's hand.

[0176] According to one embodiment, each of the first attribute and the second attribute may include brightness. The brightness may be identified based on at least one of an exposure time for acquiring an image through the camera (520), a gain value for acquiring the image, or a brightness level of a light source of the wearable device (103).

[0177] According to one embodiment, in response to the first attribute indicating the first brightness and the second attribute indicating the second brightness, the first brightness may be brighter than the second brightness.

[0178] According to one embodiment, the images may be acquired based on a specified frame per second (FPS). The first FPS of the at least one first image may correspond to the second FPS of the at least one second image. The at least one first image having the first property and the at least one second image having the second property may be alternately obtained.

[0179] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to execute the first mode when a specified condition is satisfied. The specified condition may include at least one of: the wearable device (103) being booted; the first quality for tracking the body part being equal to or greater than a first reference value and the second quality for tracking the external electronic device (420) being equal to or greater than a second reference value; or the first quality being less than the first reference value and the second quality being less than the second reference value.

[0180] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to obtain first feature values ​​that include a boundary or a point of an object within the at least one first image. The instructions, when executed by the at least one processor (510), may cause the wearable device (103) to obtain second feature values ​​that include at least one light source of the external electronic device (420) within the at least one second image.

[0181] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to obtain a first quality for tracking the body part based on the first feature values. The instructions, when executed by the at least one processor (510), may cause the wearable device (103) to obtain a second quality for tracking the external electronic device (420) based on the second feature values. The first quality may be obtained based on a first variation of the first feature values ​​for the at least one first image. The second quality may be obtained based on a second variation of the second feature values ​​for the at least one second image.

[0182] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to change the mode from the first mode to the second mode in which the third number of the at least one third image having the first property of the other images is greater than the fourth number of the at least one fourth image having the second property, in response to the first quality being less than the first threshold value and the second quality being greater than the second threshold value.

[0183] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to change the mode from the first mode to the second mode in which the third number of the at least one third image having the first property of the other images is less than the fourth number of the at least one fourth image having the second property, when the first quality is greater than or equal to a first reference value and the second quality is less than a second reference value.

[0184] In one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to execute a software application after acquiring the other images. The instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to change the mode from the second mode to the first mode based on a setting of the software application. The setting may include a first FPS for the at least one first image of the first attribute to be acquired through the camera (520) and a second FPS for the at least one second image having the second attribute.

[0185] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to change the mode from the first mode to the second mode in which the third number of the at least one third image having the first property of the other images is greater than the fourth number of the at least one fourth image having the second property, based on identifying a movement of the external electronic device (420) that is less than a reference frequency.

[0186] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to change the mode from the second mode to the first mode based on identifying that the movement of the external electronic device (420) identified by a sensor of the wearable device (103) within the second mode is greater than or equal to the reference frequency. The sensor may include an inertial measurement unit (IMU) sensor.

[0187] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to change the mode from the first mode to the second mode, wherein the third number of the at least one third image having the first property of the other images is less than the fourth number of the at least one fourth image having the second property, based on execution of a software application providing a virtual environment using the external electronic device (420). While the second mode is executed, a first quality based on the at least one third image having the first property of the other images may be greater than or equal to a first reference value.

[0188] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to change the mode from the second mode to a third mode when a specified condition is satisfied. The specified condition may include setting tracking of the external electronic device (420) among tracking of the body part and tracking of the external electronic device (420). In the third mode, images having the second attribute among the first attribute and the second attribute may be acquired through the camera (520).

[0189] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to change the mode from the second mode to a third mode when a specified condition is satisfied. The specified condition may include setting tracking of the body part among tracking of the body part and tracking of the external electronic device (420). In the third mode, images having the first attribute among the first attribute and the second attribute may be acquired through the camera (520).

[0190] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (510), may cause the wearable device (103) to change the mode from the third mode to the first mode based on an input from the user. The input from the user may include at least one of an input to a physical button of the wearable device (103) or a designated gesture of the user.

[0191] As described above, a method performed by a wearable device (103) may include, in a first mode, an operation of acquiring images including at least one first image having a first attribute for tracking a body part of a user and at least one second image having a second attribute for tracking an external electronic device (420) that is different from the first attribute. A first number of the at least one first image may correspond to a second number of the at least one second image. The method may include an operation of acquiring first feature values ​​for tracking the body part from the at least one first image and second feature values ​​for tracking the external electronic device (420) from the at least one second image. The method may include an operation of changing a mode of the wearable device (103) from the first mode to the second mode based on the first feature values ​​and the second feature values. The method may include, in the second mode, an operation of acquiring another image through the camera (520). A third number of at least one third image having the first property of the other images may differ from a fourth number of at least one fourth image having the second property of the other images.

[0192] According to one embodiment, each of the first attribute and the second attribute may include brightness. The brightness may include at least one of an exposure time for acquiring an image through the camera (520), a gain value for acquiring the image, or a brightness level of a light source of the wearable device (103).

[0193] As described above, a non-transitory computer-readable storage medium may store one or more programs including instructions that, when individually or collectively executed by at least one processor (510) of a wearable device (103) including a camera (520), cause the wearable device (103) to, in a first mode, acquire, through the camera (520), images including at least one first image having a first attribute for tracking a body part of a user and at least one second image having a second attribute for tracking an external electronic device (420) different from the first attribute. A first number of the at least one first image may correspond to a second number of the at least one second image. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to obtain first feature values ​​for tracking the body part from the at least one first image and second feature values ​​for tracking the external electronic device (420) from the at least one second image. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to change a mode of the wearable device (103) from the first mode to the second mode based on the first feature values ​​and the second feature values.The non-transitory computer-readable storage medium may store one or more programs comprising instructions that, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to, in the second mode, acquire another set of images through the camera (520). A third number of at least one third image having the first property of the other images may differ from a fourth number of at least one fourth image having the second property of the other images.

[0194] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0195] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0196] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0197] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0198] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0199] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separately arranged in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In a wearable device (103), A memory (560) storing instructions and including one or more storage media; camera (520); and At least one processor (510) comprising a processing circuit; The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: In a first mode, images including at least one first image having a first attribute for tracking a body part of a user and at least one second image having a second attribute for tracking an external electronic device (420) different from the first attribute are acquired through the camera (520), wherein a first number of the at least one first image corresponds to a second number of the at least one second image; Obtaining first feature values ​​for tracking the body part from the at least one first image and second feature values ​​for tracking the external electronic device (420) from the at least one second image; Based on the first feature values ​​and the second feature values, changing the mode of the wearable device (103) from the first mode to the second mode; and In the second mode, other images are caused to be acquired through the camera (520), A third number of at least one third image having the first property of said other images differs from a fourth number of at least one fourth image having the second property of said other images. Wearable device (103).

2. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Obtaining the first feature values ​​including a boundary or point of an object within at least one first image; and causing said second feature values ​​to be acquired, said second feature values ​​including at least one light source of said external electronic device (420) within said at least one second image; Wearable device (103).

3. In claim 2, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Obtaining a first quality for tracking the body part based on the first feature values; and To obtain a second quality for tracking the external electronic device (420) based on the second feature values, The first quality is obtained based on a first variation of the first feature values ​​of the at least one first image, The second quality is obtained based on a second variation of the second feature values ​​for at least one second image. Wearable device (103).

4. In claim 3, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: In response to the first quality being less than the first criterion value and the second quality being greater than or equal to the second criterion value, causing the mode to be changed from the first mode to the second mode in which the third number of the at least one third image having the first property of the other images is greater than the fourth number of the at least one fourth image having the second property. Wearable device (103).

5. In claim 3, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: In response to the first quality being greater than or equal to the first criterion value and the second quality being less than the second criterion value, causing the mode to be changed from the first mode to the second mode, wherein the third number of the at least one third image having the first property of the other images is less than the fourth number of the at least one fourth image having the second property. Wearable device (103).

6. In claim 1, The body part comprises at least one of the user's head or the user's hand. Wearable device (103).

7. In claim 1, The first property and the second property each include brightness, The brightness is identified based on at least one of the exposure time for acquiring an image through the camera (520), the gain value for acquiring the image, or the brightness level of the light source of the wearable device (103). Wearable device (103).

8. In claim 1, The above images are acquired based on the specified FPS (frame per second), The first FPS of the at least one first image corresponds to the second FPS of the at least one second image, and At least one first image having the first property of the above images and at least one second image having the second property are alternately obtained, Wearable device (103).

9. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: If the specified condition is satisfied, cause the first mode to be executed, The above specified condition includes at least one of: the wearable device (103) is booted; the first quality for tracking the body part is equal to or greater than a first reference value and the second quality for tracking the external electronic device (420) is equal to or greater than a second reference value; or the first quality is less than the first reference value and the second quality is less than the second reference value. Wearable device (103).

10. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Based on identifying the movement of the external electronic device (420) below the reference frequency, causing the mode to be changed from the first mode to the second mode, wherein the third number of the at least one third image having the first property of the other images is greater than the fourth number of the at least one fourth image having the second property. Wearable device (103).

11. In claim 10, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: In the second mode, based on identifying that the movement of the external electronic device (420) identified by the sensor of the wearable device (103) is greater than or equal to the reference frequency, causing the mode to be changed from the second mode to the first mode, The above sensor includes an IMU (inertial measurement unit) sensor. Wearable device (103).

12. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: Based on the execution of a software application providing a virtual environment using the external electronic device (420), causing the mode to be changed from the first mode to the second mode in which the third number of the at least one third image having the first property of the other images is less than the fourth number of the at least one fourth image having the second property, While the second mode is running, the first quality based on the at least one third image having the first property of the other images is greater than or equal to the first reference value. Wearable device (103).

13. In claim 1, The above instructions, when individually or collectively executed by the at least one processor (510), cause the wearable device (103) to: If a specified condition is satisfied, causing the mode to be changed from the second mode to the third mode, The above specified conditions include setting of tracking of the external electronic device (420) among tracking of the body part and tracking of the external electronic device (420), and In the third mode, images having the second property among the first property and the second property are acquired through the camera (520). Wearable device (103).

14. In a method performed by a wearable device (103), In a first mode, an operation of acquiring images including at least one first image having a first attribute for tracking a body part of a user and at least one second image having a second attribute for tracking an external electronic device (420) different from the first attribute, wherein a first number of the at least one first image corresponds to a second number of the at least one second image; An operation of obtaining first feature values ​​for tracking the body part from the at least one first image and second feature values ​​for tracking the external electronic device (420) from the at least one second image; An operation of changing the mode of the wearable device (103) from the first mode to the second mode based on the first feature values ​​and the second feature values; and In the second mode, an operation for acquiring another image is included, A third number of at least one third image having the first property of said other images differs from a fourth number of at least one fourth image having the second property of said other images. method.

15. A non-transitory computer-readable storage medium, when individually or collectively executed by at least one processor (510) of a wearable device (103) including a camera (520), causes the wearable device (103) to: In a first mode, images including at least one first image having a first attribute for tracking a body part of a user and at least one second image having a second attribute for tracking an external electronic device (420) different from the first attribute are acquired through the camera (520), wherein a first number of the at least one first image corresponds to a second number of the at least one second image; Obtaining first feature values ​​for tracking the body part from the at least one first image and second feature values ​​for tracking the external electronic device (420) from the at least one second image; Based on the first feature values ​​and the second feature values, changing the mode of the wearable device (103) from the first mode to the second mode; and In the second mode, instructions are included that cause other images to be acquired through the camera (520). A third number of at least one third image having the first property of said other images differs from a fourth number of at least one fourth image having the second property of said other images. A non-transitory computer-readable storage medium.

Images