Electronic device, method and storage medium for acquiring image for tracking

Abstract

A wearable device may include a memory for storing instructions, a camera, and at least one processor. When executed individually or collectively by the at least one processor, the instructions may cause the wearable device to: acquire, in a first mode, an image comprising at least one first image having a first property for tracking a body part of a user and at least one second image having a second property for tracking a body part of the user; the at least one first image has a first attribute for tracking the external electronic device, the at least one second image has a second attribute for tracking the external electronic device, the second attribute being different from the first attribute, and acquiring a first feature value for tracking the body part from the at least one first image and a second feature value for tracking the external electronic device from the at least one second image; changing a mode of the wearable device from a first mode to a second mode based on the first feature value and the second feature value; and in the second mode, acquiring other images.

Description

Technical Field

[0001] The following description relates to electronic devices, methods, and storage media for acquiring images for tracking. Background Technology

[0002] To provide an enhanced user experience, electronic devices are being developed that offer extended reality services, displaying information generated by computers in combination with external objects in the real world or virtual objects in a virtual world. These electronic devices may include wearable devices that can be worn by a user. For example, electronic devices may include user devices, augmented reality (AR) glasses, virtual reality (VR) glasses, and / or head-mounted displays (HMDs) (e.g., video see-through (VST) HMDs and optical see-through (OST) HMDs).

[0003] The above information may be provided as relevant technology to aid in understanding this disclosure. No claim is made or any judgment is made regarding whether any of the above information can be used as prior art in connection with this disclosure. Summary of the Invention

[0004] Technical solution A wearable device may include: a memory storing instructions, a camera, and at least one processor. When executed by the at least one processor, the instructions may cause the wearable device, in a first mode, to acquire an image via the camera, comprising at least one first image and at least one second image, wherein the at least one first image has a first attribute for tracking a user's body part, and the at least one second image has a second attribute different from the first attribute for tracking external electronic devices. The number of at least one first image may correspond to the number of at least one second image. When executed by the at least one processor, the instructions may cause the wearable device to: obtain a first feature value from the at least one first image for tracking the body part, and obtain a second feature value from the at least one second image for tracking external electronic devices. When executed by the at least one processor, the instructions may cause the wearable device to: change the mode of the wearable device from a first mode to a second mode based on the first and second feature values. When executed by the at least one processor, the instructions may cause the wearable device to: acquire other images via the camera in the second mode. The number of other images having the first attribute may differ from the number of other images having the second attribute.

[0005] A method performed by a wearable device may include: acquiring an image comprising at least one first image and at least one second image in a first mode, the at least one first image having a first attribute for tracking a user's body part, and the at least one second image having a second attribute different from the first attribute for tracking an external electronic device. The number of at least one first image may correspond to the number of at least one second image. The method may include: obtaining a first feature value for tracking the body part from the at least one first image, and obtaining a second feature value for tracking the external electronic device from the at least one second image. The method may include: changing the mode of the wearable device from the first mode to the second mode based on the first and second feature values. The method may include: acquiring additional images via a camera in the second mode. The number of images with the first attribute in the additional images may differ from the number of images with the second attribute in the additional images.

[0006] A non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by at least one processor of a wearable device including a camera, cause the wearable device to: acquire an image including at least one first image and at least one second image via the camera in a first mode, the at least one first image having a first attribute for tracking a user's body parts, and the at least one second image having a second attribute different from the first attribute for tracking external electronic devices. The number of at least one first image may correspond to the number of at least one second image. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by at least one processor, cause the wearable device to: acquire a first feature value from the at least one first image for tracking body parts and acquire a second feature value from the at least one second image for tracking external electronic devices. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by at least one processor, cause the wearable device to: change the mode of the wearable device from a first mode to a second mode based on the first and second feature values. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by at least one processor, cause the camera in the second mode to acquire additional images. The number of images with the first attribute among the other images may differ from the number of images with the second attribute among the other images. Attached Figure Description

[0007] Figure 1 This is a block diagram of an electronic device in a network environment according to various embodiments.

[0008] Figure 2a Examples of perspective views of wearable devices according to various embodiments are shown.

[0009] Figure 2b Examples of settings in one or more hardware components of a wearable device according to various embodiments are shown.

[0010] Figure 3a and Figure 3b Examples of the appearance of wearable devices according to various embodiments are shown.

[0011] Figure 4 Examples of methods for obtaining images for tracking body parts and for tracking external electronic devices are shown.

[0012] Figure 5 An exemplary block diagram of a wearable device is shown.

[0013] Figure 6 An example of the operational flow of a method for obtaining images based on the patterns of a wearable device is shown.

[0014] Figure 7a , Figure 7b and Figure 7c An example of a method for obtaining images based on the patterns of a wearable device is shown.

[0015] Figure 8a An example of a method for altering the mode of a wearable device based on the movement of an external electronic device is shown.

[0016] Figure 8b An example of a method for changing the mode of a wearable device based on user input is shown.

[0017] Figure 9 An example of the operational flow of a method for obtaining images for tracking body parts and images including those for tracking external electronic devices based on the mode of a wearable device is shown. Detailed Implementation

[0018] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the scope of other embodiments. Singular expressions may include plural expressions unless the context clearly indicates otherwise. The terminology used herein, including technical or scientific terms, may have the same meaning as commonly understood by one of ordinary skill in the art described in this disclosure. Among the terminology used in this disclosure, unless expressly defined herein, terms defined in a general dictionary may be interpreted in the same or similar sense as they have in the context of the relevant art, and not in an ideal or overly formal sense. In some cases, even terms defined in this disclosure may not be construed as excluding embodiments of this disclosure.

[0019] In the various embodiments of this disclosure described below, hardware methods will be described as examples. However, since the various embodiments of this disclosure include techniques using both hardware and software, software-based methods are not excluded.

[0020] Furthermore, in this disclosure, the terms "greater than" or "less than" can be used to determine whether a particular condition is met or achieved, but this is merely a description of examples and does not exclude descriptions of "greater than or equal to" or "less than or equal to". A condition described as "greater than or equal to" can be replaced by "greater than", a condition described as "less than or equal to" can be replaced by "less than", and a condition described as "greater than or equal to and less than" can be replaced by "greater than and less than or equal to". Additionally, in the following text, "A" through "B" means at least one of the elements from A (inclusive) to B (inclusive).

[0021] Figure 1 This is a block diagram illustrating an electronic device 101 in a network environment 100 according to various embodiments.

[0022] Reference Figure 1 In network environment 100, electronic device 101 can communicate with electronic device 102 via a first network 198 (e.g., a short-range wireless communication network), or with at least one of electronic device 104 or server 108 via a second network 199 (e.g., a long-range wireless communication network). According to an embodiment, electronic device 101 can communicate with electronic device 104 via server 108. According to an embodiment, electronic device 101 may include a processor 120, memory 130, input module 150, sound output module 155, display module 160, audio module 170, sensor module 176, interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, user identification module (SIM) 196, or antenna module 197. In some embodiments, at least one of the above components (e.g., connection terminal 178) may be omitted from electronic device 101, or one or more other components may be added to electronic device 101. In some embodiments, some of the components described above (e.g., sensor module 176, camera module 180, or antenna module 197) may be implemented as a single integrated component (e.g., display module 160).

[0023] Processor 120 may run software (e.g., program 140) to control at least one other component (e.g., hardware or software component) of electronic device 101 connected to processor 120, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, processor 120 may store commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132, process the commands or data stored in volatile memory 132, and store the resulting data in non-volatile memory 134. According to embodiments, processor 120 may include a main processor 121 (e.g., central processing unit (CPU) or application processor (AP)) or an auxiliary processor 123 (e.g., graphics processing unit (GPU), neural processing unit (NPU), image signal processor (ISP), sensor central processor, or communication processor (CP)) that is operationally independent of or combined with the main processor 121. For example, when electronic device 101 includes a main processor 121 and an auxiliary processor 123, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or to be dedicated to a specific function. The auxiliary processor 123 may be implemented separately from the main processor 121, or may be implemented as part of the main processor 121.

[0024] When the main processor 121 is inactive (e.g., in sleep) state, the auxiliary processor 123 (rather than the main processor 121) can control at least some of the functions or states associated with at least one component of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190), or when the main processor 121 is active (e.g., running an application), the auxiliary processor 123 can work with the main processor 121 to control at least some of the functions or states associated with at least one component of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190). According to embodiments, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., camera module 180 or communication module 190) functionally associated with the auxiliary processor 123. According to embodiments, the auxiliary processor 123 (e.g., a neural processing unit) may include hardware architecture dedicated to artificial intelligence model processing. Artificial intelligence models can be generated through machine learning. For example, such learning can be performed via electronic device 101 where artificial intelligence is performed or via a separate server (e.g., server 108). The learning algorithm may include, but is not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The artificial intelligence model may include multiple layers of artificial neural networks. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q-network, or a combination of two or more thereof, but is not limited thereto. Additionally or optionally, the artificial intelligence model may include software structures in addition to hardware structures.

[0025] Memory 130 may store various data used by at least one component of electronic device 101 (e.g., processor 120 or sensor module 176). The various data may include, for example, software (e.g., program 140) and input or output data for commands associated with it. Memory 130 may include volatile memory 132 or non-volatile memory 134.

[0026] The program 140 may be stored as software in the memory 130, and the program 140 may include, for example, an operating system (OS) 142, middleware 144, or application 146.

[0027] The input module 150 can receive commands or data from outside the electronic device 101 (e.g., a user) that will be used by other components of the electronic device 101 (e.g., processor 120). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (e.g., buttons), or digital pen (e.g., stylus).

[0028] The sound output module 155 can output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records. The receiver can be used to receive incoming calls. According to an embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0029] Display module 160 can visually provide information to the outside of electronic device 101 (e.g., to a user). Display device 160 may include, for example, a display, a holographic device, or a projector, and control circuitry for controlling a respective one of the display, holographic device, and projector. According to an embodiment, display module 160 may include a touch sensor adapted to detect touch or a pressure sensor adapted to measure the intensity of the force caused by touch.

[0030] The audio module 170 can convert sound into electrical signals and vice versa. According to an embodiment, the audio module 170 can obtain sound via the input module 150, or output sound via the sound output module 155 or headphones of an external electronic device (e.g., electronic device 102) that is directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0031] Sensor module 176 can detect the operating state of electronic device 101 (e.g., power or temperature) or the environmental state outside electronic device 101 (e.g., user state), and then generate an electrical signal or data value corresponding to the detected state. According to embodiments, sensor module 176 may include, for example, a gesture sensor, gyroscope sensor, atmospheric pressure sensor, magnetic sensor, accelerometer, grip sensor, proximity sensor, color sensor, infrared (IR) sensor, biometric sensor, temperature sensor, humidity sensor, or illuminance sensor.

[0032] Interface 177 may support one or more specific protocols used to enable electronic device 101 to connect directly (e.g., wired) or wirelessly to external electronic devices (e.g., electronic device 102). According to embodiments, interface 177 may include, for example, a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital Card (SD) interface, or an audio interface.

[0033] Connection 178 may include a connector, through which electronic device 101 may be physically connected to an external electronic device (e.g., electronic device 102). According to embodiments, connection 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0034] The haptic module 179 can convert electrical signals into mechanical stimuli (e.g., vibration or motion) or electrical stimuli that can be recognized by a user through his touch or kinesthesia. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0035] Camera module 180 can capture still or moving images. According to an embodiment, camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0036] The power management module 188 manages the power supply to the electronic device 101. According to an embodiment, the power management module 188 may be implemented as at least part of, for example, a power management integrated circuit (PMIC).

[0037] Battery 189 can power at least one component of electronic device 101. According to an embodiment, battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable rechargeable battery, or a fuel cell.

[0038] Communication module 190 can support the establishment of a direct (e.g., wired) or wireless communication channel between electronic device 101 and external electronic devices (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. Communication module 190 may include one or more communication processors capable of operating independently of processor 120 (e.g., application processor (AP)) and support direct (e.g., wired) or wireless communication. According to embodiments, communication module 190 may include wireless communication module 192 (e.g., cellular communication module, short-range wireless communication module, or Global Navigation Satellite System (GNSS) communication module) or wired communication module 194 (e.g., local area network (LAN) communication module or power line communication (PLC) module). One of these communication modules can communicate with an external electronic device via a first network 198 (e.g., a short-range communication network such as Bluetooth, Wi-Fi Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a traditional cellular network, 5G network, next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip) or as multiple components separate from each other (e.g., multiple chips). The wireless communication module 192 can identify and verify the electronic device 101 in the communication network (such as the first network 198 or the second network 199) using user information (e.g., the International Mobile Subscriber Identity (IMSI)) stored in the user identification module 196.

[0039] Wireless communication module 192 can support 5G networks following 4G networks and next-generation communication technologies (such as new radio (NR) access technologies). NR access technologies can support enhanced mobile broadband (eMBB), massive machine-type communication (mMTC), or ultra-reliable low-latency communication (URLLC). Wireless communication module 192 can support high-frequency bands (e.g., millimeter-wave bands) to achieve, for example, high data transmission rates. Wireless communication module 192 can support various technologies used to ensure performance in high-frequency bands, such as, for example, beamforming, massive MIMO, full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. Wireless communication module 192 can support various requirements specified in electronic device 101, external electronic devices (e.g., electronic device 104), or network systems (e.g., second network 199). According to an embodiment, the wireless communication module 192 may support peak data rates (e.g., 20 Gbps or greater) for implementing eMBB, lost coverage (e.g., 164 dB or less) for implementing mMTC, or U-plane latency (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less round trip) for implementing URLLC.

[0040] Antenna module 197 can transmit or receive signals or power to or from the exterior of electronic device 101 (e.g., external electronic device). According to an embodiment, antenna module 197 may include an antenna comprising a radiating element formed of a conductive material or conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, antenna module 197 may include multiple antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as a first network 198 or a second network 199) can be selected from the multiple antennas by, for example, communication module 190 (e.g., wireless communication module 192). Signals or power can then be transmitted or received between communication module 190 and the external electronic device via the selected at least one antenna. According to an embodiment, additional components besides the radiating element (e.g., a radio frequency integrated circuit (RFIC)) may be additionally incorporated into antenna module 197.

[0041] According to various embodiments, antenna module 197 may form a millimeter-wave antenna module. According to embodiments, the millimeter-wave antenna module may include a printed circuit board, a radio frequency integrated circuit (RFIC), and multiple antennas (e.g., an array antenna), wherein the RFIC is disposed on or adjacent to a first surface (e.g., a bottom surface) of the printed circuit board and is capable of supporting a specified high-frequency band (e.g., a millimeter-wave band), and the multiple antennas are disposed on or adjacent to a second surface (e.g., a top surface or a side surface) of the printed circuit board and are capable of transmitting or receiving signals in the specified high-frequency band.

[0042] At least some of the aforementioned components can be interconnected and communicate signals (e.g., commands or data) between them via an inter-peripheral communication scheme (e.g., bus, general purpose input / output (GPIO), serial peripheral interface (SPI), or mobile industrial processor interface (MIPI)).

[0043] According to an embodiment, commands or data can be sent or received between electronic device 101 and external electronic device 104 via server 108 connected to a second network 199. Each of electronic device 102 or electronic device 104 can be a device of the same type as electronic device 101, or a device of a different type. According to an embodiment, all or some operations that would be performed on electronic device 101 can be performed on one or more of external electronic devices 102, external electronic devices 104, or server 108. For example, if electronic device 101 is required to automatically perform a function or service, or is required to perform a function or service in response to a request from a user or another device, electronic device 101 may request the one or more external electronic devices to perform at least a portion of the function or service, instead of running the function or service, or electronic device 101 may request the one or more external electronic devices to perform at least a portion of the function or service in addition to running the function or service. Upon receiving the request, one or more external electronic devices may perform at least a portion of the requested function or service, or perform additional functions or services related to the request, and transmit the result of the execution to electronic device 101. Electronic device 101 may provide the result as at least a partial response to the request, with or without further processing of the result. For this purpose, technologies such as cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing may be used. Electronic device 101 may use, for example, distributed computing or mobile edge computing to provide ultra-low latency services. In another embodiment, external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and / or neural networks. According to embodiments, external electronic device 104 or server 108 may be included in a second network 199. Electronic device 101 may be applied to intelligent services based on 5G communication technology or IoT-related technologies (e.g., smart homes, smart cities, smart cars, or healthcare).

[0044] Figure 2a Examples of perspective views of wearable devices according to various embodiments are shown. Figure 2b Examples of settings in one or more hardware components of a wearable device according to various embodiments are shown.

[0045] According to an embodiment, the wearable device 103 may have the shape of glasses that can be worn on a part of a user's body (e.g., the head). Figure 2a and Figure 2b The wearable device 103 can be Figure 1Examples of electronic devices 101. Wearable device 103 may include a head-mounted display (HMD). For example, the housing of wearable device 103 may include a flexible material, such as rubber and / or silicone, having a shape that closely contacts a portion of the user's head (e.g., the portion of the face covering the eyes). For example, the housing of wearable device 103 may include one or more straps capable of wrapping around (e.g., wrapping, bending, or adapting) around the user's head and / or one or more temples attached to the ears of the head.

[0046] refer to Figure 2a According to an embodiment, the wearable device 103 may include at least one display 250 and a frame 200 supporting the at least one display 250.

[0047] According to an embodiment, the wearable device 103 can be worn on a user's body part. The wearable device 103 can provide extended reality (XR) to the user wearing the wearable device 103. For example, extended reality can provide augmented reality (AR), virtual reality (VR), or mixed reality (MR) that combines augmented reality and virtual reality. For example, the wearable device 103 can respond to... Figure 2b The gesture recognition cameras 260-2 and 263 acquire the user's specified gesture and display it on at least one display 250. Figure 2b Virtual reality images provided by at least one optical device 282 and 284.

[0048] According to an embodiment, at least one display 250 can provide visual information to a user. For example, at least one display 250 may include a transparent or translucent lens. At least one display 250 may include a first display 250-1 and / or a second display 250-2 spaced apart from the first display 250-1. For example, the first display 250-1 and the second display 250-2 may be positioned corresponding to the user's left and right eyes, respectively.

[0049] refer to Figure 2bAt least one display 250 may provide a user with visual information transmitted from external light and other visual information distinct from the aforementioned visual information via lenses included in at least one display 250. The lenses may be formed based on at least one of Fresnel lenses, pancake lenses, or multi-channel lenses. For example, at least one display 250 may include a first surface 231 and a second surface 232 opposite to the first surface 231. A display area may be formed on the second surface 232 of at least one display 250. When a user wears the wearable device 103, ambient light may be transmitted to the user by incident on the first surface 231 and passing through the second surface 232. In another example, at least one display 250 may display an augmented reality image on a display area formed on the second surface 232, wherein a virtual reality image provided by at least one optical device 282 and 284 is combined with a real-world screen transmitted via external light.

[0050] According to an embodiment, at least one display 250 may include at least one waveguide 233 and 234, which transmit light transmitted from at least one optical device 282 and 284 to a user via diffraction. At least one waveguide 233 and 234 may be formed based on at least one of glass, plastic, or polymer. Nanopatterns may be formed on at least a portion of the exterior or interior of at least one waveguide 233 and 234. The nanopatterns may be formed based on a grating structure having a polygonal or curved shape. Light incident on one end of at least one waveguide 233 and 234 may be propagated through the nanopattern to the other end of at least one waveguide 233 and 234. At least one waveguide 233 and 234 may include at least one of at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) and a reflective element (e.g., a mirror). For example, at least one waveguide 233 and 234 may be disposed in a wearable device 103 to guide a screen displayed by at least one display 250 to the user's eyes. For example, the screen can transmit to the user's eye via total internal reflection (TIR) ​​generated in at least one waveguide 233 and 234.

[0051] Wearable device 103 can analyze objects included in real images collected by cameras 260-4, combine them with virtual objects corresponding to objects provided as objects in augmented reality, and display them on at least one display 250. The virtual objects may include at least one of text and images containing various information associated with objects included in the real images. Wearable device 103 can analyze objects based on multiple cameras, such as stereo cameras. For object analysis, wearable device 103 can perform spatial recognition (e.g., simultaneous localization and mapping (SLAM)) using multiple cameras and / or Time-of-Flight (ToF). A user wearing wearable device 103 can view the images displayed on at least one display 250.

[0052] According to an embodiment, the frame 200 may be configured with a physical structure in which the wearable device 103 may be worn on a user's body. According to an embodiment, the frame 200 may be configured such that when the user wears the wearable device 103, the first display 250-1 and the second display 250-2 may be positioned corresponding to the user's left and right eyes. The frame 200 may support at least one display 250. For example, the frame 200 may support the first display 250-1 and the second display 250-2 positioned corresponding to the user's left and right eyes.

[0053] refer to Figure 2a According to an embodiment, when a user wears the wearable device 103, the frame 200 may include a region 220 that at least partially contacts a part of the user's body. For example, the region 220 of the frame 200 that contacts a part of the user's body may include a region that contacts a part of the user's nose, a part of the user's ear, and a part of the wearable device 103 that contacts the side of the user's face. According to an embodiment, the frame 200 may include a nose pad 210 that contacts a part of the user's body. When the wearable device 103 is worn by the user, the nose pad 210 may contact a part of the user's nose. The frame 200 may include a first temple 204 and a second temple 205 that contact another part of the user's body, different from the part of the user's body.

[0054] For example, frame 200 may include a first bezel 201 surrounding at least a portion of a first display 250-1, a second bezel 202 surrounding at least a portion of a second display 250-2, a bridging member 203 disposed between the first bezel 201 and the second bezel 202, a first pad 211 disposed from one end of the bridging member 203 along a portion of the edge of the first bezel 201, a second pad 212 disposed from the other end of the bridging member 203 along a portion of the edge of the second bezel 202, a first temple 204 extending from the first bezel 201 and secured to a portion of the wearer's ear, and a second temple 205 extending from the second bezel 202 and secured to a portion of the ear opposite the aforementioned ear. The first pad 211 and the second pad 212 may contact a portion of the user's nose, and the first temple 204 and the second temple 205 may contact a portion of the user's face and a portion of the user's ear. Temples 204 and 205 can be... Figure 2b Hinges 206 and 207 are rotatably connected to the frame. A first temple 204 is rotatably connected relative to the first frame 201 via a first hinge unit 206 disposed between the first frame 201 and the first temple 204. A second temple 205 is rotatably connected relative to the second frame 202 via a second hinge unit 207 disposed between the second frame 202 and the second temple 205. According to an embodiment, the wearable device 103 can identify an external object (e.g., a user's fingertip) touching the frame 200 and / or a gesture performed by the external object by using touch sensors, grip sensors, and / or proximity sensors formed on at least a portion of the surface of the frame 200.

[0055] According to embodiments, the wearable device 103 may include hardware that performs various functions (e.g., based on...). Figure 5 The block diagram describes the hardware. For example, the hardware may include a battery module 270, an antenna module 275, at least one optical device 282 and 284, a speaker (e.g., speaker 255-1 and 255-2), a microphone (e.g., microphone 265-1, 265-2 and 265-3), a light-emitting module (not shown), and / or a printed circuit board (PCB) 290 (e.g., a printed circuit board). Various hardware components may be disposed within frame 200.

[0056] According to an embodiment, the microphones of the wearable device 103 (e.g., microphones 265-1, 265-2, and 265-3) can obtain sound signals by being disposed on at least a portion of the frame 200. Figure 2b The diagram shows a first microphone 265-1 mounted on the bridging member 203, a second microphone 265-2 mounted on the second frame 202, and a third microphone 265-3 mounted on the first frame 201. However, the number and arrangement of the microphones 265 are not limited to these specifications. Figure 2b In an embodiment where the number of microphones 265 included in the wearable device 103 is two or more, the wearable device 103 can identify the direction of the sound signal by using multiple microphones disposed on different parts of the frame 200.

[0057] According to an embodiment, at least one optical device 282 and 284 can project virtual objects onto at least one display 250 to provide various image information to a user. For example, at least one optical device 282 and 284 can be a projector. At least one optical device 282 and 284 can be configured to be adjacent to at least one display 250, or can be included in at least one display 250 as part of at least one display 250. According to an embodiment, the wearable device 103 can include a first optical device 282 corresponding to a first display 250-1 and a second optical device 284 corresponding to a second display 250-2. For example, at least one optical device 282 and 284 can include a first optical device 282 disposed at the periphery of the first display 250-1 and a second optical device 284 disposed at the periphery of the second display 250-2. The first optical device 282 can transmit light to a first waveguide 233 disposed on the first display 250-1, and the second optical device 284 can transmit light to a second waveguide 234 disposed on the second display 250-2.

[0058] In an embodiment, camera 260 may include a capturing camera 260-4, an eye-tracking camera (ET CAM) 260-1, and / or motion recognition cameras 260-2 and 260-3. The capturing camera 260-4, eye-tracking camera 260-1, and motion recognition cameras 260-2 and 260-3 may be positioned at different locations on frame 200 and may perform different functions. Eye-tracking camera 260-1 may output data indicating the position or gaze of a user wearing wearable device 103. For example, wearable device 103 may detect a gaze from an image including the user's pupils obtained through eye-tracking camera 260-1. Wearable device 103 may perform gaze interaction with at least one object by using the user's gaze obtained through eye-tracking camera 260-1. Wearable device 103 may present portions corresponding to the eyes of an avatar indicating the user in virtual space by using the user's gaze obtained through eye-tracking camera 260-1. Wearable device 103 may render an image (or screen) displayed on at least one display 250 based on the position of the user's eyes. For example, the visual quality (e.g., resolution, brightness, saturation, grayscale, and PPI) of a first region within an image that is gaze-related and a second region that is distinct from the first region can differ. For example, when wearable device 103 supports iris recognition, user authentication can be performed based on iris information obtained using eye-tracking camera 260-1. Figure 2b An example of the eye-tracking camera 260-1 being set toward the user's right eye is shown, but the embodiments are not limited thereto, and the eye-tracking camera 260-1 may be set toward the user's left eye alone or toward both eyes.

[0059] In an embodiment, the camera 260-4 can capture a real image or background to be matched with a virtual image to enable augmented reality or mixed reality content. The camera 260-4 can capture an image of a specific object present at the user's viewing location and can provide that image to at least one display 250. The at least one display 250 can display an image in which the virtual image provided by at least one optical device 282 and 284 is superimposed with information about a real image or background, including an image of a specific object obtained using the camera. The wearable device 103 can compensate for depth information (e.g., the distance between the wearable device 103 and external objects obtained by a depth sensor) by using the image obtained by the camera 260-4. The wearable device 103 can perform object recognition by using the image obtained by the camera 260-4. When displaying a screen representing a virtual space on at least one display 250, the wearable device 103 can perform a pass-through function to display an image obtained by the camera 260-4 that overlaps at least a portion of the screen. In one embodiment, the camera may be mounted on the bridging member 203, which is positioned between the first frame 201 and the second frame 202.

[0060] The eye-tracking camera 260-1 can achieve more realistic augmented reality by tracking the gaze of a user wearing the wearable device 103 and matching the user's gaze with visual information provided on at least one display 250. For example, when the user looks forward, the wearable device 103 can naturally display environmental information associated with the area in front of the user on at least one display 250 at the user's location. The eye-tracking camera 260-1 can be configured to capture images of the user's pupils to determine the user's gaze. For example, the eye-tracking camera 260-1 can receive gaze detection light reflected from the user's pupils and can track the user's gaze based on the position and movement of the received gaze detection light. In an embodiment, the eye-tracking camera 260-1 can be positioned corresponding to the user's left and right eyes. For example, the eye-tracking camera 260-1 can be positioned within a first bezel 201 and / or a second bezel 202 to face the direction in which the user wearing the wearable device 103 is located.

[0061] Motion recognition cameras 260-2 and 260-3 can provide specific events to a screen on at least one display 250 by recognizing movement of the whole or parts of the user's body (such as the user's torso, hands, or face). Motion recognition cameras 260-2 and 260-3 can obtain signals corresponding to user movements (e.g., gesture recognition) and can provide a display corresponding to the signals to at least one display 250. Wearable device 103 can recognize signals corresponding to operations and can perform preset functions based on such recognition. Motion recognition cameras 260-2 and 260-3 can be used to perform simultaneous localization and mapping (SLAM) and / or spatial recognition functions for six degrees of freedom (6DOF) poses using depth maps. Wearable device 103 can perform gesture recognition and / or object tracking functions using motion recognition cameras 260-2 and 260-3. In an embodiment, motion recognition cameras 260-2 and 260-3 can be mounted on a first frame 201 and / or a second frame 202.

[0062] The camera 260 included in the wearable device 103 is not limited to the eye-tracking camera 260-1 and motion recognition cameras 260-2 and 260-3 described above. For example, the wearable device 103 can identify external objects included in the field of view (FoV) using a camera set towards the user. The wearable device 103 can identify external objects based on sensors (e.g., depth sensors and / or time-of-flight (ToF) sensors) used to identify the distance between the wearable device 103 and the external object. The camera 260 set towards the FoV can support autofocus and / or optical image stabilization (OIS) functions. For example, to obtain an image including the face of a user wearing the wearable device 103, the wearable device 103 can include a camera 260 set towards the face (e.g., a face-tracking (FT) camera).

[0063] Although not shown, the wearable device 103 according to an embodiment may also include a light source (e.g., an LED) that emits light toward an object (e.g., the user's eyes, face, and / or an external object in FoV) captured by the camera 260. The light source may include an LED having an infrared wavelength. The light source may be disposed on at least one of the frame 200 and hinge units 206 and 207.

[0064] According to an embodiment, the battery module 270 can power the electronic components of the wearable device 103. In an embodiment, the battery module 270 can be disposed in the first temple 204 and / or the second temple 205. For example, there can be multiple battery modules 270. The multiple battery modules 270 can be disposed on each of the first temple 204 and the second temple 205 respectively. In an embodiment, the battery module 270 can be disposed at the end of the first temple 204 and / or the second temple 205.

[0065] Antenna module 275 can transmit signals or power to the outside of wearable device 103, or can receive signals or power from the outside. In an embodiment, antenna module 275 may be disposed in the first temple 204 and / or the second temple 205. For example, antenna module 275 may be disposed close to a surface of the first temple 204 and / or the second temple 205.

[0066] The speaker 255 can output sound signals to the outside of the wearable device 103. The sound output module may be referred to as a speaker. In an embodiment, the speaker 255 may be disposed in the first temple 204 and / or the second temple 205 so as to be positioned near the ear of the user wearing the wearable device 103. For example, the speaker 255 may include a second speaker 255-2 disposed in the first temple 204 near the user's left ear, and a first speaker 255-1 disposed in the second temple 205 near the user's right ear.

[0067] The light-emitting module (not shown) may include at least one light-emitting element. The light-emitting module may emit light of a color corresponding to a specific state, or may emit light through operations corresponding to a specific state, in order to visually provide the user with information about a specific state of the wearable device 103. For example, when the wearable device 103 needs charging, it may emit red light at a constant period. In embodiments, the light-emitting module may be disposed on a first frame 201 and / or a second frame 202.

[0068] refer to Figure 2b According to an embodiment, the wearable device 103 may include a printed circuit board (PCB) 290. The PCB 290 may be included in at least one of a first temple 204 or a second temple 205. The PCB 290 may include an interposer layer disposed between at least two sub-PCBs. On the PCB 290, one or more hardware components (e.g., those provided by reference) included in the wearable device 103 may be disposed. Figure 5 The hardware is shown in the box described. Wearable device 103 may include a flexible PCB (FPCB) for interconnecting the hardware.

[0069] According to an embodiment, the wearable device 103 may include at least one of a gyroscope sensor, a gravity sensor, and / or an accelerometer sensor for detecting the posture of the wearable device 103 and / or the posture of a body part (e.g., head) of the user wearing the wearable device 103. Each of the gravity sensor and the accelerometer sensor may measure gravitational acceleration and / or acceleration based on preset three-dimensional axes (e.g., x-axis, y-axis, and z-axis) perpendicular to each other. The gyroscope sensor may measure the angular velocity of each of the preset three-dimensional axes (e.g., x-axis, y-axis, and z-axis). At least one of the gravity sensor, the accelerometer sensor, and the gyroscope sensor may be referred to as an inertial measurement unit (IMU). According to an embodiment, the wearable device 103 may use the IMU to identify the movements and / or gestures of a user performing actions to start or stop a specific function of the wearable device 103.

[0070] Figure 3a and Figure 3b Examples of the appearance of wearable devices according to various embodiments are shown.

[0071] Figure 3a and Figure 3b The wearable device 103 can be Figure 1 An example of an electronic device 101. According to an embodiment, Figure 3a An example of the appearance of the first surface 310 of the housing of the wearable device 103 is shown, and Figure 3b An example of the appearance of the second surface 320, which is opposite to the first surface 310, is shown.

[0072] refer to Figure 3a According to an embodiment, the first surface 310 of the wearable device 103 may have a shape that is attachable to or conforms to a user's body part (e.g., the user's face). Although not shown, the wearable device 103 may also include a strap and / or one or more temples for securing to the user's body part (e.g., [missing information]). Figures 2a to 2b The first temple 204 and / or the second temple 205. A first display 250-1 for outputting an image to the left eye of the user and a second display 250-2 for outputting an image to the right eye of the user can be disposed on the first surface 310. The wearable device 103 may also include a rubber or silicone filler formed on the first surface 310 to prevent interference caused by light different from the light emitted from the first display 250-1 and the second display 250-2 (e.g., ambient light).

[0073] According to an embodiment, the wearable device 103 may include a camera 260-1 for capturing and / or tracking the two eyes of a user adjacent to each of the first display 250-1 and the second display 250-2. The camera 260-1 may be referenced... Figure 2b The wearable device 103 may include a gaze-tracking camera 260-1. According to an embodiment, the wearable device 103 may include cameras 260-5 and 260-6 for capturing and / or recognizing the user's face. Cameras 260-5 and 260-6 may be referred to as FT cameras. The wearable device 103 may control an avatar representing the user in virtual space based on the movement of the user's face recognized using cameras 260-5 and 260-6.

[0074] refer to Figure 3b Cameras (e.g., cameras 260-7, 260-8, 260-9, 260-10, 260-11, and 260-12) and / or sensors (e.g., depth sensor 330) used to obtain information associated with the external environment of the wearable device 103 may be positioned in conjunction with... Figure 3a The first surface 310 is positioned on the opposite second surface 320. For example, cameras 260-7, 260-8, 260-9, and 260-10 can be mounted on the second surface 320 to identify external objects. Cameras 260-7, 260-8, 260-9, and 260-10 can be referenced... Figure 2b The motion recognition cameras 260-2 and 260-3.

[0075] For example, by using cameras 260-11 and 260-12, wearable device 103 can acquire images and / or videos to be transmitted to each of the user's two eyes. Camera 260-11 can be disposed on the second surface 320 of wearable device 103 to acquire an image to be displayed on the second display 250-2 corresponding to the right eye. Camera 260-12 can be disposed on the second surface 320 of wearable device 103 to acquire an image to be displayed on the first display 250-1 corresponding to the left eye. Cameras 260-11 and 260-12 can be referenced... Figure 2b The camera used for shooting is a 260-4.

[0076] According to an embodiment, the wearable device 103 may include a depth sensor 330 disposed on a second surface 320 to identify the distance between the wearable device 103 and an external object. By using the depth sensor 330, the wearable device 103 can obtain spatial information (e.g., a depth map) about at least a portion of the FoV of the user wearing the wearable device 103. Although not shown, a microphone for obtaining sound output from an external object may be disposed on the second surface 320 of the wearable device 103. According to an embodiment, the number of microphones may be one or more.

[0077] Figure 4 Examples of methods for obtaining images for tracking body parts and for tracking external electronic devices are shown.

[0078] Figure 4 The wearable device 103 can represent Figure 1 Electronic device 101 and Figures 2a to 3b An example of wearable device 103. Figure 4 Example 400 illustrates a case where wearable device 103 provides an augmented reality (AR) environment, but embodiments of this disclosure are not limited thereto. For example, wearable device 103 may provide an extended reality (XR) environment that includes a virtual reality (VR) environment.

[0079] Figure 4 Example 400 shows a method in which a user 410 wearing a wearable device 103 obtains images for tracking an external electronic device 420 and for tracking body parts.

[0080] For example, tracking of body parts can be used to obtain information about the position or movement of a body part of the user 410. For example, tracking of body parts can be referred to as first tracking. For example, a body part can include at least one of the user's head or the user's hand. For example, tracking of the hand can be referred to as hand tracking. Similarly, for example, tracking of the head can be referred to as head tracking.

[0081] For example, tracking of the external electronic device 420 can be used to obtain information about the location, movement, and / or input of the external electronic device 420. For example, tracking of the external electronic device 420 can be referred to as second tracking. For example, the external electronic device 420 can be connected to the wearable device 103. For example, the wearable device 103 can use communication technology to establish a connection with the external electronic device 420. For example, the communication technology can include wired communication technology or wireless communication technology. For example, the wearable device 103 can obtain information from the external electronic device 420 based on the established connection. For example, the external electronic device 420 can be connected to the wearable device 103 and used to obtain input from the user 410. For example, the external electronic device 420 can be referred to as a controller, control device, or input device of the wearable device 103.

[0082] According to an embodiment, the wearable device 103 can perform second tracking by an image of a hand (e.g., right hand) grasping the external electronic device 420, and can perform first tracking by an image of a hand (e.g., left hand) not grasping the external electronic device 420.

[0083] Referring to Example 400, the wearable device 103 can be worn by a user 410. For example, the user 410 can wear the wearable device 103 on their head. Furthermore, referring to Example 400, the external electronic device 420 can be grasped by the user 410. For example, the user 410 can grasp the external electronic device 420 with their right hand. Figure 4 Example 400 is for illustrative purposes only, and the embodiments disclosed herein are not limited thereto.

[0084] Referring to Example 400, wearable device 103 can acquire a first image 450 to perform tracking on body parts (e.g., head) of user 410. For example, wearable device 103 can use its camera (e.g., ...) Figure 1 The first image 450 is obtained from the field of view (FoV) of the camera module 180. For example, the first image 450 for tracking a body part may include an external object 430 and an external electronic device 420. The wearable device 103 may perform body part tracking based on changes in the boundary of the external object 430 in the first image 450 and the external electronic device 420. For example, the wearable device 103 may perform body part tracking based on the position of the boundary of the external object 430 and the external electronic device 420 in the first image 450 at a first timing and in the first image 450 at a second timing different from the first timing. The boundary may include the position, outline, placement, etc. of the object. In the above examples, the boundary and / or external electronic device 420 constituting the appearance of the external object 430 have been described by way of example, but the embodiments of this disclosure are not limited thereto. For example, the wearable device 103 may perform body part tracking based on at least one point of the external object 430.

[0085] Referring to Example 400, wearable device 103 can acquire a second image 460 to perform tracking on external electronic device 420 of user 410. For example, wearable device 103 can use its camera (e.g., Figure 1The second image 450 is obtained from the field of view (FoV) of the camera module 180. For example, the second image 460 for tracking the external electronic device 420 may include light sources 421, 422, 423, and 424 of the external electronic device 420. For example, light sources 421, 422, 423, and 424 may include light-emitting devices of the external electronic device 420. The wearable device 103 may perform tracking of the external electronic device 420 based on changes in the boundaries of the light sources 421, 422, 423, and 424 in the second image 460. For example, the wearable device 103 may perform tracking of the external electronic device 420 based on the positions of the boundaries of the light sources 421, 422, 423, and 423 in the second image 460 at a first timing and the positions of the boundaries of the light sources 421, 422, 423, and 423 in the second image 460 at a second timing different from the first timing. In the above examples, the boundaries of light sources 421, 422, 423, and 423 are described as examples, but the embodiments of this disclosure are not limited thereto. For example, wearable device 103 may perform tracking of external electronic device 420 based on at least one point of light sources 421, 422, 423, and 423.

[0086] For example, the first image 450 and the second image 460 can have different attributes. For example, the first image 450 can have a first attribute, and the second image 460 can have a second attribute different from the first attribute. For example, attributes can include the brightness, color, and saturation of the image. For example, the first image 450 can have a first brightness as a first attribute. Conversely, the second image 460 can have a second brightness as a second attribute. The second brightness can be darker than the first brightness. For example, the attribute can be referred to as brightness, characteristic, parameter, or characteristic parameter.

[0087] Referring to Example 400, the wearable device 103 can use a first image 450 with a first brightness to perform tracking of body parts. The first image 450 may have a first brightness such that the appearance of the external object 430 and the external electronic device 420 are clearly visible. In this case, the boundaries (or points) of the external electronic device 420 and the external object 430 are clearly visible in the first image 450 with the first brightness. Conversely, the wearable device 103 can use a second image 460 with a second brightness to perform tracking of the external electronic device 420. The second image 460 may have a second brightness such that the shape of the external object 430 is invisible or blurred. In this case, the light sources 421, 422, 423, and 424 of the external electronic device 420 are clearly visible in the second image 460 with the second brightness. The light sources 421, 422, 423, and 424 can be displayed more clearly in the second image 460 with the second brightness than in the first image 450 with the first brightness.

[0088] In Example 400, an example is shown where the wearable device 103 acquires a first image 450 and a second image 460; however, embodiments of this disclosure are not limited thereto. For example, the wearable device 103 may acquire multiple first images 450 and multiple second images 460. In this case, the multiple first images 450 and multiple second images 460 may be acquired based on the frames per second (FPS) of the images (or videos) acquired by the camera of the wearable device 103. For example, when the FPS is set to 60 FPS in the camera, the wearable device 103 may acquire 60 frames per second. A frame may be referred to as an image or a frame image. Each frame may include at least one of the first image 450 and the second image 460. In one example, the wearable device 103 may acquire the first image 450 and the second image 460 alternately. Details related to this are described below. Figures 7a to 7c As described in the text.

[0089] Embodiments of this disclosure present electronic devices and methods for controlling the acquisition of images (or videos) used for first and second tracking in an XR environment. In an XR environment, wearable device 103 can utilize a camera (e.g., Figure 1 The camera module 180) or sensor (e.g., Figure 1 The wearable device 103 uses information obtained from the sensor module 176 to identify the position and orientation of the user 410 (or a part of the user 410's body). The technique used to identify the position and orientation of the user 410 can be referred to as first tracking or head tracking. Furthermore, the wearable device 103 can use the external electronics 420 and the user 410's hand to interact with virtual objects. When using the external electronics 420, the wearable device 103 can identify the position and orientation of the external electronics 420 and perform interactions with virtual objects. The technique used to identify the position and orientation of the external electronics 420 can be referred to as second tracking or controller tracking.

[0090] In both the first and second tracking, images (or videos) from a camera can be used. The image used for the first tracking (e.g., first image 450) may differ from other images used for the second tracking (e.g., second image 460). The image used for the first tracking can be acquired using a camera dedicated to the first tracking, and other images used for the second tracking can be acquired using a different camera dedicated to the second tracking of the external electronic device 420. Alternatively, images and other images can be acquired using the same camera. In the case of acquiring images and other images using multiple cameras, including one camera and another, the power consumption of the wearable device 103 may increase, and the cost of manufacturing the wearable device 103 may increase. Therefore, acquiring images and other images using a single camera can be highly efficient.

[0091] When acquiring images via a camera and other images, the attributes of the acquired image may differ from those of the other images. For example, the brightness of the acquired image may be brighter than that of other images. In the case of the acquired image, a bright image can be used because feature values ​​including the boundaries or points of objects in the acquired image are used. Conversely, in the case of the other images, a dark image can be used because feature values ​​including the light emitted by the external electronic device 420 (e.g., light sources 421, 422, 423, and 424) in the other images are used. A bright image may represent an image with a first attribute, and a dark image may represent an image with a second attribute. The wearable device 103 can acquire an image for first tracking and other images for second tracking via a camera to provide both first tracking and second tracking. Therefore, the wearable device 103 can alternately use a first shooting scheme or a first shooting scheme (hereinafter referred to as the first scheme) to acquire a bright image for first tracking, and use a second shooting scheme or a second shooting scheme (hereinafter referred to as the second scheme) to acquire other dark images for second tracking. That is, the wearable device 103 can alternately repeat acquiring images via the first scheme and acquiring other images via the second scheme. The first and second schemes can be used to obtain the shooting mode for the image.

[0092] For example, based on the exposure algorithm used to acquire the image, a first scheme for acquiring the image and a second scheme for acquiring other images can be identified. For example, the first scheme could represent the use of an automatic exposure algorithm. Conversely, the second scheme could represent the use of a fixed exposure algorithm. For example, in the case of using an automatic exposure algorithm, the exposure time can be longer than in the case of using a fixed exposure algorithm. In this case, the wearable device 103 can calculate the exposure time used in the automatic exposure algorithm. In other words, compared to acquiring other images through the second scheme, the wearable device 103 can acquire images by obtaining light through the camera for a longer time using the first scheme, and acquire images based on the first scheme.

[0093] Alternatively and / or alternatively, for example, the first and second schemes can be identified based on the gain value used to acquire the image. For example, the first scheme might represent the use of a relatively high gain value. Conversely, the second scheme might represent the use of a relatively low gain value. The wearable device 103 can acquire image information about the actual environment via a camera and perform processing on the image information based on the gain value. Based on this processing, the wearable device 103 can acquire an image or other images. In this case, the higher the gain value, the brighter the acquired image. In other words, the wearable device 103 can acquire an image using a high gain value via or by means of the first scheme. Conversely, the wearable device 103 can acquire other images using a low gain value via or by means of the second scheme.

[0094] Additionally and / or alternatively, for example, the first and second scenarios can be identified based on the brightness of the light source included in the wearable device 103. For example, the light source included in the wearable device 103 may include an infrared radiation (IR) illuminator. For example, the first scenario may represent a case where the brightness of the light source is relatively high. Conversely, the second scenario may represent a case where the brightness of the light source is relatively low. The light source can be used as light by the wearable device 103 to acquire an image. In other words, the wearable device 103 can acquire an image using the high brightness of the light source in the first scenario. Conversely, the wearable device 103 can acquire a different image using the low brightness of the light source in the second scenario.

[0095] Typically, wearable device 103 may alternately use a first scheme and a second scheme to acquire relatively bright images and other relatively dark images through or using a single camera. In this case, wearable device 103 may struggle to support situations where relatively more of the images are needed than the other images (or vice versa). Therefore, inefficiencies in the first and second tracking of wearable device 103 may occur. For example, if external electronics 420 is outside the FoV of the camera, second tracking cannot be performed, making it impossible to use the second and first schemes at the same frequency, which could lead to inefficiency. Furthermore, for example, in situations requiring accuracy and fast response times for external electronics 420, more additional images might be needed for second tracking. However, since wearable device 103 acquires images and other images alternately through a single camera, it cannot acquire a relatively large number of additional images, which could lead to inefficiency.

[0096] In the following, the electronic devices and methods according to embodiments of this disclosure may use a variable sequence (e.g., alternating acquisition of two bright images and one dark image) instead of a fixed sequence (e.g., alternating acquisition of one bright image and one dark image). For example, the variable sequence may be operated based on whether specified conditions are met. For example, specified conditions may be related to the state (e.g., quality) of tracking using the tracking module in wearable device 103, user input to wearable device 103 (e.g., gestures or input to physical buttons), specific software applications or settings in wearable device 103, and / or movement of external electronic device 420. In the case of using first and second tracking via a single camera, the electronic devices and methods according to embodiments of this disclosure can flexibly acquire the images required for each situation. The electronic devices and methods according to embodiments of this disclosure may use shooting modes corresponding to the performance of the tracking module or the user's usage scenario. Therefore, the electronic devices and methods according to embodiments of this disclosure can provide an efficient user experience by using appropriate shooting modes for each service.

[0097] Figure 5 An exemplary block diagram of a wearable device is shown.

[0098] Figure 5 The wearable device 103 can be Figure 1 Electronic device 101 Figures 2a to 3b Wearable device 103 or Figure 4 Example of wearable device 103. Figure 5 The external electronic device 420 may be Figure 4 Example of external electronic device 420. In Figure 5 The present disclosure describes a wearable device 103 that can be worn by a user as an example, but the embodiments thereof are not limited thereto. For example, wearable device 103 may represent Figure 1 Example of electronic device 101.

[0099] refer to Figure 5 The illustration shows an exemplary scenario where wearable device 103 and external electronic device 420 are connected to each other via a wired and / or wireless network. For example, a wired network may include networks such as the Internet, a local area network (LAN), a wide area network (WAN), or combinations thereof. A wireless network may include networks such as Long Term Evolution (LTE), 5G New Radio (NR), Wi-Fi, Zigbee, Near Field Communication (NFC), Bluetooth, Bluetooth Low Energy (BLE), or combinations thereof. Although wearable device 103 and external electronic device 420 are shown as directly connected, they can be indirectly connected via one or more routers and / or access points (APs).

[0100] refer to Figure 5 According to an embodiment, the wearable device 103 may include at least one of a processor 510, a camera 520, a sensor 530, a display 540, a communication circuit 550, and a memory 560. The camera 520, sensor 530, display 540, communication circuit 550, and memory 560 may be electronically and / or operatively connected to each other via a communication bus. In the following, an operational combination of hardware components may mean establishing a direct or indirect connection between hardware components via wired or wireless means, such that a second hardware component is controlled by a first hardware component. Although illustrated in different blocks, the embodiments are not limited thereto, and Figure 5 At least a portion of the hardware components shown (processor 510, memory 560, and communication circuitry 550) may be included in a single integrated circuit, such as a system-on-a-chip (SoC). The type and / or number of hardware components included in wearable device 103 are not limited to... Figure 5 Those shown. For example, wearable device 103 may only include... Figure 5A portion of the hardware components shown.

[0101] According to an embodiment, the processor 510 of the wearable device 103 may include hardware components for processing data based on one or more instructions. For example, the hardware components for processing data may include an arithmetic and logic unit (ALU), a floating-point unit (FPU), and a field-programmable gate array (FPGA). As an example, the hardware components for processing data may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing unit (DSP), and / or a neural processing unit (NPU). The number of processors 510 may be one or more. For example, the processor 510 may have a multi-core processor architecture, such as dual-core, quad-core, or hexa-core. Figure 5 The processor 510 may include Figure 1 The processor is 120.

[0102] According to an embodiment, the camera 520 of the wearable device 103 may include one or more optical sensors (e.g., charge-coupled device (CCD) sensors and complementary metal-oxide-semiconductor (CMOS) sensors) that generate electrical signals representing the color and / or brightness of light. The plurality of optical sensors included in the camera 520 may be arranged in a 2D array. The camera 520 can generate an image comprising a plurality of pixels arranged in 2D, corresponding to light arriving at the 2D array of optical sensors, by substantially simultaneously acquiring the electrical signals of each of the plurality of optical sensors. For example, photographic data captured using the camera 520 may represent an image obtained from the camera 520. For example, video data captured using the camera 520 may represent a sequence of multiple images obtained from the camera 520 at a specified frame rate. The wearable device 103 according to an embodiment may also include a flash lamp positioned in the direction in which the camera 520 receives light and outputs light in that direction. The number of cameras 520 included in the wearable device 103 may be one or more, as referenced above. Figure 2a and Figure 2b and / or Figure 3a and Figure 3b As stated above.

[0103] According to an embodiment, the sensor 530 of the wearable device 103 may include at least one sensor. For example, the sensor 530 may include... Figure 1 At least a portion of the sensor module 176. For example, sensor 530 may include an IMU (or an IMU sensor). For example, sensor 530 may include a gyroscope sensor, a gravity sensor, and / or an acceleration sensor.

[0104] According to an embodiment, the display 540 of the wearable device 103 can output visual information to a user. The number of displays 540 included in the wearable device 103 can be one or more. For example, the display 540 can output visual information to the user by being controlled by a processor 510 and / or a graphics processing unit (GPU) (not shown). The display 540 may include a flat panel display (FPD) and / or electronic paper. A flat panel display (FPD) may include a liquid crystal display (LCD), a plasma display panel (PDP), a digital mirror device (DMD), one or more light-emitting diodes (LEDs) and / or micro-LEDs. Light-emitting diodes (LEDs) may include organic LEDs (OLEDs). Figure 5 The display 540 may include Figure 1 The display module 160.

[0105] According to an embodiment, the communication circuit 550 of the wearable device 103 may include hardware for supporting the transmission and / or reception of electrical signals between the wearable device 103 and an external electronic device 520. For example, the communication circuit 550 may include at least one of a modulator and demodulator (MODEM), an antenna, and an optical / electronic (O / E) converter. The communication circuit 550 may support the transmission and / or reception of electrical signals based on various types of communication means such as Ethernet, Bluetooth (BT), Bluetooth Low Energy (BLE), ZigBee, Long Term Evolution (LTE), and 5G New Radio (NR). Figure 5 The communication circuit 550 may include Figure 1 The communication module 190 and / or antenna module 197.

[0106] According to an embodiment, the memory 560 of the wearable device 103 may include hardware components for storing data and / or instructions input to and / or output from the processor 510. For example, the memory 560 may include volatile memory such as random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM). For example, the volatile memory may include at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). For example, the non-volatile memory may include at least one of programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, optical disk, and embedded multimedia card (eMMC). Figure 5 The memory 560 may include Figure 1 The memory 130.

[0107] Although not shown, according to embodiments, wearable device 103 may include output means for outputting information in a form different from that shown in a visual format. For example, wearable device 103 may include a speaker for outputting acoustic signals. For example, wearable device 103 may include a motor for providing haptic feedback based on vibration.

[0108] refer to Figure 5 In the memory 560 of the wearable device 103, according to an embodiment, one or more instructions (or commands) representing calculations and / or operations to be performed on data by the processor 510 of the wearable device 103 may be stored. The set of one or more instructions may be referred to as a program, firmware, operating system, process, routine, subroutine, and / or application. Hereinafter, installing an application within an electronic device (e.g., wearable device 103) may mean that one or more instructions provided in the form of an application are stored in the memory 560, and that one or more applications are stored in an executable format (e.g., a file with an extension specified by the operating system of the wearable device 103). According to an embodiment, the wearable device 103 can perform operations by executing one or more instructions stored in the memory 560. Figure 6 and Figure 9 The operation.

[0109] refer to Figure 5 According to an embodiment, programs installed on the wearable device 103 can be categorized based on their target into any of the different layers including the application layer 570, the frame layer 580, and / or the hardware abstraction layer (HAL) 590. For example, within the hardware abstraction layer 590, programs (e.g., drivers) designed for the hardware of the wearable device 103 (e.g., camera 520, display 540, and / or communication circuitry 550) can be categorized. For example, within the frame layer 580, programs designed for at least one of the hardware abstraction layer 590 and / or the application layer 570 (e.g., image acquisition module 581, tracking module 583, and / or mode changing module 585) can be categorized. Programs categorized as frame layer 580 can provide an executable application programming interface (API) based on another program.

[0110] refer to Figure 5 According to embodiments, within application layer 570, programs designed for user control of wearable device 103 can be categorized. For example, programs categorized as application layer 570 may include at least one application providing an XR environment. However, embodiments of this disclosure are not limited thereto. For example, programs categorized as application layer 570 may call APIs and cause the execution of functions supported by programs categorized as framework layer 580.

[0111] refer to Figure 5According to an embodiment, the wearable device 103 can acquire images via the camera 520 based on the execution of the image acquisition module 581 within the frame layer 580. For example, the wearable device 103 can acquire images via the image acquisition module 581 based on a set mode. For example, this mode can include a first mode, a second mode, and a third mode. For example, the first mode can represent a shooting mode that alternately acquires at least one first image for first tracking and at least one second image for second tracking at the same FPS (or number of images). In other words, the first mode can represent a shooting mode that alternately uses a first scheme and a second scheme at the same frequency. The first mode can be referred to as the base mode. For example, the second mode can represent a shooting mode that alternately acquires at least one first image for first tracking and at least one second image for second tracking at different FPS (or number of images). In other words, the second mode can represent a shooting mode that alternately uses a first scheme and a second scheme at different frequencies. For example, the third mode can represent a shooting mode that acquires one of at least one first image for first tracking and at least one second image for second tracking. In other words, the third mode can represent a shooting mode that uses one of the first scheme and the second scheme. The wearable device 103 can determine whether the image information acquired from the camera 520 via the image acquisition module 581 is data for a first tracking or data for a second tracking. For example, the image information may indicate visual information about the external environment and attributes of the image information. For example, attributes may include a bright image (or an image obtained through automatic exposure) or a dark image (or an image obtained through fixed exposure).

[0112] refer to Figure 5 According to embodiments, wearable device 103 can perform at least one tracking based on the execution of tracking module 583 in frame layer 580. For example, at least one tracking may include head tracking, hand tracking, and / or controller tracking. However, embodiments of this disclosure are not limited thereto. For example, wearable device 103 can identify the quality of each of the at least one tracking based on the execution of tracking module 583. For example, the quality can be identified based on feature values ​​in images obtained for each of the at least one tracking. For example, feature values ​​may include boundaries or points of external objects (or external electronics). For example, feature values ​​may include boundary lines or points of light sources included in external electronics. For example, the quality can be compared with at least one set of reference values ​​for each tracking (e.g., head tracking, hand tracking, and / or controller tracking). A mode can be selected based on the comparison of quality with at least one reference value.

[0113] refer to Figure 5According to an embodiment, the wearable device 103 can perform a mode change based on the execution of the mode change module 585 in the frame layer 580. For example, the wearable device 103 can perform a mode change based on whether specified conditions are met. For example, the specified conditions may be related to the quality of each track of the tracking module 583, user input to the wearable device 103 (e.g., gesture input or physical button), specific software applications or settings within the wearable device 103, or movement of the external electronic device 420.

[0114] refer to Figure 5 According to an embodiment, external electronic device 420 can be connected to wearable device 103. External electronic device 420 can be used to provide input to the user's wearable device 103 after being connected to it. For example, external electronic device 420 may include a button. For example, a button can be used to change the mode of wearable device 103.

[0115] Figure 6 An example of the operational flow of a method for obtaining images based on the patterns of a wearable device is shown.

[0116] Figure 6 At least a part of the method can be derived from Figure 5 The method is executed by the wearable device 103. For example, at least a portion of the method may be controlled by the processor 510 of the wearable device 103. In the following embodiments, each operation may be executed sequentially, but not necessarily sequentially. For example, the order of each operation may be changed, and at least two operations may be executed in parallel.

[0117] In operation 600, according to an embodiment, the wearable device 103 can perform startup. For example, the wearable device 103 can perform startup based on input. For example, the input may include input from a user wearing the wearable device 103 or input from a physical button used to start the wearable device 103.

[0118] In operation 605, according to an embodiment, the wearable device 103 may execute a basic mode. For example, the wearable device 103 may execute a basic mode upon startup. For example, the basic mode may be referred to as a first mode. For example, the basic mode may refer to a shooting mode that alternately acquires an image for tracking (or first tracking) a part of the user's body and an image for tracking (or second tracking) an external electronic device 420 connected to the wearable device 103.

[0119] In operation 610, according to an embodiment, wearable device 103 can acquire images in a basic mode. For example, wearable device 103 can acquire images set to a specified FPS. For example, the specified FPS can indicate the number of images (or frames) per hour set for camera 520 of wearable device 103. For example, the specified FPS can be a value set for the performance of camera 520 or wearable device 103.

[0120] For example, an image based on a specified FPS may include at least one first image for first tracking and at least one second image for second tracking. For example, the number of at least one first image in the images acquired within the base mode may correspond to the number of at least one second image in the images. In other words, the FPS of at least one first image may be the same as the FPS of at least one second image. Referring to the above, in the base mode, the wearable device 103 may sequentially acquire images for first tracking, images for second tracking, images for first tracking, images for second tracking, and so on, via or using the camera 520.

[0121] For example, at least one first image may have a first attribute, and at least one second image may have a second attribute different from the first attribute. For example, each of the first and second attributes may include a first brightness and a second brightness. For example, the first brightness is brighter than the second brightness. For example, at least one first image having a first brightness may be used to obtain a first feature value for first tracking. For example, the first feature value may include the boundary or point of an external object for first tracking. Furthermore, for example, at least one second image having a second brightness may be used to obtain a second feature value for second tracking. For example, the second feature value may include the light source of the external electronic device 420 for second tracking.

[0122] In operation 615, according to an embodiment, the wearable device 103 can identify whether specified conditions are met. For example, the wearable device 103 can identify whether specified conditions for changing the mode (or shooting mode) of the wearable device 103 are met. For example, the specified conditions may be related to the quality of each of the tracking (e.g., first tracking and second tracking), user input to the wearable device 103 (e.g., gestures or physical button input), settings in a particular software application or wearable device 103, or movement of the external electronic device 420. The shooting mode may include a first mode (or base mode), a second mode, and a third mode. For example, the first mode may represent a shooting mode in which at least one first image for the first tracking and at least one second image for the second tracking are alternately acquired at the same FPS (or number of images). The first mode may be referred to as the base mode. For example, the second mode may represent a shooting mode in which at least one first image for the first tracking and at least one second image for the second tracking are alternately acquired at different FPS (or number of images). For example, the third mode may represent a shooting mode in which at least one of the first tracking and at least one second tracking image is acquired.

[0123] According to an embodiment, the wearable device 103 can identify the quality of each track based on feature values. This quality can be referred to as attitude quality. For example, the wearable device 103 can identify a first quality of a first track based on a first feature value. For example, the wearable device 103 can identify a second quality of a second track based on a second feature value.

[0124] According to an embodiment, the wearable device 103 can change its mode (or shooting mode) based on a comparison between the quality of each tracked item and a reference value for each tracked item. For example, the wearable device 103 can compare a first quality with a first set of reference values ​​used for a first tracked item. Furthermore, for example, the wearable device 103 can compare a second quality with a second set of reference values ​​used for a second tracked item. The first set of reference values ​​may include at least one reference value. The second set of reference values ​​may include at least one reference value. For example, it is assumed that the first set of reference values ​​includes a first reference value, and the second set of reference values ​​includes a second reference value. For example, the wearable device 103 can execute a first mode when the first quality is higher than or equal to the first reference value and the second quality is higher than or equal to the second reference value. For example, the wearable device 103 can execute a first mode when the first quality is lower than the first reference value and the second quality is lower than the second reference value. For example, the wearable device 103 can execute a second mode when the first quality is higher than or equal to the first reference value and the second quality is lower than the second reference value, or when the first quality is lower than the first reference value and the second quality is higher than or equal to the second reference value. For example, in a second mode executed in response to a first quality being higher than or equal to a first reference value and a second quality being lower than a second reference value, the number of images (or FPS) used for the second tracking may be greater than (or higher than) the number of images (or FPS) used for the first tracking. Additionally and / or alternatively, for example, in a second mode executed in response to a first quality being lower than a first reference value and a second quality being higher than or equal to a second reference value, the number of images (or FPS) used for the first tracking may be greater than (or higher than) the number of images (or FPS) used for the second tracking. In the above examples, the case where each of the first and second reference value sets includes one reference value is described, but embodiments of this disclosure are not limited thereto. For example, each of the first and second reference value sets may include multiple reference values. Based on multiple reference values, the number of images (or FPS) obtained by camera 520 for a particular tracking can be adjusted. For example, it is assumed that the first reference value set includes multiple reference values ​​(e.g., a first reference value and a second reference value). In the case where the first quality is lower than the first reference value, the wearable device 103 may obtain images for the first tracking based on the first FPS. When the first quality is higher than or equal to the first reference value and lower than the second reference value, the wearable device 103 can obtain an image for the first tracking based on a second FPS that is less than the first FPS. In this case, each of the first FPS and the second FPS can be a value higher than the FPS of the image used for the second tracking (i.e., when the number of images used for the first tracking is greater than the number of images used for the second tracking).

[0125] According to embodiments, the wearable device 103 can change modes based on input. For example, the wearable device 103 can change modes based on gestures set in the wearable device 103. For example, the wearable device 103 may include a first gesture for a first mode, a second gesture for a second mode, and a third gesture for a third mode. Additionally and / or alternatively, for example, the wearable device 103 can change modes based on input from a physical button included in the wearable device 103. Specific details related to this are described below. Figure 8b As described in the text.

[0126] According to an embodiment, the wearable device 103 can change its mode based on a software application or settings running in the wearable device 103. For example, the wearable device 103 can identify a mode set in the software application in response to execution of the software application. The wearable device 103 can change its shooting mode to a setting mode. Furthermore, for example, the wearable device 103 can identify a setting mode based on settings within the wearable device 103. For example, settings in the wearable device 103 can be changed based on at least a portion of user input. The wearable device 103 can change its shooting mode to a setting mode. For example, when executing a software application (e.g., a game) that requires a relatively large amount of movement from an external electronic device 420 (or controller) or changing settings in the wearable device 103, the wearable device 103 can change its shooting mode from a first mode to a second mode. In this case, the second mode may represent a shooting mode where the number of images used for the first tracking is less than the number of images used for the second tracking. This can be to improve the accuracy or response speed of tracking the movement of the external electronic device 420. Specific details related to this are described below. Figure 8a As described in the text.

[0127] According to an embodiment, the wearable device 103 can change its shooting mode based on movement of an external electronic device 420 (or controller) connected to the wearable device 103. For example, the wearable device 103 can change its shooting mode from a first mode to a second mode based on recognizing that the movement is below a reference frequency. The second mode executed in response to the movement being below the reference frequency can represent a shooting mode in which the number of images used for first tracking is greater than the number of images used for second tracking. This is because when the movement is below the reference frequency, second tracking of the external electronic device 420 is not required. Specific details related to this are described below. Figure 8a As described in the text.

[0128] According to an embodiment, the wearable device 103 can identify the movement of the external electronic device 420 based on the sensor 530. For example, the wearable device 103 can obtain information about the movement of the external electronic device 420 through the sensor 530 (e.g., an IMU sensor). The wearable device 103 can change the shooting mode based on the movement identified according to the information.

[0129] Referring to the foregoing, wearable device 103 can identify whether specified conditions for changing the shooting mode are met. For example, wearable device 103 can periodically (or non-periodically) identify whether the timing of acquiring an image in operation 610 meets the specified conditions. Additionally and / or alternatively, for example, if the event occurs after image acquisition, wearable device 103 can identify whether the specified conditions are met. For example, the event may include the execution of a specific software application. Additionally and / or alternatively, for example, when a signal requesting a mode change is received from external electronics 420 (or a controller) after image acquisition, wearable device 103 can identify whether the specified conditions are met.

[0130] In operation 615, if the specified conditions are met, the wearable device 103 can perform operation 620. Conversely, in operation 615, if the specified conditions are not met, the wearable device 103 can perform operation 610. For example, the wearable device 103 can acquire an image in the current shooting mode (e.g., basic mode) and perform tracking based on the image. Tracking may include first tracking and second tracking. In other words, by returning to operation 610, the wearable device 103 can acquire an image without changing the basic mode (or while maintaining the basic mode).

[0131] In operation 620, the wearable device 103 according to the embodiment can change modes. For example, the wearable device 103 can identify the mode to be changed based on specified conditions. The wearable device 103 can change the shooting mode from a basic mode (or a first mode) to an identification mode.

[0132] In operation 625, the wearable device 103 according to the embodiment can acquire additional images in the modified mode. For example, the wearable device 103 can acquire additional images based on the FPS (or number of images) set within the modified mode.

[0133] For example, if the modified mode is the second mode, the other images may include at least one third image for the first tracking and at least one fourth image for the second tracking. For example, the number (or FPS) of the at least one third image may differ from the number (or FPS) of the at least one fourth image.

[0134] Furthermore, for example, when the modified mode is a third mode, the other images may include one of at least one third image for the first tracking and at least one fourth image for the second tracking. For example, when the first tracking is unnecessary (e.g., when the user's head is fixed), the other images may include at least one fourth image for the second tracking, among the at least one third image for the first tracking and at least one fourth image for the second tracking. Conversely, when the second tracking is unnecessary (e.g., when the user is not holding the external electronic device 420), the other images may include at least one third image for the first tracking and at least one fourth image for the second tracking, among the at least one third image for the first tracking and at least one third image for the second tracking.

[0135] Figures 7a to 7c An example of a method for obtaining images based on the patterns of a wearable device is shown.

[0136] Figures 7a to 7c The mode can represent the shooting mode of the wearable device 103. For example, the mode can include a first mode, a second mode, and a third mode. Figure 7a Example 700 of a method for obtaining an image based on a first mode is shown. Figure 7b Examples 720 and 740 show methods for obtaining images based on a second mode. Figure 7c Examples 760 and 780 show methods for obtaining images based on a third mode.

[0137] refer to Figure 7a In Example 700, the wearable device 103 can acquire images in a first mode. For example, the wearable device 103 can acquire images via camera 520 in the first mode. For example, the images acquired in the first mode may include a first image 701 for first tracking and a second image 703 for second tracking. For example, the first image 701 may be a frame for first tracking (e.g., head tracking or hand tracking). For example, the second image 703 may be a frame for second tracking (e.g., controller tracking). For example, the first image 701 for first tracking may have a first attribute. For example, the first attribute may include a first brightness. For example, the second image 703 for second tracking may have a second attribute different from the first attribute. For example, the second attribute may include a second brightness that is darker than the first brightness. Figure 7a Example 700 describes an example where the wearable device 103 alternately acquires images for first tracking and images for second tracking, but embodiments of this disclosure are not limited thereto. For example, the wearable device 103 may alternately acquire two images for first tracking and two images for second tracking. Even in this case, the FPS of the first image 701 may be the same as the FPS of the second image 703. Figure 7aFor ease of description, examples of first images 701 and second images 703 having the same length of time are shown, but embodiments of this disclosure are not limited thereto. For example, since each first image 701 having a first attribute uses a first scheme that requires a relatively longer exposure time compared to each second image 703 having a second attribute, the time length (or duration) required to obtain each first image 701 may be configured to be longer than the time length (or duration) required to obtain each second image 703.

[0138] refer to Figure 7b In Example 720, the wearable device 103 can acquire images in a second mode. For example, the wearable device 103 can acquire images via camera 520 in the second mode. For example, the images acquired in the second mode may include a first image 721 for first tracking and a second image 723 for second tracking. For example, the first image 721 may be a frame for first tracking (e.g., head tracking or hand tracking). For example, the second image 723 may be a frame for second tracking (e.g., controller tracking). For example, the first image 721 for first tracking may have a first attribute. For example, the first attribute may include a first brightness. For example, the second image 723 for second tracking may have a second attribute different from the first attribute. For example, the second attribute may include a second brightness that is darker than the first brightness. In the second mode of Example 720, the FPS of the first image 721 may be higher than the FPS of the second image 723. In other words, in the second mode of Example 720, the wearable device 103 can acquire images including a greater number of first images 721 than the number of second images 723. Figure 7b In Example 720, the case where the FPS of the first image 721 is twice that of the FPS of the second image 723 is shown, but embodiments of this disclosure are not limited thereto. For example, the FPS of the first image 721 may be three or four times that of the FPS of the second image 723. Furthermore, for example, the FPS of the first image 721 may be 1.5 times that of the FPS of the second image 723. In this case, the wearable device 103 may alternately acquire three first images 721 and two second images 723. Figure 7b For ease of description, examples of first images 721 and second images 723 having the same length of time are shown, but embodiments of this disclosure are not limited thereto. For example, since each first image 721 having a first attribute uses a first scheme that requires a relatively longer exposure time compared to each second image 723 having a second attribute, the time length required to obtain each first image 721 (e.g., the length of time) can be configured to be longer than the time length required to obtain each second image 723 (e.g., the length of time).

[0139] refer to Figure 7b In Example 740, the wearable device 103 can acquire images in a second mode. For example, the wearable device 103 can acquire images via camera 520 in the second mode. For example, the images acquired in the second mode may include a first image 741 for first tracking and a second image 743 for second tracking. For example, the first image 741 may be a frame for first tracking (e.g., head tracking or hand tracking). For example, the second image 743 may be a frame for second tracking (e.g., controller tracking). For example, the first image 741 for first tracking may have a first attribute. For example, the first attribute may include a first brightness. For example, the second image 743 for second tracking may have a second attribute different from the first attribute. For example, the second attribute may include a second brightness that is darker than the first brightness. In the second mode of Example 740, the FPS of the first image 741 may be lower than the FPS of the second image 743. In other words, in the second mode of Example 740, the wearable device 103 can acquire an image that includes fewer elements of the first image 741 than the second image 743. Figure 7b In Example 740, the case where the FPS of the second image 743 is twice that of the first image 741 is shown, but embodiments of this disclosure are not limited thereto. For example, the FPS of the second image 743 could be three or four times that of the first image 741. Furthermore, for example, the FPS of the second image 743 could be 1.5 times that of the first image 741. In this case, the wearable device 103 could alternately acquire three second images 743 and two first images 741. Figure 7b For ease of description, examples of first images 741 and second images 743 having the same length of time are shown, but embodiments of this disclosure are not limited thereto. For example, since each first image 741 having a first attribute uses a first scheme, the first pattern requires a relatively longer exposure time compared to each second image 743 having a second attribute. Therefore, the time length required to obtain each first image 741 (e.g., the length of time) can be configured to be longer than the time length required to obtain each second image 743 (e.g., the length of time).

[0140] Referring to 7c, the wearable device 103 can acquire images in a third mode. For example, the wearable device 103 can acquire images via camera 520 in the third mode. The image acquired in the third mode of example 760 may include a first image 761 for first tracking. The image acquired in the third mode of example 760 may not include a second image 763 for second tracking. For example, the first image 761 may be a frame for first tracking (e.g., head tracking or hand tracking). For example, the second image 763 may be a frame for second tracking (e.g., controller tracking), although a frame for the second image 763 is not shown in example 760. For example, the first image 761 for first tracking may have a first attribute. For example, the first attribute may include a first brightness. Conversely, the image acquired in the third mode of example 780 may include a second image 783 for second tracking. The image acquired in the third mode of example 780 may not include the first image 781 for first tracking. For example, the first image 781 may be a frame for first tracking (e.g., head tracking or hand tracking), although a frame for the first image 781 is not shown in example 780. For example, the second image 783 may be a frame used for second tracking (e.g., controller tracking). For example, the second image 783 used for second tracking may have a second attribute different from the first attribute. For example, the second attribute may include a second brightness that is darker than the first brightness. Figure 7c For ease of description, examples of a first image 761 or 781 and a second image 763 or 783 having the same length of time are shown, but embodiments of this disclosure are not limited thereto. For example, since each of the first images 761 or 781 having a first attribute uses a first scheme that requires a relatively longer exposure time compared to each of the second images 763 or 783 having a second attribute, the time length required to obtain each of the first images 761 or 781 (e.g., the length of time) may be configured to be longer than the time length required to obtain each of the second images 763 or 783 (e.g., the length of time).

[0141] Referring to the foregoing, wearable device 103 can execute or change modes based on specified conditions. Examples of mode execution or change are as follows. However, the use of variable-sequence electronic devices and methods according to embodiments of this disclosure is not limited to the following examples.

[0142] For example, wearable device 103 may execute a first mode in response to activation of wearable device 103. Additionally and / or alternatively, for example, wearable device 103 may execute the first mode when it is configured to use a shooting sequence fixed within wearable device 103. Additionally and / or alternatively, for example, wearable device 103 may execute the first mode when a first quality of a first track is higher than or equal to a first reference value and a second quality of a second track is higher than or equal to a second reference value. Additionally and / or alternatively, for example, wearable device 103 may execute the first mode when a first quality is lower than the first reference value and a second quality is lower than the second reference value.

[0143] For example, it can be based on a first image used for the first tracking (e.g., Figure 7a First image 701 Figure 7b The first images 721 and 741 and Figure 7c The first quality is identified by obtaining a first feature value from each of the first images 761 and 781 for the first tracking. For example, the first feature value may include the boundary or point of an external object included in each of the first images for the first tracking. For example, the wearable device 103 may identify the first quality based on the change in the position of the boundary line (or point) between the first images. For example, the first quality may be identified based on the second image used for the second tracking (e.g., Figure 7a The second image 703 Figure 7b The second images 723 and 743 and Figure 7c The second quality is identified by obtaining a second feature value from each of the second images 763 and 783. For example, the second feature value may include the light source of the external electronic device 420 included in each of the second images used for the second tracking. For example, the wearable device 103 may identify the second quality based on the change in the position of the light source between the second images. Referring to the above, the first quality of the first tracking can be improved when there are many external objects in the FoV of the camera 520 of the wearable device 103 that can identify the boundaries. In contrast, the first quality may be reduced when the environment within the FoV (e.g., an area without any pattern, such as a white wall) is difficult to identify the boundaries (e.g., an area without any pattern, such as a white wall). Alternatively and / or additionally, the second quality of the second tracking may be reduced when the external electronic device 420 is difficult to identify because it is located outside the FoV.

[0144] For example, suppose that in an environment where boundaries are difficult to discern, the external electronics 420 is located within the FoV of the camera 520 of the wearable device 103. When boundaries are difficult to discern, the wearable device 103 can identify a low first quality, and when the external electronics 420 is within the FoV, the wearable device 103 can identify a high second quality. If the second quality is higher than or equal to a reference value for the second tracking (e.g., a second reference value), the wearable device 103 can execute (or change to) a second mode to improve the first quality. In this case, in the executed second mode, the FPS of the image used for the first tracking can be higher than the FPS of the image used for the second tracking.

[0145] For example, if the external electronic device 420 is not used or does not move within a specified time, the wearable device 103 may reduce the number of images acquired for the second tracking and increase the number of images acquired for the first tracking. The wearable device 103 may execute a second mode. In this second mode, the FPS of the images used for the first tracking may be higher than the FPS of the images used for the second tracking. Subsequently, the wearable device 103 may execute the first mode from the second mode (e.g., after the second mode) because the wearable device 103 detects movement of the external electronic device 420 based on sensors or input to the wearable device 103. In the above examples, an example of changing from the second mode to the first mode is described, but the embodiments of this disclosure are not limited thereto. For example, the wearable device 103 may change from the second mode of example 740 to the second mode of example 720.

[0146] For example, when the external electronic device 420 is located outside the FoV, the wearable device 103 can execute a second mode. In this second mode, as in Example 740, the FPS of the image used for the second tracking can be higher than the FPS of the image used for the first tracking. This could be to identify the external electronic device 420 located outside the FoV and perform a second tracking of the external electronic device 420.

[0147] Additionally and / or alternatively, for example, wearable device 103 may execute a second mode to increase the accuracy or responsiveness of the second tracking of external electronic device 420 required for the operation or service of a particular software application. In this case, in the executed second mode, as in example 740, the FPS of the image used for the second tracking may be higher than the FPS of the image used for the first tracking.

[0148] For example, wearable device 103 may execute a third mode based on settings of a specific software application or service. For example, wearable device 103 may execute a third mode based on settings for activating input using only external electronics 420. In this case, the executed third mode may represent a shooting mode for acquiring images for second tracking, as in example 780. Additionally and / or alternatively, for example, based on settings being disabled, wearable device 103 may change from a third mode to a first mode, which is a base mode. For example, wearable device 103 may execute a third mode based on settings indicating that external electronics 420 is not used. In this case, the executed third mode may represent a shooting mode for acquiring images for first tracking, as in example 760.

[0149] Figure 8a An example of a method for changing the mode of a wearable device based on the movement of an external electronic device is shown.

[0150] Figure 8a The external electronic device 420 can represent Figure 4 External electronic devices 420 and Figure 5 External electronic device 420. Figure 8a The wearable device 103 can represent Figure 1 Electronic device 101 Figure 2a , Figure 2b , Figure 3a and Figure 3b Wearable device 103 Figure 4 Wearable device 103 and Figure 5 The wearable device 103. For example, the wearable device 103 may be connected to an external electronic device 420. The external electronic device 420 may be used to provide input to the wearable device 103. For example, the input may be initiated by a user 410.

[0151] Figure 8a Examples 800 and 805 are shown that change the mode based on the movement of an external electronic device 420. This mode could represent a shooting mode, in which the wearable device 103 acquires images for tracking via camera 520.

[0152] Referring to Example 800, user 410 can wear wearable device 103 and grasp external electronic device 420. For example, user 410 can sit in a chair while wearing wearable device 103. At this time, user 410's body part 810 can be supported by external object 820. For example, body part 810 can include the elbow of the right hand grasping external electronic device 420. For example, external object 820 can include the armrest of the chair. When body part 810 is supported by external object 820, wearable device 103 can anticipate less movement of external electronic device 420. For example, based on recognizing that user 410's body part 810 is supported by external object 820, wearable device 103 can anticipate less movement. Therefore, wearable device 103 can change modes. For example, wearable device 103 can execute a second mode based on recognizing that movement is below a reference frequency while executing a first mode as a base mode. At this time, in the executed second mode, as in Example 720, the FPS of the image used for the first tracking can be higher than the FPS of the image used for the second tracking.

[0153] Referring to Example 805, user 410 can wear wearable device 103 and grasp external electronic device 420. For example, user 410 can use a virtual environment 830 provided by wearable device 103. For example, virtual environment 830 can be used for games that require large movements of external electronic device 420 or a high level of accuracy or responsiveness in second tracking of movements. For example, wearable device 103 can change modes when executing a software application that provides virtual environment 830. For example, wearable device 103 can execute a second mode based on the recognition of the execution of the software application providing virtual environment 830 while executing a first mode as a base mode. In this case, in the executed second mode, as in Example 740, the FPS of the image used for second tracking can be higher than the FPS of the image used for first tracking.

[0154] Figure 8b An example of a method for changing the mode of a wearable device based on user input is shown.

[0155] Figure 8b The external electronic device 420 can represent Figure 4 External electronic devices 420 and Figure 5 External electronic device 420. Figure 8b The wearable device 103 can represent Figure 1 Electronic device 101 Figure 2a , Figure 2b , Figure 3a and Figure 3b Wearable device 103 Figure 4 Wearable device 103 and Figure 5 The wearable device 103. For example, the wearable device 103 may be connected to an external electronic device 420. The external electronic device 420 may be used to provide input to the wearable device 103. For example, the input may be initiated or entered by a user 410.

[0156] Figure 8b Examples 850 and 855 are shown based on the input changing mode of the wearable device 103. The mode can represent a shooting mode in which the wearable device 103 obtains images for tracking via the camera 520.

[0157] Referring to Example 850, wearable device 103 can receive input 860 from user 410. For example, input 860 may include a specified gesture. For example, the specified gesture may be associated with a specific pattern. In Example 850, an "X"-shaped input 860 may be associated with a first pattern. In Example 850, when wearable device 103 performs a different pattern than the first pattern (e.g., a second or third pattern), the pattern may change to the first pattern in response to receiving input 860. However, embodiments of this disclosure are not limited thereto. Furthermore, in Example 850, the example demonstrates input 860 by grasping external electronics 420 with the right hand, but embodiments of this disclosure are not limited thereto. For example, wearable device 103 may recognize left-hand-based input 860 based on tracking (i.e., hand tracking) of the left hand and change the pattern based on input 860.

[0158] Referring to Example 855, wearable device 103 may include a physical button 870. The physical button 870 can be used to change the mode of wearable device 103. Wearable device 103 may change its mode based on user 410's input to the physical button 870. However, embodiments of this disclosure are not limited thereto. For example, although not shown, the physical button 870 may be included in an external electronic device 420. Wearable device 103 can recognize user 410's input to the physical button 870 included in the external electronic device 420. For example, external electronic device 420 may provide (or send) information about the input to wearable device 103 in response to receiving user 410's input to the physical button 870. Wearable device 103 may change its mode based on this information.

[0159] Figure 9 An example of the operational flow of a method for obtaining images, including images for tracking body parts and images for tracking external electronic devices, based on the mode of a wearable device is shown.

[0160] Figure 9 At least a part of the method can be derived from Figure 5The method is executed by the wearable device 103. For example, at least a portion of the method may be controlled by the processor 510 of the wearable device 103. In the following embodiments, each operation may be executed sequentially, but not necessarily sequentially. For example, the order of each operation may be changed, and at least two operations may be executed in parallel.

[0161] In operation 900, according to an embodiment, the wearable device 103 can acquire an image including at least one first image having a first attribute and at least one second image having a second attribute in a first mode. For example, the wearable device 103 can acquire at least one first image and at least one second image via camera 520 in the first mode. For example, the first mode can represent a shooting mode that alternately acquires an image having a first attribute and an image having a second attribute. The first mode can be referred to as the basic mode of the wearable device 103.

[0162] For example, at least one first image having a first attribute can be used to track (e.g., first tracking) a body part (e.g., head or hand) of a user wearing the wearable device 103. Furthermore, at least one second image having a second attribute can be used to track (e.g., second tracking) an external electronic device 420 connected to the wearable device 103. For example, the number of at least one first image in the images obtained in the base mode can correspond to the number of at least one second image in the images. In other words, the FPS of at least one first image can be the same as the FPS of at least one second image.

[0163] In operation 910, according to an embodiment, the wearable device 103 can obtain a first feature value from at least one first image and a second feature value from at least one second image. For example, the wearable device 103 can obtain either the first feature value or the second feature value from each image.

[0164] For example, each of the first and second attributes may include a first brightness and a second brightness. For example, the first brightness is brighter than the second brightness. For example, at least one first image having the first brightness may be used to obtain a first feature value for the first tracking. For example, the first feature value may include the boundary or point of an external object for the first tracking. Furthermore, for example, at least one second image having the second brightness may be used to obtain a second feature value for the second tracking. For example, the second feature value may include the light source of the external electronic device 420 for the second tracking.

[0165] In operation 920, according to an embodiment, the wearable device 103 can change its mode from a first mode to a second mode. For example, the wearable device 103 can change its mode from the first mode to the second mode based on a first feature value and a second feature value. For example, the second mode can represent a shooting mode in which at least one first image for first tracking and at least one second image for second tracking are alternately acquired at different FPS (or number of images).

[0166] According to an embodiment, the wearable device 103 can identify the quality of each track based on feature values. This quality can be referred to as attitude quality. For example, the wearable device 103 can identify a first quality of a first track based on a first feature value. For example, the wearable device 103 can identify a second quality of a second track based on a second feature value.

[0167] According to an embodiment, the wearable device 103 can change its mode (or shooting mode) based on a comparison between the quality of each track and a reference value for each track. For example, the wearable device 103 can compare a first quality with a first set of reference values ​​for a first track. Furthermore, for example, the wearable device 103 can compare a second quality with a second set of reference values ​​for a second track. The first set of reference values ​​may include at least one reference value. The second set of reference values ​​may include at least one reference value. For example, it is assumed that the first set of reference values ​​includes a first reference value, and the second set of reference values ​​includes a second reference value. For example, the wearable device 103 can execute a first mode when the first quality is higher than or equal to the first reference value and the second quality is higher than or equal to the second reference value. For example, the wearable device 103 can execute a first mode when the first quality is lower than the first reference value and the second quality is lower than the second reference value. For example, the wearable device 103 can execute a second mode when the first quality is higher than or equal to the first reference value and the second quality is lower than the second reference value, or when the first quality is lower than the first reference value and the second quality is higher than or equal to the second reference value. For example, in a second mode executed in response to a first quality being higher than or equal to a first reference value and a second quality being lower than a second reference value, the number of images (or FPS) used for the second tracking may be greater than (or higher than) the number of images (or FPS) used for the first tracking. Additionally and / or alternatively, for example, in a second mode executed in response to a first quality being lower than a first reference value and a second quality being higher than or equal to a second reference value, the number of images (or FPS) used for the first tracking may be greater than (or higher than) the number of images (or FPS) used for the second tracking. In the above examples, the case where each of the first and second reference value sets includes one reference value is described, but embodiments of this disclosure are not limited thereto. For example, each of the first and second reference value sets may include multiple reference values. Based on the multiple reference values, the number of images (or FPS) obtained by the camera 520 for a specific tracking (e.g., the first tracking and the second tracking) can be adjusted. For example, suppose the first reference value set includes multiple reference values ​​(e.g., the first reference value and the second reference value). In the case where the first quality is lower than the first reference value, the wearable device 103 may obtain images for the first tracking based on the first FPS. When the first quality is higher than or equal to the first reference value and lower than the second reference value, the wearable device 103 can obtain an image for the first tracking based on a second FPS that is less than the first FPS. At this time, each of the first FPS and the second FPS can be a value higher than the FPS of the image used for the second tracking (i.e., when the number of images used for the first tracking is greater than the number of images used for the second tracking).

[0168] Despite Figure 9 Not shown, but according to an embodiment, wearable device 103 can change modes based on input. For example, wearable device 103 can change modes based on gestures set in wearable device 103. For example, wearable device 103 may include a first gesture for a first mode, a second gesture for a second mode, and a third gesture for a third mode. Furthermore, for example, wearable device 103 can change modes based on input from physical buttons included in wearable device 103. Specific details related to this may be referred to as... Figure 8b Examples.

[0169] Despite Figure 9 Not shown, but according to an embodiment, wearable device 103 can change its mode based on the executed software application or settings within wearable device 103. For example, wearable device 103 can identify a mode set in a software application in response to execution of the software application. Wearable device 103 can change its mode to a setting mode. Additionally and / or alternatively, for example, wearable device 103 can identify a setting mode based on settings within wearable device 103. For example, settings in wearable device 103 can be changed based on at least a portion of user input. Wearable device 103 can change its mode to a setting mode. For example, when executing a software application (e.g., a game) that requires a relatively large amount of movement (compared to a predetermined amount of movement) from an external electronic device 420 (or controller) or when settings in wearable device 103 are changed, wearable device 103 can change its mode from a first mode to a second mode. In this case, the second mode can represent a shooting mode where the number of images used for the first tracking is less than the number of images used for the second tracking. This could be to improve the accuracy or response speed of tracking the movement of the external electronic device 420. Specific details related to this are referred to below. Figure 8a Examples.

[0170] Despite Figure 9 Not shown, but according to an embodiment, the wearable device 103 can change its mode based on movement of an external electronic device 420 (or controller) connected to the wearable device 103. For example, the wearable device 103 can change its mode from a first mode to a second mode based on recognizing that the movement is below a reference frequency. The second mode executed in response to the movement being below the reference frequency can represent a shooting mode in which the number of images used for the first tracking is greater than the number of images used for the second tracking. This is because, in the case of movement below the reference frequency, the second tracking of the external electronic device 420 is not required. Specific details related to this may be referred to below as Figure 8a Examples.

[0171] Despite Figure 9Although not shown in the diagram, according to an embodiment, the wearable device 103 can identify the movement of the external electronic device 420 based on the sensor 530. For example, the wearable device 103 can obtain information about the movement of the external electronic device 420 through the sensor 530 (e.g., an IMU sensor). The wearable device 103 can change its mode based on the movement identified according to the information.

[0172] Referring to the above, wearable device 103 can identify whether specified conditions for changing modes are met. If the specified conditions are met, wearable device 103 can change modes. In contrast, if the specified conditions are not met, wearable device 103 can acquire an image in the current mode (e.g., a first mode) and perform tracking based on the image. Tracking may include first tracking and second tracking.

[0173] In operation 930, the wearable device 103 according to the embodiment can acquire additional images in the second mode. For example, when the modified mode is the second mode, the additional images may include at least one third image for the first tracking and at least one fourth image for the second tracking. For example, the number (or FPS) of at least one third image may differ from the number (or FPS) of at least one fourth image. However, the embodiments of this disclosure are not limited thereto. For example, when the modified mode is the third mode, the additional images may include one of the at least one third image for the first tracking and at least one fourth image for the second tracking. For example, when the first tracking is unnecessary (e.g., when the user's head is fixed), the additional images may include at least one of the at least one third image for the first tracking and at least one fourth image for the second tracking. Conversely, when the second tracking is unnecessary (e.g., when the user is not holding the external electronic device 420), the additional images may include at least one of the at least one third image for the first tracking and at least one fourth image for the second tracking.

[0174] As described above, wearable device 103 may include a memory 560 that stores instructions and includes one or more storage media. Wearable device 103 may include a camera 520. Wearable device 103 may include at least one processor 510 that includes processing circuitry. When executed individually or jointly by at least one processor 510, the instructions may cause wearable device 103 to: in a first mode, acquire an image via camera 520 including at least one first image and at least one second image, the at least one first image having a first attribute for tracking a user's body parts, and the at least one second image having a second attribute different from the first attribute for tracking an external electronic device 420. A first number of at least one first image may correspond to a second number of at least one second image. When executed individually or jointly by at least one processor 510, the instructions may cause wearable device 103 to: obtain a first feature value from at least one first image for tracking a body part, and obtain a second feature value from at least one second image for tracking an external electronic device 420. When executed individually or jointly by at least one processor 510, the instructions can cause the wearable device 103 to change its mode from a first mode to a second mode based on a first feature value and a second feature value. When executed individually or jointly by at least one processor 510, the instructions can also cause the wearable device 103 to acquire additional images via camera 520 in the second mode. The third number of at least one third image having a first attribute among the additional images may differ from the fourth number of at least one fourth image having a second attribute among the additional images.

[0175] According to an embodiment, the body part may include at least one of the user's head or the user's hand.

[0176] According to an embodiment, each of the first and second attributes may include brightness. Brightness may be identified based on at least one of the exposure time used to obtain an image via camera 250, the gain value used to obtain the image, or the brightness level of a light source of wearable device 103.

[0177] According to an embodiment, in response to a first attribute representing a first brightness and a second attribute representing a second brightness, the first brightness can be brighter than the second brightness.

[0178] According to an embodiment, an image can be obtained based on a specified FPS. A first FPS of at least one first image may correspond to a second FPS of at least one second image. At least one first image having a first attribute and at least one second image having a second attribute can be obtained alternately.

[0179] According to an embodiment, when executed individually or jointly by at least one processor 510, the instructions may cause the wearable device 103 to execute a first mode when specified conditions are met. The specified conditions may include at least one of the following: the wearable device 103 is activated to track a first mass of a body part that is greater than or equal to a first reference value and a second mass of an external electronic device 420 that is greater than or equal to a second reference value, or the first mass is less than the first reference value and the second mass is less than the second reference value.

[0180] According to an embodiment, when executed individually or jointly by at least one processor 510, the instructions can cause the wearable device 103 to: obtain first feature values ​​including the boundaries or points of objects in at least one first image. When executed by at least one processor 510, the instructions can also cause the wearable device 103 to: obtain second feature values ​​including at least one light source including external electronic device 420 in at least one second image.

[0181] According to an embodiment, when executed individually or jointly by at least one processor 510, the instructions can cause the wearable device 103 to: obtain a first quality for tracking body parts based on a first feature value. When executed by at least one processor 510, the instructions can cause the wearable device 103 to: obtain a second quality for tracking external electronic devices 420 based on a second feature value. The first quality can be obtained based on a first change in the first feature value for each of at least one first image. The second quality can be obtained based on a second change in the second feature value for each of at least one second image.

[0182] According to an embodiment, when the instructions are executed individually or jointly by at least one processor 510, the wearable device 103 may change its mode from a first mode to a second mode in response to a first quality being lower than a first reference value and a second quality being higher than or equal to a second reference value, wherein in the second mode, the third number of at least one third image having a first attribute of other images is greater than the fourth number of at least one fourth image having a second attribute.

[0183] According to an embodiment, when executed individually or jointly by at least one processor 510, the instructions can cause the wearable device 103 to change its mode from a first mode to a second mode in response to a first quality being higher than or equal to a first reference value and a second quality being lower than a second reference value, wherein in the second mode, the third number of at least one third image having a first attribute of other images is less than the fourth number of at least one fourth image having a second attribute.

[0184] According to an embodiment, when executed individually or jointly by at least one processor 510, the instructions can cause the wearable device 103 to: execute a software application after acquiring additional images. When executed individually or jointly by at least one processor 510, the instructions can also cause the wearable device 103 to: change a mode from a second mode to a first mode based on settings of the software application. These settings may include a first FPS of at least one first image with a first attribute to be acquired by the camera 250 and a second FPS of at least one second image with a second attribute.

[0185] According to an embodiment, when the instructions are executed individually or jointly by at least one processor 510, the wearable device 103 may change its mode from a first mode to a second mode based on the recognition that the movement of the external electronic device 420 is below a reference frequency, in which the third number of at least one third image having a first attribute of other images is greater than the fourth number of at least one fourth image having a second attribute.

[0186] According to an embodiment, when executed individually or jointly by at least one processor 510, the instructions can cause the wearable device 103 to change its mode from a second mode to a first mode based on the recognition in the second mode that the movement of the external electronic device 420, which has been identified by sensors of the wearable device 103, is higher than or equal to a reference frequency. The sensors may include inertial measurement unit (IMU) sensors.

[0187] According to an embodiment, when executed individually or jointly by at least one processor 510, the instructions can cause the wearable device 103 to change its mode from a first mode to a second mode based on the execution of a software application providing a virtual environment using an external electronic device 420. In this second mode, the third number of at least one third image with a first attribute is lower than the fourth number of at least one fourth image with a second attribute. During the execution of the second mode, the first quality of the at least one third image with the first attribute can be higher than or equal to a first reference value.

[0188] According to an embodiment, when executed individually or jointly by at least one processor 510, the instructions can cause the wearable device 103 to change its mode from a second mode to a third mode when specified conditions are met. The specified conditions may include setting the tracking external electronics 420 between tracking body parts and tracking external electronics 420. In the third mode, an image having a second attribute among a first attribute and a second attribute can be obtained via the camera 520.

[0189] According to an embodiment, when executed individually or jointly by at least one processor 510, the instructions can cause the wearable device 103 to change its mode from a second mode to a third mode when specified conditions are met. The specified conditions may include setting the tracking body part in the tracking body part and the tracking external electronics 420. In the third mode, an image having a first attribute among a first attribute and a second attribute can be obtained via the camera 520.

[0190] According to an embodiment, when executed individually or jointly by at least one processor 510, the instructions can cause the wearable device 103 to change its mode from a third mode to a first mode based on user input. User input may include at least one of the following: input to a physical button on the wearable device 103 or a user-specified gesture.

[0191] As described above, a method performed by a wearable device 103 may include: acquiring an image comprising at least one first image and at least one second image in a first mode, the at least one first image having a first attribute for tracking a user's body parts, and the at least one second image having a second attribute different from the first attribute for tracking an external electronic device 420. A first number of at least one first image may correspond to a second number of at least one second image. The method may include: obtaining a first feature value from at least one first image for tracking body parts, and obtaining a second feature value from at least one second image for tracking the external electronic device 420. The method may include: changing the mode of the wearable device 103 from the first mode to the second mode based on the first and second feature values. The method may include: acquiring additional images via a camera 520 in the second mode. A third number of at least one third image having the first attribute of the additional images may differ from a fourth number of at least one fourth image having the second attribute of the additional images.

[0192] According to an embodiment, each of the first attribute and the second attribute may include brightness. Brightness may include at least one of the following: exposure time for obtaining an image via camera 250, gain value for obtaining an image, or brightness level of a light source of wearable device 103.

[0193] As described above, a non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed individually or jointly by at least one processor 510 of a wearable device 103 including a camera 520, cause the wearable device 103 to: in a first mode, acquire images via the camera 520 including at least one first image and at least one second image, the at least one first image having a first attribute for tracking body parts of a user, and the at least one second image having a second attribute different from the first attribute for tracking an external electronic device 420. A first number of at least one first image may correspond to a second number of at least one second image. The non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by at least one processor 510, cause the wearable device 103 to: obtain first feature values ​​from at least one first image for tracking body parts and obtain second feature values ​​from at least one second image for tracking an external electronic device 420. A non-transitory computer-readable storage medium may store one or more programs including instructions that, when executed by at least one processor 510, cause the wearable device 103 to change its mode from a first mode to a second mode based on a first feature value and a second feature value. The non-transitory computer-readable storage medium may also store one or more programs including instructions that, when executed by at least one processor 510, cause the wearable device 103 to acquire additional images via camera 520 in the second mode. The third number of at least one third image having a first attribute among the additional images may differ from the fourth number of at least one fourth image having a second attribute among the additional images.

[0194] The electronic device according to various embodiments can be one of a variety of types of electronic devices. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. According to embodiments of this disclosure, the electronic device is not limited to those described above.

[0195] It should be understood that the various embodiments of this disclosure and the terminology used therein are not intended to limit the technical features set forth herein to the specific embodiments, but rather to include various changes, equivalents, or substitutions to the respective embodiments. In the description of the drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that nouns in the singular form corresponding to terms may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least one of A, B, or C” may include any one or all possible combinations of the items enumerated together with the corresponding phrase among the plurality of phrases. As used herein, terms such as “first” and “second” or “first” and “second” may be used to simply distinguish the respective component from another component and do not limit the component in other respects (e.g., importance or order). It will be understood that, whether the terms “operably” or “communically” are used or not, if an element (e.g., a first element) is referred to as “connected to another element (e.g., a second element)” or “connected to another element (e.g., a second element)”, it means that the element can be directly (e.g., wiredly) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0196] As used in connection with various embodiments of this disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms such as "logic," "logic block," "part," or "circuit." A module may be a single integrated component adapted to perform one or more functions, or the smallest unit or part of such a single integrated component. For example, according to embodiments, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0197] The various embodiments set forth herein can be implemented as software (e.g., program 140) containing one or more instructions readable by a machine (e.g., electronic device 101) stored in a storage medium (e.g., internal memory 136 or external memory 138). For example, under the control of a processor, the processor (e.g., processor 120) of the machine (e.g., electronic device 101) can invoke and execute at least one of the one or more instructions stored in the storage medium, with or without the use of one or more other components. This enables the machine to operate to perform at least one function according to the invoked at least one instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. Machine-readable storage media may be provided in the form of non-transitory storage media. The term "non-transitory" simply means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but this term does not distinguish between data being stored semi-permanently in the storage medium and data being temporarily stored in the storage medium.

[0198] According to embodiments, methods according to various embodiments of this disclosure may be included and provided in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an app store (e.g., the Play Store™), or may be distributed directly between two user devices (e.g., smartphones) (e.g., downloaded or uploaded). If distributed online, at least a portion of the computer program product may be temporarily generated, or at least a portion of the computer program product may be stored at least temporarily in a machine-readable storage medium (such as the memory of a manufacturer's server, an app store's server, or a forwarding server).

[0199] According to various embodiments, each of the above-described components (e.g., a module or program) may include a single entity or multiple entities, and some of the multiple entities may be separately disposed in different components. According to various embodiments, one or more of the above-described components may be omitted, or one or more other components may be added. Optionally or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each of the multiple components in the same or similar manner as the corresponding component of the multiple components performed one or more functions before integration. According to various embodiments, the operations performed by a module, program, or other component may be performed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be run in a different order or omitted, or one or more other operations may be added.

Claims

1. A wearable device (103), the wearable device comprising: A memory (560), the memory comprising one or more storage media, the memory storing instructions; Camera (520); as well as At least one processor (510), said at least one processor including processing circuitry, The instructions, when executed individually or jointly by the at least one processor (510), cause the wearable device (103) to: In a first mode, an image including at least one first image and at least one second image is obtained by the camera (520), the at least one first image having a first attribute for tracking body parts of the user, and the at least one second image having a second attribute different from the first attribute for tracking external electronic devices (420), wherein a first number of the at least one first image corresponds to a second number of the at least one second image; A first feature value for tracking the body part is obtained from the at least one first image, and a second feature value for tracking the external electronic device (420) is obtained from the at least one second image; Based on the first feature value and the second feature value, the mode of the wearable device (103) is changed from the first mode to the second mode; and In the second mode, other images are obtained through the camera (520). The third number of at least one third image with the first attribute in the other images is different from the fourth number of at least one fourth image with the second attribute in the other images.

2. The wearable device (103) according to claim 1. in, When the instructions are executed individually or jointly by the at least one processor (510), the wearable device (103) is made to: In the at least one first image, the first feature value, including the boundary or points of the object, is obtained; as well as The second characteristic value of at least one light source including the external electronic device (420) is obtained in the at least one second image.

3. The wearable device (103) according to claim 2. in, When the instructions are executed individually or jointly by the at least one processor (510), the wearable device (103) is made to: Based on the first feature value, a first mass for tracking the body part is obtained; as well as Based on the second characteristic value, a second mass for tracking the external electronic device (420) is obtained. The first quality is obtained based on a first change in the first feature value for each of the at least one first image. The second quality is obtained based on a second change in the second feature value for each of the at least one second image.

4. The wearable device (103) according to claim 3. in, When the instructions are executed individually or jointly by the at least one processor (510), the wearable device (103) is made to: In response to the first quality being lower than a first reference value and the second quality being higher than or equal to a second reference value, the mode is changed from the first mode to the second mode, in which the third number of the at least one third image having the first attribute among the other images is greater than the fourth number of the at least one fourth image having the second attribute.

5. The wearable device (103) according to claim 3. in, When the instructions are executed individually or jointly by the at least one processor (510), the wearable device (103) is made to: In response to the first quality being higher than or equal to a first reference value and the second quality being lower than a second reference value, the mode is changed from the first mode to the second mode, in which the third number of the at least one third image having the first attribute among the other images is less than the fourth number of the at least one fourth image having the second attribute.

6. The wearable device (103) according to claim 1. in, The body part includes at least one of the user's head or the user's hand.

7. The wearable device (103) according to claim 1. in, Each of the first and second attributes has a brightness. The brightness is identified based on at least one of the following: the exposure time for obtaining an image by the camera (520), the gain value for obtaining the image, or the brightness level of the light source of the wearable device (103).

8. The wearable device (103) according to claim 1. in, The image was obtained based on a specified number of frames per second (FPS). Wherein, the first FPS of the at least one first image corresponds to the second FPS of the at least one second image, and Wherein, the at least one first image having the first attribute and the at least one second image having the second attribute in the image are obtained alternately.

9. The wearable device (103) according to claim 1. in, When the instructions are executed individually or jointly by the at least one processor (510), the wearable device (103) is made to: When the specified conditions are met, execute the first mode. The specified conditions include at least one of the following: the wearable device (103) is activated to track a first mass of the body part that is higher than or equal to a first reference value and to track a second mass of the external electronic device (420) that is higher than or equal to a second reference value, or the first mass is lower than the first reference value and the second mass is lower than the second reference value.

10. The wearable device (103) according to claim 1. in, When the instructions are executed individually or jointly by the at least one processor (510), the wearable device (103) is made to: Based on the identification that the movement of the external electronic device (420) is below the reference frequency, the mode is changed from the first mode to the second mode, in which the third number of the at least one third image with the first attribute in the other images is greater than the fourth number of the at least one fourth image with the second attribute.

11. The wearable device (103) according to claim 10. in, When the instructions are executed individually or jointly by the at least one processor (510), the wearable device (103) is made to: Based on the fact that the movement of the external electronic device (420), which has been identified by the sensors of the wearable device (103) in the second mode, is higher than or equal to the reference frequency, the mode is changed from the second mode to the first mode. The sensor includes an inertial measurement unit (IMU) sensor.

12. The wearable device (103) according to claim 1. in, When the instructions are executed individually or jointly by the at least one processor (510), the wearable device (103) is made to: Based on the execution of the software application providing a virtual environment using the external electronic device (420), the mode is changed from the first mode to the second mode, in which the third number of the at least one third image having the first attribute among the other images is lower than the fourth number of the at least one fourth image having the second attribute. When the second mode is executed, the first quality of the at least one third image having the first attribute among the other images is higher than or equal to a first reference value.

13. The wearable device (103) according to claim 1. in, When the instructions are executed individually or jointly by the at least one processor (510), the wearable device (103) is made to: When specified conditions are met, the mode is changed from the second mode to the third mode. The specified conditions include: setting the tracking of the external electronic device (420) among tracking the body part and tracking the external electronic device (420), and In the third mode, an image having the second attribute, which is one of the first attribute and the second attribute, is obtained by the camera (520).

14. A method performed by a wearable device (103), the method comprising: In a first mode, an image is obtained comprising at least one first image and at least one second image, wherein the at least one first image has a first attribute for tracking body parts of a user, and the at least one second image has a second attribute different from the first attribute for tracking external electronic devices (420), wherein a first number of the at least one first image corresponds to a second number of the at least one second image; A first feature value for tracking the body part is obtained from the at least one first image, and a second feature value for tracking the external electronic device (420) is obtained from the at least one second image; Based on the first feature value and the second feature value, the mode of the wearable device (103) is changed from the first mode to the second mode; and Other images are obtained in the second mode. The third number of at least one third image with the first attribute in the other images is different from the fourth number of at least one fourth image with the second attribute in the other images.

15. A non-transitory computer-readable storage medium comprising instructions that, when executed alone or jointly by at least one processor (510) of a wearable device including a camera (520), cause the wearable device (103) to: In a first mode, an image comprising at least one first image and at least one second image is acquired via the camera (520), wherein the at least one first image has a first attribute for tracking body parts of the user, and the at least one second image has a second attribute different from the first attribute for tracking external electronic devices (420), wherein... The first number of the at least one first image corresponds to the second number of the at least one second image; A first feature value for tracking the body part is obtained from the at least one first image, and a second feature value for tracking the external electronic device (420) is obtained from the at least one second image; Based on the first feature value and the second feature value, the mode of the wearable device (103) is changed from the first mode to the second mode; as well as In the second mode, other images are obtained through the camera (520). The third number of at least one third image with the first attribute in the other images is different from the fourth number of at least one fourth image with the second attribute in the other images.

Images