Wearable electronic device controlling camera module and operating method thereof

Abstract

A wearable electronic device is provided. The wearable electronic device may include: a housing; at least one processor included in the housing; at least one first camera module configured to capture an image for a first field of view (FOV); and at least one second camera module configured to capture an image for a second FOV different from the first FOV. The at least one processor may be configured to: control the at least one first camera module to acquire a first image frame at a first frame rate; controlling the at least one first camera module to acquire a second image frame at a second frame rate greater than the first frame rate based on confirming that the user's hand is located in the specified area of the first FOV from at least some of the first image frames; and controlling the at least one second camera module to maintain the at least one second camera module in a deactivated state or to activate the at least one second camera module and to operate the at least one second camera module at the first frame rate.

Description

Technical Field

[0001] The present disclosure relates to a wearable electronic device for controlling a camera module and an operating method thereof. Background Art

[0002] Many various services and additional functions provided by wearable electronic devices, such as augmented reality (AR) glasses, head mounted display (HMD) devices, or video see-through (VST) devices, are gradually increasing. In order to improve the practical value of such electronic devices and meet the needs of various users, communication service providers or electronic device manufacturers are providing various functions and competing to develop electronic devices in order to distinguish themselves from other companies. Accordingly, various functions provided by wearable electronic devices are also gradually advancing.

[0003] The wearable electronic device may include at least one camera module capable of capturing a foreground image. The wearable electronic device may perform simultaneous localization and mapping (SLAM) and / or 6 degrees of freedom (DoF) based on the image captured using the at least one camera module. The wearable electronic device may perform recognition and / or tracking of an object based on the image captured using the at least one camera module, and perform operations based on the results of the recognition and / or tracking.

[0004] The above information is presented as background information only to assist with an understanding of the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above might be applicable as prior art with respect to the present disclosure. Summary of the invention

[0005] Solution

[0006] Aspects of the present disclosure will at least address the above-mentioned problems and / or disadvantages and provide at least the advantages described below. Accordingly, one aspect of the present disclosure provides a wearable electronic device for controlling a camera module and an operating method thereof.

[0007] Additional aspects will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the presented embodiments.

[0008] According to aspects of the present disclosure, a wearable electronic device is provided. The wearable electronic device may include: a housing including a structure that enables the housing to be worn on at least a portion of a user's head; at least one processor, the at least one processor included in the housing; at least one first camera module, the at least one first camera module configured to capture an image for a first field of view (FOV) configured relative to a point on a first surface of the housing, and at least one second camera module, the at least one second camera module configured to capture an image for a second FOV different from the first FOV configured relative to a point on the first surface. The at least one processor may be configured to control the at least one first camera module to obtain a first image frame at a first frame rate. The at least one processor may be configured to control the at least one first camera module to obtain a second image frame at a second frame rate greater than the first frame rate based on recognizing that a user's hand is placed in a specified area of ​​the first FOV according to at least a portion of the first image frame. The at least one processor may be configured to control the at least one second camera module to maintain the at least one second camera module in an inactive state, or to activate the at least one second camera module and operate the at least one second camera module at the first frame rate.

[0009] According to the present disclosure, a method for operating a wearable electronic device is provided, the wearable electronic device comprising at least one first camera module and at least one second camera module, the at least one first camera module being configured to capture images for a first FOV, and the at least one second camera module being configured to capture images for a second FOV different from the first FOV. The method may include controlling the at least one first camera module to obtain a first image frame at a first frame rate. The method may include: based on recognizing that a user's hand is placed in a specified area of ​​the first FOV according to at least a portion of the first image frame, controlling the at least one first camera module to obtain a second image frame at a second frame rate greater than the first frame rate. The method may include controlling the at least one second camera module to maintain the at least one second camera module in an inactive state, or activating the at least one second camera module and operating the at least one second camera module at the first frame rate.

[0010] Other aspects, advantages, and salient features of the disclosure will become apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the annexed drawings, discloses various embodiments of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0012] Figure 1 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0013] Figure 2 is a perspective view showing an internal configuration of an electronic device according to an embodiment of the present disclosure.

[0014] Figure 3 is a diagram illustrating a field of view (FOV) corresponding to a plurality of camera modules according to an embodiment of the present disclosure.

[0015] Figure 4a and Figure 4b is a perspective view illustrating a configuration of an electronic device according to various embodiments of the present disclosure.

[0016] Figure 5a is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0017] Figure 5b is a diagram illustrating FOVs corresponding to a plurality of camera modules according to an embodiment of the present disclosure.

[0018] Figure 5c is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0019] Figure 6a is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0020] Figure 6b is a diagram illustrating adjusting a frame rate of at least one first camera module according to an embodiment of the present disclosure.

[0021] Figure 6c is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0022] Figure 7a is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0023] Figure 7b is a diagram illustrating adjusting a frame rate of at least one second camera module according to an embodiment of the present disclosure.

[0024] Figure 8 is a diagram illustrating adjustment of the FOV and frame rate of an electronic device of AR glasses according to an embodiment of the present disclosure.

[0025] Fig. 9 is a diagram illustrating adjusting the FOV and frame rate of a VST device in VR mode according to an embodiment of the present disclosure.

[0026] Fig.10 is a diagram illustrating adjusting the FOV and frame rate of a VST device in a VST mode according to an embodiment of the present disclosure.

[0027] Fig.11 is a diagram illustrating adjusting the FOV and frame rate of a VST device in a VST mode connected to a controller according to an embodiment of the present disclosure.

[0028] Fig.12 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0029] Fig.13 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0030] Throughout the drawings, it should be noted that like reference numbers are used to depict the same or similar elements, features, and structures. DETAILED DESCRIPTION

[0031] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of the various embodiments of the present disclosure as defined by the claims and their equivalents. It includes various specific details to assist in understanding, but these are to be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications to the various embodiments described herein may be made without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0032] The terms and words used in the following description and claims are not limited to the bibliographical meanings, but are merely used by the inventor to enable a clear and consistent understanding of the present disclosure. Therefore, it should be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustration purposes only and is not intended to limit the present disclosure as defined by the appended claims and their equivalents.

[0033] Figure 1 is a block diagram 100 illustrating an electronic device 101 according to an embodiment of the present disclosure.

[0034] Reference Figure 1 , the electronic device 101 may be, for example, a wearable electronic device that can be worn on the user's head, such as AR glasses, an HMD device, and / or a VST device. The electronic device 101 may be named a wearable electronic device. The position where the user wears the electronic device is not limited, and all devices including multiple camera modules for capturing images in multiple FOVs as described below are possible.

[0035] Each of the external electronic devices 102 and 103 may be the same or different from the type of the electronic device 101. According to an embodiment, all or some operations running in the electronic device 101 may be run in one or more external electronic devices (external electronic devices 102 and 103 or server 108) among the external electronic devices. For example, if the electronic device 101 is required to automatically or in response to a request from a user or another device to perform a function or service, as a supplement or replacement for running a function or service by itself, the electronic device 101 may request one or more external electronic devices to perform at least a part of the function or service. The one or more external electronic devices that have received the request may run at least a part of the requested function or service, or another function or service related to the request, and may transmit the running result to the electronic device 101. The electronic device 101 may or may not process the result in addition, and may provide the processed result as at least a part of the response to the request. For example, the external electronic device 102 may render the content data running in the application and then transmit the rendered content data to the electronic device 101, and the electronic device 101 that has received the data may output the content data to the display module. If the electronic device 101 detects the movement of the user through an inertial measurement unit (IMU) sensor, the processor 120 of the electronic device 101 may correct the rendering data received from the external electronic device 102 based on the information about the movement, and output the corrected rendering data to the display module 160. Alternatively, the electronic device 101 may transmit the movement information to the external electronic device 102, and request rendering so that the screen data is updated according to the information. According to various embodiments, the external electronic device 102 may be various types of devices, such as a smart phone or a housing device capable of storing and charging the electronic device 101.

[0036] According to an embodiment, the electronic device 101 may communicate with the external electronic device 102 through a first network 198 (e.g., a short-range wireless communication network), and / or may communicate with the external electronic device 103 or the server 108 through a second network 199 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 101 may communicate with another electronic device through the server 108. According to an embodiment, the electronic device 101 may include at least one of a processor 120, a memory 130, an input module 150, a sound output module 155, a display module 160, an audio module 170, a sensor module 176, an interface 177, a connection terminal 178, a haptic module 179, a camera module 180, a power management module and / or a battery 189, a communication module 190, or an antenna module 197. In an embodiment, at least one of the above elements (e.g., the connection terminal 178) may be omitted from the electronic device 101, or one or more other elements may be added thereto. In some embodiments, some of the above-described elements (eg, sensor module 176, camera module 180, or antenna module 197) may be integrated into a single element (eg, display module 160).

[0037] According to an embodiment, the processor 120 may, for example, run software (e.g., program 140) to control different elements (e.g., hardware or software elements) of the electronic device 101 connected to the processor 120, and perform various data processing or calculations. According to an embodiment, as at least part of the data processing or calculation, the processor 120 may store commands or data received from another element (e.g., sensor module 176 or communication module 190) in a volatile memory, process the commands or data stored in the volatile memory, and store the result data in a non-volatile memory. According to an embodiment, the processor 120 may include a main processor (e.g., a central processing unit or an application processor), or an auxiliary processor (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate alone or in conjunction with the main processor. For example, if the electronic device 101 includes a main processor and an auxiliary processor, the auxiliary processor may be configured to use less power than the main processor, or be dedicated to a specified function. The auxiliary processor may be implemented as separate from the main processor, or as part of it. The auxiliary processor can, for example, represent the main processor when the main processor is in an inactive (e.g., sleep) state, or together with the main processor when the main processor is in an active (e.g., application running) state, to control at least some of the functions or states related to at least one element among the elements of the electronic device 101 (e.g., display module 160 sensor module 176, or communication module 190). According to an embodiment, the auxiliary processor (e.g., image signal processor or communication processor) can be implemented as a part of a different element (e.g., camera module 180 or communication module 190) that is functionally related to the auxiliary processor. According to an embodiment, the auxiliary processor (e.g., neural processing unit) may include a hardware structure dedicated to processing an artificial intelligence model. The artificial intelligence model can be generated by machine learning. For example, such learning can be automatically performed in the electronic device 101 running artificial intelligence, or can be performed by a separate server (e.g., server 108). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the above examples. The artificial intelligence model may include multiple artificial neural network layers. The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q network, or a combination of two or more of the above networks. However, the artificial neural network is not limited to the above examples. In addition to the hardware structure, or as a replacement for the hardware structure, the artificial intelligence model may include a software structure.

[0038] According to an embodiment, the memory 130 may store various data used by at least one element of the electronic device 101 (e.g., the processor 120 or the sensor module 176). The data may include, for example, software (e.g., the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory and / or a nonvolatile memory.

[0039] According to an embodiment, the program 140 may be stored as software in the memory 130 , and may include, for example, an operating system 142 , middleware 144 , or an application 146 .

[0040] According to an embodiment, the input module 150 may receive a command or data to be used for an element (e.g., processor 120) of the electronic device 101 from outside (e.g., from a user) of the electronic device 101. The input module 150 may include a microphone, a button, and / or a touch pad, but there is no limitation.

[0041] According to an embodiment, the sound output module 155 may output a sound signal to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker. The speaker may be used for general purposes such as multimedia playback or recording.

[0042] According to an embodiment, the display module 160 may visually provide information to the outside of the electronic device 101 (e.g., a user). For example, if the electronic device 101 is implemented as AR glasses, the display module 160 may include, for example, a liquid crystal display (LCD), a digital micromirror device (DMD), a liquid crystal on silicon (LCoS), a light emitting diode (LED) on silicon (LEDoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED), but there is no limitation. The display module 160 may be implemented differently according to the type of the electronic device 101. For example, if the electronic device 101 is implemented as AR glasses, the display module may also include a display module that displays the reference image. Figure 2 Lenses and / or optical waveguides described.

[0043] According to an embodiment, the audio module 170 may convert sound into an electrical signal, or conversely, may convert an electrical signal into sound. According to an embodiment, the audio module 170 may obtain sound through the input module 150, or may output sound through the sound output module 155 or an external electronic device (e.g., external electronic device 102) (e.g., a speaker or earphone) directly or wirelessly connected to the electronic device 101. The external electronic device 102 may be, for example, a mobile device such as a smart phone or a tablet PC, but there is no limitation.

[0044] According to an embodiment, the sensor module 176 may detect an operating state (e.g., power or temperature) of the electronic device 101 or an external environmental state (e.g., a user's state), and generate an electrical signal or data value corresponding to the detected state. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyroscope sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, and / or an illumination sensor.

[0045] The interface 177 may support one or more designated protocols that may be used for the electronic device 101 to be directly or wirelessly connected to an external electronic device (e.g., the external electronic device 102). According to an embodiment, the interface 177 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0046] According to an embodiment, the connection terminal 178 may include a connector via which the electronic device 101 can be physically connected to an external electronic device (e.g., the external electronic device 102). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0047] According to an embodiment, the haptic module 179 may convert the electric signal into electric stimulation or mechanical stimulation (eg, vibration or movement) that the user can recognize through his / her own recognition. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electric stimulator.

[0048] According to an embodiment, the camera module 180 may capture a still image or a moving image. According to an embodiment, the camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash. According to an embodiment, the electronic device 101 may include a plurality of camera modules for capturing images in a plurality of FOVs, which will be described later.

[0049] According to an embodiment, the power management module and / or the battery 189 may manage the power provided to the electronic device 101. According to an embodiment, the power management module and / or the battery 189 may be implemented, for example, as at least a portion of a power management integrated circuit (PMIC). The power management module and / or the battery 189 may supply power to at least one element of the electronic device 101. According to an embodiment, the power management module and / or the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0050] According to an embodiment, the communication module 190 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (e.g., an external electronic device 102, an external electronic device 103, or a server 108), and performing communication through the established communication channel. The communication module 190 may include one or more communication processors that operate independently of the processor 120 (e.g., an application processor) and support direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 190 may include a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (e.g., a local area network (LAN) communication module or a power line communication module). The corresponding communication module in the communication modules described above can communicate with the external electronic device 103 through a first network 198 (e.g., a short-range communication network, such as Bluetooth™, Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network, such as a traditional cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be integrated into one component (e.g., a single chip), or can be implemented as multiple separate components (e.g., multiple chips). The wireless communication module can identify and authenticate the electronic device 101 in a communication network (such as the first network 198 or the second network 199) using user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in a user identification module.

[0051] The wireless communication module can support the next generation communication technology of 5G network and super 4G network, for example, new radio (NR) access technology. NR access technology can support high-speed transmission of large amounts of data (enhanced mobile broadband (eMBB)), minimization of terminal power consumption and multi-terminal access (massive machine type communication (mMTC)) or high reliability and low latency (ultra-reliable and low latency communication (URLLC)). The wireless communication module can, for example, support high frequency bands (e.g., millimeter wave (mmWave) bands) to achieve high data transmission rates. The wireless communication module can support various technologies for ensuring performance in high frequency bands, such as beamforming, massive multiple input and multiple output (massive MIMO), full dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. The wireless communication module 192 can support various requirements specified in the electronic device 101, an external electronic device (e.g., an external electronic device 103) or a network system (e.g., a second network 199). According to an embodiment, the wireless communication module may support a peak data rate for implementing eMBB (e.g., 20 Gbps or higher), a loss coverage for implementing mMTC (e.g., 164 dB or lower), or a U-plane delay for implementing URLLC (e.g., 0.5 ms or lower for each of the downlink (DL) and uplink (UL) or 1 ms or lower for round trip). As described above, if the electronic device 101 supports cellular communications such as 4G or 5G, the electronic device 101 may be called an independent (SA) type electronic device. At the same time, the electronic device 101 may be implemented to not support cellular communications such as 4G or 5G. In this case, the electronic device 101 may use the first network 198 to use the Internet through an external electronic device 102 that supports cellular communications, and may be named a non-independent type electronic device.

[0052] According to an embodiment, the antenna module 197 may send a signal or power to the outside (e.g., an external electronic device) or receive a signal or power from the outside. According to an embodiment, the antenna module 197 may include an antenna or a conductor, the antenna including a radiator configured by a conductive pattern, and the conductor is provided on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, the antenna module 197 may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network such as the first network 198 or the second network 199 may be selected from among the plurality of antennas by the communication module 190, for example. A signal or power may be sent or received between the communication module 190 and the external electronic device through the selected at least one antenna. According to an embodiment, a component other than a radiator (e.g., a radio frequency integrated circuit (RFIC)) may be additionally provided as part of the antenna module 197.

[0053] According to an embodiment, the antenna module 197 may be configured as a millimeter wave antenna module. According to an embodiment, the millimeter wave antenna module may include a printed circuit board, an RFIC, and a plurality of antennas (e.g., array antennas), wherein the RFIC is disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and is capable of supporting a specified high frequency band (e.g., millimeter wave band), and the plurality of antennas (e.g., array antennas) are disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and are capable of sending or receiving signals in the specified high frequency band.

[0054] At least some of the elements may be connected to each other and exchange signals (e.g., commands or data) with each other through a communication scheme between peripheral devices (e.g., a bus, a general purpose input / output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)).

[0055] According to an embodiment, a command or data may be sent or received between the electronic device 101 and the external electronic device 103 through the server 108 connected to the second network 199. Each of the external electronic devices 102 or 103 may be a device of the same type or different from the electronic device 101. According to an embodiment, all or some operations executed in the electronic device 101 may be executed in one or more external electronic devices among the external electronic devices 102, 103 or 108. For example, in the case where the electronic device 101 is required to automatically or in response to a request from a user or another device to perform a function or service, as a supplement or replacement for executing the function or service by itself, the electronic device 101 may request one or more external electronic devices to execute at least a part of the function or service. The one or more external electronic devices that have received the request may execute at least a part of the requested function or service, or another function or service related to the request, and may transmit the operation result to the electronic device 101. The electronic device 101 may or may not process the result in addition, and may provide the processed result as at least a part of the response to the request. To this end, for example, cloud computing, distributed computing, mobile edge computing (MEC) or client server computing technology can be used. The electronic device 101 can provide, for example, ultra-low latency services by using distributed computing or mobile edge computing. In another embodiment, the external electronic device 103 may include an Internet of Things (IoT) device. The server 108 may be an intelligent server using machine learning and / or neural networks. According to an embodiment, the external electronic device 103 or the server 108 may be included in the second network 199. The electronic device 101 can be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or health care) based on 5G communication technology and IoT-related technologies.

[0056] The communication module 191 included in the external electronic device 102 can be implemented as, for example, the same as or similar to the communication module 190, but there is no limitation. In addition, all devices capable of communicating by wire and / or wireless are possible. The memory 193 can be implemented as the same as or similar to the memory 130, but there is no limitation, and all devices capable of storing / loading data are possible. The processor 192 can be implemented as the same as or similar to the processor 120, but there is no limitation, and all devices capable of running instructions (or programs) are possible. The battery / power management module 194 can be implemented as the same as or similar to the battery / power management module 189, but there is no limitation, and all devices capable of supplying power are possible.

[0057] Figure 2 is a perspective view showing the internal configuration of an electronic device according to an embodiment of the present disclosure. Figure 3 describe Figure 2 Embodiment of the invention. Figure 3 is a diagram illustrating a field of view (FOV) corresponding to a plurality of camera modules according to an embodiment of the present disclosure. Figure 2 The electronic device 200 may be Figure 1 An electronic device 101.

[0058] Reference Figure 2 , the electronic device 200 according to the embodiment may include components accommodated in the housings 210a, 210b, and 210c, arranged on the housings 210a, 210b, and 210c, and / or exposed through openings provided on the housings 210a, 210b, and 210c.

[0059] According to an embodiment, the electronic device 200 can obtain a visual image of an object or environment located in a direction (e.g., -Y direction) in which the user is viewing or the electronic device 200 is oriented by utilizing a plurality of camera modules 253, 254, 255, and 256. The camera modules 253 and 254 can be arranged on substantially the upper part of the housings 210b and 210c (or can be exposed through openings provided on the housings 210b and 210c). The camera modules 253 and 254 can capture an image corresponding to a FOV based on at least one point on the housings 210b and 210c (e.g., a FOV corresponding to substantially the upper side when the user wears the electronic device 200). For example, referring to Figure 3 , the camera modules 253 and 254 may capture images corresponding to the first FOV 310. The images obtained by the camera modules 253 and 254 may be used, for example, for SLAM and / or 6DoF, and / or for identifying and / or tracking a subject corresponding to the first FOV 310. The images obtained by the camera modules 253 and 254 may also be used for head tracking. Figure 3As shown on the right side of , the range of the portion 330a where the first FOV 310a and the second FOV 320a overlap each other can be variously implemented, and there is no limitation.

[0060] According to an embodiment, the camera modules 255 and 256 may be arranged on a substantially lower portion of the housings 210b and 210c (or may be exposed through openings provided on the housings 210b and 210c). The upper portion corresponding to the camera modules 253 and 254 and the lower portion corresponding to the camera modules 255 and 256 are defined for a case where the user wears the electronic device 200. Those of ordinary skill in the art will understand that the portion relatively close to the ground is referred to as the lower portion and the portion relatively far from the ground is referred to as the upper portion only for convenience of explanation. The camera modules 255 and 256 may capture an image corresponding to a FOV based on at least one point on the housings 210b and 210c (e.g., a FOV corresponding to the substantially lower side when the user wears the electronic device 200). For example, referring to Figure 3 , the camera modules 255 and 256 may capture an image corresponding to the second FOV 320. The images obtained by the camera modules 255 and 256 may be used to identify and / or track a subject corresponding to the second FOV 320. For example, the images obtained by the camera modules 255 and 256 may be used to identify and / or track a subject that is relatively lower than a portion corresponding to the user's head when the user wears the electronic device 200 (for example, the subject is the user's hand), but the present disclosure is not limited thereto.

[0061] like Figure 3 As shown in , the arrangement positions and / or arrangement directions of the camera modules 253 and 254 and the camera modules 255 and 256 may be determined so that there is a portion 330 where the first FOV 310 and the second FOV 320 overlap each other, but these correspond only to examples. The arrangement positions and / or arrangement directions of the camera modules 253 and 254 and the camera modules 255 and 256 may be determined so that there is no portion where the first FOV 310 and the second FOV 320 overlap each other, and there is no limitation on whether the FOVs 310 and 320 overlap each other.

[0062] According to an embodiment, the electronic device 200 can recognize and / or track a subject by using at least one image captured by the camera modules 253, 254, 255 and 256. The electronic device 200 can perform an operation recognized based on the result of the recognition and / or tracking, and can, for example, provide a visual object at a position corresponding to the subject. However, there is no limitation on the operation. For example, if the electronic device 200 provides a virtual keyboard, the key specified on the virtual keyboard can be recognized based on the result of tracking the user's hand. The operation corresponding to the recognition and / or tracking result can be, for example, performed only by the electronic device 200, but this corresponds to an example. The operation can be based on an external electronic device (e.g., Figure 1 It is executed by collaboration between the external electronic device 102, the electronic device 103 and / or the server 108) and the electronic device 200.

[0063] According to an embodiment, camera modules 253, 254, 255 and 256 are used for head tracking, hand detection, hand tracking and / or spatial recognition using 3DoF or 6DoF, may be global shutter (GS) cameras, but there is no limitation, and may also be implemented as rolling shutter (RS) cameras.

[0064] According to an embodiment, the camera modules 251 and 252 are eye tracking (ET) cameras, and the images captured by the camera modules 251 and 252 may be used to detect and / or track pupils. For example, by using the captured images, the position of the virtual image projected on the electronic device 200 may be determined to be located according to the direction in which the pupil of the wearer of the electronic device 200 is oriented. The camera modules 251 and 252 may be implemented as GS cameras to detect and / or track pupils, but there is no limitation.

[0065] According to an embodiment, the display module 240 may include, for example, a liquid crystal display (LCD), a digital micromirror device (DMD), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED). Although not shown, if the display module 240 is configured by one of a liquid crystal display, a digital micromirror device, or a liquid crystal on silicon, the electronic device 200 may include a light source that emits light to the screen output area of ​​the display module 240. In another embodiment, if the display module 240 is capable of autonomously generating light, such as by one of an organic light emitting diode or a micro LED, the electronic device 200 may provide a user with a virtual image with good quality without including a separate light source. In an embodiment, if the display module 240 is implemented by an organic light emitting diode or a micro LED, a light source is not required, so a weight reduction of the electronic device 200 can be achieved. The electronic device 200 may include the display module 240, the first transparent member 201, and / or the second transparent member 202, and the user may use the electronic device 200 by wearing it on the face. The first transparent member 201 and / or the second transparent member 202 may be made of a glass plate, a plastic plate or a polymer, and may be made transparent or translucent. The optical waveguide may transmit the light generated from the display module 240 to the user's eyes. The optical waveguide may be made of glass, plastic or a polymer, and may include a nano pattern disposed on a portion of its inner surface or outer surface, for example, a grating structure having a polygonal shape or a curved surface shape. According to an embodiment, the light incident into one end of the waveguide tube may be provided to the user by propagating in the display waveguide via a nano pattern. In addition, the optical waveguide configured by a free-form prism may provide incident light to the user through a reflector. The optical waveguide may include at least one diffraction element (e.g., a diffraction optical element (DOE) or a holographic optical element (HOE)) or at least one of a reflective element (e.g., a reflector). The optical waveguide may guide the display light emitted from the light source unit to the user's eyes by using at least one diffraction element or a reflective element included in the optical waveguide. According to various embodiments, the diffraction element may include an input optical member / output optical member (not shown). For example, the input optical member may represent an input grating region, and the output optical member (not shown) may represent an output grating region. The input grating region may act as an input node, which diffracts (or reflects) light output from a light source unit (e.g., a micro LED) so as to be transmitted to a transparent member (e.g., a first transparent member 201 and a second transparent member 202) of a screen display portion. The output grating region may act as an exit, which diffracts (or reflects) light transmitted to a transparent member (e.g., a first transparent member 201 and a second transparent member 202) of a waveguide to a user's eyes. According to various embodiments, the reflective element may include a total reflection optical element or a total reflection waveguide for total internal reflection (TIR).For example, total reflection is a method of guiding light, and may mean that the incident angle allows the light (e.g., virtual image) input through the input grating area to be 100% reflected on one surface (e.g., a specific surface) of the waveguide, and thus 100% transmitted to the output grating area. In an embodiment, the light emitted from the display module 240 may be guided to the waveguide along the optical path by the input optical component. The light moving in the waveguide may be guided toward the user's eyes by the output optical component. The screen display portion may be determined based on the light emitted toward the eyes. According to an embodiment, the first transparent member 201 may be arranged to face the user's right eye, and the second transparent member 202 may be arranged to face the user's left eye. According to various embodiments, if the display module 240 is transparent, the display module may be arranged at a position facing the user's eyes and configure the screen display portion. The electronic device 200 may also include a lens. The lens may adjust the focus so that the picture output to the display module 240 is visible to the user's eyes. For example, the lens may be configured by a Fresnel lens, a Pancake lens, or a multi-channel lens.

[0066] According to an embodiment, the circuit board 241 may include components for driving the electronic device 200. For example, the circuit board 241 may include at least one integrated circuit chip, and the processor 120, a memory (eg, Figure 1 130 in the memory), a power management module (e.g., Figure 1 power management module / battery 189) or a communication module (e.g., Figure 1 At least one of the communication modules 190 in the electronic device may be provided in at least one integrated circuit chip. According to an embodiment, the circuit board 241 may be provided in a wearing member and / or a lens frame of the housing 210a, 210b, 210c. According to an embodiment, the circuit board 241 may be electrically connected to the battery 243 through a power transmission structure. According to an embodiment, the circuit board 241 may be connected to a flexible printed circuit board, and an electrical signal may be transmitted to an electronic component of the electronic device (e.g., a light output module, a camera module 251, 252, 253, 254, 255, and 256 and a light emitting unit) through the flexible printed circuit board. According to an embodiment, the circuit board 241 may include a circuit board having a plugboard.

[0067] According to an embodiment, the battery 243 may be electrically connected to components of the electronic device 200 through a power transmission structure, and may supply power to the components of the electronic device 200. According to an embodiment, at least a portion of the battery 243 may be provided in the wear member.

[0068] According to an embodiment, the speaker module 245 may convert an electrical signal into sound. At least a portion of the speaker module 245 according to an embodiment may be disposed in a wearing member and / or a lens frame of the housing 210a, 210b, 210c. According to an embodiment, the speaker module 245 may be disposed between the circuit board 241 and the battery 243 corresponding to the user's ear. The speaker module 245 according to an embodiment may send auditory information to the user through low-frequency vibrations of the user's skin and bones.

[0069] According to an embodiment, the microphone module 247 may convert sound into an electrical signal. According to an embodiment, the microphone module 247 may be provided on at least a portion of the lens frame and / or the wearing member.

[0070] According to an embodiment, the electronic device 200 may recognize the user's voice and / or external sound by using at least one microphone module 247. According to an embodiment, the electronic device 200 may distinguish between voice information and environmental noise based on voice information and / or additional information (e.g., low-frequency vibrations of the user's skin and bones) obtained through at least one microphone module. For example, the electronic device 200 may perform a function (e.g., noise cancellation) that enables clear recognition of the user's voice while reducing environmental noise.

[0071] According to an embodiment, the camera module 259 may include an infrared (IR) camera module (e.g., a time-of-flight (TOF) camera or a structured light camera). For example, the IR camera may work as at least a part of a sensor module (e.g., a sensor module or a lidar sensor) for sensing the distance to a subject. According to an embodiment, the electronic device 101 may also include a sensor module (e.g., a lidar sensor). For example, the sensor module may include at least one of a vertical cavity surface emitting laser (VCSEL), an infrared sensor, and / or a photodiode.

[0072] The illumination LED 242 can have various uses depending on its attachment location. For example, when tracking eye movement using the ET camera modules 251 and 252, the illumination LED 242 attached around the frame can be used as an auxiliary device to facilitate gaze detection, and mainly uses IR LEDs using infrared wavelengths. As another example, the illumination LED 242 can be attached to be adjacent to a camera module installed around a bridge connecting the frame or around a hinge 229 connecting the frame and the temples, so as to be used as a device to supplement the ambient brightness during camera image capture. The image capture camera module 260 can, for example, capture a relatively high quality image of the foreground of the electronic device 200.

[0073] Figure 4a and Figure 4b is a perspective view illustrating a configuration of an electronic device according to various embodiments of the present disclosure. Figure 4a and Figure 4b The electronic device 400 may be Figure 1 electronic device 200.

[0074] Reference Figure 4a and Figure 4b , the electronic device 400 according to the embodiment may include housings 410 and 420. Video see-through (VST) camera modules 411 and 412 for VST, multiple camera modules 413, 414, 415 and 416, and / or a depth sensor 417 (for example, this may be a lidar sensor, but is not limited thereto) may be arranged on the housing 410 (or the VST camera modules 411 and 412, multiple camera modules 413, 414, 415 and 416, and / or the depth sensor 417 may be exposed through an opening provided on the housing 410). For example, the camera modules 413 and 414 may capture an image corresponding to a relatively upper FOV (for example, Figure 3 For example, camera modules 415 and 416 may capture images corresponding to relatively lower FOVs (e.g., Figure 3 320 in the second FOV). As shown in FIG. Figure 2 As described, the arrangement positions and / or arrangement directions of the camera modules 413 and 414 and the camera modules 415 and 416 may be determined so that there is a portion 330 where the first FOV 310 and the second FOV 320 overlap each other, but these correspond only to examples. The arrangement positions and / or arrangement directions of the camera modules 413 and 414 and the camera modules 415 and 416 may be determined so that there is no portion where the first FOV 310 and the second FOV 320 overlap each other, and there is no limitation on whether the FOVs 310 and 320 overlap each other. The depth sensor 417 may be used to identify the distance to an object, such as ToF.

[0075] According to an embodiment, the display module and / or lenses 421 and 422 and / or facial tracking camera modules 425 and 426 may be disposed on the housing 420 (or the display module and / or lenses 421 and 422 and / or facial tracking camera modules 425 and 426 may be exposed through an opening provided on the housing 420). The facial tracking camera modules 425 and 426 may be used to detect and track the user's facial expressions.

[0076] Figure 2 The camera modules 255 and 256 and / or Figure 4a and Figure 4bThe camera modules 415 and 416 may be used to identify and / or track a subject that is relatively lower than the portion corresponding to the user's head when the user wears the electronic device 200 or 400 (for example, the subject is the user's hand). Therefore, when identification and / or tracking of the hand is not required, turning on the camera modules 255 and 256 and / or the camera modules 415 and 416 may cause a waste of power and / or resources. In addition, when identification and / or tracking of the hand is not required, controlling the camera modules 255 and 256 and / or the camera modules 415 and 416 to capture images at a relatively high frame rate may also cause a waste of power and / or resources. Therefore, when identification and / or tracking of the hand is not required (or this may be a different type of object and there is no limitation), the camera module used to capture images in the relatively lower FOV may be turned off (for example, Figure 2 The camera modules 255 and 256 and / or Figure 4a and Figure 4b camera modules 415 and 416), and / or their frame rates may be controlled, thereby preventing waste of power and / or resources.

[0077] Figure 5a is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure. Figure 5b describe Figure 5a Embodiment of the invention. Figure 5b is a diagram illustrating FOVs corresponding to a plurality of camera modules according to an embodiment of the present disclosure.

[0078] Reference Figure 5a According to an embodiment, the electronic device 101 (eg, the processor 120) may control at least one first camera module (eg, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The at least one first camera module can, for example, capture images with a first FOV (e.g., Figure 3 For example, referring to Figure 5b , the first sub-FOV 531 and the second sub-FOV 532 may correspond to Figure 3 The first FOV 310 in. Figure 3 3. The FOVs are shown viewed from the side, so the first FOV 310 is shown as 1. However, Figure 5b As shown in , it is possible to define multiple camera modules (for example, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The first sub-FOV 531 and the second sub-FOV 532 corresponding to the camera modules 413 and 414 of FIG. 4 are shown in FIG. Figure 2The camera module 253 may capture an image corresponding to the first sub-FOV 531, and Figure 2 The camera module 254 can capture an image corresponding to the second sub-FOV 532. Figure 2 The camera modules 253 and 254 are arranged at substantially the same height, and the first sub-FOV 531 and the second sub-FOV 532 can also be defined as being positioned at substantially the same height. Figure 4a and Figure 4b The camera module 413 may capture an image corresponding to the first sub-FOV 531, and Figure 4a and Figure 4b The camera module 414 can capture an image corresponding to the second sub-FOV 532. Figure 4a and Figure 4b The camera modules 413 and 414 are arranged at substantially the same height, and the first sub FOV 531 and the second sub FOV 532 may also be defined to be positioned at substantially the same height.

[0079] According to an embodiment, the electronic device 101 can use at least one first camera module (eg, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b For example, the electronic device 101 can use at least one first camera module (e.g., Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The electronic device 101 can perform 6DoF and / or SLAM-based spatial recognition with respect to the image frames captured by the first sub-FOV 531 and the second sub-FOV 532 by using at least one first camera module (e.g., Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4bThe electronic device 101 may control the display module 160 to display a visual object corresponding to the first subject, based on the image frames captured by the camera modules 413 and 414 of the electronic device 101 with respect to the first sub-FOV 531 and the second sub-FOV 532. For example, if the first subject is located in the first sub-FOV 531 and the second sub-FOV 532, the electronic device 101 may control the display module 160 to display a visual object corresponding to the first subject. For example, the visual object corresponding to the first subject may be determined based on the result of identifying and / or tracking the first subject, and the above operation may be performed by the electronic device 101 and / or the server 108. For example, the visual object corresponding to the first subject may be the associated information of the subject for the AR service. For example, the visual object corresponding to the subject may be the visual object specified for the first subject for the VR service. The provision of the visual object corresponds only to an example, and thus there is no limitation on the operation of the electronic device 101 associated with the result of identifying and / or tracking the first subject.

[0080] According to an embodiment, in at least one first camera module (eg, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b When the camera modules 413 and 414) are being controlled to obtain image frames, at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416 of the embodiment may be maintained in a closed state. The at least one second camera module may, for example, capture images related to the second FOV (e.g., Figure 3 For example, referring to Figure 5b , the third sub-FOV 533 and the fourth sub-FOV 534 may correspond to Figure 3 The second FOV 320 in. Figure 3 3. The FOV is shown viewed from the side, so the second FOV 320 is shown as 1. However, Figure 5b As shown in , it is possible to define multiple camera modules (for example, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The third sub-FOV 533 and the fourth sub-FOV 534 corresponding to the camera modules 415 and 416 of FIG. 4 are shown in FIG. Figure 2 The camera module 255 may capture an image corresponding to the third sub-FOV 533, and Figure 2 The camera module 256 can capture an image corresponding to the fourth sub-FOV 534. Figure 2The camera modules 255 and 256 are arranged at substantially the same height, and the third sub-FOV 533 and the fourth sub-FOV 534 may also be defined as being positioned at substantially the same height. Figure 4a and Figure 4b The camera module 415 may capture an image corresponding to the third sub-FOV 533, and Figure 4a and Figure 4b The camera module 416 can capture an image corresponding to the fourth sub-FOV 534. Figure 4a and Figure 4b The camera modules 415 and 416 are arranged at substantially the same height, and the third sub-FOV 533 and the fourth sub-FOV 534 may also be defined as being positioned at substantially the same height. As described above, the third sub-FOV 533 and the fourth sub-FOV 534 may be defined as being lower than the first sub-FOV 531 and the second sub-FOV 532. Therefore, in a case where it is not required to identify and / or track the subject at a relatively lower side, at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416) are turned off, thereby saving power and / or resources. Figure 5b The third sub-FOV 533 and the fourth sub-FOV 534 are shown as dashed lines, which may mean that at least one camera module (eg, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b camera modules 415 and 416).

[0081] According to an embodiment, the electronic device 101 may identify in operation 503 whether a user's hand (this is an example, and there is no limitation on the object to be identified) is placed within a specified area in the first FOV. The specified area may be defined as a position, for example, relatively lower than the first FOV. For example, referring to FIG. 5 , an image 560 corresponding to the first FOV is shown. The image 560 may be an image captured by at least one first camera. In an example, the image 560 may be an image captured by Figure 2 One of an image captured by camera module 253 and an image captured by camera module 254. In an example, image 560 may be a Figure 2 The image captured by the camera module 253 and the image generated by the image captured by the camera module 254. Figure 5b , defines a first region 561 in the image 560, which may correspond to a specified region of the FOV. The specified region in the first FOV may include, for example Figure 3At least a portion of the area 330 where the FOVs overlap with each other, but this corresponds to an example and there is no limitation. The first FOV and the second FOV may not overlap with each other, so a person of ordinary skill in the art will understand that the designated area is a relatively lower area of ​​the first FOV and is not limited thereto. The operation 503 of identifying whether the user's hand is placed within the designated area in the first FOV may be interpreted as an operation of identifying whether the user's hand is placed within the first area 561 of the image 560 (or may be achieved by replacing the operation). The determination of whether the user's hand is placed within the first area 561 may be performed by whether the entirety of the user's hand is located within the first area 561 or whether the user's hand is included in the first area 561 in a specific ratio or greater. However, a person of ordinary skill in the art will understand that the determination method is not limited. For example, as Figure 5b As shown in , the user's hand 551 may be located at a relatively upper side of the first sub-FOV 531 at a first point in time, and then move to a relatively lower side of the first sub-FOV 531 at a second point in time. The electronic device 101 may identify the user's hand based on an analysis of the image captured at the first point in time. The electronic device 101 may track the identified hand. The electronic device 101 may identify that the identified hand is placed in a designated area of ​​the first FOV based on the tracking result (for example, an object corresponding to the hand moves to the first area 561 of the image 560). Alternatively, the electronic device 101 may determine that the result of identifying the object included in the first area 561 of the image 560 indicates that the object is the user's hand, and there is no limitation on the method by which the electronic device 101 identifies whether the user's hand is located in the designated area of ​​the first FOV.

[0082] If it is determined that the user's hand is located within the specified area of ​​the first FOV (operation 503-Yes), according to an embodiment, in operation 505, the electronic device 101 may control at least one first camera module (eg, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The camera modules 413 and 414) are used to obtain image frames, and at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b 415 and 416) to control at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4bThe camera modules 415 and 416) are used to obtain image frames. The turning on and off of the camera modules may be represented by, for example, an active state or an inactive state. For example, the active state of the camera module may indicate a state in which the camera module obtains image frames in the turned-on state. For example, the inactive state of the camera module may indicate a state in which the camera module does not obtain image frames in the turned-on state. The turned-on state of the camera module in the present disclosure may be represented by an active state or replaced by an active state. The turned-off state of the camera module in the present disclosure may be represented by an inactive state or replaced by an inactive state. The user's hand is located within the specified area of ​​the first FOV, so there is a possibility that the user's hand may move to the second FOV. Therefore, it is necessary to turn on at least one second camera module (for example, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b This is because if the user's hand is on at least one of the second camera modules (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b If the user moves out of the first FOV while the camera modules 415 and 416 of the electronic device 101 are turned off, the tracking of the user's hand may fail. The electronic device 101 may use at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b In an example, the electronic device 101 may control at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416 of FIG. 4 capture image frames at a relatively small frame rate, and in this case, the image frames can be captured by at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The recognition is performed on the image frames captured by the camera modules 415 and 416. This will refer to Figure 6a The above examples are illustrative examples, and the electronic device 101 may also control at least one second camera module (eg, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416) capture image frames at a relatively high frame rate, and in this case, the image frames can be captured by using at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4bFor example, in the case of performing recognition and / or tracking of an image, the image frame captured by at least one first camera module (e.g., Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b and images captured by at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b Recognition and / or tracking can be performed for each of the images captured by the at least one first camera module (e.g., Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The image captured by the camera modules 413 and 414) is compared with the image captured by at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The image obtained by the camera module 415 and 416) is used to perform recognition and / or tracking. There is no limitation on the method of recognizing and / or tracking images captured by multiple camera modules.

[0083] Although not shown, at least one second camera module (eg, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b After the user's hand 551 is moved to the remaining area outside the designated area in the first FOV, the electronic device 101 can track the object corresponding to the user's hand 551 by using the image frame corresponding to the first FOV, and can recognize that the user's hand 551 has moved to the remaining area outside the designated area 561 in the image 560 based on the tracking result. In this case, the electronic device 101 can turn off the at least one second camera module (for example, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b For example, with respect to the hysteresis characteristic, at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4bThe camera modules 415 and 416 of the user's hand 551 may be configured to be turned off again when the object corresponding to the hand 551 moves out of the wider area including the first area 561. For example, even if the object corresponding to the user's hand 551 moves out of the first area 561, the electronic device 101 maintains at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b Based on the fact that no object is recognized to move back into the first area 561, the electronic device may turn off at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416), and the above operations are illustrative and thus there is no limitation.

[0084] Turn on at least one second camera module (eg, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b 415 and 416) and controls at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b Operation 505 of obtaining an image frame by using the camera modules 415 and 416 corresponds only to an example. In another embodiment, even before operation 503, at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b Alternatively to operation 505, the electronic device 101 may control at least one second camera module (eg, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b camera modules 415 and 416) to increase at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The frame rate of camera modules 415 and 416).

[0085] Figure 5c is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0086] Reference Figure 5c According to an embodiment, in operation 551, the electronic device 101 (eg, the processor 120) may control at least one first camera module (eg, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b In operation 553, the electronic device 101 may identify whether the user's hand is placed in the designated area in the first FOV. When it is identified that the user's hand is placed in the designated area of ​​the first FOV (operation 553-Yes), in operation 555, the electronic device 101 may identify whether an additional condition for turning on the second camera module is satisfied. If it is identified that the additional condition is satisfied (operation 555-Yes), in operation 557, the electronic device 101 may control at least one first camera module (e.g., Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The camera modules 413 and 414) are used to obtain image frames, and at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b camera modules 415 and 416) to control a second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b camera modules 415 and 416) to obtain image frames.

[0087] In an example, the electronic device 101 may identify whether an application requiring recognition and / or tracking of a specific type of object (e.g., an object corresponding to a hand) is being run as whether the additional condition is satisfied. If the application requiring recognition and / or tracking of an object (e.g., an object corresponding to a hand) is not being run, it may not be necessary to turn on at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b In this case, even if it is recognized that the hand is placed in a specific area of ​​the first FOV, the electronic device 101 may maintain at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416 are in a closed state.

[0088] In an example, the electronic device 101 may identify whether a function associated with recognition and / or tracking of a specific type of object (e.g., an object corresponding to a hand) is called (e.g., this corresponds to whether a visual object associated with recognition and / or tracking is displayed, but there is no limitation), as whether an additional condition is met. For example, as a function associated with recognition and / or tracking, a virtual keyboard based on hand recognition and / or tracking operations may be provided. For example, as a function associated with recognition and / or tracking, a control function based on recognition and / or tracking of a hand may be called (run), but there is no limitation on the called function. If the function associated with recognition and / or tracking of a specific type of object (e.g., an object corresponding to a hand) is not run, it may not be necessary to turn on at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b In this case, even if it is recognized that the hand is placed in a specific area of ​​the first FOV, the electronic device 101 may maintain at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416 are in a closed state.

[0089] In an example, the electronic device 101 may identify whether the position of a virtual object (e.g., a virtual keyboard) associated with recognition and / or tracking of a specific type of object (e.g., an object corresponding to a hand) is included in a designated area as whether an additional condition is satisfied. For example, if the position of a virtual object (e.g., a virtual keyboard) associated with recognition and / or tracking is located on the relatively upper side, the user is likely to place the hand on the relatively upper side in order to manipulate the virtual object located on the upper side, and is unlikely to place the hand on the relatively lower side. If the position of a virtual object (e.g., a virtual keyboard) associated with recognition and / or tracking of a specific type of object (e.g., an object corresponding to a hand) is not included in a designated area (e.g., an area corresponding to at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b 415 and 416)), it may not be necessary to turn on at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4bIn this case, even if it is recognized that the hand is placed in a specific area of ​​the first FOV, the electronic device 101 may maintain at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The above additional conditions correspond only to examples and may be configured in association with the state of the electronic device 101 (such as whether the state is a low power state or whether the state is an over temperature state), and those skilled in the art will understand that there is no limitation. In addition, those skilled in the art will understand that if a hand is recognized to be placed in a specific area of ​​the first FOV, at least one of the second camera modules (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416) may also be configured to turn on immediately, regardless of whether additional conditions are met.

[0090] Figure 6a is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure. Figure 6b describe Figure 6a Embodiment of the invention. Figure 6b is a diagram illustrating adjusting a frame rate of at least one first camera module according to an embodiment of the present disclosure.

[0091] Reference Figure 6a and Figure 6b According to an embodiment, in operation 601, the electronic device 101 (eg, the processor 120) may control at least one first camera module (eg, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The camera modules 413 and 414) obtain image frames 621, 622, 623 and 624 at a first frame rate. For ease of explanation, Figure 6b 621, 622, 623, and 624 having a first frame rate corresponding to one camera module are shown in FIG. However, a person of ordinary skill in the art will appreciate that the camera module corresponding to the first sub-FOV 531 (e.g., Figure 2 Camera module 253 or Figure 4a and Figure 4b The camera module 413 of the second sub-FOV 532 obtains an image frame, and the camera module corresponding to the second sub-FOV 532 (e.g., Figure 2 Camera module 254 or Figure 4a and Figure 4bThe camera module 414 of the embodiment of the present invention obtains image frames. The first frame rate may be a relatively low value (e.g., 30fps). The first frame rate having a relatively low value may be, for example, a value for 6DoF, SLAM, and / or head tracking, but there is no limitation. The power and / or resources required for the camera module to obtain image frames at the first frame rate having a relatively low value may be relatively small. For example, the first frame rate may be configured to be a relatively low value but sufficient to achieve recognition of a specific object (e.g., an object corresponding to a hand).

[0092] According to an embodiment, in operation 603, the electronic device 101 may identify whether the user's hand is placed within the first FOV. The electronic device 101 may identify whether the hand is placed based on the recognition results of the image frames 621, 622, 623, and 624. In order to prevent the increase in frame rate caused by unconscious hand recognition, the electronic device 101 may be configured to perform subsequent operations based on the recognition of a special shape of the hand (e.g., the left palm and / or the back of the right hand, but there is no limitation) and / or a specified gesture associated with the hand. As described above, for example, the first frame rate may be configured to be a relatively low value but sufficient to achieve recognition of a specific object (e.g., an object corresponding to the hand), and therefore, whether the hand is placed may be identified based on the recognition results of the image frames 621, 622, 623, and 624. If it is identified that the user's hand 551 is placed within the first FOV (operation 603-yes), then in operation 605, the electronic device 101 may control at least one first camera module (e.g., Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4bThe electronic device 101 may obtain image frames 641, 642, 643, 644, 645, 646, 647, 648, 649, 650, 651, and 652 at a second frame rate (e.g., camera modules 413 and 414 of the electronic device 101). In operation 607, the electronic device 101 may recognize the movement of the hand based on the image frames 641, 642, 643, 644, 645, 646, 647, 648, 649, 650, 651, and 652. The second frame rate may be a value (e.g., 90fps) relatively higher than the first frame rate. The power and / or resources required for the camera module to obtain image frames at a first frame rate having a relatively low value may be relatively small. However, in the case of obtaining image frames at the first frame rate, there may be a higher probability of object tracking failure. When the hand moves at a relatively low speed, the user's hand may be tracked even at the first frame rate. However, the case where the user's hand moves at a relatively high speed may have a probability that hand tracking may fail based on the first frame rate. Therefore, in order to reduce the possibility of tracking failure, the image frame can be obtained at a relatively high second frame rate. The second frame rate can be, for example, a value that enables tracking of the opponent, but there is no limitation. Therefore, after recognizing that the hand 551 is placed, the electronic device 101 can use at least one first camera module (e.g., Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The frame rate of the camera modules 413 and 414) is increased from the first frame rate to the second frame rate. The electronic device 101 can track the hand 551 by using the image frames 641, 642, 643, 644, 645, 646, 647, 648, 649, 650, 651 and 652 obtained at the second frame rate of a relatively high value. The electronic device 101 can perform an operation based on the tracking result. Figure 6b In an embodiment, at least one second camera module (eg, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416 of the first camera module 415 and 416 are shown as being turned off, but this is an example and a person skilled in the art will understand that they can be turned on in certain circumstances. Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b 413 and 414) and at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4bIn a state where both the camera modules 415 and 416 of the electronic device 101 are turned on, image frames can be obtained at a first frame rate. In this case, based on recognizing that the user's hand 551 is placed within the first FOV, in order to track the hand 551, the electronic device 101 can use at least one first camera module (e.g., Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b If the hand 551 is detected in an area (e.g., area 561) where the first sub-FOV 531 and the third sub-FOV 533 overlap each other, the electronic device 101 may change the frame rate of at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The frame rate of camera modules 415 and 416) is changed from a first frame rate to a second frame rate.

[0093] In at least one second camera module (eg, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b When the camera modules 415 and 416 of the electronic device 101 according to the embodiment obtain image frames at a first frame rate, the electronic device 101 according to the embodiment may detect the user's hand 551 in the second FOV and at least one second camera module (for example, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The frame rate of the camera modules 415 and 416) of the electronic device 101 is changed from the first frame rate to the second frame rate. The electronic device 101 may be based on the at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416) track the hand 551 using image frames obtained at a second frame rate.

[0094] The electronic device 101 according to the embodiment may control at least one first camera module (eg, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b 413 and 414) and at least one second camera module (e.g., Figure 1 Camera modules 255 and 256 or Figure 4a and Figure 4b For example, if a hand 551 is detected in the first area 561, the electronic device 101 may use at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b Alternatively, in another example, if the hand 551 is detected in the first area 561, the electronic device 101 may be configured to maintain at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b In this case, the electronic device 101 may recognize that the hand is placed in a designated area of ​​the second FOV (or the hand has moved out of the first FOV), and maintain the frame rate of at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The frame rate of the camera modules 415 and 416) is changed from the first frame rate to the second frame rate. The electronic device 101 can recognize that the hand is placed in the specified area of ​​the second FOV (or the hand has moved out of the first FOV), and control at least one of the first camera modules (e.g., Figure 2 The camera modules 253 and 254 of the first FOV are operated at a first frame rate. For example, before recognizing that a hand is placed in a designated area of ​​the second FOV (or that the hand has moved out of the first FOV), at least one of the first camera modules (e.g., Figure 2 The frame rate of the camera modules 253 and 254) may be the second frame rate. In this case, based on recognizing that the hand is placed in the designated area of ​​the second FOV (or the hand has moved out of the first FOV), at least one of the first camera modules (e.g., Figure 2 The frame rate of the camera modules 253 and 254) can be changed from the second frame rate to the first frame rate.

[0095] Figure 6c is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0096] According to an embodiment, in operation 671, the electronic device 101 (eg, the processor 120) may control at least one first camera module (eg, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4bThe electronic device 101 may control at least one first camera module (e.g., camera modules 413 and 414) to obtain image frames at a first frame rate. In operation 673, the electronic device 101 may identify whether the user's hand is placed within the first FOV. If it is identified that the user's hand is placed within the first FOV (operation 673-Yes), in operation 675, the electronic device 101 may identify whether an additional condition for changing the frame rate is satisfied. If it is identified that the additional condition is satisfied (operation 673-Yes), in operation 677, the electronic device 101 may control at least one first camera module (e.g., Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The electronic device 101 may further include camera modules 413 and 414 of the first camera module 413 and 414 to obtain image frames at a second frame rate. In operation 679, the electronic device 101 may recognize the movement of the hand based on the image frames. For example, the additional condition may be a call to a function of tracking the hand, and there is no limitation. For example, if a virtual object for tracking the hand (e.g., a virtual keyboard) is displayed, the electronic device 101 may change the frame rate as an additional condition. If the virtual object for tracking the hand (e.g., a virtual keyboard) is not displayed, the electronic device 101 may maintain the at least one first camera module (e.g., a virtual keyboard) even when the hand has been recognized. Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The frame rate of camera modules 413 and 414).

[0097] Figure 7a is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure. Figure 7b describe Figure 7a Embodiment of the invention. Figure 7b is a diagram illustrating adjusting a frame rate of at least one second camera module according to an embodiment of the present disclosure.

[0098] According to an embodiment, in operation 701, the electronic device 101 (eg, the processor 120) may control at least one first camera module (eg, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The electronic device 101 may control at least one first camera module (e.g., camera modules 413 and 414) to obtain image frames at a first frame rate. In operation 703, the electronic device 101 may identify whether the user's hand is placed within the first FOV. If it is identified that the user's hand is placed within the first FOV (operation 703-Yes), in operation 705, the electronic device 101 may control at least one first camera module (e.g., Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The camera modules 413 and 414) obtain image frames at a second frame rate greater than the first frame rate, and turn on at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b camera modules 415 and 416) to control a second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416) acquire at a first frame rate Figure 7b At least one image frame 721, 722, 723, and 724 of the first frame rate may be captured. For example, the first frame rate may be a relatively low value (e.g., 30 fps), and the second frame rate may be a relatively high value (e.g., 90 fps). The power and / or resources required by the camera module to obtain the image frames at the first frame rate having a relatively low value may be relatively small. For example, the first frame rate may be configured to be a relatively low value but sufficient to achieve recognition of a specific object (e.g., an object corresponding to a hand).

[0099] According to an embodiment, in operation 707, the electronic device 101 may generate a plurality of images based on images from at least one first camera module (eg, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b 413 and 414) and / or image frames from at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b 415 and 416), and identifying a change in at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The condition for changing the frame rate may correspond to, for example, whether the user's hand is detected in a specified area in the second FOV (e.g., a relatively lower area in the second FOV, but not limited thereto), such as Figure 7b Based on the condition for changing the frame rate being satisfied, in operation 709, the electronic device 101 may control at least one first camera module (eg, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The camera modules 413 and 414) are used to obtain image frames, and control at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416) acquire at a second frame rate Figure 7bAt least one image frame 741, 742, 743, 744, 745, 746, 747, 748, 749, 750, 751 and 752. Alternatively, based on the detection of the user's hand in the designated area of ​​the second FOV and the satisfaction of the additional condition of changing the frame rate, the electronic device 101 may use at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The electronic device 101 may adjust the frame rate of the camera modules 415 and 416 of the first camera module 415 and 416 from the first frame rate to the second frame rate. Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The at least one second camera module (e.g., camera modules 415 and 416) tracks the user's hand 551 with image frames obtained at a second frame rate. Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416 of the embodiment of the present invention can obtain image frames at a relatively low first frame rate at the turn-on time point, and then when the user's hand 551 has moved to the relatively lower side, at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The frame rate of camera modules 415 and 416) can be increased to a second frame rate.

[0100] According to an embodiment, based on at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The electronic device 101 may adjust the frame rate of at least one first camera module (eg, camera modules 415 and 416) to a second frame rate. Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The frame rate of the camera modules 413 and 414 of the electronic device 101 is changed from the second frame rate to the first frame rate. The electronic device 101 can change the frame rate of the camera modules 413 and 414 of the electronic device 101 by using at least one second camera module (for example, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The image frames obtained by the camera modules 415 and 416) are used to track the target object, so at least one first camera module (for example, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4bThe frame rate of the camera modules 413 and 414) of the electronic device 101 is adjusted from the second frame rate to the first frame rate. The above description is an example, and the electronic device 101 may adjust the frame rate of at least one first camera module (e.g., Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The frame rate of camera modules 413 and 414) is maintained at the second frame rate.

[0101] Figure 8 is a diagram illustrating adjustment of the FOV and frame rate of an electronic device of AR glasses according to an embodiment of the present disclosure.

[0102] According to an embodiment, the electronic device 200 of the AR glasses may recognize at a first time point that the user's hand 551 is located in the first sub-FOV 531. For example, the electronic device 200 may control at least one first camera module (e.g., Figure 2 The electronic device 200 may obtain image frames at a first frame rate based on at least one first camera module (e.g., Figure 2 The electronic device 200 can use the image frame obtained by the camera modules 253 and 254 of the electronic device 200 to recognize that the user's hand 551 is located in the first sub-FOV 531. Based on recognizing that the hand 551 is in the first sub-FOV 531 as a part of the first FOV, the electronic device 200 can use at least one first camera module (for example, Figure 2 The frame rate of the camera modules 253 and 254 of the electronic device 200 is adjusted from the first frame rate (e.g., 30 fps) to the second frame rate (e.g., 90 fps). The electronic device 200 can track the hand 551 based on the image frames captured at the second frame rate. The at least one second camera module (e.g., Figure 2 The camera modules 255 and 256 in the image processing unit may be maintained in a closed state.

[0103] At the second time point, the user's hand 551 may move to an area where the first sub-FOV 531 and the third sub-FOV 533 overlap each other (e.g., a designated area of ​​the first FOV). The electronic device 200 may turn on at least one second camera module (e.g., Figure 2 The electronic device 200 may control at least one second camera module (eg, Figure 2 The camera modules 255 and 256 in the image processing apparatus obtain image frames at a first frame rate (e.g., 30fps).

[0104] At a third time point, the user's hand 551 may move to a designated area of ​​the third sub-FOV 533. For example, the electronic device 200 may capture the image based on at least one first camera module (e.g., Figure 2The electronic device 200 can detect that the hand 551 has moved out of the designated area of ​​the first FOV based on the image frame obtained by at least one second camera module (e.g., Figure 2 The electronic device 200 can recognize that the user's hand 551 is located in the specified area of ​​the second FOV based on the image frame obtained by the camera modules 255 and 256 in the second FOV. For example, the electronic device 200 can recognize that the user's hand 551 is located in the specified area of ​​the second FOV based on the image frame obtained by synthesizing the at least one first camera module (for example, Figure 2 The image obtained by the camera modules 253 and 254) and the image obtained by at least one second camera module (e.g., Figure 2 The electronic device 200 can obtain an image obtained by using the image obtained by the camera modules 255 and 256 in the third sub-FOV to identify that the user's hand 551 is located in the designated area of ​​the second FOV, and there is no limitation on the method of identifying whether the user's hand 551 is located in the designated area of ​​the second FOV. Based on the user's hand 551 being placed in the designated area of ​​the third sub-FOV 533, the electronic device 200 can use at least one first camera module (e.g., Figure 2 The frame rate of the camera modules 253 and 254) is adjusted from the second frame rate to the first frame rate, and at least one second camera module (eg, Figure 2 The electronic device 101 may adjust the frame rate of the camera modules 255 and 256 in the electronic device 101 from the first frame rate to the second frame rate based on the frame rate of the camera modules 255 and 256 in the electronic device 101. Figure 2 The hand 551 is tracked using image frames obtained by the camera modules 255 and 256 in FIG.

[0105] like Figure 8As shown in , the hand 551 may be located in the second sub-FOV 532 at a first time point, move to a designated area of ​​the second sub-FOV 532 at a second time point, and then move to the fourth sub-FOV 534 at a third time point. Alternatively, although not shown, the hand 551 may be located in the second sub-FOV 532 at a first time point, move to a designated area of ​​the second sub-FOV 532 at a second time point, and then move to the third sub-FOV 533 at a third time point. Alternatively, although not shown, the hand 551 may be located in the second sub-FOV 532 at a first time point, move to a designated area of ​​the first sub-FOV 531 at a second time point, and then move to the third sub-FOV 533 at a third time point. As described above, the second sub-FOV 532 may correspond to the first FOV together with the first sub-FOV 531, and the fourth sub-FOV 534 may correspond to the second FOV together with the third sub-FOV 533. The operation based on the recognition and / or tracking for the second sub-FOV 532 may be at least partially the same as the operation based on the recognition and / or tracking for the first sub-FOV 531. The operation based on the recognition and / or tracking of a designated area (e.g., a relatively lower area) for the second sub-FOV 532 may be at least partially the same as the operation based on the recognition and / or tracking of the designated area (e.g., a relatively lower area) for the first sub-FOV 531. The operation based on the recognition and / or tracking for the fourth sub-FOV 534 may be at least partially the same as the operation based on the recognition and / or tracking for the second sub-FOV 532. The operation based on the recognition and / or tracking of a designated area (e.g., a relatively upper area) for the fourth sub-FOV 534 may be at least partially the same as the operation based on the recognition and / or tracking of the designated area (e.g., a relatively upper area) for the second sub-FOV 532. Therefore, even in the case where the hand 551 moves along “the second sub-FOV 532→the designated area of ​​the second sub-FOV 532→the fourth sub-FOV 534”, “the second sub-FOV 532→the designated area of ​​the second sub-FOV 532→the third sub-FOV 533”, or “the second sub-FOV 532→the designated area of ​​the first sub-FOV 531→the third sub-FOV 533”, the electronic device 101 can execute the above reference operation. Figure 8 The operations corresponding to the first time point, the second time point, and the third time point are described. A person of ordinary skill in the art will understand that the identity between the operations corresponding to the first sub-FOV 531 and the second sub-FOV 532 and / or the identity between the operations corresponding to the third sub-FOV 533 and the fourth sub-FOV 534 can be applied to other embodiments as well as this embodiment.

[0106] Although not shown, reference Figure 8, the operation of the electronic device 101 in the case where the hand 551 moves from the relatively upper side to the relatively lower side has been described, but this corresponds to an example only. For example, when the user wears the electronic device 101 on the head, the hand 551 to be tracked is likely to be located relatively lower than the head. In the case where the hand 551 performs a specific operation (for example, typing on a virtual keyboard) while being located on the relatively lower side, it may be necessary to perform tracking of the hand 551 at a relatively high frame rate. Therefore, the operation of the electronic device 101 in the case where the hand 551 is located on the relatively lower side has been described.

[0107] However, operation may also be possible in the case where the target to be tracked is likely to be located relatively higher than the location of the wearable electronic device 101. For example, with respect to a specific object, the electronic device 101 may adjust at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b For example, the electronic device 101 may turn on at least one first camera module (e.g., the first camera module 415 and the first camera module 416) based on the fact that the specific object has moved to a specified area (e.g., a relatively upper area) of the second FOV at a second time point. Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b For example, the electronic device 101 may adjust at least one first camera module (e.g., the camera modules 413 and 414) based on the fact that the specific object has moved to a specified area (e.g., a relatively upper area) of the first FOV at a third time point (or the specific object has moved out of the second FOV). Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The frame rate of camera modules 413 and 414).

[0108] Alternatively, an operation may also be possible in a case where the target to be tracked is likely to be located relatively to the right of the position of the wearable electronic device 101. For example, with respect to a specific object, the electronic device 101 may adjust at least one left camera module (e.g., Figure 2 Camera modules 253 and 255 or Figure 4a and Figure 4b For example, the electronic device 101 may turn on at least one right camera module (e.g., the camera module 413 and the camera module 415) based on the specific object having moved to a specified area of ​​the left FOV (e.g., relative to the right area) at the second time point. Figure 2 The camera modules 254 and 256 or Figure 4a and Figure 4bFor example, the electronic device 101 may adjust at least one first camera module (e.g., the camera modules 414 and 416) based on the fact that the specific object has moved to a specified area (e.g., a relatively right area) of the right FOV at a third time point (or the specific object has moved out of the left FOV). Figure 2 The camera modules 254 and 256 or Figure 4a and Figure 4b Alternatively, the electronic device 101 may also perform an operation of moving a specific object from right to left, and the operation may be an operation of moving from left to right, wherein the left camera module and the right camera module are replaced by each other.

[0109] As described above, there is no restriction on the moving direction of the object. In an embodiment, the electronic device 101 may turn on the camera module and / or adjust the frame rate according to the object recognition result. For example, the electronic device 101 may Figure 8 The strategies for turning on the camera module and / or adjusting the frame rate are applied to a target to be tracked that is very likely to be located at the relatively lower side, such as a hand. For example, as described above, the electronic device 101 may apply the various strategies for turning on the camera module and / or adjusting the frame rate described above to a target to be tracked that is very likely to be located at the relatively upper side, or to a target to be tracked that is very likely to be located at the relatively right side.

[0110] Turning on the camera module and / or adjusting the frame rate can be performed based on the movement of the target to be tracked from the relatively upper side to the relatively lower side as described above, with no limitation on its moving direction, and a person skilled in the art will understand that it can also be applied to different embodiments of the present disclosure as well as this embodiment.

[0111] Fig. 9 is a diagram illustrating adjusting the FOV and frame rate of a VST device in VR mode according to an embodiment of the present disclosure.

[0112] According to an embodiment, the electronic device 400 of the VST device may operate in VR mode or VST mode. In VR mode, the electronic device 400 may display images corresponding to both eyes, and the user may watch the images corresponding to both eyes to experience the VR environment. The content provided in VR mode may be content that is not related to the user's surroundings. In contrast, in VST mode, the content provided to the user may include at least a portion of the captured image of the user's surroundings (or an object processed based on at least a portion thereof and / or an object corresponding to at least a portion thereof). For example, the electronic device 400 may display an image associated with the surroundings and captured by the VST camera modules 411 and 412 (or an object processed based on at least a portion thereof and / or an object corresponding to at least a portion thereof) as at least a portion of the VST content. Therefore, the user may recognize at least a portion of the image associated with the surroundings and captured by the VST camera modules 411 and 412. For example, the VST content may be generated by mixing content for the VR environment and at least a portion of the image captured by the VST camera modules 411 and 412. For example, the VST content may be generated by mixing the content for the VR environment and the result of processing at least a portion of the image captured by the VST camera modules 411 and 412 (or a corresponding object). For example, the VST content may be generated based on at least a portion of the image captured by the VST camera modules 411 and 412. For example, the VST content may be generated based on the result of processing at least a portion of the image captured by the VST camera modules 411 and 412 (or a corresponding object).

[0113] exist Fig. 9 In the embodiment of FIG. 4 , an electronic device 400 running a VR mode is used. As described above, the content corresponding to the VR mode may not be related to, for example, the surrounding environment of the electronic device 400, and thus the images captured by the VST camera modules 411 and 412 may not be needed in the VR mode. Fig. 9 As shown in , the VST camera modules 411 and 412 can remain in a closed state in the VR mode regardless of the position of the hand 551 and / or whether the hand is recognized.

[0114] According to an embodiment, the electronic device 400 may recognize at a first time point that the user's hand 551 is located in the first sub-FOV 531. For example, the electronic device 400 may control at least one first camera module (eg, Figure 4a and Figure 4b The electronic device 400 may obtain image frames at a first frame rate based on at least one first camera module (e.g., Figure 4a and Figure 4bThe electronic device 400 can use at least one first camera module (e.g., a first camera module 413 and a second camera module 414) to obtain an image frame obtained by the camera modules 413 and 414 of the electronic device 400, and recognize that the user's hand 551 is located in the first sub-FOV 531. The electronic device 400 can use at least one first camera module (e.g., a first camera module 413 and a second camera module 414) to obtain an image frame obtained by the camera modules 413 and 414 of the electronic device 400, and recognize that the user's hand 551 is located in the first sub-FOV 531 based on recognizing the hand 551 in the first sub-FOV 531 as a part of the first FOV. Figure 4a and Figure 4b The frame rate of the camera modules 413 and 414 of the electronic device 400 is adjusted from the first frame rate (e.g., 30 fps) to the second frame rate (e.g., 90 fps). The electronic device 400 may track the hand 551 based on the image frames captured at the second frame rate. The at least one second camera module (e.g., Figure 4a and Figure 4b The camera modules 415 and 416 in the VR mode can be maintained in an off state. The electronic device 400 can perform operations based on the tracking results, such as manipulating objects included in the VR content, and its operation is not limited. As described above, in the VR mode, the VST camera modules 411 and 412 can be maintained in an off state.

[0115] At the second time point, the user's hand 551 may move to an area where the first sub-FOV 531 and the third sub-FOV 533 overlap each other (e.g., a designated area of ​​the first FOV). The electronic device 400 may turn on at least one second camera module (e.g., Figure 4a and Figure 4b The electronic device 400 may control at least one second camera module (eg, Figure 4a and Figure 4b The camera modules 415 and 416 in the VR mode obtain image frames at a first frame rate (e.g., 30fps). As described above, in the VR mode, the VST camera modules 411 and 412 can maintain a closed state.

[0116] At a third time point, the user's hand 551 may move to a designated area of ​​the third sub-FOV 533. For example, the electronic device 400 may capture the image based on at least one first camera module (e.g., Figure 4a and Figure 4b Based on the user's hand 551 being placed in the designated area of ​​the third sub-FOV 533, the electronic device 400 may use at least one first camera module (e.g., Figure 4a and Figure 4b The frame rate of the camera modules 413 and 414) is adjusted from the second frame rate to the first frame rate, and at least one second camera module (eg, Figure 4a and Figure 4bThe electronic device 101 may adjust the frame rate of the camera modules 415 and 416 in the electronic device 101 from the first frame rate to the second frame rate based on the frame rate of the camera modules 415 and 416 in the electronic device 101. Figure 4a and Figure 4b The hand 551 is tracked using image frames obtained by the camera modules 415 and 416 in the VR mode. As described above, in the VR mode, the VST camera modules 411 and 412 can be kept turned off.

[0117] Fig.10 is a diagram illustrating adjustment of the FOV and frame rate of a VST device in a VST mode according to an embodiment of the present disclosure.

[0118] exist Fig.10 In the embodiment of the present invention, the electronic device 400 can operate in VST mode. As described above, in VST mode, the content to be provided to the user may include at least a portion of the captured image of the user's surroundings (or an object processed based on at least a portion thereof and / or an object corresponding to at least a portion thereof). Therefore, the electronic device 400 can display the image associated with the surroundings and captured by the VST camera modules 411 and 412 (or an object processed based on at least a portion thereof and / or an object corresponding to at least a portion thereof) as at least a portion of the VST content. Fig.10 As shown, the fifth sub-FOV 535 corresponding to the VST camera module 411 may include most of the first sub-FOV 531, and the sixth sub-FOV 536 corresponding to the VST camera module 412 may include most of the second sub-FOV 532. Therefore, the electronic device 101 can perform 6DoF and / or SLAM and / or recognize and / or track an object by using image frames obtained by the VST camera modules 411 and 412. When the VST camera modules 411 and 412 are turned on and obtain image frames, at least one of the first camera modules (e.g., Figure 4a and Figure 4b For example, in VR mode, at least one first camera module (e.g., Figure 4a and Figure 4b The image frames obtained by the camera modules 413 and 414 in the VST mode can be used to perform 6DoF and / or SLAM and / or identify and / or track objects. For example, in the VST mode, the image frames obtained by the VST camera modules 411 and 412 can be used to perform 6DoF and / or SLAM and / or identify and / or track objects. Fig.10 As shown in , at least one first camera module (eg, Figure 4a and Figure 4b 413 and 414) and at least one second camera module (e.g., Figure 4a and Figure 4b The camera modules 415 and 416 in FIG. 4 are maintained turned off.

[0119] According to an embodiment, the electronic device 400 may recognize that the user's hand 551 is located in the fifth sub-FOV 535 at a first point in time. For example, the electronic device 400 may control the VST camera modules 411 and 412 to obtain image frames at a first frame rate. The electronic device 400 may recognize that the user's hand 551 is located in the fifth sub-FOV 535 based on the image frames obtained by the VST camera modules 411 and 412. As described above, the fifth sub-FOV 535 includes at least a portion of the first sub-FOV 531, and the sixth sub-FOV 536 includes at least a portion of the second sub-FOV 532, so the fifth sub-FOV 535 and the sixth sub-FOV 536 may correspond to the first FOV. The electronic device 400 may use the image frames obtained by the VST camera modules 411 and 412 as images corresponding to the first FOV. For example, the electronic device 101 may perform operations such as turning on the camera module and / or controlling the frame rate based on the images obtained by the VST camera modules 411 and 412. Figure 6a In an embodiment, this is based on at least one first camera module (e.g., Figure 4a and Figure 4b For example, at least one first camera module (e.g., Figure 4a and Figure 4b The camera modules 413 and 414 in the VST camera modules 411 and 412 are in a turned-off state, so the electronic device 101 can operate based on the images obtained by the VST camera modules 411 and 412.

[0120] According to an embodiment, the electronic device 400 may adjust the frame rate of the VST camera modules 411 and 412 from the first frame rate (e.g., 30 fps) to the second frame rate (e.g., 90 fps) based on recognizing the hand 551 in the fifth sub-FOV 535. The electronic device 400 may track the hand 551 based on the image frames captured at the second frame rate. The at least one second camera module (e.g., Figure 4a and Figure 4b The camera modules 415 and 416 in the VST mode may be maintained in an off state. The electronic device 400 may perform operations based on the tracking results, such as manipulating objects included in the VR content, and its operation is not limited. As described above, in the VST mode, at least one first camera module (e.g., Figure 4a and Figure 4b 413 and 414) and at least one second camera module (e.g., Figure 4a and Figure 4b The camera modules 415 and 416 in the image processing unit may be maintained in a closed state.

[0121] At the second time point, the user's hand 551 may move to an area where the fifth sub-FOV 535 and the third sub-FOV 533 overlap each other (e.g., a designated area of ​​the first FOV). The electronic device 400 may turn on at least one second camera module (e.g., Figure 4a and Figure 4b The electronic device 400 may control at least one second camera module (eg, Figure 4a and Figure 4b The camera modules 415 and 416 in the VST mode) obtain image frames at a first frame rate (e.g., 30fps). As described above, in the VST mode, at least one first camera module (e.g., Figure 4a and Figure 4b The camera modules 413 and 414 in the image processing unit may be maintained in a closed state.

[0122] At a third time point, the user's hand 551 may move to a designated area of ​​the third sub-FOV 533. For example, the electronic device 400 may recognize that the hand 551 has moved out of the designated area of ​​the first FOV based on the image frames obtained by the VST camera modules 411 and 412. Based on the placement of the user's hand 551 in the designated area of ​​the third sub-FOV 533, the electronic device 400 may adjust the frame rate of the VST camera modules 411 and 412 from the second frame rate to the first frame rate, and adjust the at least one second camera module (e.g., Figure 4a and Figure 4b The electronic device 101 may adjust the frame rate of the camera modules 415 and 416 in the electronic device 101 from the first frame rate to the second frame rate based on the frame rate of the camera modules 415 and 416 in the electronic device 101. Figure 4a and Figure 4b The hand 551 is tracked using image frames obtained by the camera modules 415 and 416 in the VST mode. As described above, in the VST mode, at least one first camera module (e.g., Figure 4a and Figure 4b The camera modules 413 and 414 in the image processing unit may be maintained in a closed state.

[0123] Fig.11 is a diagram illustrating adjusting the FOV and frame rate of a VST device in a VST mode connected to a controller according to an embodiment of the present disclosure.

[0124] exist Fig.11 In the embodiment of the present invention, the electronic device 400 may be connected to the controller by wire or wirelessly. The electronic device 400 may establish wireless communication with the controller based on short-range communication, for example, but there is no limitation. Fig.10 In the VST mode, at least one first camera module (eg, Figure 4a and Figure 4bThe camera modules 413 and 414 in the embodiment are kept closed. Fig.11 When the controller is connected, in VST mode, the electronic device 101 can turn on at least one first camera module (eg, Figure 4a and Figure 4b For example, the electronic device 101 may be based on at least one first camera module (eg, Figure 4a and Figure 4b The electronic device 101 may also maintain the VST camera modules 411 and 412 turned on for the VST content. For example, in Fig.11 In the embodiment, VST content may be provided using image frames obtained by the VST camera modules 411 and 412. The images obtained by the VST camera modules 411 and 412 are not used for tracking the controller, so the VST camera modules 411 and 412 may operate at a frame rate having a relatively low value (eg, 30 fps).

[0125] According to an embodiment, the electronic device 400 may recognize at a first time point that the controller 553 is located in the first sub-FOV 531. For example, the electronic device 400 may control at least one first camera module (eg, Figure 4a and Figure 4b The electronic device 400 may obtain image frames at a first frame rate based on at least one first camera module (e.g., Figure 4a and Figure 4b The electronic device 400 can identify the controller 553 in the first sub-FOV 531 based on the recognition of the controller 553 in the first sub-FOV 531 as a part of the first FOV, and can also identify the controller 553 in the first sub-FOV 531 as a part of the first FOV. Figure 4a and Figure 4b The frame rate of the camera modules 413 and 414 of the electronic device 400 is adjusted from the first frame rate (e.g., 30 fps) to the second frame rate (e.g., 90 fps). The electronic device 400 may track the controller 553 based on the image frames captured at the second frame rate. Figure 4a and Figure 4b The camera modules 415 and 416 in the electronic device 400 may maintain a closed state. The electronic device 400 may perform an operation associated with the manipulation of the controller based on the tracking result, and its operation is not limited.

[0126] At the second time point, the controller 553 may move to an area where the first sub-FOV 531 and the third sub-FOV 533 overlap each other (e.g., a designated area of ​​the first FOV). The electronic device 400 may turn on at least one second camera module (e.g., Figure 4a and Figure 4b The electronic device 400 may control at least one second camera module (eg, Figure 4a and Figure 4b The camera modules 415 and 416 in the image processing apparatus obtain image frames at a first frame rate (e.g., 30fps).

[0127] At a third time point, the controller 553 may move to a designated area of ​​the third sub-FOV 533. For example, the electronic device 400 may be based on at least one first camera module (eg, Figure 4a and Figure 4b The electronic device 400 can detect the image frame obtained by the camera modules 413 and 414 of the first sub-FOV and recognize that the controller 553 has moved out of the designated area of ​​the first FOV. Based on the controller 553 being placed in the designated area of ​​the third sub-FOV 533, the electronic device 400 can detect the image frame obtained by the camera modules 413 and 414 of the first sub-FOV and recognize that the controller 553 has moved out of the designated area of ​​the first FOV. Figure 4a and Figure 4b The frame rate of the camera modules 413 and 414) is adjusted from the second frame rate to the first frame rate, and at least one second camera module (eg, Figure 4a and Figure 4b The electronic device 101 may adjust the frame rate of the camera modules 415 and 416 in the electronic device 101 from the first frame rate to the second frame rate based on the frame rate of the camera modules 415 and 416 in the electronic device 101. Figure 4a and Figure 4b The controller 553 is tracked using image frames obtained by the camera modules 415 and 416 in the image frame.

[0128] Fig.12 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0129] According to an embodiment, the electronic device 101 (eg, the processor 120) may control at least one first camera module (eg, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b413 and 414) to obtain at least one image frame. In operation 1203, the electronic device 101 may identify whether the specified content has been called and / or whether the specified application has been run. For example, the specified content may be content that requires the recognition and / or tracking of an object (e.g., a hand) that is relatively lower than the electronic device 101 when the user wears the electronic device 101. For example, if the object to be recognized and / or tracked is a hand, the content may be, for example, a virtual keyboard. For example, the specified application may be an application that requires the recognition and / or tracking of an object (e.g., a hand) that is relatively lower than the electronic device 101 when the user wears the electronic device 101. For example, if the object to be recognized and / or tracked is a hand, the application may be a piano playing application. Based on identifying that the specified content is called and / or the specified application is run (operation 1203-yes), in operation 1205, the electronic device 101 may control at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b If the designated content call and / or designated application execution is not recognized (operation 1203-No), the electronic device 101 may control at least one first camera module (e.g., Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b If the second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416 of the electronic device 101 are in the off state, and the electronic device 101 can turn on the second camera module (for example, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b According to an embodiment, the electronic device 101 may turn off the second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b camera modules 415 and 416).

[0130] Fig.13 is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0131] According to an embodiment, in operation 1301, the electronic device 101 (eg, the processor 120) may control at least one first camera module (eg, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b In operation 1303, the electronic device 101 may identify whether the user's gaze moves out of the at least one first camera module (eg, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b For example, the electronic device 101 may be configured to generate a FOV corresponding to the camera modules 413 and 414 of the ET. Figure 2 For example, the electronic device 101 can recognize the user's gaze based on the images obtained by the ET camera modules (e.g., Figure 2 ET camera modules 251 and 252) and the images obtained with the camera modules for capturing the foreground image (e.g., Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The electronic device 101 may process the user's gaze by processing the user's gaze in a FOV corresponding to the camera modules 413 and 414 of the first camera module. The electronic device 101 may identify whether the user's gaze has moved out of, for example, the first FOV. Based on the user's gaze having moved out of the FOV corresponding to the at least one first camera module (for example, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b 413 and 414) corresponding to the FOV (eg, which has moved out of the first FOV) (operation 1303-Yes), in operation 1305, the electronic device 101 may control at least one second camera module (eg, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b If the user's gaze does not move out of the at least one first camera module (e.g., Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b 414) corresponding to the FOV (eg, it does not move out of the first FOV) (operation 1303-no), in operation 1301, the electronic device 101 may control at least one first camera module (eg, Figure 1 The camera modules 253 and 254 or Figure 4a and Figure 4b If the second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416 of the electronic device 101 are in a closed state, and the electronic device 101 can be based on the data from the first camera module (for example, Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The camera modules 413 and 414) corresponding to the FOV escape to open the second camera module (for example, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b camera modules 415 and 416).

[0132] According to an embodiment, a wearable electronic device ( Figure 1 The electronic device 101, Figure 2 The electronic device 200 or Figure 4a and Figure 4b The electronic device 400 in the embodiment may include: a housing 210a, 210b, 210c, which includes a structure that enables the device to be worn on at least a portion of a user's head; at least one processor 120, which is included in the housing 210a, 210b, 210c; at least one first camera module ( Figure 2 Camera modules 253 and 254 or Figure 4a and Figure 4b 4 and 414) configured to capture images for a first FOV configured relative to a point on a first surface of housing 210a, 210b, 210c, and at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 in or Figure 4a and Figure 4b The at least one processor 120 may be configured to control at least one first camera module ( Figure 2 Camera modules 253 and 254 or Figure 4a and Figure 4b The at least one processor 120 may be configured to control at least one first camera module ( 413 and 414 in the embodiment) to obtain a first image frame at a first frame rate. The at least one processor 120 may be configured to control at least one first camera module ( Figure 2 Camera modules 253 and 254 or Figure 4a and Figure 4bThe at least one processor 120 may be configured to control at least one second camera module ( Figure 2 Camera modules 255 and 256 in or Figure 4a and Figure 4b camera modules 415 and 416) to at least one second camera module ( Figure 2 Camera modules 255 and 256 in or Figure 4a and Figure 4b 415 and 416) in an inactive state, or activating at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 in or Figure 4a and Figure 4b 415 and 416) and operating at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 in or Figure 4a and Figure 4b 415 and 416 in FIG.

[0133] According to an embodiment, as a method of activating at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b At least one processor 120 may be configured to, based on recognizing that a user's hand is placed in a designated area of ​​the first FOV and satisfies the conditions configured to activate at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b 415 and 416) to activate at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b camera modules 415 and 416).

[0134] According to an embodiment, a distance from the second FOV to a designated area of ​​the first FOV may be shorter than a distance from the second FOV to the remaining areas of the first FOV except the designated area.

[0135] According to an embodiment, as a method for controlling at least one first camera module ( Figure 2 Camera modules 253 and 254 or Figure 4a and Figure 4bThe at least one processor 120 may be configured to, based on recognizing that the user's hand is placed in the first FOV and satisfying the change of at least one first camera module ( Figure 2 Camera modules 253 and 254 or Figure 4a and Figure 4b at least one second additional condition of the frame rate of the camera modules 413 and 414 in the control unit 413 and 414, controlling at least one first camera module ( Figure 2 Camera modules 253 and 254 or Figure 4a and Figure 4b The camera modules 413 and 414 in the image processing module 410 obtain a second image frame at a second frame rate.

[0136] According to an embodiment, the at least one processor 120 may be configured to control at least one second camera module (eg, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b Camera modules 415 and 416) obtain a third image frame at the first frame rate.

[0137] According to an embodiment, the at least one processor 120 may be configured to recognize whether the hand is placed in a designated area of ​​the second FOV and / or whether the hand has moved out of the first FOV based on at least one second image frame and / or at least a portion of at least one third image frame. The at least one processor 120 may be configured to control at least one first camera module ( Figure 2 Camera modules 253 and 254 or Figure 4a and Figure 4b The at least one processor 120 may be configured to control at least one second camera module (eg, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b Camera modules 415 and 416) obtain a fifth image frame at a second frame rate greater than the first frame rate.

[0138] According to an embodiment, as a control method for controlling at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The at least one processor 120 may be configured to, based on recognizing that the hand is placed in a specified area of ​​the second FOV and / or the hand has moved out of the first FOV and satisfying the change of at least one second camera module ( Figure 2 Camera modules 255 and 256; Figure 4a and Figure 4b at least one third additional condition of the frame rate of the camera modules 415 and 416), controlling at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 in or Figure 4a and Figure 4b The camera modules 415 and 416 in the image processing module 410 obtain a fifth image frame at a second frame rate.

[0139] According to an embodiment, as controlling at least one first camera module ( Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The at least one processor 120 may be configured to control at least one first camera module ( Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b Camera modules 413 and 414) obtain a fourth image frame at the first frame rate.

[0140] According to an embodiment, the wearable electronic device 101 may further include at least one third camera (VST camera modules 411 and 412), which is configured to capture an image for at least a portion of the first FOV configured relative to a point on the first surface of the housing 210a, 210b, 210c.

[0141] According to an embodiment, at least one processor 120 may be configured to perform 6DoF, SLAM, object recognition and / or object tracking based on at least a portion of the first image frame, the second image frame and / or the third image frame based on the recognition that the VR mode is running. At least one processor 120 may be configured to perform at least one operation based on the VR mode based on the result of object recognition and / or object tracking. When the VR mode is running, at least one third camera (VST camera modules 411 and 412) may remain in an inactive state.

[0142] According to an embodiment, at least one processor 120 may be configured to, based on identifying the VST mode, control at least one third camera (VST camera modules 411 and 412) to obtain a sixth image frame. When the sixth image frame is being obtained, at least one first camera module ( Figure 2 Camera modules 253 and 254; Figure 4a and Figure 4b camera modules 413 and 414) and at least one second camera module ( Figure 2 Camera modules 255 and 256; Figure 4a and Figure 4bThe at least one processor 120 may be configured to, based on recognizing that the user's hand is placed in a designated area of ​​the first FOV according to at least a portion of the sixth image frame, control at least one third camera (VST camera modules 411 and 412) to obtain a seventh image frame. When the seventh image frame is being obtained, at least one first camera module ( Figure 2 Camera modules 253 and 254; Figure 4a and Figure 4b The camera modules 413 and 414 in the embodiment may be deactivated. The at least one processor 120 may be configured to control at least one second camera module ( Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b camera modules 415 and 416) to activate at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b When obtaining the eighth image frame, at least one first camera module ( Figure 2 Camera modules 253 and 254; Figure 4a and Figure 4b The camera modules 413 and 414) in the image may be deactivated.

[0143] According to an embodiment, the at least one processor 120 may be configured to, based on recognizing that the VST mode is in operation and at least one controller is connected to the wearable electronic device, control at least one first camera module ( Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The at least one second camera module (VST camera modules 413 and 414) is used to obtain the ninth image frame, and controls at least one third camera (VST camera modules 411 and 412) to obtain the tenth image frame. When the tenth image frame is being obtained, at least one second camera module ( Figure 2 Camera modules 255 and 256; Figure 4a and Figure 4b The camera modules 415 and 416 of the first FOV may be deactivated, and the tenth image frame may be used to configure the VST content. The at least one processor 120 may be configured to, based on recognizing that the controller is placed in a designated area of ​​the first FOV according to at least a portion of the ninth image frame, control the at least one first camera module ( Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4bThe at least one processor 120 may be configured to control at least one second camera module ( Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b camera modules 415 and 416) to activate at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The VST camera modules 415 and 416) are used to obtain the thirteenth image frame, and the at least one third camera (VST camera modules 411 and 412) is controlled to obtain the fourteenth image frame. The fourteenth image frame can be used to configure the VST content.

[0144] According to an embodiment, a method for operating a wearable electronic device ( Figure 1 The electronic device 101, Figure 2 The electronic device 200 or Figure 4a and Figure 4b The wearable electronic device comprises: at least one first camera module ( Figure 2 Camera modules 253 and 254; Figure 4a and Figure 4b 413 and 414) and at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 in or Figure 4a and Figure 4b The method may include: controlling at least one first camera module ( Figure 2 Camera modules 253 and 254 or Figure 4a and Figure 4b The method may include: based on recognizing that a user's hand is placed in a specified area of ​​the first FOV according to at least a portion of the first image frame, controlling at least one first camera module ( Figure 2 Camera modules 253 and 254 or Figure 4a and Figure 4b The method may include: controlling at least one second camera module ( Figure 2 Camera modules 255 and 256 in or Figure 4a and Figure 4b415 and 416) to at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 in or Figure 4a and Figure 4b 415 and 416) in an inactive state, or activating at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 in or Figure 4a and Figure 4b 415 and 416 in FIG. 4 ), and operating at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 in or Figure 4a and Figure 4b 415 and 416 in FIG.

[0145] According to an embodiment, at least one second camera module (eg, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416 may include: based on recognizing that the user's hand is placed in a specified area of ​​the first FOV and satisfies the conditions configured to activate at least one second camera module (for example, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b 415 and 416), activating at least one first additional condition of at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b camera modules 415 and 416).

[0146] According to an embodiment, at least one first camera module ( Figure 2 Camera modules 253 and 254 or Figure 4a and Figure 4b The camera modules 413 and 414 in the first FOV) obtaining the second image frame at a second frame rate greater than the first frame rate may include: based on recognizing that the user's hand is placed in the first FOV and satisfying the change of at least one first camera module ( Figure 2 Camera modules 253 and 254; Figure 4a and Figure 4b at least one second additional condition of the frame rate of the camera modules 413 and 414 in the control unit 413 and 414, controlling at least one first camera module ( Figure 2 Camera modules 253 and 254 or Figure 4a and Figure 4b The camera modules 413 and 414 in the image processing module 410 obtain a second image frame at a second frame rate.

[0147] According to an embodiment, the method may further include: controlling at least one second camera module (eg, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b Camera modules 415 and 416) obtain a third image frame at the first frame rate.

[0148] According to an embodiment, the method may include: based on at least one second image frame and / or at least a portion of at least one third image frame, identifying whether the hand is placed in a specified area of ​​the second FOV and / or whether the hand has moved out of the first FOV. The method may include: based on identifying that the hand is placed in the specified area of ​​the second FOV and / or the hand has moved out of the first FOV, controlling at least one first camera module ( Figure 2 Camera modules 253 and 254 or Figure 4a and Figure 4b The method may include controlling at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b Camera modules 415 and 416) obtain a fifth image frame at a second frame rate greater than the first frame rate.

[0149] According to an embodiment, at least one second camera module (eg, Figure 2 Camera modules 255 and 256 or Figure 4a and Figure 4b The camera modules 415 and 416 of the embodiment of the present invention) obtaining a fifth image frame at a second frame rate may include: based on recognizing that the hand is placed in a specified area of ​​the second FOV and / or the hand has moved out of the first FOV and satisfying the change of at least one second camera module ( Figure 2 Camera modules 255 and 256; Figure 4a and Figure 4b at least one third additional condition of the frame rate of the camera modules 415 and 416), controlling at least one second camera module (e.g., Figure 2 Camera modules 255 and 256 in or Figure 4a and Figure 4b The camera modules 415 and 416 in the image processing module 410 obtain a fifth image frame at a second frame rate.

[0150] According to an embodiment, controlling at least one first camera module ( Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b The camera modules 413 and 414) to obtain the fourth image frame may include: controlling at least one first camera module ( Figure 2 The camera modules 253 and 254 or Figure 4a and Figure 4b Camera modules 413 and 414) obtain a fourth image frame at the first frame rate.

[0151] According to an embodiment, the wearable electronic device may further include at least one third camera (VST camera modules 411 and 412) configured to capture images for at least a portion of a first FOV configured relative to a point on the first surface of the housing 210a, 210b, 210c. The method of the wearable electronic device may include: based on the recognition of the operation of the VR mode, based on at least a portion of the first image frame, the second image frame and / or the third image frame, performing 6DoF, SLAM, object recognition and / or object tracking. The method may include: based on the result of object recognition and / or object tracking, performing at least one operation based on the VR mode. When the VR mode is running, at least one third camera (VST camera modules 411 and 412) may remain in an inactive state.

[0152] The electronic device according to various embodiments disclosed herein may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. The electronic device according to an embodiment of the present disclosure is not limited to those electronic devices described above.

[0153] It should be understood that the embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, and include various changes, equivalents or replacements for corresponding embodiments. Regarding the description of the accompanying drawings, similar reference numerals may be used to specify similar or related elements. The noun corresponding to the singular form of an item may include one or more items, unless the relevant context clearly indicates otherwise. As used herein, each phrase in the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include all possible combinations of items listed together with a corresponding phrase in the multiple phrases. As used herein, terms such as "first", "second", "the first", "the second" may be used to simply distinguish a corresponding element from another element, and do not limit elements in other aspects (e.g., importance or order). It should be understood that if an element (for example, a first element) is referred to as being "coupled to another element (for example, the second element)", "coupled to another element (for example, the second element)", "connected to another element (for example, the second element)" or "connected to another element (for example, the second element)" when the terms "operably" or "communicatively" are used or when the terms "operably" or "communicatively" are not used, it means that the element may be directly (for example, wired) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0154] As used in the embodiments of the present disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "component," or "circuit." A "module" may be the smallest unit of a single integrated component suitable for performing one or more functions, or a portion thereof. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0155] The embodiments set forth herein may be implemented as software (e.g., program 140) including one or more instructions stored in a storage medium (e.g., internal memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). For example, a processor (e.g., processor 120) of a machine (e.g., electronic device 101) may call at least one of the one or more instructions stored in the storage medium and run it. This enables the machine to operate to perform at least one function according to at least one instruction called. One or more instructions may include code generated by a compiler or code that can be run by an interpreter. A machine-readable storage medium may be provided in the form of a non-transitory storage medium. Among them, the term "non-transitory" only means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but the term does not distinguish between data being semi-permanently stored in a storage medium and data being temporarily stored in a storage medium.

[0156] According to an embodiment, the method according to various embodiments of the present disclosure may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be downloaded via an application store (e.g., Play Store TM ) the computer program product may be distributed (e.g., downloaded or uploaded) online, or may be distributed (e.g., downloaded or uploaded) directly between two user devices (e.g., smart phones). If distributed online, at least a portion of the computer program product may be temporarily generated, or at least a portion of the computer program product may be at least temporarily stored in a machine-readable storage medium (such as a memory of a manufacturer's server, an application store's server, or a relay server).

[0157] According to an embodiment, each element (for example, module or program) in the element described above can include a single entity or multiple entities, and some of the multiple entities can be separately arranged in any other element. According to various embodiments, one or more elements in the above-mentioned elements can be omitted, or one or more other elements can be added. As another option or in addition, multiple elements (for example, module or program) can be integrated into a single element. In this case, according to various embodiments, the integrated element can still perform one or more functions of each element in multiple elements in the same or similar manner as one or more functions performed by a corresponding element in multiple elements before integration. According to various embodiments, the operation performed by a module, program or another element can be performed sequentially, in parallel, repeatedly or in a heuristic manner, or one or more operations in the operation can be performed or omitted in different orders, or one or more other operations can be added.

[0158] While the present disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.

Claims

1. A wearable electronic device (the electronic device 101 in FIG. 1 ; the electronic device 200 in FIG. 2 ; the electronic device 400 in FIG. 4 a and FIG. 4 b ), the wearable electronic device comprising: A housing (210a; 210b; 210c), wherein the housing (210a; 210b; 210c) comprising structure enabling wear on at least a portion of a user's head; at least one processor (120), the at least one processor (120) being included in the housing (210a; 210b; 210c); at least one first camera module (camera modules 253 and 254 in FIG. 2 ; camera modules 413 and 414 in FIGS. 4 a and 4 b ), the at least one first camera module (camera modules 253 and 254 in FIG. 2 ; camera modules 413 and 414 in FIGS. 4 a and 4 b ) being configured to capture an image with respect to a first field of view FOV configured at a point on a first surface of the housing (210 a; 210 b; 210 c); as well as at least one second camera module (camera modules 255 and 256 in FIG. 2 ; camera modules 415 and 416 in FIGS. 4 a and 4 b ), the at least one second camera module (camera modules 255 and 256 in FIG. 2 ; camera modules 415 and 416 in FIGS. 4 a and 4 b ) configured to capture images for a second FOV different from the first FOV configured relative to a point on the first surface, Wherein, the at least one processor (120) is configured to: controlling the at least one first camera module (camera modules 253 and 254 in FIG. 2 ; camera modules 413 and 414 in FIG. 4 a and FIG. 4 b ) to obtain a first image frame at a first frame rate, and Based on recognizing, based on at least a portion of the first image frame, that a hand of a user is placed in a designated area of ​​the first FOV: Controlling the at least one first camera module (camera modules 253 and 254 in FIG. 2 ; The camera modules 413 and 414 in FIGS. 4a and 4b) obtain a second image frame at a second frame rate greater than the first frame rate, and Controlling the at least one second camera module (camera modules 255 and 256 in FIG. 2 ; 4a and 4b) to maintain the at least one second camera module (camera modules 255 and 256 in FIG. 2; camera modules 415 and 416 in FIG. 4a and 4b) in an inactive state, or to activate the at least one second camera module (camera modules 255 and 256 in FIG. 2; camera modules 415 and 416 in FIG. 4a and 4b) and operate the at least one second camera module (camera modules 255 and 256 in FIG. 2; camera modules 415 and 416 in FIG. 4a and 4b) at the first frame rate.

2. The wearable electronic device according to claim 1, wherein: As at least part of activating the at least one second camera module (camera modules 255 and 256 of FIG. 2 ; camera modules 415 and 416 of FIGS. 4 a and 4 b ), the at least one processor (120) is configured to: Based on recognizing that the user's hand is placed in the specified area of ​​the first FOV and satisfying at least one first additional condition configured to activate the at least one second camera module (camera modules 255 and 256 of Figure 2; camera modules 415 and 416 of Figures 4a and 4b), the at least one second camera module (camera modules 255 and 256 of Figure 2; camera modules 415 and 416 of Figures 4a and 4b) is activated.

3. The wearable electronic device according to one of claims 1 and 2, wherein: A distance from the second FOV to the designated area of ​​the first FOV is shorter than a distance from the second FOV to remaining areas of the first FOV except the designated area.

4. The wearable electronic device according to one of claims 1 to 3, wherein: As controlling the at least one first camera module (camera modules 253 and 254 in FIG. 2 ; camera modules 413 and 414 in FIG. 4 a and FIG. 4 b ) to obtain at least a portion of the second image frame at the second frame rate greater than the first frame rate based on recognizing that the user's hand is placed in the first FOV, the at least one processor (120) is configured to: Based on identifying that the user's hand is placed in the first FOV and satisfying at least one second additional condition for changing the frame rate of the at least one first camera module (camera modules 253 and 254 in Figure 2; camera modules 413 and 414 in Figures 4a and 4b), the at least one first camera module (camera modules 253 and 254 in Figure 2; camera modules 413 and 414 in Figures 4a and 4b) is controlled to obtain the second image frame at the second frame rate.

5. The wearable electronic device according to one of claims 1 to 4, wherein: The at least one processor (120) is also configured to control the at least one second camera module (camera modules 255 and 256 of FIG. 2; camera modules 415 and 416 of FIGS. 4a and 4b) to obtain a third image frame at the first frame rate.

6. The wearable electronic device according to claim 5, wherein: The at least one processor (120) is further configured to: identifying whether a user's hand is placed in a designated area of ​​the second FOV and / or whether the user's hand has moved out of the first FOV based on at least a portion of the second image frame and / or the third image frame, and Based on recognizing that the user's hand is placed in the designated area of ​​the second FOV and / or the user's hand has moved out of the first FOV: Controlling the at least one first camera module (camera modules 253 and 254 in FIG. 2 ; 4a and 4b) to obtain a fourth image frame, and Controlling the at least one second camera module (camera modules 255 and 256 of FIG. 2 ; The camera modules 415 and 416 of Figures 4a and 4b) obtain a fifth image frame at a second frame rate greater than the first frame rate.

7. The wearable electronic device according to claim 6, wherein: As controlling the at least one second camera module (camera modules 255 and 256 of FIG. 2 ; camera modules 415 and 416 of FIG. 4 a and FIG. 4 b ) to obtain at least a portion of the fifth image frame at the second frame rate, the at least one processor (120) is configured to: Based on recognizing that the user's hand is placed in the specified area of ​​the second FOV and / or the user's hand has moved out of the first FOV and satisfying at least one third additional condition for changing the frame rate of the at least one second camera module (camera modules 255 and 256 in Figure 2; camera modules 415 and 416 in Figures 4a and 4b), the at least one second camera module (camera modules 255 and 256 in Figure 2; camera modules 415 and 416 in Figures 4a and 4b) is controlled to obtain the fifth image frame at the second frame rate.

8. The wearable electronic device according to one of claims 6 to 7, wherein: The at least one processor (120) is configured to: control the at least one first camera module (camera modules 253 and 254 in Figure 2; camera modules 413 and 414 in Figures 4a and 4b) to obtain at least a portion of the fourth image frame, and control the at least one first camera module (camera modules 253 and 254 in Figure 2; camera modules 413 and 414 in Figures 4a and 4b) to obtain the fourth image frame at the first frame rate.

9. The wearable electronic device according to one of claims 1 to 8, further comprising at least one third camera (VST camera modules 411 and 412), wherein the at least one third camera (VST camera modules 411 and 412) is configured to capture an image of at least a portion of the first FOV configured relative to a point on the first surface of the housing (210a; 210b; 210c).

10. The wearable electronic device according to claim 9, wherein: The at least one processor (120) is further configured to: based on identifying the operation of the virtual reality (VR) mode: Based on at least a portion of the first image frame, the second image frame, and / or the third image frame, performing 6 degrees of freedom (DoF), simultaneous localization and mapping (SLAM), object recognition, and / or object tracking, and Based on the result of the object recognition and / or the object tracking, performing at least one operation based on the VR mode, and Wherein, when the VR mode is running, the at least one third camera (VST camera modules 411 and 412) remains in an inactive state.

11. The wearable electronic device according to one of claims 9 to 10, wherein: The at least one processor (120) is further configured to: based on identifying that the video perspective VST mode is in operation: controlling the at least one third camera (VST camera modules 411 and 412) to obtain a sixth image frame, wherein, while the sixth image frame is being obtained, the at least one first camera module (camera modules 253 and 254 in FIG. 2 ; camera modules 413 and 414 in FIGS. 4 a and 4 b ) and the at least one second camera module (camera modules 255 and 256 in FIG. 2 ; camera modules 415 and 416 in FIGS. 4 a and 4 b ) are deactivated, and Based on recognizing, according to at least a portion of the sixth image frame, that a hand of the user is placed in the designated area of ​​the first FOV: controlling the at least one third camera (VST camera modules 411 and 412) to obtain a seventh image frame, wherein, while the seventh image frame is being obtained, the at least one first camera module (camera modules 253 and 254 in FIG. 2 ; camera modules 413 and 414 in FIGS. 4 a and 4 b ) is deactivated, and Controlling the at least one second camera module (camera modules 255 and 256 of FIG. 2 ; 4a and 4b) to activate at least one second camera module (camera modules 255 and 256 in FIG. 2; camera modules 415 and 416 in FIG. 4a and 4b) and obtain an eighth image frame, wherein, when the eighth image frame is being obtained, at least one first camera module (camera modules 253 and 254 in FIG. 2; camera modules 413 and 414 in FIG. 4a and 4b) is deactivated.

12. The wearable electronic device according to one of claims 1 to 11, wherein: The at least one processor (120) is further configured to, based on recognizing that the video see-through (VST) mode is in operation and at least one controller is connected to the wearable electronic device: controlling the at least one first camera module (camera modules 253 and 254 of FIG. 2 ; camera modules 413 and 414 of FIGS. 4 a and 4 b ) to obtain a ninth image frame, and controlling the at least one third camera (VST camera modules 411 and 412) to obtain a tenth image frame, wherein when the tenth image frame is being obtained, the at least one second camera module (camera modules 255 and 256 of FIG. 2 ; camera modules 415 and 416 of FIGS. 4 a and 4 b ) is deactivated, and wherein the tenth image frame is used to configure VST content, and Based on identifying, based on at least a portion of the ninth image frame, that the controller is disposed in the designated area of ​​the first FOV: Controlling the at least one first camera module (camera modules 253 and 254 of FIG. 2 ; 4a and 4b) to obtain an eleventh image frame, and controlling the at least one third camera (VST camera modules 411 and 412) to obtain a twelfth image frame, wherein the twelfth image frame is used to configure the VST content, and Controlling the at least one second camera module (camera modules 255 and 256 of FIG. 2 ; 4a and 4b) to activate the at least one second camera module (camera modules 255 and 256 of FIG. 2; camera modules 415 and 416 of FIG. 4a and 4b) and obtain a thirteenth image frame, and control the at least one third camera (VST camera modules 411 and 412) to obtain a fourteenth image frame, wherein the fourteenth image frame is used to configure the VST content.

13. A method of operating a wearable electronic device (electronic device 101 in FIG. 1 ; electronic device 200 in FIG. 2 ; electronic device 400 in FIG. 4 a and FIG. 4 b ), the wearable electronic device comprising at least one first camera module (camera modules 253 and 254 in FIG. 2 ; camera modules 413 and 414 in FIG. 4 a and FIG. 4 b ) and at least one second camera module (camera modules 255 and 256 in FIG. 2 ; camera modules 415 and 416 in FIG. 4 a and FIG. 4 b ), the at least one first camera module (camera modules 253 and 254 in FIG. 2 ; camera modules 413 and 414 in FIG. 4 a and FIG. 4 b ) being configured to capture images for a first field of view (FOV), the at least one second camera module (camera modules 255 and 256 in FIG. 2 ; camera modules 415 and 416 in FIG. 4 a and FIG. 4 b ) being configured to capture images for a second FOV different from the first FOV, the method comprising: Controlling the at least one first camera module (camera modules 253 and 254 in FIG. 2 ; The camera modules 413 and 414 in FIG. 4a and FIG. 4b) obtain a first image frame at a first frame rate; as well as Based on recognizing, based on at least a portion of the first image frame, that a hand of a user is placed in a designated area of ​​the first FOV: Controlling the at least one first camera module (camera modules 253 and 254 in FIG. 2 ; The camera modules 413 and 414 in FIGS. 4a and 4b) obtain a second image frame at a second frame rate greater than the first frame rate, and Controlling the at least one second camera module (camera modules 255 and 256 in FIG. 2 ; 4a and 4b) to maintain the at least one second camera module (camera modules 255 and 256 in FIG. 2; camera modules 415 and 416 in FIG. 4a and 4b) in an inactive state, or to activate the at least one second camera module (camera modules 255 and 256 in FIG. 2; camera modules 415 and 416 in FIG. 4a and 4b) and operate the at least one second camera module (camera modules 255 and 256 in FIG. 2; camera modules 415 and 416 in FIG. 4a and 4b) at the first frame rate.

14. The method for operating a wearable electronic device according to claim 13, wherein: Based on recognizing that the user's hand is placed in the first FOV, controlling the at least one first camera module (camera modules 253 and 254 in FIG. 2 ; camera modules 413 and 414 in FIG. 4 a and FIG. 4 b ) to obtain the second image frame at the second frame rate greater than the first frame rate, comprising: Based on identifying that the user's hand is placed in the first FOV and satisfying at least one second additional condition for changing the frame rate of the at least one first camera module (camera modules 253 and 254 in Figure 2; camera modules 413 and 414 in Figures 4a and 4b), the at least one first camera module (camera modules 253 and 254 in Figure 2; camera modules 413 and 414 in Figures 4a and 4b) is controlled to obtain the second image frame at the second frame rate.

15. The method of operating a wearable electronic device according to one of claims 13 to 14, the method further comprising controlling the at least one second camera module (camera modules 255 and 256 of FIG. 2 ; camera modules 415 and 416 of FIG. 4 a and FIG. 4 b ) to obtain a third image frame at the first frame rate; Based on at least a portion of the at least one second image frame and / or the at least one third image frame, identifying whether the hand is placed in a designated area of ​​the second FOV and / or whether the hand has moved out of the first FOV; as well as Based on recognizing that the hand is placed in the designated area of ​​the second FOV and / or the hand has moved out of the first FOV: Controlling the at least one first camera module (camera modules 253 and 254 in FIG. 2 ; 4a and 4b) to obtain a fourth image frame; as well as Controlling the at least one second camera module (camera modules 255 and 256 of FIG. 2 ; The camera modules 415 and 416 of Figures 4a and 4b) obtain a fifth image frame at the second frame rate that is greater than the first frame rate.

Images