Wearable electronic device including camera and method of operating same

Abstract

A wearable electronic device may include a first camera, a second camera including image processing circuitry, a memory storing at least one instruction that, when executed by the at least one processor, generates at least one image on the second camera, and at least one processor. Causing the wearable electronic device to: recognize a gaze area of a user through the first camera when the wearable electronic device is worn by the user; acquiring an image having a first resolution by capturing an external object by a second camera; determining a first region corresponding to a gaze region of the user and a second region corresponding to a region different from the gaze region in the image; acquiring a first image corresponding to the first region from the image through a first channel of a plurality of channels between the processor and the second camera, the first image having a first resolution; and acquiring a second image corresponding to the second region through a second channel of the plurality of channels, the second image having a second resolution lower than the first resolution. Various other embodiments are possible.

Description

Technical Field

[0001] Various embodiments of the present disclosure relate to a wearable electronic device including a camera and an operating method thereof. Background Art

[0002] Various services and additional functions provided by wearable electronic devices (eg, augmented reality glasses devices) are increasing.

[0003] The augmented reality glasses device is a wearable device worn on the user's head, and can provide augmented reality services to the user by providing content through a display. The display of the augmented reality glasses device can be implemented as a light output device to display content through glasses elements.

[0004] The above information may be provided as related art to help understand the present disclosure. No statement or judgment is made as to the applicability of any of the above content as prior art of the present disclosure. Summary of the invention

[0005] According to an embodiment, a wearable electronic device may include a first camera, a second camera including an image processing circuit, a memory, and at least one processor.

[0006] According to an embodiment, the memory may store at least one instruction which, when executed by at least one processor, enables the wearable electronic device to recognize a gaze area of ​​the user through the first camera when the wearable electronic device is worn by the user.

[0007] According to an embodiment, the memory may store at least one instruction which, when executed by the at least one processor, causes the wearable electronic device to acquire an image having a first resolution generated by photographing an external object by the second camera.

[0008] According to an embodiment, the memory may store at least one instruction which, when executed by at least one processor, causes the wearable electronic device to determine in the image a first area corresponding to a gaze area of ​​the user and a second area corresponding to an area different from the gaze area.

[0009] According to an embodiment, the memory may store at least one instruction which, when executed by at least one processor, enables the wearable electronic device to acquire a first image having a first resolution and corresponding to a first area from an image through a first channel among multiple channels between the processor and the second camera.

[0010] According to an embodiment, the memory may store at least one instruction which, when executed by at least one processor, enables the wearable electronic device to acquire a second image having a second resolution lower than the first resolution and corresponding to the second area through a second channel among the plurality of channels.

[0011] According to an embodiment, the operating method of the wearable electronic device may include the following operation: in a state where the wearable electronic device is worn by the user, recognizing a gaze area of ​​the user through a first camera included in the wearable electronic device.

[0012] According to an embodiment, an operating method of a wearable electronic device may include an operation of acquiring an image having a first resolution generated by photographing an external object by a second camera included in the wearable electronic device.

[0013] According to an embodiment, the operating method of the wearable electronic device may include the following operation: determining, in an image, a first area corresponding to a gaze area of ​​a user and a second area corresponding to an area different from the gaze area.

[0014] According to an embodiment, the operating method of the wearable electronic device may include the following operation: acquiring a first image having a first resolution and corresponding to a first area from an image through a first channel among a plurality of channels between a processor included in the wearable electronic device and a second camera.

[0015] According to an embodiment, the operating method of the wearable electronic device may include the following operation: acquiring, through a second channel among the plurality of channels, a second image having a second resolution lower than the first resolution and corresponding to the second area.

[0016] According to an embodiment, a non-transitory computer-readable recording medium may store at least one instruction that may perform the following operations: in a state where the wearable electronic device is worn by the user, recognizing a gaze area of ​​the user through a first camera included in the wearable electronic device.

[0017] According to an embodiment, a non-transitory computer-readable recording medium may store at least one instruction which may perform the following operations: acquiring an image having a first resolution generated by photographing an external object by a second camera included in the wearable electronic device.

[0018] According to an embodiment, a non-transitory computer-readable recording medium may store at least one instruction that may perform the following operations: determining a first area corresponding to a gaze area of ​​a user and a second area corresponding to an area different from the gaze area in an image.

[0019] According to an embodiment, a non-transitory computer-readable recording medium may store at least one instruction that may perform the following operations: acquiring a first image having a first resolution and corresponding to a first area from an image through a first channel among multiple channels between a processor included in a wearable electronic device and a second camera.

[0020] According to an embodiment, a non-transitory computer-readable recording medium may store at least one instruction that may perform the following operations: acquiring a second image having a second resolution lower than the first resolution and corresponding to the second area through a second channel among the plurality of channels. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a block diagram illustrating electronic devices in a network environment according to various embodiments.

[0022] Figure 2 is a perspective view showing an internal configuration of a wearable electronic device according to an embodiment.

[0023] Figure 3A is a schematic block diagram showing a configuration of a wearable electronic device according to an embodiment.

[0024] Figure 3B is a schematic block diagram showing a configuration of a second camera according to an embodiment.

[0025] Figure 4 is a flowchart illustrating an operating method of a wearable electronic device according to an embodiment.

[0026] Figure 5 is a flowchart illustrating an operating method of a wearable electronic device according to an embodiment.

[0027] Fig. 6A is a diagram illustrating an operation of a conventional wearable electronic device according to a comparative example to acquire an image corresponding to a gaze area of ​​a user and an image corresponding to an area other than the gaze area.

[0028] Figure 6B 2 is a diagram illustrating an operation of a wearable electronic device acquiring an image corresponding to a gaze area of ​​a user and an image corresponding to an area different from the gaze area according to an embodiment.

[0029] Figure 6C 2 is a diagram illustrating an operation of a wearable electronic device acquiring an image corresponding to a gaze area of ​​a user and an image corresponding to an area different from the gaze area according to an embodiment.

[0030] Fig. 7A is a flowchart illustrating an operating method of a wearable electronic device according to an embodiment.

[0031] Figure 7B 2 is a diagram illustrating an operation of a wearable electronic device acquiring an image corresponding to a gaze area of ​​a user and an image corresponding to an area different from the gaze area according to an embodiment.

[0032] Figure 7C2 is a diagram illustrating an operation of a wearable electronic device acquiring an image corresponding to a gaze area of ​​a user and an image corresponding to an area different from the gaze area according to an embodiment.

[0033] Figure 8 is a diagram showing that a wearable electronic device according to an embodiment synthesizes an image corresponding to a gaze area of ​​a user and an image corresponding to an area different from the gaze area and displays them through a glasses element.

[0034] Fig. 9 is a diagram showing that a wearable electronic device according to an embodiment synthesizes an image corresponding to a gaze area of ​​a user and an image corresponding to an area different from the gaze area and displays them through a glasses element. DETAILED DESCRIPTION

[0035] Figure 1 1 is a block diagram illustrating an electronic device 101 in a network environment 100 according to various embodiments. Figure 1 , the electronic device 101 in the network environment 100 may communicate with the electronic device 102 via the first network 198 (e.g., a short-range wireless communication network), or communicate with at least one of the electronic device 104 or the server 108 via the second network 199 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 via the server 108. According to an embodiment, the electronic device 101 may include a processor 120, a memory 130, an input module 150, a sound output module 155, a display module 160, an audio module 170, a sensor module 176, an interface 177, a connection terminal 178, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a user identification module (SIM) 196, or an antenna module 197. In some embodiments, at least one of the above components (e.g., the connection terminal 178) may be omitted from the electronic device 101, or one or more other components may be added to the electronic device 101. In some embodiments, some of the above-described components (eg, sensor module 176, camera module 180, or antenna module 197) may be implemented as a single integrated component (eg, display module 160).

[0036] The processor 120 may run, for example, software (e.g., program 140) to control at least one other component (e.g., hardware component or software component) of the electronic device 101 connected to the processor 120, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, the processor 120 may store a command or data received from another component (e.g., sensor module 176 or communication module 190) in the volatile memory 132, process the command or data stored in the volatile memory 132, and store the resultant data in the non-volatile memory 134. According to an embodiment, the processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)) or an auxiliary processor 123 (e.g., a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is operationally independent or combined with the main processor 121. For example, when the electronic device 101 includes a main processor 121 and an auxiliary processor 123, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or adapted to be dedicated to a specific function. The auxiliary processor 123 may be implemented separately from the main processor 121, or as part of the main processor 121.

[0037] When the main processor 121 is in an inactive (e.g., sleep) state, the auxiliary processor 123 (rather than the main processor 121) may control at least some of the functions or states related to at least one component among the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190), or when the main processor 121 is in an active state (e.g., running an application), the auxiliary processor 123 may control at least some of the functions or states related to at least one component among the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) together with the main processor 121. According to an embodiment, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., a camera module 180 or a communication module 190) that is functionally related to the auxiliary processor 123. According to an embodiment, the auxiliary processor 123 (e.g., a neural processing unit) may include a hardware structure dedicated to artificial intelligence model processing. The artificial intelligence model may be generated by machine learning. For example, such learning may be performed by the electronic device 101 where the artificial intelligence is executed or via a separate server (e.g., server 108). The learning algorithm may include, but is not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, for example. The artificial intelligence model may include multiple artificial neural network layers. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q network or a combination of two or more thereof, but is not limited thereto. Additionally or optionally, the artificial intelligence model may include a software structure in addition to a hardware structure.

[0038] The memory 130 may store various data used by at least one component of the electronic device 101 (e.g., the processor 120 or the sensor module 176). The various data may include, for example, software (e.g., the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a nonvolatile memory 134.

[0039] The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system (OS) 142 , middleware 144 , or applications 146 .

[0040] The input module 150 may receive commands or data to be used by other components (e.g., the processor 120) of the electronic device 101 from outside (e.g., a user) of the electronic device 101. The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus).

[0041] The sound output module 155 can output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records. The receiver can be used to receive incoming calls. Depending on the embodiment, the receiver can be implemented as a separate part from the speaker, or as part of the speaker.

[0042] The display module 160 may visually provide information to the outside of the electronic device 101 (e.g., a user). The display device 160 may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling a corresponding one of the display, the holographic device, and the projector. According to an embodiment, the display module 160 may include a touch sensor adapted to detect a touch or a pressure sensor adapted to measure the strength of a force caused by a touch.

[0043] The audio module 170 may convert sound into an electrical signal, or vice versa. According to an embodiment, the audio module 170 may obtain sound via the input module 150, or output sound via the sound output module 155 or an earphone of an external electronic device (e.g., electronic device 102) directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0044] The sensor module 176 may detect an operating state (e.g., power or temperature) of the electronic device 101 or an environmental state (e.g., a state of a user) outside the electronic device 101, and then generate an electrical signal or data value corresponding to the detected state. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0045] The interface 177 may support one or more specific protocols to be used to connect the electronic device 101 directly (e.g., wired) or wirelessly to an external electronic device (e.g., the electronic device 102). According to an embodiment, the interface 177 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0046] The connection end 178 may include a connector, wherein the electronic device 101 can be physically connected to an external electronic device (e.g., the electronic device 102) via the connector. According to an embodiment, the connection end 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0047] The haptic module 179 may convert the electrical signal into mechanical stimulation (eg, vibration or motion) or electrical stimulation that can be recognized by the user via his sense of touch or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0048] The camera module 180 may capture still images or moving images. According to an embodiment, the camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0049] The power management module 188 may manage power supply to the electronic device 101. According to one embodiment, the power management module 188 may be implemented as, for example, at least a portion of a power management integrated circuit (PMIC).

[0050] The battery 189 may power at least one component of the electronic device 101. According to an embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0051] The communication module 190 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. The communication module 190 may include one or more communication processors capable of operating independently from the processor 120 (e.g., an application processor (AP)) and supporting direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules can communicate with an external electronic device via a first network 198 (e.g., a short-range communication network such as Bluetooth, Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a traditional cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip), or these various types of communication modules can be implemented as multiple components separated from each other (e.g., multiple chips). The wireless communication module 192 can identify and authenticate the electronic device 101 in a communication network (such as the first network 198 or the second network 199) using user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the user identification module 196.

[0052] The wireless communication module 192 can support 5G networks after 4G networks and next-generation communication technologies (e.g., new radio (NR) access technology). NR access technology can support enhanced mobile broadband (eMBB), massive machine type communication (mMTC), or ultra-reliable low-latency communication (URLLC). The wireless communication module 192 can support high-frequency bands (e.g., millimeter wave bands) to achieve, for example, high data transmission rates. The wireless communication module 192 can support various technologies for ensuring performance on high-frequency bands, such as, for example, beamforming, massive multiple-input multiple-output (massive MIMO), full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. The wireless communication module 192 can support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., a second network 199). According to an embodiment, the wireless communication module 192 may support a peak data rate for implementing eMBB (e.g., 20 Gbps or greater), loss coverage for implementing mMTC (e.g., 164 dB or less), or U-plane delay for implementing URLLC (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less round trip).

[0053] The antenna module 197 may transmit a signal or power to the outside of the electronic device 101 (e.g., an external electronic device) or receive a signal or power from the outside of the electronic device 101 (e.g., an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a radiating element, the radiating element being formed of a conductive material or a conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, the antenna module 197 may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as the first network 198 or the second network 199) may be selected from the plurality of antennas by, for example, the communication module 190 (e.g., the wireless communication module 192). A signal or power may then be transmitted or received between the communication module 190 and the external electronic device via the selected at least one antenna. According to an embodiment, another component (e.g., a radio frequency integrated circuit (RFIC)) other than the radiating element may be additionally formed as a part of the antenna module 197.

[0054] According to various embodiments, the antenna module 197 may form a millimeter wave antenna module. According to an embodiment, the millimeter wave antenna module may include a printed circuit board, a radio frequency integrated circuit (RFIC), and a plurality of antennas (e.g., array antennas), wherein the RFIC is disposed on a first surface (e.g., bottom surface) of the printed circuit board, or adjacent to the first surface and capable of supporting a specified high frequency band (e.g., millimeter wave band), and the plurality of antennas are disposed on a second surface (e.g., top surface or side surface) of the printed circuit board, or adjacent to the second surface and capable of transmitting or receiving signals of the specified high frequency band.

[0055] At least some of the above components may be connected to each other via an inter-peripheral communication scheme (e.g., a bus, general purpose input output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and communicatively transmit signals (e.g., commands or data) therebetween.

[0056] According to an embodiment, a command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 via the server 108 connected to the second network 199. Each of the electronic device 102 or the electronic device 104 may be a device of the same type as the electronic device 101, or a device of a different type from the electronic device 101. According to an embodiment, all or some operations to be executed in the electronic device 101 may be executed in one or more of the external electronic device 102, the external electronic device 104, or the server 108. For example, if the electronic device 101 should automatically execute a function or service or should execute a function or service in response to a request from a user or another device, the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service instead of executing the function or service, or the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service in addition to executing the function or service. The one or more external electronic devices receiving the request may execute at least part of the function or service requested, or execute another function or another service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may provide the result as at least a partial reply to the request in the case of further processing the result or in the case of not further processing the result. To this end, cloud computing technology, distributed computing technology, mobile edge computing (MEC) technology or client-server computing technology, for example, may be used. The electronic device 101 may use, for example, distributed computing or mobile edge computing to provide ultra-low latency services. In another embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. The server 108 may be an intelligent server using machine learning and / or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart homes, smart cities, smart cars or health care) based on 5G communication technology or IoT-related technologies.

[0057] Figure 2 is a perspective view showing an internal configuration of a wearable electronic device according to an embodiment.

[0058] Reference Figure 2 According to an embodiment, the wearable electronic device 200 may include the following components: these components are accommodated in the housing 210a, 210b or 210c, are disposed on the housing 210a, 210b or 210c, and / or are exposed through openings disposed on the housing 210a, 210b or 210c.

[0059] According to an embodiment, the wearable electronic device 200 can obtain a visual image of an object or environment in a direction (e.g., -Y direction) that a user is viewing or that the wearable electronic device (200) is facing by using a plurality of camera modules 253, 254, 255, and 256. The camera modules 253 and 254 can be disposed on relatively upper portions of the housings 210b and 210c (or exposed through openings disposed on the housings 210b and 210c). The camera modules 253 and 254 can capture an image corresponding to a field of view (FOV) relative to at least one point on the housings 210b and 210c, such as a FOV corresponding to a relatively upper side when the user wears the wearable electronic device 200. The images obtained by the camera modules 253 and 254 can be used, for example, for simultaneous localization and mapping (SLAM) and / or 6 degrees of freedom (6DoF), and / or for identifying and / or tracking an object corresponding to the FOV. The images obtained by the camera modules 253 and 254 can be used for head tracking.

[0060] According to an embodiment, the camera modules 255 and 256 may be disposed on the relatively lower portion of the housings 210b and 210c (or exposed through an opening disposed on the housings 210b and 210c). Here, when the user wears the wearable electronic device 200, the upper portion corresponding to the camera modules 253 and 254 and the lower portion corresponding to the camera modules 255 and 256 are defined, and it should be understood by those skilled in the art that for ease of description, the portion relatively close to the ground is referred to as the lower portion, and the portion relatively far from the ground is referred to as the upper portion. The camera modules 255 and 256 may capture an image of a FOV corresponding to at least one point on the housings 210b and 210c, such as a FOV corresponding to the relatively lower side when the user wears the wearable electronic device 200. The images acquired by the camera modules 255 and 256 may be used to identify and / or track an object corresponding to the FOV. For example, in the case where the user wears the wearable electronic device 200, the images acquired by the camera modules 255 and 256 may be used for, but not limited to, identifying and / or tracking an object located relatively lower than the portion corresponding to the head, such as the user's hand.

[0061] According to an embodiment, the wearable electronic device 200 may perform recognition and / or tracking of an object by using at least one image captured by the camera modules 253, 254, 255, and 256. The wearable electronic device 200 may perform an operation of recognition based on the result of recognition and / or tracking, and provide a visual object at a position corresponding to the object, but is not limited to the operation. For example, in the case where the wearable electronic device 200 provides a virtual keyboard, based on the tracking result of the user's hand, a designated key may be identified in the virtual keyboard. The operation corresponding to the result of recognition and / or tracking may be performed independently by the wearable electronic device 200, but this is illustrative, and the operation may be based on the wearable electronic device 200 and an external electronic device (e.g., Figure 1 It is executed by collaboration of the electronic device 102, the electronic device 104 and / or the server 108).

[0062] According to an embodiment, camera modules 253, 254, 255 and 256 are used for 3DoF or 6DoF head tracking, hand detection, hand tracking and / or space recognition, and may be global shutter (GS) cameras, but are not limited thereto, and may be implemented as rolling shutter (RS) cameras.

[0063] According to an embodiment, the camera modules 251 and 252 may be eye tracking (ET) cameras, and the images captured by the camera modules 251 and 252 may be used to detect and / or track pupils. For example, using the captured images, the position of the virtual image projected on the wearable electronic device 200 may be determined so that the pupil of the wearer of the wearable electronic device 200 is located according to the gaze direction. The camera modules 251 and 252 may be implemented as GS cameras for detecting and / or tracking pupils, but are not limited thereto.

[0064] According to an embodiment, the display module 240 may include, for example, a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), a light emitting diode (LED) on silicon (LEDoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED). Although not shown in the drawings, in the case where the display module 240 includes one of a liquid crystal display, a digital mirror display device, or a liquid crystal on silicon, the wearable electronic device 200 may include a light source configured to emit light to a screen output area of ​​the display module 240. In an embodiment, if the display module 240 is capable of generating light by itself, for example, including one of an organic light emitting diode or a micro LED, the wearable electronic device 200 may provide a user with a high-quality virtual image even if the wearable electronic device does not include a separate light source. According to an embodiment, in the case where the display panel module 240 is implemented as an organic light emitting diode or a micro LED, a light source is not necessary, so the wearable electronic device 200 may become lighter in weight. The wearable electronic device 200 may include a display module 240, a first transparent element 201, and / or a second transparent element 202, and the user may use the wearable electronic device 200 while wearing the wearable electronic device. The first transparent element 201 and / or the second transparent element 202 may include a glass plate, a plastic plate, or a polymer, and may be manufactured to be transparent or translucent. The optical waveguide may transmit the light source generated by the display module 240 to the user's eyes. The optical waveguide may be formed of glass, plastic, or a polymer, and may include a nano-pattern, such as a grating structure having a polygonal or curved shape disposed on an inner or outer surface. According to an embodiment, light incident to one end of the waveguide may be propagated within the display optical waveguide through the nano-pattern and provided to the user. In addition, the optical waveguide including a free-form type prism may provide incident light to the user through a reflector. The optical waveguide may include at least one of a diffractive element (e.g., a diffractive optical element (DOE) and a holographic optical element (HOE)) or a reflective element (e.g., a reflector). The optical waveguide can guide the light emitted from the light source of the display to the user's eyes by using at least one diffraction element or a reflective element. According to various embodiments, the diffraction element may include an input optical element / output optical element (not shown). As an example, the input optical element (not shown) may indicate an input grating area, and the output optical element (not shown) may indicate an output grating area. The input grating area may be used as an input end of a transparent element (e.g., a first transparent element 201 and a second transparent element 202) that diffracts (or reflects) the light output from the light source (e.g., a micro LED) to transmit the light to the screen display portion. The output grating area may be used as an exit for diffracting (or reflecting) the light of the transparent element (e.g., the first transparent element 201 and the second transparent element 202) that has been transmitted to the waveguide (optical waveguide) to the user's eyes.According to various embodiments, the reflective element may include a total internal reflection (TIR) ​​optical element or a total internal reflection waveguide for total internal reflection. For example, total internal reflection corresponds to a method of guiding light, and may indicate that an incident angle is generated so that light (e.g., a virtual image) input through an input grating region is 100% reflected from one surface (e.g., a predetermined surface) of a waveguide and 100% transmitted to an output grating region. In an embodiment, the light output from the display module 240 may have an optical path guided to the waveguide through an input optical element. The light propagating in the waveguide may be guided to the user's eyes through an output optical element. The screen display portion may be determined based on the light output to the eyes. According to an embodiment, the first transparent element 201 may be arranged to face the user's right eye, and the second transparent element 202 may be arranged to face the user's left eye. According to various embodiments, in the case where the display module 240 is transparent, the display module may be arranged at a position facing the user's eyes to configure the screen display portion. The wearable electronic device 200 may also include a lens. The lens may adjust the screen focus output to the display module 240 so that it is visible to the user's eyes. For example, the lens may include a Fresnel lens, a flat lens, a multi-channel lens, etc.

[0065] According to an embodiment, the circuit board 241 may include components for driving the wearable electronic device 200. For example, the circuit board 241 may include at least one integrated circuit chip, and the processor 120, a memory (eg, Figure 1 130 in the memory), a power management module (e.g., Figure 1 power management module 188 / battery 189) or a communication module (e.g., Figure 1 According to an embodiment, the circuit board 241 may be disposed in the housing 210a. According to an embodiment, the circuit board 241 may be electrically connected to the battery 243 through a power transmission structure. According to an embodiment, the circuit board 241 may be connected to a flexible printed circuit board and may transmit electrical signals to light sources and electronic components (e.g., display module 240 and camera modules 251, 252, 253, 254, 255, 256, 259, and 260) of the electronic device through the flexible printed circuit board. According to an embodiment, the circuit board 241 may include a circuit board with an interposer.

[0066] According to an embodiment, the battery 243 may be electrically connected to components of the wearable electronic device 200 through a power transmission structure and may supply power to the components of the wearable electronic device 200. According to an embodiment, at least a portion of the battery 243 may be provided in the wearable element.

[0067] According to an embodiment, the speaker module 245 may convert an electrical signal into sound. The speaker module 245 according to an embodiment may be disposed in the housing 210a, 210b, or 210c. According to an embodiment, the speaker module 245 may be disposed between the circuit board 241 and the battery 243 to correspond to the user's ear. The speaker module 245 according to an embodiment may also transmit auditory information to the user through low-frequency vibrations in the user's skin and bones.

[0068] According to an embodiment, the microphone module 247 may convert sound into an electric signal. According to an embodiment, the microphone module 247 may be provided on at least a portion of the housing 210b or 210c.

[0069] According to an embodiment, the wearable electronic device 200 may recognize the user's voice and / or external sound by using at least one microphone module 247. According to an embodiment, the wearable electronic device 200 may distinguish voice information from peripheral noise based on voice information and / or additional information (e.g., low-frequency vibrations of the user's skin and bones) acquired through the at least one microphone module 247. For example, the wearable electronic device 200 may clearly recognize the user's voice and perform a function for reducing peripheral noise (e.g., noise cancellation).

[0070] According to an embodiment, the camera module 259 may include an infrared (IR) camera module (e.g., a time-of-flight (TOF) camera or a structured light camera). For example, the IR camera may operate as at least a part of a sensor module (e.g., a sensor module or a light detection and ranging (LiDAR) sensor) for detecting the distance of an object. According to an embodiment, the wearable electronic device 200 may also include a sensor module (e.g., a LiDAR sensor). For example, the sensor module may include at least one of a vertical cavity surface emitting laser (VCSEL), an infrared sensor, and / or a photodiode.

[0071] LED 242 (e.g., an illumination LED) can have a variety of uses depending on the location to which the LED is attached. As an example, LED 242 attached to housing 210b or 210c can be used as an auxiliary component for assisting gaze detection when tracking the movement of the eye by using camera module 251 or 252, and IR LEDs with infrared wavelengths are mainly used for this purpose. For another example, LED 242 can be attached around hinge 229, which connects housing 210b or 210c of transparent element 201 or 202 to housing 210a of temple (eyeglass leg), or attached adjacent to the camera module as a component for supplementing ambient brightness during camera shooting. For example, camera module 260 for shooting can capture relatively high-quality images of the foreground of wearable electronic device 200.

[0072] Figure 3A is a schematic block diagram showing a configuration of a wearable electronic device according to an embodiment.

[0073] According to an embodiment, the wearable electronic device 301 (eg, Figure 1 The electronic device 101 or Figure 2 The wearable electronic device 200 in FIG. 1 may include a processor 320 , a memory 330 , a first camera 370 , a second camera 380 , and a display 360 .

[0074] According to an embodiment, the wearable electronic device 301 may display an augmented reality (AR) image through the display 360. For example, the wearable electronic device 301 may display information associated with a real thing (or object) as a virtual object (e.g., an AR image) surrounding the real thing. For example, a user wearing the wearable electronic device 301 may display an augmented reality (AR) image through the display 360 (e.g., Figure 1 The display module 160 or Figure 2 The display module 240 in the embodiment of the present invention can be used to see the virtual image (or virtual object) displayed by the wearable electronic device 301 and the real space together, and the display 360 is configured to be connected through the eyeglass element (for example, Figure 2 The wearable electronic device 301 may be implemented as a smart glasses or a head mounted display device.

[0075] According to an embodiment, the processor 320 may control general operations of the electronic device 301. As an example, the processor 320 may be implemented as Figure 1 The processor 120 is the same as or similar to the processor 120 in FIG.

[0076] According to an embodiment, the display 360 may be Figure 1 According to an embodiment, the display 360 may be implemented as the same as or similar to the display module 160 in FIG. Figure 2 The display module 240 in FIG. 3 is the same as or similar to the display module 240 in FIG. According to an embodiment, the display 360 may display content or an object according to the control of the processor 320. For example, in the case where the wearable electronic device 301 is implemented as smart glasses, the display 360 may be implemented as a light output device configured to display content or an object through a glasses element.

[0077] According to an embodiment, when the wearable electronic device 301 is worn by the user, the processor 320 may identify the user's gaze area through the first camera 370. According to an embodiment, the first camera module 370 may capture the user's eyes. According to an embodiment, the processor 320 may identify the user's gaze area based on an image generated by capturing the user's eyes. According to an embodiment, the first camera 370 may be implemented as a gaze tracking camera (eye tracking camera (ET camera)) for identifying the user's gaze. For example, the first camera 370 may be connected to Figure 1 The camera module 180 and / or Figure 2 The camera modules 251 and 252 in FIG. 1 are implemented identically or similarly.

[0078] According to an embodiment, the processor 320 may acquire an image having a first resolution generated by photographing an external object by the second camera 380. According to an embodiment, the second camera 380 may photograph the external object according to the control of the processor 320. According to an embodiment, the first resolution may be a resolution preconfigured by the processor 320 or a user among resolutions supported by the second camera 380. As an example, the first resolution may correspond to a maximum resolution (or full resolution) supported by the second camera 380. For example, the second camera 380 may be configured to Figure 1 The camera module 180 and / or Figure 2 The camera modules 253, 254, 255, 256 or 260 in the second camera 380 are implemented identically or similarly. According to an embodiment, the processor 320 may control data of an image having a first resolution to be stored in a volatile memory included in the second camera 380.

[0079] According to an embodiment, the processor 320 may determine a first area corresponding to the gaze area of ​​the user and a second area corresponding to an area different from the gaze area in the image acquired by the second camera 380. Alternatively, according to an embodiment, the first area and the second area may be determined by the second camera 380. Here, the second camera 380 may determine the first area and the second area based on the gaze area of ​​the user recognized by the processor 320.

[0080] According to an embodiment, the processor 320 may acquire a first image corresponding to a first area (an area corresponding to the gaze area) having the first resolution from an image having the first resolution through a first channel among a plurality of channels between the processor 320 and the second camera 380 .

[0081] According to an embodiment, the processor 320 may control an image processing circuit (eg, Figure 3BThe processor 320 may further include an image processing circuit 315 in the processor 320, and reads data for the image having the first resolution from the volatile memory to perform pixel binning of the second region at the second resolution. According to an embodiment, the second resolution may be lower than the first resolution. According to an embodiment, the processor 320 may acquire a second image corresponding to the second region having the second resolution from the image through a second channel among a plurality of channels between the processor 320 and the second camera 380.

[0082] According to an embodiment, the processor 320 may acquire a first image corresponding to the first region and a second image corresponding to the second region in parallel through the first channel and the second channel. Alternatively, according to an implementation, the processor 320 may acquire the first image and the second image sequentially.

[0083] According to an embodiment, the first channel and the second channel may be different from each other. According to an embodiment, each of the first channel and the second channel may be implemented as a physically separated channel. Alternatively, each of the first channel and the second channel may be implemented as a logically separated channel (e.g., a virtual channel).

[0084] According to an embodiment, the processor 320 may divide the second area into a plurality of areas. According to an embodiment, the processor 320 may divide the second area into a plurality of areas based on the distance from the first area corresponding to the gaze area. According to an embodiment, the processor 320 may control the second camera 380 to perform pixel merging of the plurality of areas at different resolutions. According to an embodiment, pixel merging may be performed so that a portion of the second area located at a relatively short distance from the first area has a relatively higher resolution than a portion of the second area located at a relatively long distance from the first area.

[0085] According to an embodiment, the processor 320 may acquire the first image and the second image in parallel through the first channel and the second channel. For example, the processor 320 may acquire the first image and the second image simultaneously or sequentially.

[0086] According to an embodiment, the first channel may include a channel associated with an improved inter-integrated circuit (I3C). The second channel may include a channel associated with a mobile industry processor interface (MIPI).

[0087] According to an embodiment, the first channel may include a first virtual channel associated with a Mobile Industry Processor Interface (MIPI).According to an embodiment, the second channel may include a second virtual channel associated with a Mobile Industry Processor Interface (MIPI).

[0088] According to an embodiment, the processor 320 may acquire a third image in which the first image and the second image are combined. As an example, the third image may correspond to an image including an external object captured by the second camera 380. For example, the third image may be an image in which pixel merging is performed at a relatively low resolution (e.g., a second resolution) on a second area in the image that is different from the user's gaze area. According to an embodiment, the processor 320 may control the display 360 to display the user's gaze area through the glasses element (e.g., Figure 2 The first transparent element 201 and the second transparent element 202) display a third image.

[0089] According to the above method, the processor 320 can quickly acquire the first image and the second image processed by the second camera 380. In this way, the processor 320 can display the third image on the display 360 without frame delay.

[0090] Figure 3B is a schematic block diagram showing a configuration of a second camera according to an embodiment.

[0091] According to an embodiment, the second camera 380 may include an image sensor 311 , a memory 313 , an image processing circuit 315 , a first channel 317 , and a second channel 319 .

[0092] According to an embodiment, the image sensor 311 may convert light emitted or reflected from an object (external object) and transmitted through a lens into an electrical signal, thereby acquiring an image having a first resolution and corresponding to the object. According to an embodiment, the image sensor 311 may include an image sensor selected from image sensors having different properties, such as an RGB sensor, a black and white (BW) sensor, an IR sensor, or a UV sensor, a plurality of image sensors having the same properties, or a plurality of image sensors having different properties. Each image sensor included in the image sensor 311 may be implemented as, for example, a charge coupled device sensor (CCD) or a complementary metal oxide semiconductor (CMOS) sensor.

[0093] According to an embodiment, the memory 313 may store data for an image having a first resolution. According to an embodiment, the memory 313 may include a volatile memory. According to an embodiment, the memory 313 may include a dynamic random access memory (DRAM).

[0094] According to an embodiment, the image processing circuit 315 may read data about an image (e.g., the entire image) having a first resolution stored in the memory 313. According to an embodiment, the image processing circuit 315 may perform defective pixel correction (DPC), frame white balance, and / or noise reduction. According to an embodiment, the image processing circuit 315 may obtain a first image corresponding to a first area (e.g., a user's gaze area) from the entire image by using the read data. According to an embodiment, the image processing circuit 315 may perform pixel merging on a second area (e.g., an area different from the gaze area) in the entire image at a second resolution lower than the first resolution by using the read data. According to an embodiment, the image processing circuit 315 may obtain a second image corresponding to the second area based on the pixel-merged image.

[0095] According to an embodiment, the image processing circuit 315 may be connected to the processor 320 through the first channel 317 and / or the second channel 319. According to an embodiment, the first channel 317 and the second channel 319 may be different from each other.

[0096] According to an embodiment, the image processing circuit 315 may transmit a first image having a first resolution and corresponding to the first area to the processor 320 through the first channel 317. According to an embodiment, the image processing circuit 315 may transmit a second image having a second resolution and corresponding to the second area to the processor 320 through the second channel 319. According to an embodiment, the image processing circuit 315 may transmit the first image and the second image to the processor 320 in parallel through the first channel 317 and the second channel 319.

[0097] According to an embodiment, the first channel 317 may include a channel associated with an improved inter-integrated circuit (I3C). According to an embodiment, the second channel 319 may include a channel associated with a mobile industry processor interface (MIPI). According to an embodiment, each of the first channel 317 and the second channel 319 may be implemented as a logically separated channel (e.g., a virtual channel). According to an embodiment, the first channel 317 may include a first virtual channel associated with a mobile industry processor interface (MIPI), and the second channel 319 may include a second virtual channel associated with MIPI.

[0098] Reference below Figure 4 The described operations of the wearable electronic device 301 may be performed by the processor 320. However, for ease of explanation, the operations performed by the processor 320 will be described as being performed by the wearable electronic device 301.

[0099] Figure 4 is a flowchart illustrating an operating method of a wearable electronic device according to an embodiment.

[0100] According to an embodiment, in operation 401, the wearable electronic device 301 (eg, Figure 3A The wearable electronic device 301 in the embodiment may be provided with a first camera 370 (eg, Figure 3A According to an embodiment, the wearable electronic device 301 may identify the user's gaze area based on an image generated by photographing the user's eyes. According to an embodiment, the first camera 370 may be implemented as a gaze tracking camera (eye tracking camera (ET camera)) for identifying the user's gaze.

[0101] According to an embodiment, in operation 403, the wearable electronic device 301 may acquire a video image of the wearable electronic device 301 through the second camera 380 (eg, Figure 3A The image with the first resolution is generated by photographing an external object with the second camera 380 in the image processing unit 300. According to an embodiment, the first resolution may be a resolution preconfigured by the processor 320 or a user among the resolutions supported by the second camera 380.

[0102] According to an embodiment, the wearable electronic device 301 may determine a first area corresponding to a gaze area of ​​the user and a second area corresponding to an area different from the gaze area in operation 405. According to an embodiment, the first area and the second area may be different areas.

[0103] According to an embodiment, in operation 407, the wearable electronic device 301 may acquire a first image corresponding to a first region having a first resolution through a first channel 317 among a plurality of channels between the processor 320 and the second camera 380. According to an embodiment, the plurality of channels may include physically separated channels, or may include virtually separated channels.

[0104] According to an embodiment, in operation 409, the wearable electronic device 301 may acquire a second image corresponding to a second area having a second resolution through a second channel 319 among a plurality of channels. According to an embodiment, the second resolution may be lower than the first resolution. According to an embodiment, the first channel 317 and the second channel 319 may include different channels. According to an embodiment, the wearable electronic device 301 may acquire the first image and the second image in parallel through the first channel 317 and the second channel 319. For example, the processor 320 may acquire the first image and the second image simultaneously or sequentially.

[0105] According to an embodiment, the first channel 317 may include a channel associated with an improved inter-integrated circuit (I3C). The second channel 319 may include a channel associated with a mobile industry processor interface (MIPI).

[0106] According to an embodiment, each of the first channel 317 and the second channel 319 may be implemented as a logically separated channel. According to an embodiment, the first channel 317 may include a first virtual channel associated with a mobile industry processor interface (MIPI). According to an embodiment, the second channel 319 may include a second virtual channel associated with a mobile industry processor interface (MIPI).

[0107] According to an embodiment, although it is described that operation 409 is performed after operation 407, operation 407 may be performed after operation 409, and operation 407 and operation 409 may also be performed simultaneously.

[0108] According to an embodiment, in operation 411, the wearable electronic device 301 may acquire a third image in which the first image and the second image are combined.

[0109] According to an embodiment, in operation 413, the wearable electronic device 301 may control the display 360 to display the image through the eyeglass element (eg, Figure 2 The third image is displayed on the first transparent element 201 and the second transparent element 202 in the image. As an example, the third image may correspond to an image including an external object photographed by the second camera 380. For example, the third image may be an image in which pixel merging is performed at a relatively low resolution (e.g., a second resolution) on a second area in the image that is different from the gaze area of ​​the user.

[0110] According to an embodiment, the wearable electronic device 301 may perform pixel merging on an image corresponding to an area different from the gaze area of ​​the user, and make the resolution of the image corresponding to the gaze area of ​​the user relatively high.

[0111] According to an embodiment, the wearable electronic device 301 may perform pixel merging on an image corresponding to an area different from a gaze area of ​​the user to reduce an amount of data, thereby relatively reducing current consumption of the wearable electronic device 301 .

[0112] According to an embodiment, the image processing circuit 315 can transmit an image corresponding to the user's gaze area and an image corresponding to an area different from the user's gaze area to the processor 330 through different channels, so that the wearable electronic device 301 does not experience frame delay, thereby relatively reducing the deterioration of the frame rate.

[0113] Reference below Figure 5 The described operations of the wearable electronic device 301 may be performed by the second camera 380. However, for convenience of explanation, the operations performed by the second camera 380 are described as being performed by the wearable electronic device 301.

[0114] Figure 5 is a flowchart illustrating an operating method of a wearable electronic device according to an embodiment.

[0115] According to an embodiment, in operation 501, the wearable electronic device 301 (eg, Figure 3A The wearable electronic device 301 in the embodiment may be provided with an image sensor 311 (eg, Figure 3B The image sensor 311 in the image sensor 311 acquires data of an image with a first resolution generated by photographing an external object, and stores the data in the memory 313 (for example, Figure 3B According to an embodiment, the second camera 380 may include a memory 313. According to an embodiment, the memory 313 may be implemented as a volatile memory. According to an embodiment, the memory 313 may be implemented as a dynamic random access memory (DRAM). However, the type of memory is merely illustrative, and the memory 313 may be implemented with various types of memory.

[0116] According to an embodiment, in operation 503, based on the slave processor 320 (eg, Figure 3A The processor 320 in the image processing circuit 315 (for example, Figure 3B The image processing circuit 315 in the image processing circuit 315 may determine a first area corresponding to the image having the first resolution and a second area corresponding to an area different from the gaze area. According to an embodiment, the processor 320 may transmit data (e.g., coordinates, etc.) associated with the gaze area to the image processing circuit 315 based on identifying the gaze area of ​​the user through the first camera 370.

[0117] According to an embodiment, in operation 504, the image processing circuit 315 may acquire only a portion corresponding to a first area in an image (e.g., the entire image) having a first resolution generated by photographing an external object as a first image. According to an embodiment, the image processing circuit 315 may read data about an image (e.g., the entire image) having a first resolution stored in the memory 313. According to an embodiment, the image processing circuit 315 may acquire a first image corresponding to a first area (e.g., a gaze area of ​​a user) from the entire image by using the read data.

[0118] According to an embodiment, in operation 505, the image processing circuit 315 (eg, Figure 3BThe image processing circuit 315 in the memory 313 may perform pixel merging on the second area at a second resolution. According to an embodiment, the second resolution may be lower than the first resolution. According to an embodiment, the image processing circuit 315 may read data of an image with a first resolution stored in the memory 313, and perform pixel merging on the image with the first resolution corresponding to the second area at the second resolution. According to an embodiment, the image processing circuit 315 may acquire only a portion corresponding to the pixel-merged second area in an image with a first resolution (e.g., the entire image) generated by photographing an external object as the second image.

[0119] According to an embodiment, in operation 507, the image processing circuit 315 may use the first channel 317 (eg, Figure 3B The first channel 317 in the processor 320 transmits the first image corresponding to the first area.

[0120] According to an embodiment, in operation 509, the image processing circuit 315 may use the second channel 319 (eg, Figure 3B The first channel 317 and the second channel 319 in the processor 320 transmit the second image corresponding to the second area to the processor 320. According to an embodiment, the first channel 317 and the second channel 319 may be different from each other.

[0121] According to an embodiment, the first channel 317 may include a channel associated with an improved inter-integrated circuit (I3C). The second channel 319 may include a channel associated with a mobile industry processor interface (MIPI).

[0122] According to an embodiment, the first channel 317 may include a first virtual channel associated with a mobile industry processor interface (MIPI). According to an embodiment, the second channel 319 may include a second virtual channel associated with a mobile industry processor interface (MIPI).

[0123] According to an embodiment, the image processing circuit 315 may transmit the first image and the second image to the processor 320 in parallel through the first channel 317 and the second channel 319. For example, the processor 320 may acquire the first image and the second image simultaneously or sequentially.

[0124] According to an embodiment, although it is described that operation 509 is performed after operation 507, operation 507 may be performed after operation 509, and operation 509 and operation 507 may also be performed simultaneously.

[0125] Fig. 6A is a diagram illustrating an operation of a conventional wearable electronic device according to a comparative example to acquire an image corresponding to a gaze area of ​​a user and an image corresponding to an area other than the gaze area.

[0126] According to a comparative example, readout may refer to the image processing circuit transmitting an image (or frame) to a processor. According to a comparative example, the first time interval t1 may refer to the time interval for the image processing circuit to transmit an image (or frame) corresponding to a gaze area of ​​an n-th frame (e.g., n is a natural number) (e.g., an image with a first resolution) and an image (or frame) corresponding to an area different from the gaze area of ​​an n'-th frame (e.g., n' is a natural number) (e.g., a pixel-binned image). For example, the n-th frame may refer to a pixel-binned image (or frame) of the n-th frame.

[0127] According to a comparative example, the second time interval t2 may refer to a time interval for the image processing circuit to transmit to the processor an image (or frame) corresponding to the fixation area of ​​the (n+1)th frame (e.g., an image having a first resolution) and an image (or frame) corresponding to an area different from the fixation area of ​​the (n'+1)th frame (e.g., n' is a natural number) (e.g., a pixel-binned image). For example, the (n'+1)th frame may refer to the pixel-binned image (or frame) of the (n+1)th frame.

[0128] According to a comparative example, acquiring an image may refer to acquiring a frame (or an image) by exposure.

[0129] According to a comparative example, after acquiring the nth frame (or image) by exposure, the wearable electronic device may acquire an image corresponding to an area different from the gaze area through a channel (e.g., an image after pixel merging). According to a comparative example, after acquiring the n'th frame by exposure, the wearable electronic device may acquire an image corresponding to the gaze area through a channel (e.g., an image having a first resolution).

[0130] According to the comparative example, since the wearable electronic device transmits an image corresponding to the gaze area of ​​the nth frame and an image corresponding to an area different from the gaze area of ​​the nth frame acquired by the camera, two transmission operations need to be performed to transmit multiple images with different resolutions to the processor.

[0131] According to a comparative example, the processor of the wearable electronic device acquires images corresponding to the gaze area and images corresponding to an area different from the gaze area through the same channel, which may cause frame delay. That is, deterioration of the frame rate may be caused by the frame delay.

[0132] Figure 6B 2 is a diagram illustrating an operation of a wearable electronic device acquiring an image corresponding to a gaze area of ​​a user and an image corresponding to an area different from the gaze area according to an embodiment.

[0133] According to an embodiment, acquiring an image may refer to an operation of acquiring a frame (or image) through exposure via the image sensor 311 included in the second camera 380 .

[0134] According to an embodiment, readout may refer to an operation of the image processing circuit 315 transmitting an image (or frame) to the processor 320. According to an embodiment, the first time interval t3 may refer to an operation of the image processing circuit 315 transmitting an image (or frame) to the processor 320 via the first channel I3C (eg, Figure 3B The first channel 317 in the MIPI and the second channel MIPI (eg, Figure 3B The second channel 319 in the image processing circuit 315 transmits an image (or frame) corresponding to an area different from the gaze area of ​​the n-th frame (for example, n is a natural number) (for example, a pixel-merged image) and an image (or frame) corresponding to the gaze area of ​​the n-th frame (for example, an image with a first resolution) to the processor 320. According to an embodiment, the second time interval t4 may refer to a time interval for the image processing circuit 315 to transmit an image (or frame) corresponding to an area different from the gaze area of ​​the (n+1)-th frame (for example, a pixel-merged image) and an image (or frame) corresponding to the gaze area of ​​the (n+1)-th frame (for example, an image with a first resolution) to the processor 320 through the first channel I3C and the second channel MIPI.

[0135] According to an embodiment, the first channel I3C may include a channel associated with an improved inter-integrated circuit (I3C). According to an embodiment, the second channel MIPI may include a channel associated with a mobile industry processor interface (MIPI). For example, each of the first channel I3C and the second channel MIPI may be a physically separated channel.

[0136] According to an embodiment, the image processing circuit 315 may transmit an image (or frame) corresponding to the gaze area (e.g., an image with a first resolution) to the processor 320 through the first channel I3C. According to an embodiment, the image processing circuit 315 may transmit an image (or frame) corresponding to an area other than the gaze area (e.g., a pixel-merged image) to the processor 320 through the second channel MIPI.

[0137] According to an embodiment, the image processing circuit 315 may transmit an image corresponding to the gaze area (e.g., an image having a first resolution) and an image corresponding to an area different from the gaze area (e.g., a pixel-merged image) in parallel through the first channel I3C and the second channel MIPI, respectively. According to an embodiment, the pixel-merged image may be an image having a lower resolution than the image of the first resolution.

[0138] According to an embodiment, the first time interval t3 may be Fig. 6AThe first time interval t1 shown in FIG. 1 is shorter, and the second time interval t4 may be shorter. Fig. 6A According to an embodiment, the wearable electronic device 301 may minimize the frame delay so as to reduce the deterioration of the frame rate.

[0139] Figure 6C 2 is a diagram illustrating an operation of a wearable electronic device acquiring an image corresponding to a gaze area of ​​a user and an image corresponding to an area different from the gaze area according to an embodiment.

[0140] According to an embodiment, the first time interval t5 may refer to the image processing circuit 315 (eg, Figure 3B The image processing circuit 315 in the embodiment transmits an image (or frame) corresponding to the gaze area of ​​the nth frame (for example, n is a natural number) (for example, an image with a first resolution) and an image (or frame) corresponding to an area different from the gaze area of ​​the nth frame (for example, a pixel-merged image) to the processor 320 through the first virtual channel VC1 and the second virtual channel VC2. According to an embodiment, the second time interval t6 may refer to a time interval for the image processing circuit 315 to transmit an image (or frame) corresponding to the gaze area of ​​the (n+1)th frame (for example, an image with a first resolution) and an image (or frame) corresponding to an area different from the gaze area of ​​the (n+1)th frame (for example, a pixel-merged image) to the processor 320 through the first virtual channel VC1 and the second virtual channel VC2.

[0141] According to an embodiment, the first virtual channel VC1 and the second virtual channel VC2 may include virtual channels associated with a Mobile Industry Processor Interface (MIPI).For example, each of the first virtual channel VC1 and the second virtual channel VC2 may be a physically separated channel (or virtual channel).

[0142] According to an embodiment, the image processing circuit 315 may transmit an image (or frame) corresponding to the gaze area (e.g., an image having a first resolution) to the processor 320 through the first virtual channel VC1. According to an embodiment, the image processing circuit 315 may transmit an image (or frame) corresponding to an area other than the gaze area (e.g., a pixel-binned image) to the processor 320 through the second virtual channel VC2.

[0143] According to an embodiment, the image processing circuit 315 may transmit an image corresponding to the gaze area (e.g., an image having a first resolution) and an image corresponding to an area other than the gaze area (e.g., a pixel-merged image) in parallel through the first virtual channel VC1 and the second virtual channel VC2. According to an embodiment, the stored image may be an image having a lower resolution than the image of the first resolution.

[0144] According to an embodiment, the first time interval t5 may be Fig. 6A The first time interval t1 shown in FIG. 1 is shorter, and the second time interval t6 may be shorter. Fig. 6A According to an embodiment, the wearable electronic device 301 may minimize the frame delay so as to reduce the deterioration of the frame rate.

[0145] Fig. 7A is a flowchart illustrating an operating method of a wearable electronic device according to an embodiment.

[0146] According to an embodiment, in operation 701, the processor 320 may use the first camera 370 (eg, Figure 3A The first camera 370 in the image is used to identify the user's gaze area.

[0147] According to an embodiment, in operation 703, the processor 320 may acquire the image captured by the second camera 380 (eg, Figure 3A An image with a first resolution is generated by photographing an external object with the second camera 380 in the image.

[0148] According to an embodiment, in operation 705, the processor 320 may determine a first area corresponding to a gaze area of ​​the user and a second area corresponding to an area different from the gaze area.

[0149] According to an embodiment, in operation 707, the processor 320 may divide the second area into a plurality of areas. According to an embodiment, the processor 320 may divide the second area into a plurality of areas based on the distance from the first area. For example, the processor 320 may divide an area at a relatively short distance from the first area into a third area, and divide an area at a relatively long distance from the first area into a fourth area. However, dividing into a plurality of areas based on distance is only an example, and the embodiment is not limited thereto, and various other methods may be used to divide into a plurality of areas.

[0150] According to an embodiment, in operation 709, the image processing circuit 315 may perform pixel merging on a plurality of regions at different resolutions. According to an embodiment, the image processing circuit 315 may perform pixel merging so that a region relatively close to the first region has a relatively higher resolution than a resolution of a region relatively far from the first region. For example, the image processing circuit 315 may perform pixel merging on an image corresponding to the third region at a second resolution lower than the first resolution. The image processing circuit 315 may perform pixel merging on an image corresponding to the fourth region at a third resolution lower than the second resolution.

[0151] According to an embodiment, in operation 711, the image processing circuit 315 may process the image through a first channel (eg, Figure 3BThe processor 320 may further include a first channel 317 in the processor 320 to transmit a first image having a first resolution and corresponding to a first region. According to an embodiment, the first channel 317 may include a channel associated with an improved inter-integrated circuit (I3C). Alternatively, the first channel 317 may include a first virtual channel associated with a mobile industry processor interface (MIPI).

[0152] According to an embodiment, in operation 713, the image processing circuit 315 may use a second channel (eg, Figure 3B According to an embodiment, the second channel 319 may refer to a channel different from the first channel 317. According to an embodiment, in the case where the first channel 317 is implemented as a channel associated with an improved inter-integrated circuit (I3C), the second channel 319 may include a channel associated with a mobile industry processor interface (MIPI). Alternatively, in the case where the first channel 317 is implemented as a first virtual channel associated with MIPI, the second channel 319 may include a second virtual channel associated with MIPI. According to an embodiment, the image processing circuit 315 may transmit an image corresponding to the third area to the processor 320 through the second channel 319. Thereafter, the image processing circuit 315 may transmit an image corresponding to the fourth area to the processor 320 through the second channel 319.

[0153] According to an embodiment, although it is described that operation 713 is performed after operation 711, operation 711 may be performed after operation 713, and operation 711 and operation 713 may also be performed simultaneously.

[0154] According to an embodiment, in operation 715, the processor 320 may merge the first image and images corresponding to the plurality of regions.

[0155] According to an embodiment, in operation 717, the processor 320 may control the display 360 to display the image through the glasses element (eg, Figure 2 The processor 320 may display the merged image by using the first transparent element 201 and the second transparent element 202 in the display 360. For example, the processor 320 may display the image with a gradually decreasing resolution centered on the user's gaze area through the display 360.

[0156] Figure 7B 2 is a diagram illustrating an operation of a wearable electronic device acquiring an image corresponding to a gaze area of ​​a user and an image corresponding to an area different from the gaze area according to an embodiment.

[0157] According to an embodiment, the first channel I3C (eg, Figure 3BThe first channel 317 in the example may include a channel associated with an improved inter-integrated circuit (I3C). According to an embodiment, the second channel MIPI (e.g., Figure 3B The second channel 319) in may include a channel associated with a Mobile Industry Processor Interface (MIPI).

[0158] According to an embodiment, the image processing circuit 315 may transmit a first image having a first resolution and corresponding to the first area to the processor 320 through the first channel 317 .

[0159] According to an embodiment, the processor 320 may divide the second area into a plurality of areas. The processor 320 may divide an area at a relatively short distance from the first area into a third area, and divide an area at a relatively long distance from the first area into a fourth area.

[0160] According to an embodiment, the image processing circuit 315 may perform pixel merging on an image corresponding to the third region at a second resolution lower than the first resolution. The image processing circuit 315 may perform pixel merging on an image corresponding to the fourth region at a third resolution lower than the second resolution.

[0161] According to an embodiment, the image processing circuit 315 may transmit an image corresponding to the third region (e.g., an image with pixel merging at the second resolution) to the processor 320 through the second channel MIPI. According to an embodiment, the image processing circuit 315 may transmit an image corresponding to the fourth region (e.g., an image with pixel merging at the third resolution) to the processor 320 through the second channel MIPI. For example, the image processing circuit 315 may sequentially transmit the third image and the fourth image to the processor 320.

[0162] Figure 7C 2 is a diagram illustrating an operation of a wearable electronic device acquiring an image corresponding to a gaze area of ​​a user and an image corresponding to an area different from the gaze area according to an embodiment.

[0163] According to an embodiment, the first channel VC1 (eg, Figure 3B The first channel 317 in the example may include a virtual channel associated with MIPI. According to an embodiment, the second channel VC2 (eg, Figure 3B The second channel 319) in may include a virtual channel associated with MIPI.

[0164] According to an embodiment, the image processing circuit 315 may transmit a first image having a first resolution and corresponding to a first area to the processor 320 through the first virtual channel VC1.

[0165] According to an embodiment, the image processing circuit 315 may transmit an image corresponding to the third region (e.g., an image with pixel merging at the second resolution) to the processor 320 through the second virtual channel VC2. According to an embodiment, the image processing circuit 315 may transmit an image corresponding to the fourth region (e.g., an image with pixel merging at the third resolution) to the processor 320 through the second virtual channel VC2. For example, the image processing circuit 315 may sequentially transmit the third image and the fourth image to the processor 320.

[0166] although Figure 7B and Figure 7C The image processing circuit 315 is shown to sequentially transmit an image corresponding to the first region and an image corresponding to the fourth region (e.g., an image with pixel merging at a third resolution) to the processor 320, but the technical concept here may not be limited to this. For example, the image processing circuit 315 may transmit the third image and the fourth image to the processor 320 in parallel through a virtual channel different from the second virtual channel.

[0167] Figure 8 is a diagram showing that a wearable electronic device according to an embodiment synthesizes an image corresponding to a gaze area of ​​a user and an image corresponding to an area different from the gaze area and displays them through a glasses element.

[0168] According to an embodiment, a processor (e.g., Figure 3A The processor 320 in the embodiment may be based on the first camera (eg, Figure 3A The first camera 370 in the image captures the user's eyes and generates an image to identify the user's gaze area.

[0169] According to an embodiment, the processor 320 may acquire the image captured by the second camera 380 (eg, Figure 3A According to an embodiment, data about the image 810 with the first resolution may be stored in a memory (e.g., Figure 3B In the memory 313).

[0170] According to an embodiment, the processor 320 may determine a first area corresponding to a gaze area of ​​the user and a second area corresponding to an area different from the gaze area in the image 810 having the first resolution.

[0171] According to an embodiment, the image processing circuit 315 may read data about an image (e.g., the entire image) having a first resolution stored in the memory 313. According to an embodiment, the image processing circuit 315 may acquire a first image corresponding to a first region from the entire image by using the read data. According to an embodiment, the image processing circuit 315 may perform pixel merging on a second region in the entire image at a second resolution lower than the first resolution by using the read data. The image processing circuit 315 may acquire a second image corresponding to the second region based on the stored image.

[0172] According to an embodiment, the image processing circuit 315 may be configured to process the image through a first channel (eg, Figure 3B The image processing circuit 315 may transmit the first image 830 corresponding to the first area to the processor 320 through the second channel (e.g., Figure 3B The second channel 319 in the processor 320 transmits a second image 820 corresponding to the second area.

[0173] According to an embodiment, the processor 320 may combine the first image 830 corresponding to the first area and the second image 820 corresponding to the second area to obtain a third image. According to an embodiment, the third image may be obtained by using a glasses element (e.g., Figure 2 The first transparent element 201 and the second transparent element 202) display a third image.

[0174] Fig. 9 is a diagram showing that a wearable electronic device according to an embodiment synthesizes an image corresponding to a gaze area of ​​a user and an image corresponding to an area different from the gaze area and displays them through a glasses element.

[0175] According to an embodiment, a processor (e.g., Figure 3A The processor 320 in the embodiment may acquire the image through the second camera (eg, Figure 3A An image 910 with a first resolution is generated by photographing an external object with the second camera 380 in FIG.

[0176] According to an embodiment, data regarding the image 910 having the first resolution may be stored in a memory (eg, Figure 3B In the memory 313).

[0177] According to an embodiment, the processor 320 may determine a first area corresponding to a gaze area of ​​the user and a second area corresponding to an area different from the gaze area in the image 910 having the first resolution.

[0178] According to an embodiment, the processor 320 may divide the second area into a plurality of areas. According to an embodiment, the processor 320 may divide an area at a relatively short distance from the first area into a third area, and divide an area at a relatively long distance from the first area into a fourth area.

[0179] According to an embodiment, an image processing circuit (eg, Figure 3B The image processing circuit 315 in the memory 313 may read data about an image (e.g., an entire image) having a first resolution stored in the memory 313. According to an embodiment, the image processing circuit 315 may obtain a first image corresponding to the first area from the entire image by using the read data. According to an embodiment, the image processing circuit 315 may perform pixel merging on a third area in the second area at a second resolution lower than the first resolution by using the read data. The image processing circuit 315 may obtain a third image corresponding to the third area based on the image that is pixel-merged at the second resolution. According to an embodiment, the image processing circuit 315 may perform pixel merging on a fourth area in the second area at a third resolution lower than the second resolution by using the read data. The image processing circuit 315 may obtain a fourth image corresponding to the fourth area based on the image that is pixel-merged at the third resolution.

[0180] According to an embodiment, the image processing circuit 315 may be configured to process the image through a first channel (eg, Figure 3B The image processing circuit 315 may transmit the first image 920 corresponding to the first area to the processor 320 through the second channel (e.g., Figure 3B The image processing circuit 315 may transmit the second image 930 corresponding to the third area to the processor 320 through the second channel 319. According to an embodiment, the image processing circuit 315 may transmit the third image 940 corresponding to the third area to the processor 320 through the second channel 319.

[0181] According to an embodiment, the processor 320 may combine the first image 920 corresponding to the first area, the third image 930 corresponding to the third area, and the fourth image 940 corresponding to the fourth area to obtain a fifth image. ... Figure 2 The first transparent element 201 and the second transparent element 202 in the image display a fifth image.

[0182] According to an embodiment, a wearable electronic device (eg, Figure 3A The wearable electronic device 301 in the embodiment may include: a first camera (eg, Figure 3A The first camera 370 in FIG. 1 includes an image processing circuit (eg, Figure 3AA second camera (eg, Figure 3A a second camera 380 in the example), a memory (e.g., Figure 3A 330) and at least one processor (e.g., Figure 3A Processor 320 in.

[0183] According to an embodiment, the memory 330 may store at least one instruction which, when executed by the at least one processor 320 , enables the wearable electronic device 301 to recognize a gaze area of ​​the user through the first camera 370 when the wearable electronic device 301 is worn by the user.

[0184] According to an embodiment, the memory 330 may store at least one instruction which, when executed by the at least one processor 320 , enables the wearable electronic device 301 to acquire an image having a first resolution by photographing an external object through the second camera 380 .

[0185] According to an embodiment, the memory 330 may store at least one instruction which, when executed by the at least one processor 320, enables the wearable electronic device 301 to determine in the image a first area corresponding to the user's gaze area and a second area corresponding to an area different from the gaze area.

[0186] According to an embodiment, the memory 330 may store at least one instruction which, when executed by the at least one processor 320, enables the wearable electronic device 301 to transmit the image through a first channel (eg, Figure 3B A first channel 317 in the image is used to obtain a first image having a first resolution and corresponding to the first area from the image.

[0187] According to an embodiment, the memory 330 may store at least one instruction, which, when executed by the at least one processor 320, enables the wearable electronic device 301 to pass through a second channel (eg, Figure 3B A second channel 319 in the method acquires a second image having a second resolution lower than the first resolution and corresponding to the second area.

[0188] According to an embodiment, the memory 330 may store at least one instruction, which, when executed by the at least one processor 320 , enables the wearable electronic device 301 to acquire the first image and the second image in parallel through the first channel and the second channel.

[0189] According to an embodiment, the wearable electronic device 301 may include eyeglass elements (eg, Figure 2 The first transparent element 201 or Figure 2 The second transparent element 202 in the embodiment of the present invention.

[0190] According to an embodiment, the wearable electronic device 301 may include a display (eg, Figure 3A The display 360 in the figure is configured to display virtual objects through the eyeglass element 201 or 202.

[0191] According to an embodiment, the memory may store at least one instruction, which, when executed by the at least one processor 320 , enables the wearable electronic device 301 to acquire a third image that combines the first image and the second image.

[0192] According to an embodiment, the memory 330 may store at least one instruction which, when executed by the at least one processor 320 , enables the wearable electronic device 301 to control the display 360 to display the third image through the glasses element 201 or 202 .

[0193] According to an embodiment, the first channel 317 may include a channel associated with an improved inter-integrated circuit (I3C).

[0194] According to an embodiment, the second channel 319 may include a channel associated with a Mobile Industry Processor Interface (MIPI).

[0195] According to an embodiment, the first channel 317 may include a first virtual channel associated with a Mobile Industry Processor Interface (MIPI).

[0196] According to an embodiment, the second channel 319 may include a second virtual channel associated with MIPI.

[0197] According to an embodiment, the second camera 380 may include a volatile memory.

[0198] According to an embodiment, the memory 330 may store at least one instruction which, when executed by the at least one processor 320, causes the wearable electronic device 301 to store data about an image having a first resolution in a volatile memory.

[0199] According to an embodiment, the memory 330 may store at least one instruction which, when executed by the at least one processor 320 , causes the wearable electronic device 301 to cause the image processing circuit 315 to read data stored in the volatile memory and perform pixel binning on the second area at a second resolution.

[0200] According to an embodiment, the memory 330 may store at least one instruction which, when executed by the at least one processor 320 , enables the wearable electronic device 301 to cause the image processing circuit 315 to perform pixel merging on a partial area of ​​the second area at a third resolution lower than the second resolution.

[0201] According to an embodiment, the partial region may include an edge region of the second region.

[0202] According to an embodiment, the first camera 370 may include a gaze tracking camera (eye tracking camera) for recognizing the gaze of the user.

[0203] According to an embodiment, the operating method of the wearable electronic device 301 may include the following operation: in a state where the wearable electronic device 301 is worn by the user, recognizing a gaze area of ​​the user through the first camera 370 included in the wearable electronic device 301 .

[0204] According to an embodiment, the operating method of the wearable electronic device 301 may include the following operation: acquiring an image having a first resolution generated by photographing an external object by the second camera 380 included in the wearable electronic device 301 .

[0205] According to an embodiment, the operating method of the wearable electronic device 301 may include the following operation: determining, in the image, a first area corresponding to a gaze area of ​​the user and a second area corresponding to an area different from the gaze area.

[0206] According to an embodiment, the operating method of the wearable electronic device 301 may include the following operations: acquiring a first image having a first resolution and corresponding to a first area from an image through a first channel 317 among multiple channels between a processor 320 and a second camera 380 included in the wearable electronic device 301 .

[0207] According to an embodiment, the operating method of the wearable electronic device 301 may include the following operation: acquiring a second image having a second resolution lower than the first resolution and corresponding to the second area through a second channel 319 among the plurality of channels.

[0208] According to an embodiment, the operating method of the wearable electronic device 301 may include the following operations: acquiring the first image and the second image in parallel through the first channel 317 and the second channel 319 .

[0209] According to an embodiment, the operating method of the wearable electronic device 301 may include the following operation: acquiring a third image in which the first image and the second image are combined.

[0210] According to an embodiment, the operating method of the wearable electronic device 301 may include the following operations: controlling the display 360 to display the third image through the glasses element 201 or 202 .

[0211] According to an embodiment, the operating method of the wearable electronic device 301 may include the following operation: storing data about an image having a first resolution in a volatile memory.

[0212] According to an embodiment, the operating method of the wearable electronic device 301 may include the following operations: causing the image processing circuit 315 to read data stored in the volatile memory, and perform pixel binning on the second area at the second resolution.

[0213] According to an embodiment, the operating method of the wearable electronic device 301 may include the following operation: causing the image processing circuit 315 to perform pixel merging on a partial area of ​​the second area at a third resolution lower than the second resolution.

[0214] According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction that may perform the following operations: in a state where the user wears the wearable electronic device, recognizing a gaze area of ​​the user through the first camera 370 included in the wearable electronic device 301 .

[0215] According to an embodiment, the non-transitory computer-readable recording medium may store at least one instruction which may perform the following operations: acquiring an image of a first resolution generated by photographing an external object by the second camera 380 included in the wearable electronic device 301 .

[0216] According to an embodiment, a non-transitory computer-readable recording medium may store at least one instruction that may perform the following operations: determining a first area corresponding to a gaze area of ​​a user and a second area corresponding to an area different from the gaze area in an image.

[0217] According to an embodiment, a non-transitory computer-readable recording medium may store at least one instruction that may perform the following operations: acquiring a first image having a first resolution and corresponding to a first area from an image through a first channel 317 among multiple channels between a processor 320 and a second camera 380 included in the wearable electronic device 301.

[0218] According to an embodiment, the non-transitory computer readable recording medium may store at least one instruction that may perform the following operations: acquiring a second image having a second resolution lower than the first resolution and corresponding to the second area through a second channel 319 among the plurality of channels.

[0219] The electronic device according to various embodiments may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. According to an embodiment of the present disclosure, the electronic device is not limited to those electronic devices described above.

[0220] It should be understood that the various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, but include various changes, equivalent forms or alternative forms for corresponding embodiments. For the description of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that the nouns in the singular form corresponding to the term may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each phrase in the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include any one or all possible combinations of the items listed together with the corresponding one of the multiple phrases. As used herein, terms such as "1st" and "2nd" or "first" and "second" may be used to simply distinguish the corresponding component from another component, and do not limit the component in other aspects (e.g., importance or order). It will be understood that if an element (e.g., a first element) is referred to as being “combined with another element (e.g., the second element)”, “combined to another element (e.g., the second element)”, “connected with another element (e.g., the second element)”, or “connected to another element (e.g., the second element)” when the terms “operably” or “communicatively” are used or when the terms “operably” or “communicatively” are not used, it means that the element may be directly (e.g., wired) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0221] As used in connection with various embodiments of the present disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "portion," or "circuit"). A module may be a single integrated component adapted to perform one or more functions or a minimum unit or portion of the single integrated component. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0222] The various embodiments described herein may be implemented as software (e.g., program 140) including one or more instructions stored in a storage medium (e.g., internal memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101, 200, 301). For example, under the control of a processor, a processor (e.g., processor 120) of the machine (e.g., electronic device 101, 200, 301) may call at least one of the one or more instructions stored in the storage medium and execute the at least one instruction with or without the use of one or more other components. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be run by an interpreter. A machine-readable storage medium may be provided in the form of a non-transitory storage medium. Among them, the term "non-transitory" only means that the storage medium is a tangible device and does not include a signal (e.g., an electromagnetic wave), but the term does not distinguish between data being semi-permanently stored in a storage medium and data being temporarily stored in a storage medium.

[0223] According to an embodiment, the method according to various embodiments of the present disclosure may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be released in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be downloaded via an application store (e.g., PlayStore). TM ) The computer program product may be published (e.g., downloaded or uploaded) online, or the computer program product may be distributed (e.g., downloaded or uploaded) directly between two user devices (e.g., smart phones). If published online, at least part of the computer program product may be temporarily generated, or at least part of the computer program product may be at least temporarily stored in a machine-readable storage medium (such as a memory of a manufacturer's server, an application store's server, or a forwarding server).

[0224] According to various embodiments, each component (e.g., module or program) in the above-mentioned components may include a single entity or multiple entities, and some entities in the multiple entities may be separately arranged in different components. According to various embodiments, one or more components in the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each component in the multiple components in the same or similar manner as the corresponding one component in the multiple components before integration. According to various embodiments, the operations performed by a module, program or another component may be performed sequentially, in parallel, repeatedly or in a heuristic manner, or one or more operations in the operations may be run or omitted in different orders, or one or more other operations may be added.

Claims

1. A wearable electronic device (101 in FIG. 1; 200 in FIG. 2) ； 301 in FIG. 3A ), including: A first camera ( 180 in FIG. 1 ; 251 , 252 in FIG. 2 ; 370 in FIG. 3A ); a second camera ( 180 in FIG. 1 ; 253 , 254 , 255 , 256 , 260 in FIG. 2 ; 380 in FIG. 3A ) including an image processing circuit ( 315 in FIG. 3A ); Memory (130 in FIG. 1; 330 in FIG. 3A); and at least one processor (120 in FIG. 1 ; 320 in FIG. 3A ), The memory stores at least one instruction, and when the at least one instruction is executed by the at least one processor, the wearable electronic device: When the wearable electronic device is worn by a user, identifying a gaze area of ​​the user through the first camera; acquiring an image having a first resolution generated by photographing an external object with the second camera; determining, in the image, a first area corresponding to the gaze area of ​​the user and a second area corresponding to an area different from the gaze area; and A first image having the first resolution and corresponding to the first area is acquired from the image through a first channel (317 in FIG. 3C ) among the multiple channels between the at least one processor and the second camera, and a second image having a second resolution lower than the first resolution and corresponding to the second area is acquired through a second channel (319 in FIG. 3C ) among the multiple channels.

2. The wearable electronic device according to claim 1, in, The wearable electronic device stores the at least one instruction, and when the at least one instruction is executed by the at least one processor, the wearable electronic device also acquires the first image and the second image in parallel through the first channel and the second channel.

3. The wearable electronic device according to claim 1 or 2, further comprising: include: Eyeglass components (201, 202 in FIG. 2); as well as a display (160 in FIG. 1 ; 240 in FIG. 2 ; 360 in FIG. 3A ) configured to display a virtual object through the eyeglass element, Wherein, when the at least one instruction is executed by the at least one processor, the wearable electronic device is also caused to: Acquire a third image that combines the first image and the second image; and The display is controlled to display the third image through the eyeglass element.

4. The wearable electronic device according to any one of claims 1 to 3, in, The first channel comprises a channel associated with an improved inter-integrated circuit I3C, and The second channel includes a channel associated with the mobile industry processor interface MIPI.

5. The wearable electronic device according to any one of claims 1 to 4, in, The first channel comprises a first virtual channel associated with a mobile industry processor interface MIPI, and The second channel includes a second virtual channel associated with the MIPI.

6. The wearable electronic device according to any one of claims 1 to 5, in, The second camera also includes a volatile memory (130 in FIG. 1 ; 313 in FIG. 3B ), and The memory stores the at least one instruction, and when the at least one instruction is executed by the at least one processor, the wearable electronic device stores data about the image with the first resolution in the volatile memory.

7. The wearable electronic device according to any one of claims 1 to 6, in, The memory stores the at least one instruction, which, when executed by the at least one processor, further causes the wearable electronic device to cause the image processing circuit to read data stored in the volatile memory and perform pixel binning on the second area at the second resolution.

8. The wearable electronic device according to any one of claims 1 to 7, in, The memory stores the at least one instruction, which, when executed by the at least one processor, further causes the wearable electronic device to cause the image processing circuit to perform pixel binning on a portion of the second area at a third resolution lower than the second resolution.

9. The wearable electronic device according to any one of claims 1 to 8, in, The partial region includes an edge region of the second region.

10. The wearable electronic device according to any one of claims 1 to 9, in, The first camera includes a gaze tracking camera (eye tracking camera) configured to identify a gaze of the user.

11. A wearable electronic device (101 in FIG. 1; 200 in FIG. 2) ； 301 in FIG. 3A), the method comprising: When the wearable electronic device is worn by a user, identifying a gaze area of ​​the user by using a first camera (180 in FIG. 1; 251, 252 in FIG. 2; 370 in FIG. 3A) included in the wearable electronic device; acquiring an image having a first resolution generated by photographing an external object by a second camera ( 180 in FIG. 1 ; 253 , 254 , 255 , 256 , 260 in FIG. 2 ; 380 in FIG. 3A ) included in the wearable electronic device; determining, in the image, a first area corresponding to the gaze area of ​​the user and a second area corresponding to an area different from the gaze area; Acquire, from the image, a first image having the first resolution and corresponding to the first area, through a first channel (317 in FIG. 3C ) of a plurality of channels between at least one processor (120 in FIG. 1 ; 320 in FIG. 3A ) included in the wearable electronic device and the second camera; and Through a second channel (319 in FIG. 3C ) among the plurality of channels, a second image having a second resolution lower than the first resolution and corresponding to the second region is acquired.

12. The method according to claim 11, further comprising: include: The first image and the second image are acquired in parallel through the first channel and the second channel.

13. The method according to any one of claims 11 to 12, in, The wearable electronic device further comprises: Eyeglass components; and a display configured to display a virtual object through the eyeglass element, and Wherein, the method further comprises: Acquire a third image that combines the first image and the second image; and The display is controlled to display the third image through the eyeglass element.

14. The method according to any one of claims 11 to 13, in, The first channel comprises a channel associated with an improved inter-integrated circuit I3C, and The second channel includes a channel associated with the mobile industry processor interface MIPI.

15. A non-transitory computer-readable recording medium storing at least one instruction capable of performing the following operations: When a wearable electronic device (101 in FIG. 1; 200 in FIG. 2; 301 in FIG. 3A) is worn by a user, identifying a gaze area of ​​the user through a first camera (180 in FIG. 1; 251, 252 in FIG. 2; 370 in FIG. 3A) included in the wearable electronic device; acquiring an image having a first resolution generated by photographing an external object by a second camera ( 180 in FIG. 1 ; 253 , 254 , 255 , 256 , 260 in FIG. 2 ; 380 in FIG. 3A ) included in the wearable electronic device; determining, in the image, a first area corresponding to the gaze area of ​​the user and a second area corresponding to an area different from the gaze area; Acquire, from the image, a first image having the first resolution and corresponding to the first area, by at least one processor (120 in FIG. 1 ; 320 in FIG. 3A ) included in the wearable electronic device and a first channel (317 in FIG. 3C ) among a plurality of channels in the second camera; as well as Through a second channel (319 in FIG. 3C ) among the plurality of channels, a second image having a second resolution lower than the first resolution and corresponding to the second region is acquired.

Images