Electronic device comprising image sensor, operation method therefor, and recording medium

Abstract

This electronic device may comprise: an image sensor; an image preprocessor comprising a circuit; a memory for storing instructions; and a processor comprising a processing circuit. The instructions, when executed individually and / or collectively by at least one processor, may instruct the electronic device to acquire raw data including biometric data by using the image sensor. The instructions, when executed individually and / or collectively by the at least one processor, may instruct the electronic device to store the raw data as first image data in a first buffer. The instructions, when executed individually and / or collectively by the at least one processor, may instruct the electronic device to store, in a second buffer, second image data generated by applying a change operation to the raw data by using the image preprocessor. The instructions, when executed individually and / or collectively by the at least one processor, may instruct the electronic device to: transmit the first image data stored in the first buffer to a first application belonging to a secure area; and apply a first algorithm for biometric authentication to the first image data. The instructions, when executed individually and / or collectively by the at least one processor, may instruct the electronic device to: transmit the second image data stored in the second buffer to a second application belonging to a non-secure area; and apply a second algorithm distinguished from the first algorithm to the second image data. Various other embodiments are possible.

Description

Electronic device including an image sensor, method of operation thereof, and recording medium

[0001] The present disclosure relates to an electronic device including an image sensor according to one embodiment, a method of operating the same, and a recording medium.

[0002] Biometric authentication is a technology that uses a person's physical and behavioral characteristics to verify their identity and grant access to specific systems or data. Unlike traditional authentication methods (e.g., PINs, patterns, and passwords), which require complex settings for enhanced security and are difficult to remember, biometric authentication eliminates the need for separate memorization and, as long as it possesses reliable recognition performance, is highly convenient for users. These advantages have led to its widespread adoption in many systems requiring user authentication, such as bank ATMs and mobile devices. While PINs, patterns, and passwords can be easily replaced with new PINs, patterns, and passwords if their settings are leaked, biometric data (e.g., face, iris, fingerprints, and vein patterns) used in biometric authentication cannot be changed once leaked, and the damage can be permanent. Therefore, securing the biometric data used in biometric authentication is crucial.

[0003] Electronic devices may include user interfaces that provide augmented reality (AR), virtual reality (VR), mixed reality (MR), and extended reality (XR) experiences.

[0004] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above-described matters constitute prior art related to the present disclosure.

[0005] Biometric authentication using a camera can perform authentication by receiving biometric data (e.g. face, iris, fingerprint, vein) from the camera and applying a biometric authentication algorithm.

[0006] Embodiments of the present disclosure can provide an electronic device capable of performing operations in a secure area (e.g., biometric authentication operations) and operations in a non-secure area through the same camera.

[0007] According to one embodiment, an electronic device may include an image sensor, an image preprocessor including circuitry, a memory storing instructions, and at least one processor including a processing circuit. The instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to acquire raw data including biometric data using the image sensor. The instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to store the raw data as first image data in a first buffer. The instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to store second image data generated by applying a change operation to the raw data using the image preprocessor in a second buffer. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to transfer the first image data stored in the first buffer to a first application belonging to a secure area and to apply a first algorithm for biometric authentication to the first image data. The instructions, when individually and / or collectively executed by at least one processor, may cause the electronic device to transfer the second image data stored in the second buffer to a second application belonging to a non-secure area and to apply a second algorithm distinct from the first algorithm to the second image data.

[0008] According to one embodiment, a method of operating an electronic device may include an operation of acquiring raw data including biometric data using an image sensor of the electronic device. The method may include an operation of storing the raw data as first image data in a first buffer. The method may include an operation of storing second image data generated by applying a change operation to the raw data in a second buffer. The method may include an operation of transferring the first image data stored in the first buffer to a first application belonging to a secure area, and applying a first algorithm for biometric authentication to the first image data. The method may include an operation of transferring the second image data stored in the second buffer to a second application belonging to a non-secure area, and applying a second algorithm distinct from the first algorithm to the second image data.

[0009] According to one embodiment, a computer-readable recording medium storing instructions that, when individually and / or collectively executed by at least one processor of an electronic device, cause the electronic device to perform at least one operation, wherein the at least one operation may include an operation of obtaining raw data including biometric data using an image sensor of the electronic device. The at least one operation may include an operation of storing the raw data as first image data in a first buffer. The at least one operation may include an operation of storing second image data generated by applying a change operation to the raw data in a second buffer. The at least one operation may include an operation of transmitting the first image data stored in the first buffer to a first application belonging to a secure area, and applying a first algorithm for biometric authentication to the first image data. The at least one operation may include transmitting the second image data stored in the second buffer to a second application belonging to a non-secure area, and applying a second algorithm distinct from the first algorithm to the second image data.

[0010] The above and other aspects, features and advantages of specific embodiments of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0011] FIG. 1 is a block diagram of an electronic device within a network environment, according to one embodiment.

[0012] FIG. 2A is a block diagram illustrating a camera module according to one embodiment.

[0013] FIG. 2b is a block diagram of an electronic device according to one embodiment.

[0014] FIG. 2c is a block diagram of an electronic device according to one embodiment.

[0015] FIG. 3 is a drawing illustrating the operation of an electronic device according to one embodiment.

[0016] FIG. 4 is a flowchart of a method of operating an electronic device according to one embodiment.

[0017] FIG. 5 is a flowchart of a method of operating an electronic device according to one embodiment.

[0018] FIG. 6 is a flowchart of a method of operating an electronic device according to one embodiment.

[0019] FIG. 7 is a flowchart of a method of operating an electronic device according to one embodiment.

[0020] FIG. 8 is a drawing illustrating the operation of an electronic device according to one embodiment.

[0021] FIG. 9 is a flowchart of a method of operating an electronic device according to one embodiment.

[0022] FIG. 10A is a drawing illustrating an electronic device according to one embodiment.

[0023] FIG. 10b is a drawing illustrating an electronic device according to one embodiment.

[0024] FIG. 10c is a drawing illustrating an electronic device according to one embodiment.

[0025] FIG. 11A is a diagram illustrating a light emitter and an image sensor of an electronic device according to one embodiment.

[0026] FIG. 11B is a diagram illustrating a light emitter and an image sensor of an electronic device according to one embodiment.

[0027] FIG. 12 is a drawing illustrating the operation of an electronic device according to one embodiment.

[0028] FIG. 13A is a diagram illustrating the operation of an electronic device according to one embodiment.

[0029] FIG. 13b is a diagram illustrating the operation of an electronic device according to one embodiment.

[0030] FIG. 14 is a drawing illustrating the operation of an electronic device according to one embodiment.

[0031] FIG. 15 is a flowchart of a method of operating an electronic device according to one embodiment.

[0032] FIG. 16 is a flowchart of a method of operating an electronic device according to one embodiment.

[0033] Embodiments of the present disclosure are described in more detail with reference to the drawings. However, the present disclosure may be implemented in many different forms and is not limited to the various embodiments described herein. In the present disclosure and drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0034] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100), according to one embodiment.

[0035] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0036] The processor (120) may include various processing circuits and / or multiple processors. For example, the term "processor" as used herein, including in the claims, may include various processing circuits including at least one processor, one or more of which may be configured to individually and / or collectively perform the various functions described herein in a distributed manner. As used herein, when "processor," "at least one processor," and "one or more processors" are described as being configured to perform various functions, these terms may encompass, for example, without limitation, a situation where one processor performs some of the recited functions and other processor(s) perform other of the recited functions, and a situation where a single processor performs all of the recited functions. Additionally, the at least one processor may include a combination of processors that perform the various functions enumerated / disclosed, for example, in a distributed manner. The at least one processor may execute program instructions to achieve or perform the various functions. The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134).According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0037] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0038] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0039] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0040] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0041] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0042] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0043] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0044] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0045] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0046] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0047] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0048] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0049] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0050] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0051] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0052] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for realizing 1eMBB, a loss coverage (e.g., 164 dB or less) for realizing mMTC, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for realizing URLLC.

[0053] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0054] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0055] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0056] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0057] FIG. 2A is a block diagram (200) illustrating a camera module (180) according to one embodiment.

[0058] Referring to FIG. 2A, a camera module (180) may include a lens assembly (210) (e.g., including a lens), a flash (220), an image sensor (230), an image stabilizer (240) (e.g., including circuitry), a memory (250) (e.g., a buffer memory), and / or an image signal processor (260) (e.g., including image processing circuitry). The lens assembly (210) may include at least one lens and may collect light emitted from a subject that is a target of image capturing. The lens assembly (210) may include one or more lenses. According to one embodiment, the camera module (180) may include a plurality of lens assemblies (210). In this case, the camera module (180) may form, for example, a dual camera, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies (210) may have the same lens properties (e.g., angle of view, focal length, autofocus, f-number, or optical zoom), or at least one lens assembly may have one or more lens properties that are different from the lens properties of the other lens assemblies. A lens assembly (210) may include, for example, a wide-angle lens or a telephoto lens.

[0059] The flash (220) can emit light used to enhance light emitted or reflected from a subject. According to one embodiment, the flash (220) can include one or more light-emitting diodes (e.g., red-green-blue (RGB) LED, white LED, infrared LED, or ultraviolet LED), or a xenon lamp. The image sensor (230) can acquire an image corresponding to the subject by converting light emitted or reflected from the subject and transmitted through the lens assembly (210) into an electrical signal. According to one embodiment, the image sensor (230) can include one image sensor selected from among image sensors having different properties, such as an RGB sensor, a black and white (BW) sensor, an IR sensor, or a UV sensor, a plurality of image sensors having the same property, or a plurality of image sensors having different properties. Each image sensor included in the image sensor (230) can be implemented using, for example, a CCD (charged coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor.

[0060] The image stabilizer (240) may include various circuits and may move at least one lens included in the lens assembly (210) or the image sensor (230) in a specific direction or control the operating characteristics of the image sensor (230) (e.g., adjust the read-out timing) in response to the movement of the camera module (180) or the electronic device (101) including the same. This allows compensating for at least some of the negative effects of the movement on the captured image. In one embodiment, the image stabilizer (240) may detect such movement of the camera module (180) or the electronic device (101) using a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera module (180). In one embodiment, the image stabilizer (240) may be implemented as, for example, an optical image stabilizer. The memory (250) can temporarily store at least a portion of the image acquired through the image sensor (230) for the next image processing task. For example, when image acquisition is delayed due to the shutter, or when multiple images are acquired at high speed, the acquired original image (e.g., a Bayer-patterned image or a high-resolution image) is stored in the memory (250), and a corresponding copy image (e.g., a low-resolution image) can be previewed through the display module (160). Thereafter, when a specified condition is satisfied (e.g., a user input or a system command), at least a portion of the original image stored in the memory (250) can be acquired and processed, for example, by the image signal processor (260). According to one embodiment, the memory (250) can be configured as at least a portion of the memory (130) or as a separate memory that operates independently therefrom.

[0061] The image signal processor (260) may include various image processing circuits and may perform one or more image processing operations on an image acquired through an image sensor (230) or an image stored in a memory (250). The one or more image processing operations may include, for example, depth map generation, 3D modeling, panorama generation, feature point extraction, image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Additionally or alternatively, the image signal processor (260) may perform control (e.g., exposure time control, or read-out timing control) on at least one of the components included in the camera module (180) (e.g., image sensor (230)). An image processed by the image signal processor (260) may be stored back in the memory (250) for further processing or provided to an external component of the camera module (180) (e.g., memory (130), display module (160), electronic device (102), electronic device (104), or server (108)). According to one embodiment, the image signal processor (260) may include at least one of the processors (120). It may be configured as a separate processor that is configured as a part of the processor (120) or operates independently of the processor (120). If the image signal processor (260) is configured as a separate processor from the processor (120), at least one image processed by the image signal processor (260) may be displayed through the display module (160) as is or after undergoing additional image processing by the processor (120).

[0062] According to one embodiment, the electronic device (101) may include a plurality of camera modules (180), each having different properties or functions. In this case, for example, at least one of the plurality of camera modules (180) may be a wide-angle camera, and at least another may be a telephoto camera. Similarly, at least one of the plurality of camera modules (180) may be a front camera, and at least another may be a rear camera.

[0063] FIG. 2b is a block diagram of an electronic device (101) according to one embodiment.

[0064] According to one embodiment, the electronic device (101) may include an image sensor (230), an image preprocessor (270) (e.g., including various circuits), a memory (130), and a processor (120) (e.g., including processing circuits).

[0065] According to one embodiment, the operation of the electronic device (101) may be controlled by the processor (120) of the electronic device (101) (e.g., the processor (120) of FIG. 1 and / or the image signal processor (260) of FIG. 2A). The electronic device (101) performing a specific operation may be that the electronic device (101) or a component included in the electronic device (101) is controlled by the processor (120) of the electronic device (101). The electronic device (101) may include one or more processors (120), and for the convenience of explanation, in the following, even when a plurality of processors (120) are implemented, the term “operation of the electronic device (101)” or “operation of the processor (120)” will be used.

[0066] According to one embodiment, the electronic device (101) (e.g., the processor (120)) may acquire raw data using the image sensor (230). The “raw data” may be data acquired through the image sensor (230) (e.g., an image corresponding to a subject (200, 201)). In FIG. 2B, the subject (200, 201) may include a human face portion (200) and / or a human eye portion (201), but this is merely an example. For example, the raw data acquired through the image sensor (230) may include biometric data (e.g., face, iris, fingerprint, vein).

[0067] According to one embodiment, the electronic device (101) (e.g., the processor (120)) may perform image processing (e.g., image preprocessing and / or change operation) on raw data using the image preprocessor (270). The image preprocessor (270) may be a hardware module including various circuits that perform image processing (e.g., image preprocessing and / or change operation). For example, the image preprocessor (270) may receive raw data acquired through the image sensor (230). The image preprocessor (270) may perform image preprocessing (e.g., 3A (auto focus, auto exposure, auto white balance)) and / or change operation on the raw data. The change operation is an operation that changes (or transforms) biometric data, and the change operation will be described later with reference to FIG. 3.

[0068] According to one embodiment, the electronic device (101) (e.g., processor (120)) may store data (e.g., image data) on which image processing (e.g., image preprocessing and / or modification operations) has been performed by the image preprocessor (270) in the memory (130). For example, the image preprocessor (270) may transmit data (e.g., image data) on which image processing (e.g., image preprocessing and / or modification operations) has been performed to the memory (130). The memory (130) may store data transmitted from the image preprocessor (270).

[0069] FIG. 2c is a block diagram of an electronic device (101) according to one embodiment.

[0070] According to one embodiment, the electronic device (101) may include a processor (120) (e.g., including a processing circuit), a camera module (180) (e.g., including a camera), a display (274) (e.g., at least one display included in the display module (160) of FIG. 1), a sensor (276) (e.g., at least one sensor included in the sensor module (176) of FIG. 1), a communication circuit (278) (e.g., at least one communication circuit included in the communication module (190) of FIG. 1), and a memory (130).

[0071] In one embodiment, the memory (130) may include an XR framework (280) and a hardware abstraction layer (299). For example, the XR framework (280) may be designed to utilize OpenXR standard-based rendering and input structures on the platform. For example, the hardware abstraction layer (299) may include an interface for hardware operations.

[0072] According to one embodiment, the XR framework (280) may include a compositing presentation manager (CPM) (281), a perception abstract layer (286), and a perception service (290).

[0073] In one embodiment, the recognition abstraction layer (286) may be an interface for the recognition service (290) and the CPM (281).

[0074] In one embodiment, the recognition service (290) may estimate tracking data through the recognition solution(s) based on the acquired sensor data. The recognition service (290) may include location tracking (291), spatial recognition (292), gesture tracking (293), eye tracking (294), face tracking (295), and other tracking (296) (e.g., tracking for other configurations). For example, the electronic device (101) may estimate the 6 degrees of freedom (6DoF) pose of the electronic device (101) (e.g., head mounted device (HMD)) using a recognition camera and IMU input through the location tracking (291) (e.g., head tracking (HeT)) of the recognition service (290). For example, the electronic device (101) can reconstruct the surrounding environment in 3D using the recognition camera input, 6DoF pose, and ToF (time of flight) through the spatial recognition (292) (e.g., SU (scene understanding)) of the recognition service (290), and perform a plane recognition operation based on this. The electronic device (101) may also utilize the photographing camera input as an additional function for the spatial recognition (292). For example, the electronic device (101) can track the user's hand pose and recognize gestures using the recognition camera and ToF input through the gesture tracking (293) (e.g., HaT (hand tracking)) of the recognition service (290). For example, the electronic device (101) can estimate and track the user's eye movement (or pupil movement) using the ET camera and the IR (infrared) LED through the eye tracking (294) (e.g., ET (eye tracking)) of the recognition service (290).For example, the electronic device (101) can estimate and track the user's facial movements using a camera for FT and an IR LED through face tracking (295) (e.g., FT (face tracking)) of the recognition service (290). For example, the electronic device (101) can estimate and track the user's iris using an IR camera (e.g., an IR sensor included in the image sensor of FIG. 2) and an IR LED through other tracking (296) (e.g., iris tracking) of the recognition service (290).

[0075] In one embodiment, the CPM (281) may include a runtime (282), a pass-through (283), an input manager (284), and a compositor (285). The CPM (281) may compose a virtual node rendered based on data (e.g., tracking data) obtained from a recognition abstraction layer (286) through the input manager (284) and a node obtained through the pass-through (283) (e.g., a pass-through library).

[0076] FIG. 3 is a drawing illustrating the operation of an electronic device (101) according to one embodiment.

[0077] Referring to FIG. 3, according to one embodiment, the memory (130) of the electronic device (101) may include a first buffer (310) (e.g., a secure buffer) and a second buffer (320) (e.g., a non-secure buffer). For example, the first buffer (310) and the second buffer (320) may be areas that are dynamically allocated in the memory (130).

[0078] According to one embodiment, the first buffer (310) (e.g., a secure buffer) may be a buffer in which data including biometric data is stored. The secure buffer (e.g., 310) may be a buffer accessible from a secure area (e.g., the first area of ​​FIG. 8). For example, the secure area may be an area corresponding to a secure execution environment (e.g., a trusted execution environment (TEE)). According to one embodiment, the electronic device (101) may include a secure processor. The secure processor may correspond to a physically separate secure area. For example, the secure area may be an area corresponding to a secure processor including secure memory or an embedded secure element (eSE). For example, the secure processor or eSE of the electronic device (101) may process data stored in the secure buffer (e.g., 310). According to one embodiment, the second buffer (320) (e.g., non-secure buffer) may be a buffer in which data on which a modification operation has been performed on biometric data is stored. The non-secure buffer (e.g., 320) may be a buffer accessible from a non-secure area (e.g., the second area of ​​FIG. 8). The non-secure area may be an area corresponding to a general execution environment (e.g., a rich execution environment (REE)). According to one embodiment, the electronic device (101) (e.g., processor (120)) may store data (e.g., image data) on which image processing (e.g., image preprocessing and / or modification operation) has been performed by the image preprocessor (270) in the first buffer (310) and / or the second buffer (320) of the memory (130).

[0079] Referring to FIG. 3, according to one embodiment, an electronic device (101) (e.g., processor (120)) may perform image processing (e.g., image preprocessing and / or modification operation) on raw data acquired through an image sensor (230) using an image preprocessor (270).

[0080] For example, the electronic device (101) can perform image preprocessing (e.g., P of FIG. 3) on raw data using an image preprocessor (270). The image preprocessing can include, for example, 3A (auto focus, auto exposure, auto white balance).

[0081] For example, the electronic device (101) can perform a change operation (e.g., C in FIG. 3) on raw data using the image preprocessor (270). FIG. 3 illustrates an example of performing image preprocessing and then performing a change operation, but the electronic device (101) can also perform a change operation directly on raw data acquired through the image sensor (230), or can perform a change operation on raw data on which image preprocessing has been performed, as shown in FIG. 3. The “change operation” may be an operation of changing (or transforming) biometric data included in raw data (or raw data on which image preprocessing has been performed). For example, the change operation may be an operation of performing image processing on biometric data or raw data including biometric data (or raw data on which image preprocessing has been performed). For example, the electronic device (101) can perform image processing on the entire raw data including biometric data (or raw data on which image preprocessing has been performed). For example, the electronic device (101) may perform image processing on an area corresponding to biometric data in raw data containing biometric data (or raw data subjected to image preprocessing). For example, a change operation (e.g., image processing performed on biometric data or raw data containing biometric data (or raw data subjected to image preprocessing)) may include at least one of blank processing (e.g., 321 in FIG. 3), removal of biometric data (e.g., 322 in FIG. 3), blur processing (e.g., 323 in FIG. 3), or hatching processing (e.g., 324 in FIG. 3). “Blank processing” may be blank processing for the entire area of ​​raw data (or raw data subjected to image preprocessing), or an area corresponding to biometric data included in the raw data (or raw data subjected to image preprocessing). “Removal of biometric data” may be an operation of removing an area corresponding to biometric data included in raw data (or raw data subjected to image preprocessing).When removal of biometric data is performed, an area other than an area corresponding to the biometric data may be identical to the original data (e.g., raw data (or raw data on which image preprocessing has been performed)). "Blur processing" may be blur processing for the entire area of ​​raw data (or raw data on which image preprocessing has been performed), or an area corresponding to biometric data included in the raw data (or raw data on which image preprocessing has been performed). For example, when blur processing is performed for an area corresponding to biometric data, an area other than an area corresponding to the biometric data may be identical to the original data (e.g., raw data (or raw data on which image preprocessing has been performed)). "Hashing processing" may be hatching processing for the entire area of ​​raw data (or raw data on which image preprocessing has been performed), or an area corresponding to biometric data included in the raw data (or raw data on which image preprocessing has been performed). For example, if hatching is performed on an area corresponding to biometric data, an area other than the area corresponding to biometric data may be identical to the original data (e.g., raw data (or raw data on which image preprocessing has been performed)).

[0082] According to one embodiment, the electronic device (101) (e.g., the processor (120)) can identify the location of biometric data in raw data (or raw data on which image preprocessing has been performed) and perform image processing (e.g., at least one of blanking, removal of biometric data, blurring, or hatching) on ​​the location of the identified biometric data. For example, the electronic device (101) (e.g., the processor (120)) can identify biometric data (e.g., the location of biometric data) in raw data (or raw data on which image preprocessing has been performed) based on information stored in the memory (130) (e.g., information on the location of biometric data). For example, the electronic device (101) (e.g., the processor (120)) can store information on the location of biometric data to be included in the raw data in the memory (130) before the raw data is acquired. The electronic device (101) can identify the location of biometric data in raw data (or raw data on which image preprocessing has been performed) based on information stored in the memory (130) (e.g., information on the location of biometric data to be included in the raw data). According to one embodiment, the electronic device (101) can include a memory (130) in which information on the location of biometric data to be included in the raw data is stored. According to one embodiment, the electronic device (101) can identify the location of biometric data at a first point in time based on first raw data or image data corresponding to the first raw data, and store information on the location of the identified biometric data in the memory (130). The electronic device (101) can acquire second raw data at a second point in time after the first point in time, and identify the location of biometric data in the second raw data (or second raw data on which image preprocessing has been performed) based on information on the location of biometric data identified before the second point in time (e.g., the first point in time).

[0083] According to one embodiment, referring to FIG. 3, the electronic device (101) may store first image data (311) (e.g., image data including biometric data (e.g., iris)) identified by performing image preprocessing (P) on raw data, in a first buffer (310) of the memory (130). For example, referring to FIG. 3, the electronic device (101) may store second image data (325) (e.g., image data in which image processing is performed on an area corresponding to biometric data) identified by performing image preprocessing (P) and a change operation (C) on raw data, in a second buffer (320) of the memory (130). The second image data (325) stored in the second buffer (320) may be data in which a change operation is performed on raw data acquired through the image sensor (230), or may be data in which a change operation is performed on first image data (311) in which image preprocessing is performed on raw data.

[0084] Fig. 4 is a flowchart of an operating method of an electronic device (101) according to one embodiment. Fig. 4 can be explained with reference to the previously described embodiment.

[0085] At least some of the operations of FIG. 4 may be omitted. The order of the operations of FIG. 4 may be changed. Operations other than those of FIG. 4 may be performed before, during, or after the operations of FIG. 4.

[0086] Referring to FIG. 4, in operation 401, according to one embodiment, the electronic device (101) (e.g., processor (120)) may obtain raw data using the image sensor (230). The raw data may be data obtained through the image sensor (230) (e.g., an image corresponding to a subject (200, 201)).

[0087] In operation 403, according to one embodiment, the electronic device (101) (e.g., processor (120)) may perform image preprocessing (e.g., 3A) on the raw data of operation 401 using the image preprocessor (270). The electronic device (101) may confirm image data (e.g., 311 of FIG. 3) by performing image preprocessing (e.g., 3A) on the raw data of operation 401.

[0088] In operation 405, according to one embodiment, the electronic device (101) (e.g., processor (120)) may store image data (e.g., 311 of FIG. 3) on which image preprocessing has been performed in a first buffer (310) (e.g., secure buffer) of the memory (130).

[0089] Fig. 5 is a flowchart of an operating method of an electronic device (101) according to one embodiment. Fig. 5 can be explained with reference to the previously described embodiment.

[0090] At least some of the operations of FIG. 5 may be omitted. The order of the operations of FIG. 5 may be changed. Operations other than those of FIG. 5 may be performed before, during, or after the operations of FIG. 5.

[0091] Referring to FIG. 5, in operation 501, according to one embodiment, the electronic device (101) (e.g., processor (120)) may obtain raw data using the image sensor (230). The raw data may be data obtained through the image sensor (230) (e.g., an image corresponding to a subject (200, 201)).

[0092] In operation 503, according to one embodiment, the electronic device (101) (e.g., processor (120)) may perform image preprocessing (e.g., 3A) on the raw data of operation 501 using the image preprocessor (270).

[0093] In operation 505, according to one embodiment, the electronic device (101) (e.g., the processor (120)) may perform a change operation on the data on which image preprocessing was performed in operation 503, using the image preprocessor (270). The change operation may be an operation of performing image processing (e.g., at least one of blank processing, removal of biometric data, blur processing, or hatching processing) on ​​the biometric data. The electronic device (101) may confirm image data (e.g., 325 of FIG. 3) by performing the image preprocessing of operation 503 and the change operation of operation 505 on the raw data of operation 501.

[0094] In operation 507, according to one embodiment, the electronic device (101) (e.g., processor (120)) may store image data (e.g., 325 of FIG. 3) on which image preprocessing and modification operations have been performed in a second buffer (320) (e.g., non-secure buffer) of the memory (130).

[0095] Fig. 6 is a flowchart of an operating method of an electronic device (101) according to one embodiment. Fig. 6 can be explained with reference to the previously described embodiment.

[0096] At least some of the operations of FIG. 6 may be omitted. The order of the operations of FIG. 6 may be changed. Operations other than those of FIG. 6 may be performed before, during, or after the operations of FIG. 6.

[0097] Referring to FIG. 6, in operation 601, according to one embodiment, the electronic device (101) (e.g., processor (120)) may obtain raw data using the image sensor (230). The raw data may be data obtained through the image sensor (230) (e.g., an image corresponding to a subject (200, 201)).

[0098] In operation 603, according to one embodiment, the electronic device (101) (e.g., processor (120)) can confirm (or generate or process or obtain) the first image data (e.g., 311 of FIG. 3) by performing image preprocessing on the raw data of operation 601 using the image preprocessor (270).

[0099] In operation 605, according to one embodiment, the electronic device (101) (e.g., processor (120)) can perform a change operation on the first image data (e.g., 311 of FIG. 3) of operation 603 using the image preprocessor (270) to confirm (or generate or process or obtain) the second image data (e.g., 325 of FIG. 3).

[0100] In operation 607, according to one embodiment, the electronic device (101) (e.g., processor (120)) may store the first image data (e.g., 311 of FIG. 3) of operation 603 in the first buffer (310). The electronic device (101) may store the first image data (e.g., 311 of FIG. 3) provided from the image preprocessor (270) in the first buffer (310).

[0101] In operation 609, according to one embodiment, the electronic device (101) (e.g., processor (120)) may store the second image data (e.g., 325 of FIG. 3) of operation 605 in the second buffer (320). The electronic device (101) may store the second image data (e.g., 325 of FIG. 3) provided from the image preprocessor (270) in the second buffer (320).

[0102] Fig. 7 is a flowchart of an operating method of an electronic device (101) according to one embodiment. Fig. 7 can be explained with reference to the previously described embodiment.

[0103] At least some of the operations of FIG. 7 may be omitted. The order of the operations of FIG. 7 may be changed. Operations other than those of FIG. 7 may be performed before, during, or after the operations of FIG. 7.

[0104] Referring to Figure 7, one can understand the change operation for raw data.

[0105] Referring to FIG. 7, in operation 701, according to one embodiment, the electronic device (101) (e.g., processor (120)) may obtain raw data including biometric data. The biometric data may include, but is not limited to, a face, an iris, a fingerprint, a vein, etc.

[0106] In operation 703, according to one embodiment, the electronic device (101) (e.g., the processor (120)) may perform image processing (e.g., at least one of blanking, removal of biometric data, blurring, or cross-hatching) on ​​the raw data of operation 701. The electronic device (101) may perform image processing (e.g., at least one of blanking, removal of biometric data, blurring, or cross-hatching) on ​​the raw data including biometric data. Image processing for the raw data including biometric data may be understood with reference to FIG. 3. For example, the electronic device (101) may perform image processing (e.g., at least one of blanking, removal of biometric data, blurring, or cross-hatching) before performing image preprocessing on the raw data. For example, the electronic device (101) may also perform image preprocessing after performing image processing (e.g., at least one of blanking, removal of biometric data, blurring, or cross-hatching) on ​​the raw data. For example, the electronic device (101) may perform image processing (e.g., at least one of blank processing, removal of biometric data, blur processing, or hatching) after performing image preprocessing on the raw data. For example, the electronic device (101) may perform image processing (e.g., at least one of blank processing, removal of biometric data, blur processing, or hatching) without performing image preprocessing on the raw data.

[0107] According to one embodiment, the electronic device (101) (e.g., processor (120)) may store data on which the change operation (e.g., 703) of FIG. 7 is performed in a non-secure buffer (e.g., 320 of FIG. 3).

[0108] FIG. 8 is a diagram illustrating the operation of an electronic device (101) according to one embodiment. FIG. 9 is a flowchart illustrating a method of operating an electronic device according to one embodiment. FIGS. 8 and 9 can be described with reference to the previously described embodiment.

[0109] With reference to FIG. 8, a secure area (e.g., first area) and a non-secure area (e.g., second area) can be understood.

[0110] Referring to FIG. 8, according to one embodiment, an application (e.g., a first application (810)) classified as a configuration belonging to a secure area (e.g., a first area) can access a first buffer (310) of a memory (130). For example, the first application (810) may be a binary operating in a secure area (e.g., an area corresponding to a TEE or an area corresponding to a secure processor (or eSE)). The secure area may be an area corresponding to a configuration(s) capable of handling data including biometric data. The configuration belonging to the secure area may handle data including biometric data. The data including biometric data may be data for which no modification operation has been performed.

[0111] Referring to FIG. 8, according to one embodiment, the second buffer (320) of the memory (130) can be accessed by an application (e.g., a second application (820)) classified as a configuration belonging to a non-secure area (e.g., a second area). For example, the second application (820) may be a binary operating in a non-secure area (e.g., a REE). The non-secure area may be an area corresponding to a configuration(s) capable of handling data that does not include biometric data. The configuration belonging to the non-secure area may handle data that does not include biometric data. The data that does not include biometric data may be data that does not include biometric data from the beginning, or data that has undergone a modification operation on original data (e.g., raw data or raw data on which image preprocessing has been performed).

[0112] According to one embodiment, the electronic device (101) may include a physically distinct separate security processor (or eSE). The security processor (or eSE) of the electronic device (101) may correspond to a secure area. The security processor (or eSE) of the electronic device (101) may access a first buffer (310). The security processor (or eSE) of the electronic device (101) may process data stored in the first buffer (310). The security processor (or eSE) of the electronic device (101) may apply a biometric authentication algorithm (e.g., the first algorithm (811)) to the data stored in the first buffer (310).

[0113] With reference to FIG. 9, the configuration of FIG. 8 and the previously described embodiment can be described in detail.

[0114] At least some of the operations of FIG. 9 may be omitted. The order of the operations of FIG. 9 may be changed. Operations other than those of FIG. 9 may be performed before, during, or after the operations of FIG. 9.

[0115] Referring to FIG. 9, in operation 901, according to one embodiment, the electronic device (101) (e.g., the processor (120)) may transmit (e.g., transfer) first image data (e.g., 311 of FIG. 3) (e.g., data including biometric data) stored in the first buffer (310) to the first application (810). The first application (810) may obtain the first image data (e.g., 311 of FIG. 3) stored in the first buffer (310). The first application (810) may be an application belonging to a secure area (e.g., the first area). The first application (810) may be an application for biometric authentication. For example, the first application (810) may perform biometric authentication on a user of the electronic device (101) by applying a first algorithm (811) to the image data.

[0116] In operation 903, according to one embodiment, the electronic device (101) (e.g., the processor (120)) may transmit (e.g., transfer) second image data (e.g., 325 of FIG. 3) (e.g., data not including biometric data or data on which image processing is performed on biometric data) stored in the second buffer (320) to the second application (820). The second application (820) may obtain the second image data (e.g., 325 of FIG. 3) stored in the second buffer (320). The second application (820) may be an application belonging to a non-secure area (e.g., the second area). The second application (820) may be an application unrelated to biometric authentication. The second application (820) may be an application for camera image processing that does not require security. For example, the second application (820) can perform gaze tracking on a user of the electronic device (101) by applying the second algorithm (821) to image data. For example, the second application (820) can perform image preview (e.g., image preview through a display (e.g., 160 of FIG. 1)) by applying the second algorithm (821) to the image data. For example, the second application (820) can perform video recording by applying the second algorithm (821) to the image data. The operations that the second application (820) can perform are not limited to gaze tracking, image preview, or video recording.

[0117] In one embodiment, at least a portion of the period during which operation 901 is performed may correspond to at least a portion of the period during which operation 903 is performed. For example, the electronic device (101) (e.g., processor (120)) may transmit second image data (e.g., 325 of FIG. 3) stored in a second buffer (320) to a second application (820) during a period that at least partially overlaps with the period during which the first image data (e.g., 311 of FIG. 3) stored in the first buffer (310) is transmitted to the first application (810).

[0118] In one embodiment, operations 901 and 903 may be performed alternately. For example, the electronic device (101) (e.g., processor (120)) may transfer first image data (e.g., 311 of FIG. 3) stored in a first buffer (310) to a first application (810) during a first period, and transfer second image data (e.g., 325 of FIG. 3) stored in a second buffer (320) to a second application (820) during a second period after the first period. For example, the electronic device (101) (e.g., the processor (120)) may transfer first image data (e.g., 311 of FIG. 3) stored in the first buffer (310) to the first application (810) during a first period, transfer second image data (e.g., 325 of FIG. 3) stored in the second buffer (320) to the second application (820) during a second period after the first period, transfer first image data (e.g., 311 of FIG. 3) stored in the first buffer (310) to the first application (810) during a third period after the second period, and transfer second image data (e.g., 325 of FIG. 3) stored in the second buffer (320) to the second application (820) during a fourth period after the third period.

[0119] In operation 905, according to one embodiment, the electronic device (101) (e.g., the processor (120)) may apply a first algorithm (811) to first image data (e.g., 311 of FIG. 3) (e.g., data including biometric data) provided from the first buffer (310) using a first application (810). The first application (810) may be an application for biometric authentication. The first algorithm (811) may be an algorithm for biometric authentication. The electronic device (101) may perform biometric authentication for a user of the electronic device (101) by applying the first algorithm (811) (e.g., an algorithm for biometric authentication) to the first image data (e.g., 311 of FIG. 3) (e.g., data including biometric data). There is no limitation on the type of the first application (810) (e.g., an application for biometric authentication). There is no limitation on the type of the first algorithm (811) (e.g., an algorithm for biometric authentication). There is no limitation on the type of biometric authentication.

[0120] In operation 907, according to one embodiment, the electronic device (101) (e.g., the processor (120)) may apply a second algorithm (821) to second image data (e.g., 325 of FIG. 3) (e.g., data that does not include biometric data or data on which a modification operation has been performed on biometric data) provided from the second buffer (320) using a second application (820). The second application (820) may be an application for processing camera images that do not require security. The second algorithm (821) may be an algorithm for processing camera images that do not require security. For example, the second algorithm (821) may be an algorithm for tracking a user's gaze. For example, the second algorithm (821) may be an algorithm for previewing an image. For example, the second algorithm (821) may be an algorithm for recording a video. For example, the electronic device (101) can perform gaze tracking on a user of the electronic device (101) by applying the second algorithm (821) to second image data (e.g., 325 of FIG. 3) (e.g., data that does not include biometric data or data that has undergone a change operation on biometric data). For example, the electronic device (101) can perform image preview by applying the second algorithm (821) to second image data (e.g., 325 of FIG. 3) (e.g., data that does not include biometric data or data that has undergone a change operation on biometric data). For example, the electronic device (101) can perform video recording by applying the second algorithm (821) to second image data (e.g., 325 of FIG. 3) (e.g., data that does not include biometric data or data that has undergone a change operation on biometric data). There is no limitation on the type of the second application (820). There is no limitation on the type of the second algorithm (821). There is no limitation on the operations that the second application (820) can perform.

[0121] FIG. 10A is a drawing illustrating an electronic device (101) according to one embodiment. FIG. 10B is a drawing illustrating an electronic device (101) according to one embodiment. FIG. 10C is a drawing illustrating an electronic device (101) according to one embodiment.

[0122] FIGS. 10A and 10B are drawings showing the front and back of an electronic device (101) according to one embodiment.

[0123] Referring to FIGS. 10A and 10B, in one embodiment, camera modules (1011, 1012, 1013, 1014, 1015, 1016) and / or depth sensors (1017) for obtaining information related to the surrounding environment of the electronic device (101) may be arranged on the first surface (1010) of the housing.

[0124] In one embodiment, the camera modules (1011, 1012) can acquire images related to the environment surrounding the electronic device.

[0125] In one embodiment, the camera modules (1013, 1014, 1015, 1016) can acquire images while the electronic device is worn by a user. The camera modules (1013, 1014, 1015, 1016) can be used for hand detection and tracking, and user gesture (e.g., hand movement) recognition. The camera modules (1013, 1014, 1015, 1016) can be used for 3DoF, 6DoF head tracking, position (spatial, environmental) recognition, and / or movement recognition. In one embodiment, the camera modules (1011, 1012) can also be used for hand detection and tracking, and user gesture.

[0126] In one embodiment, a depth sensor (1017) may be configured to transmit a signal and receive a signal reflected from a subject, and may be used for purposes such as time of flight (TOF) to determine the distance to an object. Instead of or in addition to the depth sensor (1017), the camera modules (1013, 1014, 1015, 1016) may determine the distance to an object.

[0127] According to one embodiment, a camera module (1025, 1026) for facial recognition and / or a display (1021) (and / or a lens) may be disposed on the second side (1020) of the housing.

[0128] In one embodiment, a face recognition camera module (1025, 1026) adjacent to the display may be used to recognize a user's face, or may recognize and / or track both eyes of the user.

[0129] In one embodiment, the display (1021) (and / or lens) may be disposed on the second side (1020) of the electronic device (101). In one embodiment, the electronic device (101) may not include camera modules (1015, 1016) among the plurality of camera modules (1013, 1014, 1015, 1016). Although not illustrated in FIGS. 10A and 10B , the electronic device (101) may further include at least one of the configurations illustrated in FIG. 10C .

[0130] FIG. 10c is a perspective view illustrating the internal configuration of an electronic device according to one embodiment.

[0131] Referring to FIG. 10c, an electronic device (101) according to one embodiment may include at least one of a light output module (1041) (e.g., including a light output circuit), a display member (1031) (e.g., including a waveguide), and a camera module (1050) (e.g., including a camera).

[0132] According to one embodiment, the light output module (1041) may include a light source capable of outputting an image, and a lens for guiding the image to the display member (1031). According to one embodiment, the light output module (1041) may include at least one of a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED).

[0133] According to one embodiment, the display member (1031) may include an optical waveguide (e.g., a waveguide). According to one embodiment, an output image of the optical output module (1041) incident on one end of the optical waveguide may be propagated inside the optical waveguide and provided to a user. According to one embodiment, the optical waveguide may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) or at least one reflective element (e.g., a reflective mirror). For example, the optical waveguide may guide the output image of the optical output module (1041) to the user's eyes by using at least one diffractive element or reflective element.

[0134] In one embodiment, the camera module (1050) (e.g., the camera module (180) of FIG. 1) can capture still images and / or moving images. In one embodiment, the camera module (1050) can be positioned within a lens frame and can be positioned around the display member (1031).

[0135] According to one embodiment, the first camera module (1051) can capture and / or recognize the trajectory of the user's eye (e.g., pupil, iris) or gaze. According to one embodiment, the first camera module (1051) can periodically or aperiodically transmit information related to the trajectory of the user's eye or gaze (e.g., trajectory information) to a processor (e.g., processor (120) of FIG. 1).

[0136] In one embodiment, the second camera module (1053) can capture an external image.

[0137] According to one embodiment, the third camera module (1055) can be used for hand detection and tracking, and user gesture (e.g., hand movement) recognition. According to one embodiment, the third camera module (1055) can be used for 3DoF (10 degrees of freedom), 6DoF head tracking, position (spatial, environmental) recognition, and / or movement recognition. According to one embodiment, the second camera module (1053) can also be used for hand detection and tracking, and user gesture recognition. According to one embodiment, at least one of the first camera module (1051) to the third camera module (1055) can be replaced with a sensor module (e.g., a LiDAR sensor). For example, the sensor module can include at least one of a vertical cavity surface emitting laser (VCSEL), an infrared sensor, and / or a photodiode.

[0138] According to one embodiment, the electronic device (101) may include a voice input device (1062-1, 1062-2) and a voice output device (1063-1, 1063-2).

[0139] As described above, according to one embodiment, the electronic device (101) may have a form factor for being worn on a user's head. For example, the electronic device (101) may include a housing for being worn on a user's head. The electronic device (101) may further include a strap and / or a wearing member for being secured on a body part of the user. For example, the electronic device (101) may provide a user experience based on augmented reality, virtual reality, and / or mixed reality while being worn on the user's head.

[0140] According to one embodiment, the electronic device (101) may have a form factor that can be carried by a user (e.g., held in the user's hand). For example, the electronic device (101) may be a cell phone or a tablet.

[0141] According to one embodiment, the electronic device (101) may be a device placed in a specific space or attached to another device. There is no limitation on the implementation method of the electronic device (101).

[0142] FIG. 11A is a diagram illustrating a light emitter (1100) and an image sensor (230) of an electronic device (101) according to one embodiment. FIG. 11B is a diagram illustrating a light emitter (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and / or 1112) and an image sensor (230) of an electronic device (101) according to one embodiment. FIG. 12 is a diagram illustrating an operation of an electronic device (101) according to one embodiment. FIG. 13A is a diagram illustrating an operation of an electronic device (101) according to one embodiment. FIG. 13B is a diagram illustrating an operation of an electronic device (101) according to one embodiment. Figures 11a, 11b, 12, 13a, and 13b can be explained with reference to the previously described embodiment.

[0143] Referring to FIG. 11A, according to one embodiment, an electronic device (101) may include a light emitter (1100) and an image sensor (230). The light emitter (1100) may be the flash (220) of FIG. 2. The light emitter (1100) may include an infrared light emitting diode (IR LED). There is no limitation on the number of light emitters (1100). For example, the light emitter (1100) of FIG. 11A may include a plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112) of FIG. 11B.

[0144] Referring to FIGS. 11A and 11B , according to one embodiment, the electronic device (101) may be implemented in a goggle, glasses, or headset type design (e.g., a form factor for being worn on a user's head). This design allows the electronic device (101) to be less sensitive to external environmental factors such as light while being closely attached and fixed to the user's eye, and the image sensor (230) (e.g., an infrared (IR) camera) may be fixed and operated at the eye position.

[0145] Referring to FIG. 11B, according to one embodiment, the electronic device (101) may include a plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112) and a plurality of image sensors (230). FIG. 11B is a diagram illustrating light emitters and image sensors corresponding to the left and right eyes of a user. For convenience of explanation, referring to FIG. 11b, a plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112) and a plurality of image sensors (230) corresponding to the user's left eye can be described. This description can be equally applied to light emitters and image sensors corresponding to the user's right eye. According to one embodiment, the electronic device (101) (e.g., processor (120)) may acquire raw data by using at least one of a plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112) and at least one of a plurality of image sensors (230). For example, the electronic device (101) (e.g., processor (120)) may control at least one of the plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112) to be turned on and control the remaining at least one to be turned off, depending on a period. This will be described later.

[0146] FIG. 12 may include raw data (1200) acquired through an image sensor (230). Referring to FIG. 12, it is possible to understand a phenomenon in which, in raw data (1200) acquired through an image sensor (230), a light emitter (e.g., at least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112 of FIG. 11B) is projected (1210) onto a pupil (1202) and an iris (1203). For example, raw data (1200) may include biometric data (e.g., iris (1203)). The raw data (1200) may include data about an eyebrow (1201), a pupil (1202), an iris (1203), and a sclera (1204). The electronic device (101) may acquire the raw data (1200) after turning on at least one of a plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112). At this time, a light source (e.g., at least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112 of FIG. 11b) may be projected (1210) onto the raw data (1200). As a light source (e.g., at least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112 of FIG. 11b) is projected (1210) onto the raw data (1200), data corresponding to the iris (1203) may be distorted. On the other hand, gaze tracking algorithms that require data on the eyelids and eyebrows (e.g., the second algorithm (821) of FIG. 8) may be less sensitive to distortion caused by such light projections (e.g., 1210). This will be described in detail in FIGS. 14 to 16.

[0147] According to one embodiment, FIG. 13A may include data (1300) on which image processing has been performed. According to one embodiment, the electronic device (101) (e.g., the processor (120)) may perform image processing on an area corresponding to biometric data (e.g., an area corresponding to the iris (1203)). For example, referring to FIG. 13A, the electronic device (101) (e.g., the processor (120)) may identify the location of the pupil (1202) (or the iris (1203)) in the raw data (1200) of FIG. 12, and based on the identified location of the pupil (1202) (or the iris (1203)), may distort or delete (1310) the surrounding data including the iris (1203). For example, the electronic device (101) (e.g., the processor (120)) can determine the location of a designated area based on the characteristics of the electronic device (101) (e.g., the design of a goggle, glasses, or headset type). The designated area may be an area corresponding to biometric data. For example, the electronic device (101) may be implemented with a design of a goggle, glasses, or headset type (e.g., a form factor for being worn on a user's head), and accordingly, the location of the area corresponding to biometric data in the raw data acquired from the electronic device (101) may be included in a designated range. For example, when the user wears the electronic device (101), the distance and angle from a part of the user's body (e.g., an eyebrow, an eyeball, a pupil, an iris) to the electronic device (101) (e.g., an image sensor (230) of the electronic device (101)) may be constant. Accordingly, raw data acquired by the electronic device (101) may include biometric data, and in this case, the biometric data may be included in a designated area (e.g., a designated range) of the raw data. For example, the electronic device (101) may store information about the location of a designated area based on the characteristics of the electronic device (101) (e.g., the design of goggles, glasses, or a headset type) in the memory (130).For example, the electronic device (101) can acquire raw data while the electronic device (101) is worn on the user's head. The electronic device (101) (e.g., the processor (120)) can identify the location of a designated area based on the characteristics of the electronic device (101) in the acquired raw data as the location of an area corresponding to biometric data. The electronic device (101) (e.g., the processor (120)) can perform image processing (e.g., distortion or deletion) on an area corresponding to biometric data corresponding to the location of the designated area. For example, the image processing on the area corresponding to biometric data can include at least one of blanking, blurring, deletion of biometric data, or cross-hatching. According to one embodiment, the electronic device (101) (e.g., the processor (120)) can identify the location of biometric data in the raw data (or raw data on which image preprocessing has been performed) and perform image processing (e.g., at least one of blanking, removal of biometric data, blurring, or cross-hatching) on ​​the location of the identified biometric data. For example, the electronic device (101) (e.g., the processor (120)) can identify biometric data (e.g., the location of biometric data) in raw data (or raw data on which image preprocessing has been performed) based on information (e.g., information on the location of biometric data) stored in the memory (130). For example, the electronic device (101) (e.g., the processor (120)) can store information on the location of biometric data to be included in the raw data in the memory (130) before the raw data is acquired. The electronic device (101) can identify the location of biometric data in the raw data (or raw data on which image preprocessing has been performed) based on information (e.g., information on the location of biometric data to be included in the raw data) stored in the memory (130). According to one embodiment, the electronic device (101) may include a memory (130) in which information on the location of biometric data to be included in the raw data is stored.According to one embodiment, the electronic device (101) can identify the location of biometric data at a first point in time based on the first raw data or image data corresponding to the first raw data, and store information about the location of the identified biometric data in the memory (130). The electronic device (101) can acquire second raw data at a second point in time after the first point in time, and identify the location of the biometric data in the second raw data (or second raw data on which image preprocessing has been performed) based on information about the location of the biometric data identified before the second point in time (e.g., the first point in time).

[0148] According to one embodiment, FIG. 13B may include data (1320) on which image processing has been performed. According to one embodiment, the electronic device (101) (e.g., the processor (120)) may perform image processing on original data (e.g., raw data or raw data on which image preprocessing has been performed). For example, the electronic device (101) may perform image processing (e.g., distortion or deletion) on the entire raw data (or raw data on which image preprocessing has been performed). For example, the image processing on the entire raw data (or raw data on which image preprocessing has been performed) may include at least one of blank processing, blur processing, or hatching processing.

[0149] FIG. 14 is a diagram illustrating the operation of an electronic device according to one embodiment. FIG. 15 is a flowchart illustrating a method of operating an electronic device according to one embodiment. FIG. 16 is a flowchart illustrating a method of operating an electronic device according to one embodiment. FIGS. 14, 15, and 16 may be described with reference to the previously described embodiments.

[0150] Referring to FIG. 14, the setting values ​​of the plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112) of FIG. 11 can be understood. The "setting value" may include information about which of the plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112) is to be controlled to be turned on, and / or information about the brightness (e.g., auto exposure gain) of the light emitter(s) to be controlled to be turned on among the plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112). For example, the setting value may include information about the number, position, and / or brightness of the light emitters to be controlled to be turned on. For example, the setting values ​​may include first setting value(s) for biometric recognition (e.g., b0, b1, b2, bn of 1410 of FIG. 14) and second setting value(s) for non-security camera image processing (e.g., c of 1420 of FIG. 14). 1410 of FIG. 14 may represent acquiring raw data based on the first setting value(s) for biometric recognition (e.g., b0, b1, b2, bn of 1410 of FIG. 14) during a period including t1, t3, and t5. 1420 of FIG. 14 may represent acquiring raw data based on the second setting value(s) for non-security camera image processing during a period including t2, t4, and t5. For example, based on the first setting value(s) for biometric recognition, the electronic device (101) (e.g., processor (120)) can acquire raw data including biometric data using the image sensor (230).For example, based on the second setting value for camera image processing that does not require security, the electronic device (101) (e.g., the processor (120)) may obtain raw data that does not include biometric data (e.g., raw data in which biometric data is distorted) using the image sensor (230). According to one embodiment, the first setting values ​​for biometric recognition may be plural, and the next setting value for biometric recognition may be determined based on feedback on the previous setting value for biometric recognition. For example, in FIG. 14, b1 may be determined based on feedback on b0, b2 may be determined based on feedback on b1, and this operation may be repeated. According to one embodiment, the first setting value for biometric recognition may be one, and the setting value for biometric recognition may not be changed. According to one embodiment, when the embodiment of FIG. 14 is applied, the electronic device (101) may not perform a change operation on the biometric data. According to one embodiment, even when the embodiment of FIG. 14 is applied, the electronic device (101) can perform a change operation on biometric data.

[0151] Fig. 15 can be explained with reference to Fig. 14 and the previously described embodiment.

[0152] At least some of the operations of FIG. 15 may be omitted. The order of the operations of FIG. 15 may be changed. Operations other than those of FIG. 15 may be performed before, during, or after the operations of FIG. 15.

[0153] Referring to FIG. 15, in operation 1501, according to one embodiment, the electronic device (101) (e.g., the processor (120)) may obtain first raw data during a first period. For example, the electronic device (101) may obtain first raw data during a first period (t1) of FIG. 14. The electronic device (101) may control first light emitters among a plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112) to turn on based on a first setting value for biometric recognition (e.g., b0 among the first setting values ​​of FIG. 14) during the first period (t1) of FIG. 14. For example, the first light emitters may include 1101, 1103, 1105, 1107, 1109, and 1111 of FIG. 11B, but there is no limitation on the selection criteria of the first light emitters. The electronic device (101) may control the first light emitters among the plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112) to be turned on based on the first setting value for biometric recognition (e.g., b0 among the first setting values ​​of FIG. 14) during the first period (t1) of FIG. 14, and then acquire the first raw data using the image sensor (230).

[0154] In operation 1503, according to one embodiment, the electronic device (101) (e.g., processor (120)) may perform image preprocessing on the first raw data of operation 1501 using the image preprocessor (270). According to one embodiment, operation 1503 may be omitted.

[0155] In operation 1505, according to one embodiment, the electronic device (101) (e.g., processor (120)) may store the first raw data of operation 1501 or the data on which image preprocessing is performed of operation 1503 in the first buffer (310) of the memory (130). In operation 1505, the data stored in the first buffer (310) may include biometric data. Thereafter, the electronic device (101) may perform biometric authentication based on the data including the biometric data stored in the first buffer (310).

[0156] In operation 1507, according to one embodiment, the electronic device (101) (e.g., processor (120)) may acquire second raw data during a second period. For example, the electronic device (101) may acquire second raw data during the second period (t2) of FIG. 14. The electronic device (101) may control second raw data among the plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112) to be turned on based on a second setting value (e.g., c of FIG. 14) for camera image processing that does not require security during the second period (t2) of FIG. 14. For example, the second light emitters may include 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 of FIG. 11b, but there is no limitation on the selection criteria of the second light emitters.

[0157] In one embodiment, the number of first light emitters (e.g., 1101, 1103, 1105, 1107, 1109, and 1111 of FIG. 11B ) of operation 1501 may be less than the number of second light emitters (e.g., 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 of FIG. 11B ) of operation 1507. For example, the number of light emitters controlled to be on in operation 1501 may be a number suitable for biometric authentication. The number of light emitters controlled to be on in operation 1507 may be greater than the number of light emitters controlled to be on in operation 1501, and thus, raw data (e.g., second raw data) obtained in operation 1507 may be raw data in which biometric data is distorted. In one embodiment, the number of first light emitters in operation 1501 and the number of second light emitters in operation 1507 may be the same, and positions of at least some of the first light emitters in operation 1501 may be different from positions of at least some of the second light emitters in operation 1507. For example, the first light emitters in operation 1501 may be selected as light emitters arranged at positions suitable for biometric authentication. The second light emitters in operation 1507 may be selected as light emitters arranged at designated positions such that data for biometric authentication may be distorted. In one embodiment, the first light emitters of operation 1501 and the second light emitters of operation 1507 may be identical, and the brightness of at least one of the first light emitters of operation 1501 may be different from the brightness of at least one of the second light emitters of operation 1507. For example, the brightness of at least one of the first light emitters of operation 1501 may be a brightness suitable for biometric authentication. The brightness of at least one of the second light emitters of operation 1507 may be a designated brightness that may cause data for biometric authentication to be distorted. The embodiments regarding the number, position, and brightness of the light emitters described above may be applied interchangeably.For example, at least one of the number, position, and brightness of the first light emitters of operation 1501 may be different from at least one of the number, position, and brightness of the second light emitters of operation 1507.

[0158] In operation 1509, according to one embodiment, the electronic device (101) (e.g., processor (120)) may perform image preprocessing and modification operations on the second raw data of operation 1507 using the image preprocessor (270). According to one embodiment, operation 1509 may be omitted. For example, the image preprocessing of operation 1509 may be performed and only the modification operation may be omitted. For example, both the image preprocessing and modification operations of operation 1509 may be omitted.

[0159] In operation 1511, according to one embodiment, the electronic device (101) (e.g., the processor (120)) may store the second raw data of operation 1507, or the data on which at least one of the image preprocessing or modification operations of operation 1509 is performed, in the second buffer (320) of the memory (130). In operation 1511, the data stored in the second buffer (320) may not include biometric data, or the data stored in the second buffer (320) may include distorted biometric data. Thereafter, the electronic device (101) may perform camera image processing that does not require security based on the data stored in the second buffer (320).

[0160] Although FIG. 15 describes the first period and the second period, this describes a part of the operation of the electronic device (101), and the operation of FIG. 15 may be repeatedly performed. For example, operations 1501 to 1505 are performed during the first period, operations 1507 to 1511 are performed during the second period after the first period, operations 1501 to 1505 are performed during the third period after the second period, and operations 1507 to 1511 are performed during the fourth period after the third period, and these operations may be repeated.

[0161] Fig. 16 can be explained with reference to Fig. 14, Fig. 15 and the previously described embodiment.

[0162] At least some of the operations of FIG. 16 may be omitted. The order of the operations of FIG. 16 may be changed. Operations other than those of FIG. 16 may be performed before, during, or after the operations of FIG. 16.

[0163] Referring to FIG. 16, in operation 1601, according to one embodiment, the electronic device (101) (e.g., processor (120)) may control the first light emitters (e.g., at least some of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112 of FIG. 11B) to be turned on during a first period. For example, the electronic device (101) can control the first light emitters among the plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112) to turn on based on a setting value for biometric recognition (e.g., b0 in FIG. 14) during the first period (t1) in FIG. 14. For example, the first light emitters can include 1101, 1103, 1105, 1107, 1109, and 1111 in FIG. 11B, but there is no limitation on the selection criteria for the first light emitters. For example, the setting value of 1601 (e.g., b0 in Fig. 14) may be the default setting value for biometric authentication.

[0164] In operation 1603, according to one embodiment, the electronic device (101) (e.g., processor (120)) may acquire first raw data using the image sensor (230) after controlling the first light emitters (e.g., at least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112 of FIG. 11B) of operation 1601 to be turned on among the plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112). According to one embodiment, the electronic device (101) (e.g., the processor (120)) may perform operation 1505, or operations 1503 and 1505, based on the first raw data of operation 1603. The first raw data of operation 1603 may include, for example, biometric data at a level for biometric authentication. Based on the inclusion of the biometric data at a level for biometric authentication in the first raw data, the setting values ​​for biometric authentication (e.g., the number, position, and / or brightness of light emitters controlled to be on) may not be changed. If the setting values ​​are not changed, the setting values ​​of operation 1601 and the setting values ​​of operation 1609 may be the same. The first raw data of operation 1603 may not include, for example, biometric data at a level for biometric authentication. Based on the fact that the first row data does not include biometric data at a level for biometric authentication, the setting values ​​for biometric authentication (e.g., the number, positions, and / or brightness of light emitters controlled to be turned on) may be updated. The electronic device (101) (e.g., the processor (120)) may update the setting values ​​for biometric authentication (e.g., the number, positions, and / or brightness of light emitters controlled to be turned on). When the setting values ​​are updated, the setting values ​​of operation 1601 and the setting values ​​of operation 1609 may be different.

[0165] In operation 1605, according to one embodiment, the electronic device (101) (e.g., processor (120)) can control the second light emitters (e.g., at least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112 of FIG. 11B) to be turned on during the second period. For example, the electronic device (101) can control the second light emitters among the plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112) to turn on based on the setting value for camera image processing that does not require security (e.g., c in FIG. 14) during the second period (t2) in FIG. 14. For example, the second light emitters may include 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 of FIG. 11b, but there is no limitation on the selection criteria of the second light emitters. The setting value of the 1601 operation (e.g., b0 of FIG. 14) may be different from the setting value of the 1605 operation (e.g., c of FIG. 14). At least one of the number, position, or brightness of the first light emitters of the 1601 operation may be different from at least one of the number, position, or brightness of the second light emitters of the 1605 operation. For example, the number of second emitters in operation 1605 may be greater than the number of first emitters in operation 1601.

[0166] In operation 1607, according to one embodiment, the electronic device (101) (e.g., processor (120)) may acquire second raw data using the image sensor (230) after controlling the second light emitters (e.g., at least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112 of FIG. 11B) of operation 1605 to turn on among the plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112). According to one embodiment, the electronic device (101) (e.g., processor (120)) may perform operation 1511, or operations 1509 and 1511, on the second raw data of operation 1607.

[0167] In operation 1609, according to one embodiment, the electronic device (101) (e.g., processor (120)) can control third light emitters (e.g., at least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112 of FIG. 11B) to be turned on during a third period. For example, the electronic device (101) can control third light emitters among a plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112) to be turned on based on a setting value for biometric recognition (e.g., b1 of FIG. 14) during the third period (t3) of FIG. 14. For example, the third light emitters can include 1102, 1104, 1106, 1108, 1110, and 1112 of FIG. 11b, but there is no limitation on the selection criteria of the third light emitters. As described above in operations 1601 and 1603, the setting value of operation 1609 may be the same as or different from the setting value of operation 1601. For example, the setting value of 1609 (e.g., b1 in FIG. 14) may be an updated setting value based on a default setting value (e.g., b0 in FIG. 14). Based on the difference between the setting value of operation 1609 and the setting value of operation 1601, at least one of the first light emitters of operation 1601 may be different from at least one of the third light emitters of operation 1609. For example, the setting value of 1609 (e.g., b1 in FIG. 14) may be the same as the default setting value for biometric authentication (e.g., b0 in FIG. 14). Based on the fact that the setting values ​​of the 1609 operation are the same as the setting values ​​of the 1601 operation, the first light emitters of the 1601 operation may be the same as the third light emitters of the 1609 operation.

[0168] In operation 1611, according to one embodiment, the electronic device (101) (e.g., processor (120)) may acquire third raw data using the image sensor (230) after controlling the third light emitters (e.g., at least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112 of FIG. 11B) of operation 1609 to turn on among the plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112). According to one embodiment, the electronic device (101) (e.g., processor (120)) may perform operation 1505, or operations 1503 and 1505, on the third row data of operation 1611.

[0169] In operation 1613, according to one embodiment, the electronic device (101) (e.g., processor (120)) can control the second light emitters (e.g., at least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112 of FIG. 11B) to be turned on during the fourth period. For example, the electronic device (101) may control the second light emitters among the plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112) to be turned on based on a setting value for camera image processing that does not require security (e.g., c in FIG. 14) during the fourth period (t4) of FIG. 14. The second light emitters of the 1613 operation may be the same as the second light emitters of the 1605 operation.

[0170] In operation 1615, according to one embodiment, the electronic device (101) (e.g., processor (120)) may acquire fourth raw data using the image sensor (230) after controlling the second light emitters (e.g., at least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112 of FIG. 11B) of operation 1613 to turn on among the plurality of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112). According to one embodiment, the electronic device (101) (e.g., processor (120)) may perform operation 1511, or operations 1509 and 1511, on the fourth raw data of operation 1615.

[0171] Those skilled in the art will appreciate that the embodiments described herein may be applied interchangeably, within the scope of their applicability. For example, those skilled in the art will appreciate that at least some operations of one embodiment described herein may be omitted and applied, or at least some operations of one embodiment may be applied in conjunction.

[0172] The present disclosure is not limited to the foregoing, and other variations not mentioned will be apparent to those skilled in the art from the present disclosure.

[0173] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs from the description below.

[0174] According to one embodiment, an electronic device (101) may include an image sensor (230), an image preprocessor (270) including circuitry, a memory (130) storing instructions, and at least one processor (120) including a processing circuit. The instructions, when individually and / or collectively executed by the at least one processor (120), may cause the electronic device (101) to acquire raw data including biometric data using the image sensor (230). The instructions, when individually and / or collectively executed by the at least one processor (120), may cause the electronic device (101) to store the raw data as first image data in a first buffer (310). The instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to store second image data generated by applying a change operation to the raw data using the image preprocessor (270) in a second buffer (320). The instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to transfer the first image data stored in the first buffer (310) to a first application belonging to a secure area, and to apply a first algorithm for biometric authentication to the first image data. The above instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to transfer the second image data stored in the second buffer (320) to a second application belonging to a non-secure area and to apply a second algorithm distinct from the first algorithm to the second image data.

[0175] According to one embodiment, the instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to transfer the second image data stored in the second buffer (320) to the second application during a period that at least partially overlaps with the period during which the electronic device transfers the first image data stored in the first buffer (310) to the first application.

[0176] According to one embodiment, the change operation may include performing image processing on the raw data. The image processing may include at least one of blank processing, blur processing, or hatching processing.

[0177] According to one embodiment, the image processing may include at least one of the blank processing, the blur processing, or the hatching processing for the entire raw data.

[0178] According to one embodiment, the image processing may include at least one of the blank processing, the blur processing, or the hatching processing for an area corresponding to the biometric data included in the raw data.

[0179] According to one embodiment, the instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to acquire the raw data while the electronic device (101) is worn on the user's head. The instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to identify, in the raw data, a location of a designated area based on a characteristic of the electronic device (101). The instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to perform the image processing on the area corresponding to the biometric data corresponding to the location of the designated area.

[0180] According to one embodiment, the instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to acquire first raw data of the raw data during a first period of time using the image sensor (230). The instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to store at least a portion of the first image data identified by performing the image preprocessing on the first raw data using the image preprocessor (270) in the first buffer (310). The instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to acquire second raw data of the raw data during a second period after the first period using the image sensor (230). The instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to store at least a portion of the second image data identified by performing the image preprocessing and the modification operation on the second raw data using the image preprocessor (270) in the second buffer (320).

[0181] According to one embodiment, the electronic device (101) may include a plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112). The above instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to control, during the first period, to turn on first light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112). The above instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to control, during the second period, to turn on second light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112). The number of the first light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) may be less than the number of the second light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112).

[0182] According to one embodiment, the instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to control third light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) to turn on during a third period following the second period. At least one of the third light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) may be different from at least one of the first light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112). The instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to acquire third raw data of the raw data during the third period using the image sensor (230). The instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to store at least a portion of the first image data identified by performing the image preprocessing on the third raw data using the image preprocessor (270) in the first buffer (310).

[0183] According to one embodiment, the instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to control the second light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) to turn on during a fourth period following the third period. The instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to acquire fourth raw data of the raw data during the fourth period using the image sensor (230). The instructions, when individually and / or collectively executed by at least one processor (120), may cause the electronic device (101) to store at least a portion of the second image data identified by performing the image preprocessing and the change operation on the fourth raw data using the image preprocessor (270), in the second buffer (320).

[0184] According to one embodiment, a method of operating an electronic device (101) may include an operation of acquiring raw data including biometric data using an image sensor (230) of the electronic device (101). The method may include an operation of storing the raw data as first image data in a first buffer (310). The method may include an operation of storing second image data generated by applying a change operation to the raw data in a second buffer (320). The method may include an operation of transferring the first image data stored in the first buffer (310) to a first application belonging to a secure area and applying a first algorithm for biometric authentication to the first image data. The method may include an operation of transferring the second image data stored in the second buffer (320) to a second application belonging to a non-secure area and applying a second algorithm, which is distinct from the first algorithm, to the second image data.

[0185] According to one embodiment, the operation of transmitting the second image data to the second application may include transmitting the second image data stored in the second buffer (320) to the second application during a period that at least partially overlaps with the period of transmitting the first image data stored in the first buffer (310) to the first application.

[0186] According to one embodiment, the change operation may include performing image processing on the raw data. The image processing may include at least one of blank processing, blur processing, or hatching processing.

[0187] According to one embodiment, the image processing may include at least one of the blank processing, the blur processing, or the hatching processing for the entire raw data.

[0188] According to one embodiment, the image processing may include at least one of the blank processing, the blur processing, or the hatching processing for an area corresponding to the biometric data included in the raw data.

[0189] According to one embodiment, the operation of acquiring the raw data may include an operation of acquiring the raw data while the electronic device (101) is worn on the user's head. The operation of changing may include an operation of confirming the location of a designated area based on characteristics of the electronic device (101) in the raw data. The operation of changing may include an operation of performing the image processing on the area corresponding to the biometric data corresponding to the location of the designated area.

[0190] According to one embodiment, the operation of acquiring the raw data may include an operation of acquiring first raw data among the raw data during a first period using the image sensor (230). The operation of storing the first image data in the first buffer (310) may include an operation of storing at least a portion of the first image data confirmed by performing the image preprocessing on the first raw data in the first buffer (310). The operation of acquiring the raw data may include an operation of acquiring second raw data among the raw data during a second period after the first period using the image sensor (230). The operation of storing the second image data in the second buffer (320) may include an operation of storing at least a portion of the second image data confirmed by performing the image preprocessing and the modification operation on the second raw data in the second buffer (320).

[0191] According to one embodiment, the method may include an operation of controlling, during the first period, to turn on first light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) of the electronic device (101). The method may include an operation of turning on second light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) during the second period. The number of the first light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) may be less than the number of the second light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112).

[0192] According to one embodiment, the method may include an operation of controlling third light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) to be turned on during a third period after the second period. At least one of the third light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) may be different from at least one of the first light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112). The method may include an operation of acquiring third raw data among the raw data during the third period using the image sensor (230). The method may include an operation of storing at least a portion of the first image data, which is confirmed by performing the image preprocessing on the third raw data using the image preprocessor (270), in the first buffer (310).

[0193] According to one embodiment, the method may include an operation of controlling the second light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) to be turned on during a fourth period after the third period. The method may include an operation of acquiring fourth raw data among the raw data during the fourth period using the image sensor (230). The method may include an operation of storing at least a portion of the second image data, which is confirmed by performing the image preprocessing and the change operation on the fourth raw data using the image preprocessor (270), in the second buffer (320).

[0194] According to one embodiment, a computer-readable recording medium storing instructions that, when individually and / or collectively executed by at least one processor (120) of an electronic device (101), cause the electronic device (101) to perform at least one operation, may include an operation of acquiring raw data including biometric data using an image sensor (230) of the electronic device (101). The at least one operation may include an operation of storing the raw data as first image data in a first buffer (310). The at least one operation may include an operation of storing second image data generated by applying a change operation to the raw data in a second buffer (320). The at least one operation may include an operation of transmitting the first image data stored in the first buffer (310) to a first application belonging to a secure area and applying a first algorithm for biometric authentication to the first image data. The at least one operation may include an operation of transmitting the second image data stored in the second buffer (320) to a second application belonging to a non-secure area and applying a second algorithm distinct from the first algorithm to the second image data.

[0195] According to one embodiment, the operation of transmitting the second image data to the second application may include transmitting the second image data stored in the second buffer (320) to the second application during a period that at least partially overlaps with the period of transmitting the first image data stored in the first buffer (310) to the first application.

[0196] According to one embodiment, the change operation may include performing image processing on the raw data. The image processing may include at least one of blank processing, blur processing, or hatching processing.

[0197] According to one embodiment, the image processing may include at least one of the blank processing, the blur processing, or the hatching processing for the entire raw data.

[0198] According to one embodiment, the image processing may include at least one of the blank processing, the blur processing, or the hatching processing for an area corresponding to the biometric data included in the raw data.

[0199] According to one embodiment, the operation of acquiring the raw data may include an operation of acquiring the raw data while the electronic device (101) is worn on the user's head. The operation of changing may include an operation of confirming the location of a designated area based on characteristics of the electronic device (101) in the raw data. The operation of changing may include an operation of performing the image processing on the area corresponding to the biometric data corresponding to the location of the designated area.

[0200] According to one embodiment, the operation of acquiring the raw data may include an operation of acquiring first raw data among the raw data during a first period using the image sensor (230). The operation of storing the first image data in the first buffer (310) may include an operation of storing at least a portion of the first image data confirmed by performing the image preprocessing on the first raw data in the first buffer (310). The operation of acquiring the raw data may include an operation of acquiring second raw data among the raw data during a second period after the first period using the image sensor (230). The operation of storing the second image data in the second buffer (320) may include an operation of storing at least a portion of the second image data confirmed by performing the image preprocessing and the modification operation on the second raw data in the second buffer (320).

[0201] According to one embodiment, the at least one operation may include an operation of controlling, during the first period, to turn on first light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) of the electronic device (101). The at least one operation may include an operation of controlling second light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) to be turned on during the second period. The number of the first light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) may be less than the number of the second light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112).

[0202] According to one embodiment, the at least one operation may include an operation of controlling third light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) to be turned on during a third period after the second period. At least one of the third light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) may be different from at least one of the first light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112). The at least one operation may include an operation of acquiring third raw data among the raw data during the third period using the image sensor (230). The at least one operation may include an operation of storing at least a portion of the first image data, which is confirmed by performing the image preprocessing on the third raw data using the image preprocessor (270), in the first buffer (310).

[0203] According to one embodiment, the at least one operation may include an operation of controlling the second light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) to be turned on during a fourth period after the third period. The at least one operation may include an operation of acquiring fourth raw data among the raw data during the fourth period using the image sensor (230). The at least one operation may include an operation of storing at least a portion of the second image data, which is confirmed by performing the image preprocessing and the change operation on the fourth raw data using the image preprocessor (270), in the second buffer (320).

[0204] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0205] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0206] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0207] Various embodiments of the present document may be implemented as software (e.g., a program) including one or more instructions stored on a storage medium that can be read by a machine (e.g., an electronic device). For example, a processor (e.g., a controller) of the machine may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

[0208] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0209] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separately arranged in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0210] While this disclosure has been described and illustrated with reference to various embodiments, it is to be understood that the various embodiments are illustrative and not limiting. It will be further understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the present disclosure, including the appended claims and their equivalents. It will also be understood that any embodiment(s) described herein can be used in conjunction with any other embodiment(s) described herein.

Claims

1. In an electronic device (101), Image sensor (230); Image preprocessor (270) including a circuit; Memory (130) for storing instructions; and At least one processor (120) comprising a processing circuit, The above instructions, when individually and / or collectively executed by at least one processor (120), cause the electronic device (101) to: By using the above image sensor (230), raw data including biometric data is acquired, The above raw data is stored as first image data in the first buffer (310), The second image data generated by applying a change operation to the raw data using the image preprocessor (270) is stored in the second buffer (320). Transferring the first image data stored in the first buffer (310) to a first application belonging to a secure area, and applying a first algorithm for biometric authentication to the first image data, Transferring the second image data stored in the second buffer (320) to a second application belonging to a non-secure area, and causing a second algorithm, which is different from the first algorithm, to be applied to the second image data. Electronic devices (101).

2. In paragraph 1, The above instructions, when individually and / or collectively executed by at least one processor (120), cause the electronic device (101) to: Causing the second image data stored in the second buffer (320) to be transmitted to the second application during a period that at least partially overlaps with the period during which the first image data stored in the first buffer (310) is transmitted to the first application. Electronic devices (101).

3. In paragraph 1 or 2, The above change action is, Includes an operation of performing image processing on the above data, The image processing comprises at least one of blank processing, blur processing, or hatching processing. Electronic devices (101).

4. In any one of paragraphs 1 to 3, The image processing includes at least one of blank processing, blur processing, or hatching processing for the entire raw data. Electronic devices (101).

5. In any one of paragraphs 1 to 4, The image processing includes at least one of the blank processing, the blur processing, or the hatching processing for the area corresponding to the biometric data included in the raw data. Electronic devices (101).

6. In any one of paragraphs 1 to 5, The above instructions, when individually and / or collectively executed by at least one processor (120), cause the electronic device (101) to: When the above electronic device (101) is worn on the user's head, the above data is acquired, In the above data, the location of the designated area based on the characteristics of the electronic device (101) is confirmed, Causing the image processing to be performed for the area corresponding to the biometric data corresponding to the location of the designated area. Electronic devices (101).

7. In any one of paragraphs 1 to 6, The above instructions, when individually and / or collectively executed by at least one processor (120), cause the electronic device (101) to: Using the image sensor (230), the first raw data among the raw data is acquired during the first period, By performing the image preprocessing on the first raw data using the image preprocessor (270), at least a part of the first image data confirmed is stored in the first buffer (310), Using the image sensor (230), the second raw data is acquired during a second period after the first period, Causing at least a portion of the second image data, which is confirmed by performing the image preprocessing and the change operation on the second raw data using the image preprocessor (270), to be stored in the second buffer (320). Electronic device (101).

8. In any one of paragraphs 1 to 7, Further comprising a plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) including a light emitting circuit, The above instructions, when individually and / or collectively executed by at least one processor (120), cause the electronic device (101) to: During the first period, among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112), the first light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) are controlled to be turned on, During the second period, causing the second light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) to be controlled to be turned on, The number of the first light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) is less than the number of the second light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112). Electronic device (101).

9. In any one of paragraphs 1 to 8, The above instructions, when individually and / or collectively executed by at least one processor (120), cause the electronic device (101) to: During a third period after the second period, among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112), third light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) are controlled to turn on - the third light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; At least one of the first light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) is different from at least one of the first light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112). Using the image sensor (230), during the third period, the third raw data is acquired among the raw data, Causing at least a portion of the first image data, which is confirmed by performing the image preprocessing on the third raw data using the image preprocessor (270), to be stored in the first buffer (310). Electronic devices (101).

10. In any one of paragraphs 1 to 9, The above instructions, when individually and / or collectively executed by at least one processor (120), cause the electronic device (101) to: During a fourth period after the third period, among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112), the second light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) are controlled to be turned on, Using the image sensor (230), during the fourth period, the fourth raw data is acquired among the raw data, Causing at least a portion of the second image data, which is confirmed by performing the image preprocessing and the change operation on the fourth raw data using the image preprocessor (270), to be stored in the second buffer (320). Electronic device (101).

11. In the operating method of an electronic device (101), An operation of obtaining raw data including biometric data using the image sensor (230) of the electronic device (101), An operation of storing the above raw data as first image data in the first buffer (310), An operation of storing second image data generated by applying a change operation to the above data in a second buffer (320), An operation of transmitting the first image data stored in the first buffer (310) to a first application belonging to a secure area and applying a first algorithm for biometric authentication to the first image data; An operation of transmitting the second image data stored in the second buffer (320) to a second application belonging to a non-secure area and applying a second algorithm distinct from the first algorithm to the second image data, method.

12. In paragraph 11, The operation of transmitting the second image data to the second application comprises: An operation of transmitting the second image data stored in the second buffer (320) to the second application during a period that at least partially overlaps with the period of transmitting the first image data stored in the first buffer (310) to the first application, method.

13. In clause 11 or 12, The above change action is, Includes an operation of performing image processing on the above data, The image processing comprises at least one of blank processing, blur processing, or hatching processing. method.

14. In any one of paragraphs 11 to 13, The image processing includes at least one of blank processing, blur processing, or hatching processing for the entire raw data. method.

15. A non-transitory computer readable recording medium storing instructions that, when individually and / or collectively executed by at least one processor (120) of an electronic device (101), cause the electronic device (101) to perform at least one operation, wherein the at least one operation: An operation of obtaining raw data including biometric data using the image sensor (230) of the electronic device (101), An operation of storing the above raw data as first image data in the first buffer (310), An operation of storing second image data generated by applying a change operation to the above data in a second buffer (320), An operation of transmitting the first image data stored in the first buffer (310) to a first application belonging to a secure area and applying a first algorithm for biometric authentication to the first image data; An operation of transmitting the second image data stored in the second buffer (320) to a second application belonging to a non-secure area and applying a second algorithm distinct from the first algorithm to the second image data, Recording medium.

Images