Electronic device including image sensor, method of operating same, and recording medium

Abstract

The electronic device may include: an image sensor; an image preprocessor including circuitry; a memory to store instructions; and a processor including processing circuitry. The instructions, when executed individually and / or collectively by the at least one processor, may instruct the electronic device to acquire raw data including biometric data by using the image sensor. The instructions, when executed individually and / or collectively by the at least one processor, may instruct the electronic device to store raw data as first image data in a first buffer. The instructions, when executed individually and / or collectively by the at least one processor, may instruct the electronic device to store in a second buffer second image data generated by applying a change operation to the raw data using the image preprocessor. The instructions, when executed individually and / or collectively by the at least one processor, may instruct the electronic device to send first image data stored in the first buffer to a first application belonging to the secure area and to apply a first algorithm for biometric authentication to the first image data. The instructions, when executed individually and / or collectively by the at least one processor, may instruct the electronic device to send second image data stored in the second buffer to a second application belonging to the non-secure area and to apply a second algorithm different from the first algorithm to the second image data. Various other embodiments are possible.

Description

Technical Field

[0001] This disclosure relates to electronic devices including image sensors, methods for operating the electronic devices, and recording media. Background Technology

[0002] Biometric authentication is a technology that uses human physical and behavioral characteristics to identify an individual and authorize access to specific systems or data. Unlike traditional authentication schemes that require complex settings that may be difficult to remember to enhance security (such as those using PINs, patterns, or passwords), biometric authentication requires no memorization and offers significant convenience to users if the identification performance is reliable. Due to these advantages, biometric authentication is widely used in systems requiring user authentication, such as bank ATMs or mobile devices. With PIN, pattern, and password schemes, the PIN, pattern, or password can be easily changed if the settings are leaked. However, in biometric authentication, biometric data (such as facial features, iris, fingerprints, or veins) is not easily altered once leaked and can cause permanent damage. For these reasons, securely protecting the biometric data used for biometric authentication is crucial.

[0003] Electronic devices may include user interfaces that provide augmented reality (AR), virtual reality (VR), mixed reality (MR), and extended reality (XR) experiences.

[0004] The above information may be provided as relevant technology for the purpose of aiding understanding of this disclosure. No statement or assertion is made as to whether any of the foregoing is applicable as background technology in relation to this disclosure. Summary of the Invention

[0005] Solution to the problem Camera-based biometric authentication schemes can perform authentication by receiving biometric data (e.g., face, iris, fingerprint, or vein) from a camera and applying that data to a biometric authentication algorithm.

[0006] Embodiments of this disclosure may provide an electronic device that can perform operations in a secure area (e.g., biometric authentication operations) and in a non-secure area using the same camera.

[0007] According to an example embodiment, an electronic device may include: an image sensor, an image preprocessor including circuitry, a memory storing instructions, and at least one processor including processing circuitry. When executed individually and / or jointly by at least one processor, the instructions may cause the electronic device to: acquire raw data including biometric data using the image sensor. When executed individually and / or jointly by at least one processor, the instructions may cause the electronic device to: store the raw data as a first image data in a first buffer. When executed individually and / or jointly by at least one processor, the instructions may cause the electronic device to: store second image data generated by applying a modification operation to the raw data using the image preprocessor in a second buffer. When executed individually and / or jointly by at least one processor, the instructions may cause the electronic device to: forward the first image data stored in the first buffer to a first application included in a secure area, and apply a first algorithm for biometric authentication to the first image data. When executed individually and / or jointly by at least one processor, the instructions may cause the electronic device to: forward the second image data stored in the second buffer to a second application included in a non-secure area, and apply a second algorithm different from the first algorithm to the second image data.

[0008] According to an example embodiment, a method for operating an electronic device may include: using an image sensor of the electronic device to obtain raw data including biometric data. The method may include: storing the raw data as a first image data in a first buffer. The method may include: storing a second image data generated by applying a modification operation to the raw data in a second buffer. The method may include: forwarding the first image data stored in the first buffer to a first application included in a secure area, and applying a first algorithm for biometric authentication to the first image data. The method may include: forwarding the second image data stored in the second buffer to a second application included in a non-secure area, and applying a second algorithm different from the first algorithm to the second image data.

[0009] According to an example embodiment, in a non-transitory computer-readable recording medium storing instructions, the instructions, when executed individually and / or jointly by at least one processor of the electronic device, cause the electronic device to perform at least one operation, which may include: acquiring raw data including biometric data using an image sensor 230 of the electronic device. The at least one operation may include: storing the raw data as a first image data in a first buffer. The at least one operation may include: storing a second image data generated by applying a modification operation to the raw data in a second buffer. The at least one operation may include: forwarding the first image data stored in the first buffer to a first application included in a secure area, and applying a first algorithm for biometric authentication to the first image data. The at least one operation may include: forwarding the second image data stored in the second buffer to a second application included in a non-secure area, and applying a second algorithm different from the first algorithm to the second image data. Attached Figure Description

[0010] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the accompanying drawings and the following detailed description.

[0011] Figure 1 This is a block diagram illustrating an electronic device in a network environment according to an embodiment.

[0012] Figure 2a This is a block diagram illustrating a camera module according to an embodiment.

[0013] Figure 2b This is a block diagram illustrating an electronic device according to an embodiment.

[0014] Figure 2c This is a block diagram illustrating an electronic device according to an embodiment.

[0015] Figure 3 This is a view illustrating the operation of an electronic device according to an embodiment.

[0016] Figure 4 This is a flowchart illustrating an operation method of an electronic device according to an embodiment.

[0017] Figure 5 This is a flowchart illustrating an operation method of an electronic device according to an embodiment.

[0018] Figure 6 This is a flowchart illustrating an operation method of an electronic device according to an embodiment.

[0019] Figure 7 This is a flowchart illustrating an operation method of an electronic device according to an embodiment.

[0020] Figure 8 This is a view illustrating the operation of an electronic device according to an embodiment.

[0021] Figure 9 This is a flowchart illustrating an operation method of an electronic device according to an embodiment.

[0022] Figure 10a This is a view showing an electronic device according to an embodiment.

[0023] Figure 10b This is a view showing an electronic device according to an embodiment.

[0024] Figure 10c This is a view showing an electronic device according to an embodiment.

[0025] Figure 11a This is a view showing the light emitter and image sensor of an electronic device according to an embodiment.

[0026] Figure 11b This is a view showing the light emitter and image sensor of an electronic device according to an embodiment.

[0027] Figure 12 This is a view illustrating the operation of an electronic device according to an embodiment.

[0028] Figure 13a This is a view illustrating the operation of an electronic device according to an embodiment.

[0029] Figure 13b This is a view illustrating the operation of an electronic device according to an embodiment.

[0030] Figure 14 This is a view illustrating the operation of an electronic device according to an embodiment.

[0031] Figure 15 This is a flowchart illustrating an operation method of an electronic device according to an embodiment.

[0032] Figure 16 This is a flowchart illustrating an operation method of an electronic device according to an embodiment. Detailed Implementation

[0033] Various exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. However, the present disclosure may be implemented in various other forms and is not limited to the various embodiments set forth herein. Throughout the present disclosure and the accompanying drawings, the same or similar reference numerals may be used to refer to the same or similar elements. Furthermore, for clarity and brevity, well-known functions and configurations in the drawings and related descriptions may not be described.

[0034] Figure 1 This is a block diagram illustrating an electronic device 101 in a network environment 100 according to various embodiments.

[0035] Reference Figure 1 In network environment 100, electronic device 101 can communicate with electronic device 102 via a first network 198 (e.g., a short-range wireless communication network), or with at least one of electronic device 104 or server 108 via a second network 199 (e.g., a long-range wireless communication network). According to an embodiment, electronic device 101 can communicate with electronic device 104 via server 108. According to an embodiment, electronic device 101 may include a processor 120, memory 130, input module 150, sound output module 155, display module 160, audio module 170, sensor module 176, interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, user identification module (SIM) 196, or antenna module 197. According to an embodiment, display module 160 may include a first display module 351 corresponding to the user's left eye and / or a second display module 353 corresponding to the user's right eye. In embodiments, at least one of the aforementioned components (e.g., connection terminal 178) may be omitted from electronic device 101, or one or more other components may be added to electronic device 101. According to embodiments, some of the aforementioned components (e.g., sensor module 176, camera module 180, or antenna module 197) may be integrated into a single component (e.g., display module 160).

[0036] Processor 120 may include various processing circuitry and / or multiple processors. For example, as used herein (including the claims), the term "processor" may include a variety of processing circuitry, including at least one processor, wherein one or more of the at least one processor may be configured individually and / or collectively in a distributed manner to perform the various functions described herein. As used herein, when "processor," "at least one processor," and "one or more processors" are described as being configured to perform multiple functions, these terms cover (e.g., but not limited to) situations where one processor performs some functions of the functions while another (or more) processors perform other functions of the functions, and also cover situations where a single processor can perform all of the functions. Additionally, at least one processor may include a combination of processors performing various described / disclosed functions in a distributed manner, for example. At least one processor may execute program instructions to implement or perform various functions. Processor 120 may run, for example, software (e.g., program 140) to control at least one other component (e.g., hardware or software component) of electronic device 101 connected to processor 120, and may perform various data processing or calculations. According to embodiments, as at least part of the data processing or computation, processor 120 may store commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132, process the commands or data stored in volatile memory 132, and store the result data in non-volatile memory 134. According to embodiments, processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or application processor (AP)) or an auxiliary processor 123 (e.g., a graphics processing unit (GPU), neural processing unit (NPU), image signal processor (ISP), sensor central processor, or communication processor (CP)) that is operationally independent of or combined with the main processor 121. For example, when electronic device 101 includes a main processor 121 and an auxiliary processor 123, the auxiliary processor 123 may be configured to consume less power than the main processor 121 or be configured to be dedicated to a specific function. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of the main processor 121.

[0037] When the main processor 121 is inactive (e.g., in sleep) state, the auxiliary processor 123 (rather than the main processor 121) can control at least some of the functions or states associated with at least one component of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190), or when the main processor 121 is active (e.g., running an application), the auxiliary processor 123 can work with the main processor 121 to control at least some of the functions or states associated with at least one component of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190). According to embodiments, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., camera module 180 or communication module 190) functionally associated with the auxiliary processor 123. According to embodiments, the auxiliary processor 123 (e.g., a neural processing unit) may include hardware architecture dedicated to artificial intelligence model processing. Artificial intelligence models can be generated through machine learning. For example, such learning can be performed via electronic device 101 where artificial intelligence is performed or via a separate server (e.g., server 108). The learning algorithm may include, but is not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The artificial intelligence model may include multiple layers of artificial neural networks. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q-network, or a combination of two or more thereof, but is not limited thereto. Additionally or optionally, the artificial intelligence model may include software structures in addition to hardware structures.

[0038] Memory 130 may store various data used by at least one component of electronic device 101 (e.g., processor 120 or sensor module 176). The various data may include, for example, software (e.g., program 140) and input or output data for commands associated with it. Memory 130 may include volatile memory 132 or non-volatile memory 134.

[0039] The program 140 may be stored as software in the memory 130, and the program 140 may include, for example, an operating system (OS) 142, middleware 144, or application 146.

[0040] The input module 150 can receive commands or data from outside the electronic device 101 (e.g., a user) that will be used by other components of the electronic device 101 (e.g., processor 120). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (e.g., buttons), or digital pen (e.g., stylus).

[0041] The sound output module 155 can output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records. The receiver can be used to receive incoming calls. According to an embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0042] Display module 160 can visually provide information to the outside of electronic device 101 (e.g., to a user). Display device 160 may include, for example, a display, a holographic device, or a projector, and control circuitry for controlling a respective one of the display, holographic device, and projector. According to an embodiment, display module 160 may include a touch sensor configured to detect a touch or a pressure sensor configured to measure the intensity of the force generated by the touch.

[0043] The audio module 170 can convert sound into electrical signals and vice versa. According to an embodiment, the audio module 170 can obtain sound via the input module 150, or output sound via the sound output module 155 or headphones of an external electronic device (e.g., electronic device 102) that is directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0044] Sensor module 176 can detect the operating state of electronic device 101 (e.g., power or temperature) or the environmental state outside electronic device 101 (e.g., user state), and then generate an electrical signal or data value corresponding to the detected state. According to embodiments, sensor module 176 may include, for example, a gesture sensor, gyroscope sensor, atmospheric pressure sensor, magnetic sensor, accelerometer, grip sensor, proximity sensor, color sensor, infrared (IR) sensor, biometric sensor, temperature sensor, humidity sensor, or illuminance sensor.

[0045] Interface 177 may support one or more specific protocols used to enable electronic device 101 to connect directly (e.g., wired) or wirelessly to external electronic devices (e.g., electronic device 102). According to embodiments, interface 177 may include, for example, a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital Card (SD) interface, or an audio interface.

[0046] Connection 178 may include a connector, through which electronic device 101 may be physically connected to an external electronic device (e.g., electronic device 102). According to embodiments, connection 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0047] The haptic module 179 can convert electrical signals into mechanical stimuli (e.g., vibration or motion) or electrical stimuli that can be recognized by a user through his touch or kinesthesia. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0048] Camera module 180 can capture still or moving images. According to an embodiment, camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0049] The power management module 188 manages the power supply to the electronic device 101. According to an embodiment, the power management module 188 may be implemented as at least part of, for example, a power management integrated circuit (PMIC).

[0050] Battery 189 can power at least one component of electronic device 101. According to an embodiment, battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable rechargeable battery, or a fuel cell.

[0051] Communication module 190 can support the establishment of a direct (e.g., wired) or wireless communication channel between electronic device 101 and external electronic devices (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. Communication module 190 may include one or more communication processors capable of operating independently of processor 120 (e.g., application processor (AP)) and support direct (e.g., wired) or wireless communication. According to embodiments, communication module 190 may include wireless communication module 192 (e.g., cellular communication module, short-range wireless communication module, or Global Navigation Satellite System (GNSS) communication module) or wired communication module 194 (e.g., local area network (LAN) communication module or power line communication (PLC) module). One of these communication modules can communicate with an external electronic device 104 via a first network 198 (e.g., a short-range communication network such as Bluetooth, Wi-Fi Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a traditional cellular network, 5G network, next-generation communication network, the Internet, or a computer network (e.g., a local area network (LAN) or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip) or as multiple components separate from each other (e.g., multiple chips). The wireless communication module 192 can identify or verify the electronic device 101 in the communication network (such as the first network 198 or the second network 199) using user information (e.g., the International Mobile Subscriber Identity (IMSI)) stored in the user identification module 196.

[0052] Wireless communication module 192 can support 5G networks following 4G networks and next-generation communication technologies (such as new radio (NR) access technologies). NR access technologies can support enhanced mobile broadband (eMBB), massive machine-type communication (mMTC), or ultra-reliable low-latency communication (URLLC). Wireless communication module 192 can support high-frequency bands (e.g., millimeter-wave bands) to achieve, for example, high data transmission rates. Wireless communication module 192 can support various technologies used to ensure performance in high-frequency bands, such as, for example, beamforming, massive MIMO, full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. Wireless communication module 192 can support various requirements specified in electronic device 101, external electronic devices (e.g., electronic device 104), or network systems (e.g., second network 199). According to an embodiment, the wireless communication module 192 may support peak data rates (e.g., 20 Gbps or greater) for implementing eMBB, lost coverage (e.g., 164 dB or less) for implementing mMTC, or U-plane latency (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less round trip) for implementing URLLC.

[0053] Antenna module 197 can transmit or receive signals or power to or from the external (e.g., external electronic device) of electronic device 101. According to an embodiment, antenna module 197 may include an antenna comprising a radiator formed of a conductor or conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, antenna module 197 may include multiple antennas (e.g., an antenna array). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as a first network 198 or a second network 199) can be selected from the multiple antennas by, for example, communication module 190. Signals or power can then be transmitted or received between communication module 190 and the external electronic device via the selected at least one antenna. According to an embodiment, additional components besides the radiator (e.g., a radio frequency integrated circuit (RFIC)) may be additionally incorporated into antenna module 197.

[0054] According to various embodiments, antenna module 197 may form a millimeter-wave antenna module. According to embodiments, the millimeter-wave antenna module may include a printed circuit board, a radio frequency integrated circuit (RFIC), and multiple antennas (e.g., an array antenna), wherein the RFIC is disposed on or adjacent to a first surface (e.g., a bottom surface) of the printed circuit board and is capable of supporting a specified high-frequency band (e.g., a millimeter-wave band), and the multiple antennas are disposed on or adjacent to a second surface (e.g., a top or side surface) of the printed circuit board and are capable of transmitting or receiving signals in the specified high-frequency band.

[0055] At least some of the aforementioned components can be interconnected and communicate signals (e.g., commands or data) between them via an inter-peripheral communication scheme (e.g., bus, general purpose input / output (GPIO), serial peripheral interface (SPI), or mobile industrial processor interface (MIPI)).

[0056] According to an embodiment, commands or data can be sent or received between electronic device 101 and external electronic device 104 via server 108 connected to a second network 199. Each of external electronic device 102 or external electronic device 104 can be the same device as electronic device 101 or a device of a different type. According to an embodiment, all or some operations that would be performed on electronic device 101 can be performed on one or more of external electronic devices 102, external electronic devices 104, or server 108. For example, if electronic device 101 is required to automatically perform a function or service or should perform a function or service in response to a request from a user or another device, electronic device 101 may request the one or more external electronic devices to perform at least a portion of the function or service instead of running the function or service, or electronic device 101 may request the one or more external electronic devices to perform at least a portion of the function or service in addition to running the function or service. Upon receiving the request, one or more external electronic devices may perform at least a portion of the requested function or service, or perform additional functions or services related to the request, and transmit the result of the execution to electronic device 101. Electronic device 101 may provide the result as at least a partial response to the request, with or without further processing of the result. For this purpose, technologies such as cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing may be used. Electronic device 101 may use, for example, distributed computing or mobile edge computing to provide ultra-low latency services. In an embodiment, external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and / or neural networks. According to an embodiment, external electronic device 104 or server 108 may be included in a second network 199. Electronic device 101 may be applied to intelligent services based on 5G communication technology or IoT-related technologies (e.g., smart homes, smart cities, smart cars, or healthcare).

[0057] Figure 2a This is a block diagram 220 showing a camera module 180 according to an embodiment.

[0058] Reference Figure 2aThe camera module 180 may include a lens assembly (e.g., including a lens) 210, a flash 220, an image sensor 230, an image stabilizer (e.g., including circuitry) 240, a memory 250 (e.g., a buffer memory), and / or an image signal processor (e.g., including image processing circuitry) 260. The lens assembly 210 may include at least one lens and capture light emitted or reflected from an object whose image is to be captured. The lens assembly 210 may include one or more lenses. According to embodiments, the camera module 180 may include multiple lens assemblies 210. In this case, the camera module 180 may form, for example, a dual-camera system, a 360-degree camera, or a spherical camera. Some of the multiple lens assemblies 210 may have the same lens properties (e.g., field of view, focal length, autofocus, f-number, or optical zoom), or at least one lens assembly may have one or more lens properties that differ from the lens properties of other lens assemblies. The lens assembly 210 may include, for example, a wide-angle lens or a telephoto lens.

[0059] Flash 220 is capable of emitting light, wherein the emitted light is used to enhance light reflected from an object. According to embodiments, flash 220 may include one or more light-emitting diodes (LEDs) (e.g., red-green-blue (RGB) LEDs, white LEDs, infrared (IR) LEDs, or ultraviolet (UV) LEDs) or xenon lamps. Image sensor 230 acquires an image corresponding to an object by converting light emitted or reflected from the object and transmitted through lens assembly 210 into an electrical signal. According to embodiments, image sensor 230 may include one image sensor selected from a plurality of image sensors with different properties (e.g., an RGB sensor, a black-and-white (BW) sensor, an IR sensor, or a UV sensor), a plurality of image sensors having the same properties, or a plurality of image sensors with different properties. Each image sensor included in image sensor 230 may be implemented using, for example, a charge-coupled device (CCD) sensor or a complementary metal-oxide-semiconductor (CMOS) sensor.

[0060] Image stabilizer 240 may include various circuits and move image sensor 230 or at least one lens included in lens assembly 210 in a specific direction, or control the operability properties of image sensor 230 (e.g., adjust readout timing) in response to movement of camera module 180 or electronics 101 including camera module 180. This can compensate for at least some of the negative effects caused by movement of the image being captured. According to embodiments, image stabilizer 240 may use a gyroscope sensor (not shown) or accelerometer sensor (not shown) arranged inside or outside camera module 180 to sense such movement of camera module 180 or electronics 101. According to embodiments, image stabilizer 240 may be implemented as, for example, an optical image stabilizer.

[0061] Memory 250 may at least temporarily store at least a portion of the images acquired via image sensor 230 for subsequent image processing tasks. For example, if multiple images are captured rapidly or image capture is delayed due to shutter lag, the acquired raw images (e.g., Bayer pattern images, high-resolution images) may be stored in memory 250, and their corresponding copy images (e.g., low-resolution images) may be previewed via display module 160. Then, if specified conditions are met (e.g., by user input or system command), at least a portion of the raw images stored in memory 250 may be acquired and processed by, for example, image signal processor 260. According to embodiments, memory 250 may be configured as at least a portion of memory 130, or memory 250 may be configured as a separate memory operating independently of memory 130.

[0062] Image signal processor 260 may include various image processing circuits and perform one or more image processing operations on images acquired via image sensor 230 or stored in memory 250. The one or more image processing operations may include, for example, depth map generation, 3D modeling, panorama generation, feature point extraction, image compositing, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Alternatively or additionally, image signal processor 260 may perform control (e.g., exposure time control or readout timing control) on at least one component included in camera module 180 (e.g., image sensor 230). Images processed by image signal processor 260 may be stored back in memory 250 for further processing, or the image may be provided to external components outside camera module 180 (e.g., memory 130, display module 160, electronics 102, electronics 104, or server 108). According to embodiments, the image signal processor 260 can be configured as at least a part of the processor 120, or the image signal processor 260 can be configured as a separate processor that operates independently of the processor 120. If the image signal processor 260 is configured as a separate processor from the processor 120, the processor 120 can display at least one image processed by the image signal processor 260 as is via the display module 160, or the at least one image can be displayed after further processing.

[0063] According to an embodiment, the electronic device 101 may include a plurality of camera modules 180 with different attributes or functions. In this case, at least one of the plurality of camera modules 180 may form, for example, a wide-angle camera, and at least another of the plurality of camera modules 180 may form a telephoto camera. Similarly, at least one of the plurality of camera modules 180 may form, for example, a front-facing camera, and at least another of the plurality of camera modules 180 may form a rear-facing camera.

[0064] Figure 2b This is a block diagram illustrating an electronic device 101 according to an embodiment.

[0065] According to an embodiment, electronic device 101 may include an image sensor 230, an image preprocessor (e.g., including various circuits) 270, a memory 130, and a processor (e.g., including processing circuitry) 120.

[0066] The operation of the electronic device 101 according to the embodiment can be controlled by the processor 120 of the electronic device 101 (e.g., Figure 1 Processor 120 and / or Figure 2a The image signal processor 260 controls the operation. Specific operations performed by the electronic device 101 may be controlled by the processor 120 of the electronic device 101, which controls the electronic device 101 or the components included in the electronic device 101. The electronic device 101 may include one or more processors 120. Hereinafter, even when multiple processors 120 are employed, for ease of description, the operation is described as "operation of the electronic device 101" or "operation of the processor 120".

[0067] According to an embodiment, electronic device 101 (e.g., processor 120) can acquire raw data using image sensor 230. "Raw data" can be data acquired by image sensor 230 (e.g., images corresponding to objects 200 and 201). Figure 2b In this context, objects 200 and 201 may include a face portion 200 and / or an eye portion 201, but this is merely an example. For instance, raw data obtained by the image sensor 230 may include biometric data (e.g., facial features, iris, fingerprints, veins).

[0068] According to an embodiment, electronic device 101 (e.g., processor 120) can use image preprocessor 270 to perform image processing (e.g., image preprocessing and / or modification operations) on raw data. Image preprocessor 270 can be a hardware module including various circuits for performing image processing (e.g., image preprocessing and / or modification operations). For example, image preprocessor 270 can receive raw data obtained through image sensor 230. Image preprocessor 270 can perform image preprocessing (e.g., autofocus, auto exposure, auto white balance (3A)) and / or modification operations on the raw data. Modification operations are operations that alter (or transform) biometric data, and modification operations are referred to below. Figure 3 To describe.

[0069] According to an embodiment, electronic device 101 (e.g., processor 120) can store data (e.g., image data) that has undergone image processing (e.g., image preprocessing and / or modification operations) by image preprocessor 270 in memory 130. For example, image preprocessor 270 can transfer data (e.g., image data) to which image processing (e.g., image preprocessing and / or modification operations) has been performed (e.g., image data) to memory 130. Memory 130 can store data received from image preprocessor 270.

[0070] Figure 2c This is a block diagram illustrating an electronic device 101 according to an embodiment.

[0071] According to an embodiment, the electronic device 101 may include: a processor (e.g., including processing circuitry) 120, a camera module (e.g., including a camera) 180, and a display 274 (e.g., ...). Figure 1 At least one display included in the display module 160), and sensor 276 (e.g., Figure 1 The sensor module 176 includes at least one sensor), and the communication circuit 278 (e.g., Figure 1 The communication module 190 includes at least one communication circuit, and the memory 130.

[0072] According to an embodiment, memory 130 may include an XR architecture 280 and a hardware abstraction layer 299. For example, the XR architecture 280 may be designed to utilize an OpenXR-based rendering and input architecture for the platform. For example, the hardware abstraction layer 299 may include an interface for hardware operation.

[0073] According to an embodiment, the XR architecture 280 may include a compositional rendering manager (CPM) 281, a perception abstraction layer 286, and a perception service 290.

[0074] According to an embodiment, the perception abstraction layer 286 may be an interface for the perception service 290 and CPM 281.

[0075] According to embodiments of this disclosure, perception service 290 can estimate tracking data based on acquired sensor data using a perception solution. Perception service 290 may include position tracking 291, spatial perception 292, gesture tracking 293, gaze tracking 294, face tracking 295, and other tracking 296 (e.g., tracking of other components). For example, electronic device 101 can estimate the 6 degrees of freedom (6DoF) pose of electronic device 101 (e.g., head tracking (HeT)) using IMU input and a perception camera through position tracking 291 of perception service 290. For example, electronic device 101 can reconstruct the surrounding environment in 3D mode using perception camera input, 6DoF pose, and time-of-flight (ToF) through spatial perception 292 of perception service 290 (e.g., scene understanding (SU)), and can perform planar perception operations based on this. Electronic device 101 may additionally utilize camera input to capture spatial perception 292. For example, electronic device 101 can use gesture tracking 293 (e.g., hand tracking (HaT)) of perception service 290, using a perception camera and ToF input to track the user's hand posture and identify gestures. For example, electronic device 101 can use eye tracking 294 (e.g., eye tracking (ET)) of perception service 290, using an ET camera and an infrared (IR) LED to estimate and track the user's eye movement (or pupil movement). For example, electronic device 101 can use face tracking 295 (e.g., face tracking (FT)) of perception service 290, using an FT camera and an IR LED to estimate and track the user's facial movement. For example, electronic device 101 can use other tracking 296 (e.g., iris tracking) of perception service 290, using an IR camera (e.g., the IR sensor included in the image sensor of FIG. 2) and an IR LED to estimate and track the user's iris.

[0076] According to an embodiment, CPM 281 may include runtime 282, pass-through 283, input manager 284, and synthesizer 285. CPM 281 may include virtual nodes rendered based on data (e.g., tracking data) obtained from the perception abstraction layer 286 via input manager 284 and nodes obtained via pass-through 283 (e.g., pass-through library).

[0077] Figure 3 This is a view illustrating the operation of the electronic device 101 according to an embodiment.

[0078] refer to Figure 3According to an embodiment, the memory 130 of the electronic device 101 may include a first buffer 310 (e.g., a secure buffer) and a second buffer 320 (e.g., a non-secure buffer). For example, the first buffer 310 and the second buffer 320 may be dynamically allocated regions in the memory 130.

[0079] According to an embodiment, the first buffer 310 (e.g., a security buffer) may be a buffer in which data including biometric data is stored. The security buffer (e.g., 310) may be a secure area (e.g., Figure 8 A secure region is a buffer accessible within a first region. For example, a secure region may correspond to a secure execution environment (e.g., a trusted execution environment (TEE)). According to an embodiment, electronic device 101 may include a security processor. The security processor may correspond to a physically divided separate secure region. For example, a secure region may correspond to a security processor that includes secure memory or an embedded secure element (eSE). For example, the security processor or eSE of electronic device 101 may process data stored in a secure buffer (e.g., 310). According to an embodiment, a second buffer 320 (e.g., a non-secure buffer) may be a buffer in which data containing biometric data has been modified. The non-secure buffer (e.g., 320) may be a non-secure region (e.g., Figure 8 The buffers are accessible in the second region. The non-secure region may be a region corresponding to a general execution environment (e.g., a rich execution environment (REE)). According to an embodiment, the electronic device 101 (e.g., processor 120) may store data (e.g., image data) obtained by performing image processing (e.g., image preprocessing and / or modification operations) by the image preprocessor 270 in the first buffer 310 and / or the second buffer 320 of the memory 130.

[0080] refer to Figure 3 According to an embodiment, electronic device 101 (e.g., processor 120) can use image preprocessor 270 to perform image processing (e.g., image preprocessing and / or modification operations) on raw data obtained by image sensor 230.

[0081] For example, electronic device 101 can use image preprocessor 270 to perform image preprocessing on the raw data (e.g., Figure 3 Image preprocessing may include, for example, 3A (autofocus, auto exposure, auto white balance).

[0082] For example, electronic device 101 can use image preprocessor 270 to perform modification operations on the raw data (e.g., Figure 3 (C). Although Figure 3An example of performing image preprocessing followed by modification is shown, but the electronic device 101 can directly perform modification operations on the raw data obtained by the image sensor 230, or it can perform modifications as follows: Figure 3 The diagram illustrates a modification operation performed on raw data that has already undergone image preprocessing. A "modification operation" can be an operation that alters (or transforms) biometric data included in the raw data (or raw data that has undergone image preprocessing). For example, a modification operation can be an image processing operation performed on biometric data or raw data including biometric data (or raw data that has undergone image preprocessing). For example, electronic device 101 can perform image processing on the entire raw data including biometric data (or raw data that has undergone image preprocessing). For example, electronic device 101 can perform image processing on a region corresponding to biometric data in the raw data including biometric data (or raw data that has undergone image preprocessing). For example, a modification operation (e.g., image processing performed on biometric data or raw data including biometric data (or raw data that has undergone image preprocessing)) can include hidden surface removal (e.g., Figure 3 321), removal of biometric data (e.g., Figure 3 322), blurring (e.g., Figure 3 323) or shading treatment (e.g., Figure 3At least one of (324). "Hidden removal" can be a process of removing a region of the original data (or the original data for which image preprocessing has been performed) or a region corresponding to biometric data included in the original data (or the original data for which image preprocessing has been performed). "Removal of biometric data" can be an operation of removing a region corresponding to biometric data included in the original data (or the original data for which image preprocessing has been performed). When biometric data is removed, the region other than the region corresponding to the biometric data may be identical to the initial data (e.g., the original data (or the original data for which image preprocessing has been performed)). "Blurring" can be a process of blurring a region of the original data (or the original data for which image preprocessing has been performed) or a region corresponding to biometric data included in the original data (or the original data for which image preprocessing has been performed). For example, when blurring is performed on a region corresponding to biometric data, the region other than the region corresponding to the biometric data may be identical to the initial data (e.g., the original data (or the original data for which image preprocessing has been performed)). "Shading" can be applied to the entire area of ​​the original data (or the original data that has undergone image preprocessing) or to the area corresponding to biometric data included in the original data (or the original data that has undergone image preprocessing). For example, when shading is applied to the area corresponding to biometric data, the area outside the area corresponding to the biometric data can be identical to the initial data (e.g., the original data (or the original data that has undergone image preprocessing)).

[0083] According to an embodiment, electronic device 101 (e.g., processor 120) can identify the location of biometric data in raw data (or raw data that has undergone image preprocessing) and can perform image processing (e.g., at least one of hidden line removal, biometric data removal, blurring, or shading) on ​​the identified location of the biometric data. For example, electronic device 101 (e.g., processor 120) can identify biometric data (e.g., the location of biometric data) from the raw data (or raw data that has undergone image preprocessing) based on information stored in memory 130 (e.g., information about the location of the biometric data). For example, electronic device 101 (e.g., processor 120) can store information about the location of biometric data to be included in the raw data in memory 130 before obtaining the raw data. Electronic device 101 can identify the location of biometric data in the raw data (or raw data that has undergone image preprocessing) based on information stored in memory 130 (e.g., information about the location of biometric data to be included in the raw data). According to an embodiment, electronic device 101 may include a memory 130 storing information about the location of biometric data to be included in raw data. According to an embodiment, electronic device 101 can identify the location of biometric data based on first raw data or image data corresponding to the first raw data at a first time, and can store information about the identified location of the biometric data in memory 130. Electronic device 101 can obtain second raw data at a second time after the first time, and can identify the location of the biometric data in the second raw data (or the second raw data that has undergone image preprocessing) based on information about the location of the biometric data identified before the second time (e.g., at the first time).

[0084] According to the embodiments, refer to Figure 3 The electronic device 101 can store first image data 311 (e.g., image data including biometric data (e.g., iris)) identified by performing image preprocessing (P) on the raw data in a first buffer 310 of the memory 130. For example, refer to Figure 3The electronic device 101 can store second image data 325 (e.g., image data obtained by performing image processing on a region corresponding to biometric data) identified by performing image preprocessing (P) and modification (C) on the original data in a second buffer 320 of the memory 130. The second image data 325 stored in the second buffer 320 can be data obtained by performing modification operations on the original data obtained via the image sensor 230, or it can be data obtained by performing modification operations on the first image data 311 obtained by performing image preprocessing on the original data.

[0085] Figure 4 This is a flowchart illustrating the operation method of the electronic device 101 according to an embodiment. Figure 4 The above embodiments can be used as a reference for description.

[0086] Figure 4 At least some operations can be omitted. It can be changed. Figure 4 The order of operations. (Can be...) Figure 4 Perform before, during or after the operation Figure 4 Operations other than those performed by the operator.

[0087] refer to Figure 4 In operation 401, according to an embodiment, electronic device 101 (e.g., processor 120) may use image sensor 230 to acquire raw data. The raw data may be data acquired by image sensor 230 (e.g., images corresponding to object 200 and object 201).

[0088] In operation 403, according to an embodiment, electronic device 101 (e.g., processor 120) can perform image preprocessing (e.g., 3A) on the raw data of operation 401 using image preprocessor 270. Electronic device 101 can recognize image data (e.g., ...) by performing image preprocessing (e.g., 3A) on the raw data of operation 401. Figure 3 (311).

[0089] In operation 405, according to an embodiment, electronic device 101 (e.g., processor 120) can process image data that has undergone image preprocessing (e.g., Figure 3 The 311) is stored in the first buffer 310 (e.g., a safety buffer) of the memory 130.

[0090] Figure 5 This is a flowchart illustrating the operation method of the electronic device 101 according to an embodiment. Figure 5 The above embodiments can be used as a reference for description.

[0091] Figure 5At least some operations can be omitted. It can be changed. Figure 5 The order of operations. (Can be...) Figure 5 Perform before, during or after the operation Figure 5 Operations other than those performed by the operator.

[0092] refer to Figure 5 In operation 501, according to an embodiment, electronic device 101 (e.g., processor 120) can use image sensor 230 to acquire raw data. The raw data may be data acquired by image sensor 230 (e.g., images corresponding to object 200 and object 201).

[0093] In operation 503, according to an embodiment, electronic device 101 (e.g., processor 120) may use image preprocessor 270 to perform image preprocessing (e.g., 3A) on the raw data of operation 501.

[0094] In operation 505, according to an embodiment, electronic device 101 (e.g., processor 120) can use image preprocessor 270 to perform operations that modify the data obtained in operation 503 by performing image preprocessing. The modification operation may be an operation of performing image processing (e.g., hidden line removal, biometric data removal, blurring, or shading) on ​​biometric data. Electronic device 101 can identify image data (e.g., by performing image preprocessing in operation 503 on the raw data of operation 501 and making modifications in operation 505) Figure 3 325).

[0095] In operation 507, according to an embodiment, electronic device 101 (e.g., processor 120) can process image data that has undergone image preprocessing and modification operations (e.g., Figure 3 The 325) is stored in the second buffer 320 (e.g., a non-safety buffer) of the memory 130.

[0096] Figure 6 This is a flowchart illustrating the operation method of the electronic device 101 according to an embodiment. Figure 6 The above embodiments can be used as a reference for description.

[0097] Figure 6 At least some operations can be omitted. It can be changed. Figure 6 The order of operations. (Can be...) Figure 6 Perform before, during or after the operation Figure 6 Operations other than those performed by the operator.

[0098] refer to Figure 6In operation 601, according to an embodiment, electronic device 101 (e.g., processor 120) can acquire raw data using image sensor 230. The raw data may be data acquired by image sensor 230 (e.g., images corresponding to object 200 and object 201).

[0099] In operation 603, according to an embodiment, electronic device 101 (e.g., processor 120) can use image preprocessor 270 to identify (or generate, process, or obtain) first image data (e.g., ...) by performing image preprocessing on the raw data of operation 601. Figure 3 (311).

[0100] In operation 605, according to an embodiment, electronic device 101 (e.g., processor 120) can use image preprocessor 270 to process the first image data (e.g., from operation 603) Figure 3 311) Perform the modified operation to identify (or generate, process, or obtain) second image data (e.g., Figure 3 325).

[0101] In operation 607, according to an embodiment, electronic device 101 (e.g., processor 120) can transmit the first image data from operation 603 (e.g., Figure 3 The first image data (311) provided by the image preprocessor 270 is stored in the first buffer 310. The electronic device 101 can store the first image data (e.g., 311) in the first buffer 310. Figure 3 311) is stored in the first buffer 310.

[0102] In operation 609, according to an embodiment, electronic device 101 (e.g., processor 120) can transmit the second image data from operation 605 (e.g., Figure 3 The second image data (325) provided by the image preprocessor 270 is stored in the second buffer 320. The electronic device 101 can store the second image data (e.g., 325) in the second buffer 320. Figure 3 325) is stored in the second buffer 320.

[0103] Figure 7 This is a flowchart illustrating the operation method of the electronic device 101 according to an embodiment. Figure 7 The above embodiments can be used as a reference for description.

[0104] Figure 7 At least some operations can be omitted. It can be changed. Figure 7 The order of operations. (Can be...) Figure 7 Perform before, during or after the operation Figure 7 Operations other than those performed by the operator.

[0105] refer to Figure 7This can be understood as an operation that modifies the original data.

[0106] refer to Figure 7 In operation 701, according to an embodiment, electronic device 101 (e.g., processor 120) can acquire raw data including biometric data. Biometric data may include, for example, but not limited to, facial features, iris, fingerprints, veins, etc.

[0107] In operation 703, according to an embodiment, electronic device 101 (e.g., processor 120) can perform image processing (e.g., at least one of hidden line removal, biometric data removal, blurring, or shading) on ​​the raw data of operation 701. Electronic device 101 can perform image processing (e.g., at least one of hidden line removal, biometric data removal, blurring, or shading) on ​​raw data including biometric data. See also... Figure 3 To understand image processing of raw data including biometric data. For example, electronic device 101 may perform image processing (e.g., at least one of hidden surface removal, biometric data removal, blurring, or shading) before performing image preprocessing on the raw data. For example, electronic device 101 may perform image preprocessing after performing image processing (e.g., at least one of hidden surface removal, biometric data removal, blurring, or shading) on ​​the raw data. For example, electronic device 101 may perform image processing (e.g., at least one of hidden surface removal, biometric data removal, blurring, or shading) after performing image preprocessing on the raw data. For example, electronic device 101 may perform image processing (e.g., at least one of hidden surface removal, biometric data removal, blurring, or shading) without performing image preprocessing on the raw data.

[0108] According to an embodiment, electronic device 101 (e.g., processor 120) can execute... Figure 7 Data from change operations (e.g., operation 703) is stored in a non-secure buffer (e.g., Figure 3 (of 320).

[0109] Figure 8 This is a view illustrating the operation of the electronic device 101 according to an embodiment. Figure 9 This is a flowchart illustrating an operation method of an electronic device according to an embodiment. The above embodiments can be used as a reference for description. Figure 8 and Figure 9 .

[0110] refer to Figure 8 This can be understood as a safe area (e.g., the first area) and an unsafe area (e.g., the second area).

[0111] refer to Figure 8 According to an embodiment, an application (e.g., first application 810) belonging to a component classified as belonging to a secure region (e.g., a first region) can access the first buffer 310 of memory 130. For example, the first application 810 can be a binary operation within a secure region (e.g., a region corresponding to a TEE or a region corresponding to a security processor (or eSE)). A secure region can be a region corresponding to a component capable of processing data including biometric data. Components belonging to a secure region can process data including biometric data. Data including biometric data can be data to which no modification operations have been performed.

[0112] refer to Figure 8 According to an embodiment, an application (e.g., second application 820) of a component classified as belonging to a non-secure region (e.g., a second region) can access the second buffer 320 of memory 130. For example, the second application 820 can be a binary operation in a non-secure region (e.g., REE). A non-secure region can be a region corresponding to a component capable of processing data that does not include biometric data. A component belonging to a non-secure region can process data that does not include biometric data. Data that does not include biometric data can be data that does not initially include biometric data, or data obtained by performing modification operations on initial data (e.g., raw data or data that has undergone image preprocessing).

[0113] According to an embodiment, electronic device 101 may include physically separated, separate security processors (or eSEs). The security processor (or eSE) of electronic device 101 may correspond to a secure area. The security processor (or eSE) of electronic device 101 may access a first buffer 310. The security processor (or eSE) of electronic device 101 may process data stored in the first buffer 310. The security processor (or eSE) of electronic device 101 may apply a biometric authentication algorithm (e.g., first algorithm 811) to the data stored in the first buffer 310.

[0114] You can refer to this. Figure 9 To describe in detail Figure 9 The configuration and the above embodiments.

[0115] Figure 9 At least some operations can be omitted. It can be changed. Figure 9 The order of operations. (Can be...) Figure 9 Perform before, during or after the operation Figure 9 Operations other than those performed by the operator.

[0116] refer to Figure 9In operation 901, according to an embodiment, electronic device 101 (e.g., processor 120) can store first image data (e.g., ...) in the first buffer 310. Figure 3 The first image data (e.g., data including biometric data) is transmitted (e.g., forwarded) to the first application 810. The first application 810 can obtain the first image data (e.g., data including biometric data) stored in the first buffer 310. Figure 3 (311). The first application 810 may be an application belonging to a secure area (e.g., a first area). The first application 810 may be an application for biometric authentication. For example, the first application 810 may perform biometric authentication for a user of the electronic device 101 by applying the first algorithm 811 to image data.

[0117] In operation 903, according to an embodiment, electronic device 101 (e.g., processor 120) can store second image data (e.g., ...) in the second buffer 320. Figure 3 The second image data (e.g., data excluding biometric data, or data for which image processing of biometric data has been performed) stored in the second buffer 320 is transmitted (e.g., forwarded) to the second application 820. The second application 820 can then access the second image data (e.g., data stored in the second buffer 320) stored in the second buffer 320. Figure 3 (325). The second application 820 may be an application belonging to a non-secure area (e.g., a second area). The second application 820 may be an application unrelated to biometric authentication. The second application 820 may be an application for camera image processing that does not require security. For example, the second application 820 may perform gaze tracking on the user of the electronic device 101 by applying the second algorithm 821 to the image data. For example, the second application 820 may perform image preview (e.g., via a display) by applying the second algorithm 821 to the image data. Figure 1 (Image preview performed by 160). For example, the second application 820 can perform video recording by applying the second algorithm 821 to the image data. The operations that can be performed by the second application 820 are not limited to gaze tracking, image preview, or video recording.

[0118] According to an embodiment, at least a portion of the time period for performing operation 901 may correspond to at least a portion of the time period for performing operation 903. For example, electronic device 101 (e.g., processor 120) may interact with first image data (e.g., stored in first buffer 310) stored in first buffer 310. Figure 3 During the period when the time intervals of the transmission of (311) to the first application 810 at least partially overlap, the second image data (e.g., ) stored in the second buffer 320 will be transmitted. Figure 3 325) is transmitted to the second application 820.

[0119] According to an embodiment, operations 901 and 903 can be performed alternately. For example, electronic device 101 (e.g., processor 120) can store first image data (e.g., ...) in a first buffer 310 during a first time period. Figure 3 (311) is transmitted to the first application 810, and the second image data (e.g., stored in the second buffer 320) can be transmitted to the second application 810 during the second time period after the first time period. Figure 3 The first image data (e.g., 325) stored in the first buffer 310 is transmitted to the second application 820. For example, the electronic device 101 (e.g., processor 120) may transmit the first image data (e.g., 325) to the second application 820 during the first time period. Figure 3 (311) is transmitted to the first application 810, and the second image data (e.g., stored in the second buffer 320) can be transmitted to the second application 810 during the second time period after the first time period. Figure 3 (325) is transmitted to the second application 820, which can transmit the first image data (e.g., stored in the first buffer 310) to the third time period after the second time period. Figure 3 (311) is transmitted to the first application 810, and the second image data (e.g., stored in the second buffer 320) can be transmitted to the first application 810 during the fourth time period after the third time period. Figure 3 325) is transmitted to the second application 820.

[0120] In operation 905, according to an embodiment, electronic device 101 (e.g., processor 120) can use first application 810 to apply first algorithm 811 to first image data (e.g., from first buffer 310) provided by first buffer 310. Figure 3 The first application 810 may be an application for biometric authentication. The first algorithm 811 may be an algorithm for biometric authentication. The electronic device 101 can apply the first algorithm 811 (e.g., an algorithm for biometric authentication) to the first image data (e.g., ...). Figure 3 The first application 810 (e.g., data including biometric data) is used to perform biometric authentication on the user of electronic device 101. The type of the first application 810 (e.g., an application for biometric authentication) is not limited. The type of the first algorithm 811 (e.g., an algorithm for biometric authentication) is not limited. The type of biometric authentication is not limited.

[0121] In operation 907, according to an embodiment, electronic device 101 (e.g., processor 120) can use second application 820 to apply second algorithm 821 to second image data (e.g., from second buffer 320) provided by second buffer 320. Figure 3(e.g., data excluding biometric data, or data obtained by performing alteration operations on biometric data). The second application 820 can be an application for camera image processing that does not require security. The second algorithm 821 can be an algorithm for camera image processing that does not require security. For example, the second algorithm 821 can be an algorithm for tracking a user's gaze. For example, the second algorithm 821 can be an algorithm for image preview. For example, the second algorithm 821 can be an algorithm for video recording. For example, the electronic device 101 can apply the second algorithm 821 to second image data (e.g., Figure 3 The electronic device 101 performs gaze tracking on the user of the second image data (e.g., data excluding biometric data, or data obtained by altering biometric data). For example, the electronic device 101 can apply the second algorithm 821 to the second image data (e.g., ...). Figure 3 Image preview can be performed using data from the second image data (e.g., data excluding biometric data, or data obtained by altering biometric data). For example, electronic device 101 can perform image preview by applying a second algorithm 821 to the second image data (e.g., Figure 3 The second application 820 performs video recording using data that is excluding biometric data (e.g., data that does not include biometric data, or data obtained by altering biometric data). The type of the second application 820 is not limited. The type of the second algorithm 821 is not limited. The operations that the second application 820 can perform are not limited.

[0122] Figure 10a This is a view showing an electronic device 101 according to an embodiment. Figure 10b This is a view showing an electronic device 101 according to an embodiment. Figure 10c This is a view showing an electronic device 101 according to an embodiment.

[0123] Figure 10a and Figure 10b This is a view showing the front and rear surfaces of the electronic device 101 according to an embodiment.

[0124] refer to Figure 10a and Figure 10b In an embodiment, camera modules 1011, 1012, 1013, 1014, 1015, 1016 and / or depth sensor 1017 for obtaining information related to the surrounding environment of electronic device 101 may be disposed on the first surface 1010 of the housing.

[0125] In an embodiment, camera module 1011 and camera module 1012 can acquire images related to the surrounding environment of the electronic device.

[0126] In embodiments, camera modules 1013, 1014, 1015, and 1016 can acquire images when the electronic device is worn by a user. Camera modules 1013, 1014, 1015, and 1016 can be used for hand detection, tracking, and recognition of user gestures (e.g., hand movements). Camera modules 1013, 1014, 1015, and 1016 can be used for 3DoF or 6DoF head tracking, position (spatial or environmental) recognition, and / or motion recognition. In embodiments, camera modules 1011 and 1012 can be used for hand detection and tracking, and recognition of user gestures.

[0127] In an embodiment, depth sensor 1017 may be configured to send and receive signals reflected from an object and to identify the distance to the object, such as time-of-flight (TOF). As an alternative to or supplement to depth sensor 1017, camera modules 1013, 1014, 1015, and 1016 may identify the distance to the object.

[0128] According to an embodiment, camera modules 1025, 1026 and / or display 1021 (and / or lens) for facial recognition may be disposed on the second surface 1020 of the housing.

[0129] In the embodiments, the face recognition camera modules 1025 and 1026 adjacent to the display can be used to recognize the user's face, or to recognize and / or track the user's eyes.

[0130] In an embodiment, the display 1021 (and / or lens) may be disposed on the second surface 1020 of the electronic device 101. In an embodiment, the electronic device 101 may not include camera modules 1015 and 1016 among a plurality of camera modules 1013, 1014, 1015, and 1016. Although in Figure 10a and Figure 10b Not shown, but electronic device 101 may also include Figure 10c At least one of the components shown.

[0131] Figure 10c This is a perspective view showing the internal configuration of an electronic device according to an embodiment.

[0132] refer to Figure 10c The electronic device 101 according to the embodiment may include at least one of an optical output module (e.g., including an optical output circuit) 1041, a display component (e.g., including a waveguide) 1031, and a camera module (e.g., including a camera) 1050.

[0133] According to an embodiment, the light output module 1041 may include a light source capable of outputting an image and a lens guiding the image to the display member 1031. According to an embodiment, the light output module 1041 may include at least one of a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light-emitting diode (OLED), or a micro light-emitting diode (micro LED).

[0134] According to an embodiment, the display component 1031 may include an optical waveguide (e.g., a waveguide). According to an embodiment, an image incident on one end of the optical waveguide and output from the light output module 1041 can propagate within the optical waveguide and be provided to the user. According to an embodiment, the optical waveguide may include at least one of a diffractive element (e.g., a diffractive optical element (DOE) or a holographic optical element (HOE)) or a reflective element (e.g., a mirror). For example, the optical waveguide may use at least one diffractive element or reflective element to guide the image output from the light output module 1041 to the user's eye.

[0135] According to an embodiment, camera module 1050 (e.g., Figure 1 The camera module 1050 can capture still images and / or video. According to an embodiment, the camera module 1050 can be disposed in a lens frame and can be arranged around the display member 1031.

[0136] According to an embodiment, the first camera module 1051 can capture and / or identify the trajectory of a user's eyes (e.g., pupil or iris) or gaze. According to an embodiment, the first camera module 1051 can periodically or non-periodically transmit signals to a processor (e.g., ...). Figure 1 The processor 120 sends information (e.g., trajectory information) related to the trajectory of the user's eye or gaze.

[0137] According to an embodiment, the second camera module 1053 can capture external images.

[0138] According to an embodiment, the third camera module 1055 can be used for hand detection and tracking, as well as the recognition of user gestures (e.g., hand movements). According to an embodiment, the third camera module 1055 can be used for 3-DOF or 6DoF head tracking, position (space, environment) recognition, and / or motion recognition. According to an embodiment, the second camera module 1053 can also be used for hand detection and tracking, and the recognition of user gestures. According to an embodiment, at least one of the first camera modules 1051 to the third camera module 1055 can be replaced by a sensor module (e.g., a LiDAR sensor). For example, the sensor module may include at least one of a vertical-cavity surface-emitting laser (VCSEL), an infrared sensor, and / or a photodiode.

[0139] According to an embodiment, the electronic device 101 may include sound input devices 1062-1 and 1062-2 and sound output devices 1063-1 and 1063-2.

[0140] As described above, according to the embodiments, the electronic device 101 may have a shape and specifications intended to be worn on a user's head. For example, the electronic device 101 may include a shell to be worn on a user's head. The electronic device 101 may also include straps and / or wearable components to be secured to a part of the user's body. For example, the electronic device 101 may provide a user experience based on augmented reality, virtual reality, and / or mixed reality when worn on a user's head.

[0141] According to an embodiment, the electronic device 101 may have a shape and form factor that a user can carry (e.g., be able to be held in a user's hand). For example, the electronic device 101 may be a cellular phone or a tablet computer.

[0142] According to an embodiment, the electronic device 101 may be a device placed in a specific space or attached to another device. The method of implementing the electronic device 101 is not limited.

[0143] Figure 11a This is a view showing the light emitter 1100 and image sensor 230 of the electronic device 101 according to an embodiment. Figure 11b This is a view showing the light emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and / or 1112 of the electronic device 101 according to an embodiment, and the image sensor 230. Figure 12 This is a view illustrating the operation of the electronic device 101 according to an embodiment. Figure 13a This is a view illustrating the operation of the electronic device 101 according to an embodiment. Figure 13b This is a view illustrating the operation of the electronic device 101 according to an embodiment. The embodiments described above can be referenced for further explanation. Figure 11a , Figure 11b , Figure 12 , Figure 13a and Figure 13b .

[0144] refer to Figure 11a According to an embodiment, electronic device 101 may include a light emitter 1100 and an image sensor 230. The light emitter 1100 may be the flash lamp 220 of FIG. 2. The light emitter 1100 may include an infrared light-emitting diode (IR LED). The number of light emitters 1100 is not limited. For example, Figure 11a The light emitter 1100 may include Figure 11bMultiple optical emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112.

[0145] refer to Figure 11a and Figure 11b According to an embodiment, the electronic device 101 can be implemented using a design similar to goggles, glasses, or headphones (e.g., a shape to be worn on a user's head). With this design, the electronic device 101 is less sensitive to external environmental factors (such as light) when in close contact with or fixed to the user's eyes, and operates in conjunction with an image sensor 230 (e.g., an infrared (IR) camera) fixed at the eye position.

[0146] refer to Figure 11b According to an embodiment, the electronic device 101 may include a plurality of light emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 and an image sensor 230. Figure 11b This is a view showing the light emitter and image sensor corresponding to the user's left and right eyes. For ease of description, refer to... Figure 11b This can describe a plurality of light emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 corresponding to the user's left eye, as well as a plurality of image sensors 230. This description can be similarly applied to the light emitters and image sensors corresponding to the user's right eye. According to an embodiment, electronic device 101 (e.g., processor 120) can use at least one of the plurality of light emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 and at least one of the plurality of image sensors 230 to acquire raw data. For example, electronic device 101 (e.g., processor 120) can control the activation of at least one of a plurality of optical emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 and control the deactivation of at least one of the remaining optical emitters according to time periods. This is described below.

[0147] Figure 12 This may include raw data 1200 obtained through image sensor 230. (Reference) Figure 12 This phenomenon can be understood as follows: light emitters (e.g., Figure 11bAt least one of the following light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112) is projected onto the pupil 1202 and iris 1203 in the raw data 1200 obtained by the image sensor 230. For example, the raw data 1200 may include biometric data (e.g., iris 1203). The raw data 1200 may include data about the eyebrow 1201, pupil 1202, iris 1203, and sclera 1204. The electronic device 101 may obtain the raw data 1200 after controlling the activation of at least one of the plurality of light emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112. In this case, the light source (e.g., Figure 11b At least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 can be projected onto the original data 1200 (1210). This is because the light source (e.g., Figure 11b At least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 is projected onto the original data 1200 (1210), so the data corresponding to the iris 1203 may be distorted. On the other hand, gaze tracking algorithms that require data on the eyelids and eyebrows (e.g., Figure 8 The second algorithm (821) may be less sensitive to distortion caused by projection from such a light source (e.g., 1210). This will refer to... Figures 14 to 16 Provide a detailed description.

[0148] According to an embodiment, Figure 13a This may include data 1300 that has already undergone image processing. According to an embodiment, electronic device 101 (e.g., processor 120) may perform image processing on a region corresponding to biometric data (e.g., a region corresponding to the iris 1203). For example, refer to... Figure 13a Electronic device 101 (e.g., processor 120) can recognize pupil 1202 (or iris 1203) in Figure 12The location in the raw data 1200 can be distorted or removed based on the location of the identified pupil 1202 (or iris 1203) (1310). For example, the electronic device 101 (e.g., processor 120) can identify the location of a designated area based on the characteristics of the electronic device 101 (e.g., goggles, glasses, or headphone type design). The designated area can be an area corresponding to biometric data. For example, the electronic device 101 can be implemented with a design of goggles, glasses, or headphones (e.g., shape specifications for being worn on a user's head), so the location of the area corresponding to the biometric data in the raw data obtained from the electronic device 101 can be included within the designated range. For example, when the user is wearing the electronic device 101, the distance and angle from the user's body parts (e.g., eyebrows, eyeballs, pupils, and irises) to the electronic device 101 (e.g., image sensor 230 of the electronic device 101) can remain constant. Therefore, the raw data obtained by electronic device 101 may include biometric data, and the biometric data may be included in a designated area (e.g., a designated range) of the raw data. For example, electronic device 101 may store information about the location of a designated area based on the characteristics of electronic device 101 (e.g., the design of goggles, glasses, or headphones) in memory 130. For example, electronic device 101 may obtain raw data when electronic device 101 is worn on a user's head. Electronic device 101 (e.g., processor 120) may identify the location of a designated area based on the characteristics of electronic device 101 from the obtained raw data as the location of the area corresponding to the biometric data. Electronic device 101 (e.g., processor 120) may perform image processing (e.g., distortion or deletion) on the area corresponding to the location of the designated area and corresponding to the biometric data. For example, image processing of the area corresponding to the biometric data may include at least one of hidden line removal, blurring, deletion of biometric data, or shading. According to an embodiment, electronic device 101 (e.g., processor 120) can identify the location of biometric data in raw data (or raw data that has undergone image preprocessing) and can perform image processing (e.g., at least one of hidden line removal, biometric data removal, blurring, or shading) on ​​the identified location of the biometric data. For example, electronic device 101 (e.g., processor 120) can identify biometric data (e.g., the location of biometric data) from the raw data (or raw data that has undergone image preprocessing) based on information stored in memory 130 (e.g., information about the location of the biometric data). For example, electronic device 101 (e.g., processor 120) can store information about the location of biometric data to be included in the raw data in memory 130 before obtaining the raw data.Electronic device 101 can identify the location of biometric data in raw data (or raw data that has undergone image preprocessing) based on information stored in memory 130 (e.g., information about the location of biometric data to be included in the raw data). According to an embodiment, electronic device 101 may include memory 130 storing information about the location of biometric data to be included in the raw data. According to an embodiment, electronic device 101 can identify the location of biometric data based on first raw data or image data corresponding to the first raw data at a first time, and can store the identified location information of the biometric data in memory 130. Electronic device 101 can obtain second raw data at a second time after the first time, and can identify the location of the biometric data in the second raw data (or second raw data that has undergone image preprocessing) based on information about the location of the biometric data identified before the second time (e.g., at the first time).

[0149] According to an embodiment, Figure 13b This may include data 1320 that has already undergone image processing. According to an embodiment, electronic device 101 (e.g., processor 120) may perform image processing on initial data (e.g., raw data or raw data that has undergone image preprocessing). For example, electronic device 101 may perform image processing (e.g., distortion or deletion) on the raw data (or raw data that has undergone image preprocessing) as a whole. For example, image processing on the raw data (or raw data that has undergone image preprocessing) as a whole may include at least one of hidden line removal, blurring, or shading.

[0150] Figure 14 This is a view illustrating the operation of an electronic device according to an embodiment. Figure 15 This is a flowchart illustrating an operation method of an electronic device according to an embodiment. Figure 16 This is a flowchart illustrating an operation method of an electronic device according to an embodiment. The above embodiments can be used as a reference for description. Figure 14 , Figure 15 and Figure 16 .

[0151] refer to Figure 14The settings of the plurality of light emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 in Figure 11 can be understood. The “settings” may include information about which of the plurality of light emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 will be controlled to be turned on, and / or information about the brightness (e.g., auto exposure gain) of the light emitter that is controlled to be turned on among the plurality of light emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112. For example, the settings may include information about the number, location, and / or brightness of the controlled light emitters. For example, the settings may include a first setting for biometric sensing (e.g., Figure 14 (b0, b1, b2, and bn of 1410) and a second setting value for camera image processing where security is not required (e.g., Figure 14 (c of 1420). Figure 14 1410 can show the first set value (e.g., for biometric sensing) based on the time periods including t1, t3, and t5. Figure 14 The original data is obtained from b0, b1, b2 and bn of 1410. Figure 14 1420 can illustrate obtaining raw data based on a second setpoint for camera image processing that does not require security during time periods including t2, t4, and tn. For example, electronic device 101 (e.g., processor 120) can use image sensor 230 to obtain raw data including biometric data based on a first setpoint for biometric sensing. For example, electronic device 101 (e.g., processor 120) can use image sensor 230 to obtain raw data excluding biometric data (e.g., raw data where the biometric data is distorted) based on a second setpoint for camera image processing that does not require security. According to embodiments, multiple first setpoints for biometric sensing may exist, and a next setpoint for biometric sensing can be determined based on feedback regarding a previous setpoint for biometric sensing. For example, in Figure 14 In this process, b1 can be determined based on the feedback from b0, and b2 can be determined based on the feedback from b1, and this operation can be repeated. According to an embodiment, a first set value for biometric sensing can exist, and this set value for biometric sensing can remain unchanged. According to an embodiment, when the application... Figure 14 In some embodiments, the electronic device 101 may not perform any alteration operations on the biometric data. According to embodiments, even in applications... Figure 14In some embodiments, the electronic device 101 may also perform modification operations on biometric data.

[0152] You can refer to this. Figure 14 Described in accordance with the above embodiments Figure 15 .

[0153] Figure 15 At least some operations can be omitted. It can be changed. Figure 15 The order of operations. (Can be...) Figure 15 Perform before, during or after the operation Figure 15 Operations other than those performed by the operator.

[0154] refer to Figure 15 In operation 1501, according to an embodiment, electronic device 101 (e.g., processor 120) can acquire first raw data during a first time period. For example, electronic device 101 can... Figure 14 The first raw data was obtained during the first time period t1. Figure 14 During the first time period t1, the electronic device 101 can, based on a first set value for biometric sensing (e.g., Figure 14 The first setting (b0) is used to control the activation of the first optical emitter among multiple optical emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112. For example, the first optical emitter may include... Figure 11b The 1101, 1103, 1105, 1107, 1109, and 1111 are examples, but the criteria used to select the first optical emitter are unrestricted. Figure 14 During the first time period t1, the electronic device 101 can, based on a first set value for biometric sensing (e.g., Figure 14 The first setting (b0) controls the first light emitter among the multiple light emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 to turn on, and then the first raw data can be obtained using the image sensor 230.

[0155] In operation 1503, according to an embodiment, electronic device 101 (e.g., processor 120) may perform image preprocessing on the first raw data of operation 1501 using image preprocessor 270. According to an embodiment, operation 1503 may be omitted.

[0156] In operation 1505, according to an embodiment, electronic device 101 (e.g., processor 120) may store the first raw data of operation 1501 or the data obtained by performing image preprocessing in operation 1503 in a first buffer 310 of memory 130. In operation 1505, the data stored in the first buffer 310 may include biometric data. Thereafter, electronic device 101 may perform biometric authentication based on the data including biometric data stored in the first buffer 310.

[0157] In operation 1507, according to an embodiment, electronic device 101 (e.g., processor 120) may acquire second raw data during a second time period. For example, electronic device 101 may... Figure 14 The second raw data was obtained during the second time period t2. Figure 14 During the second time period t2, the electronic device 101 can perform image processing for cameras that do not require security (e.g., Figure 14 c) controls the activation of a second optical emitter among multiple optical emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112. For example, the second optical emitter may include... Figure 11b The numbers 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 are used, but the criteria for selecting the second optical emitter are not limited to these.

[0158] According to an embodiment, the first light emitter of 1501 is operated (e.g., Figure 11b The number of 1101, 1103, 1105, 1107, 1109, and 1111 can be less than that of the second light emitter of operation 1507 (e.g., Figure 11bThe number of light emitters (1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112). For example, the number of light emitters controlled to be turned on in operation 1501 can be a number suitable for biometric authentication. The number of light emitters controlled to be turned on in operation 1507 can be greater than the number of light emitters controlled to be turned on in operation 1501. Therefore, the raw data obtained in operation 1507 (e.g., second raw data) can be raw data in which the biometric data has been distorted. According to an embodiment, the number of first light emitters in operation 1501 and the number of second light emitters in operation 1507 can be the same, and the positions of at least some of the first light emitters in operation 1501 and at least some of the second light emitters in operation 1507 can be different. For example, the first light emitters in operation 1501 can be selected as light emitters positioned at locations suitable for biometric authentication. The second light emitter in operation 1507 can be selected as a light emitter positioned at a designated location where data used for biometric authentication may be distorted. According to embodiments, the first light emitter in operation 1501 and the second light emitter in operation 1507 can be the same, and the brightness of at least one of the first light emitters in operation 1501 and the brightness of at least one of the second light emitters in operation 1507 can be different. For example, the brightness of at least one of the first light emitters in operation 1501 can be a brightness suitable for biometric authentication. The brightness of at least one of the second light emitters in operation 1507 can be a designated brightness that distorts data used for biometric authentication. The embodiments described above regarding the number, position, and brightness of light emitters can be applied interchangeably. For example, at least one of the number, position, and brightness of the first light emitter in operation 1501 can be different from at least one of the number, position, and brightness of the second light emitter in operation 1507.

[0159] In operation 1509, according to an embodiment, electronic device 101 (e.g., processor 120) can perform image preprocessing and modification operations on the second raw data of operation 1507 using image preprocessor 270. According to an embodiment, operation 1509 can be omitted. For example, image preprocessing in operation 1509 can be performed, and only the modification operations can be omitted. For example, both image preprocessing and modification operations in operation 1509 can be omitted.

[0160] In operation 1511, according to an embodiment, electronic device 101 (e.g., processor 120) may store the second raw data of operation 1507 or the data obtained by performing at least one of the image preprocessing or modification operations of operation 1509 in a second buffer 320 of memory 130. In operation 1511, the data stored in the second buffer 320 may not include biometric data, or the data stored in the second buffer 320 may include distorted biometric data. Thereafter, electronic device 101 may perform camera image processing that does not require security based on the data stored in the second buffer 320.

[0161] Figure 15 The first and second time periods are described, but only some aspects of the operation of electronic device 101 are described. Figure 15 The operations can be repeated. For example, operations 1501 to 1505 can be performed during the first time period, operations 1507 to 1511 can be performed during the second time period after the first time period, operations 1501 to 1505 can be performed during the third time period after the second time period, and operations 1507 to 1511 can be performed during the fourth time period after the third time period, and such operations can be repeated.

[0162] You can refer to this. Figure 14 and Figure 15 And the above embodiments are described Figure 16 .

[0163] Figure 16 At least some operations can be omitted. It can be changed. Figure 16 The order of operations. (Can be...) Figure 16 Perform before, during or after the operation Figure 16 Operations other than those performed by the operator.

[0164] refer to Figure 16 In operation 1601, according to an embodiment, electronic device 101 (e.g., processor 120) can control a first light emitter (e.g., ...) during a first time period. Figure 11b At least some of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 are enabled. For example, in Figure 14 During the first time period t1, the electronic device 101 can, based on set values ​​used for biometric sensing (e.g., Figure 14b0) controls the activation of a first optical emitter among multiple optical emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112. For example, the first optical emitter may include... Figure 11b The values ​​1101, 1103, 1105, 1107, 1109, and 1111 are acceptable, but the criteria used to select the first optical emitter are not limited. For example, setting value 1601 (e.g., Figure 14 b0) can be the default setting for biometric authentication.

[0165] In operation 1603, according to an embodiment, electronic device 101 (e.g., processor 120) can control the first light emitter of operation 1601 among a plurality of light emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 (e.g., Figure 11b At least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 is activated, and then the first raw data can be obtained using the image sensor 230. According to an embodiment, the electronic device 101 (e.g., processor 120) can perform operation 1505 on the first raw data of operation 1603, or perform both operation 1503 and operation 1505. The first raw data of operation 1603 may include biometric data at the level of biometric authentication. Based on the first raw data including biometric authentication-level biometric data, the settings for biometric authentication (e.g., the number, position, and / or brightness of controlled light emitters) may remain unchanged. When the settings remain unchanged, the settings for operation 1601 and operation 1609 may be the same. The first raw data of operation 1603 may not include biometric data at the level of biometric authentication. Based on first raw biometric data excluding biometric authentication levels, settings for biometric authentication (e.g., the number, location, and / or brightness of controlled light emitters) can be updated. Electronic device 101 (e.g., processor 120) can update the settings for biometric authentication (e.g., the number, location, and / or brightness of controlled light emitters). When the settings are updated, the settings for operation 1601 and operation 1609 can be different.

[0166] In operation 1605, according to an embodiment, electronic device 101 (e.g., processor 120) can control a second light emitter (e.g., ...) during the second time period. Figure 11bAt least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 is enabled. For example, in Figure 14 During the second time period t2, the electronic device 101 can perform image processing based on settings for cameras that do not require security (e.g., Figure 14 c) controls the activation of a second optical emitter among multiple optical emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112. For example, the second optical emitter may include... Figure 11b The values ​​1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 are used, but the criteria for selecting the second optical emitter are not limited to these. The settings for operation 1601 (e.g., Figure 14 b0) can be different from the setting value of operation 1605 (e.g., Figure 14 (c) The number, position, or brightness of the first light emitter in operation 1601 may differ from the number, position, or brightness of the second light emitter in operation 1605. For example, the number of the second light emitters in operation 1605 may be greater than the number of the first light emitters in operation 1601.

[0167] In operation 1607, according to an embodiment, electronic device 101 (e.g., processor 120) can control a second light emitter of operation 1605 among a plurality of light emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 (e.g., Figure 11b At least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 is turned on, and then the second raw data can be obtained using the image sensor 230. According to an embodiment, the electronic device 101 (e.g., processor 120) can perform operation 1511 on the second raw data of operation 1607, or perform operations 1509 and operation 1511.

[0168] In operation 1609, according to an embodiment, electronic device 101 (e.g., processor 120) can control a third light emitter (e.g., ...) during the third time period. Figure 11b At least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 is enabled. For example, in Figure 14 During the third time period t3, the electronic device 101 can, based on set values ​​used for biometric sensing (e.g., Figure 14 b1) controls the activation of a third optical transmitter among multiple optical transmitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112. For example, the third optical transmitter may include... Figure 11b The values ​​1102, 1104, 1106, 1108, 1110, and 1112 are not limited to these, but the criteria used to select the third light emitter are not limited to these. As described above in operations 1601 and 1603, the setting value of operation 1609 may be the same as or different from the setting value of operation 1601. For example, the setting value of 1609 (e.g., Figure 14 b1) can be based on default settings (e.g., Figure 14 The setting value is updated based on the fact that the setting value of operation 1609 is different from the setting value of operation 1601. At least one of the first light emitters in operation 1601 may be different from at least one of the third light emitters in operation 1609. For example, the setting value of operation 1609 (e.g., b0) is different from the setting value of operation 1601. Figure 14 b1) can be used with the default settings for biometric authentication (e.g., Figure 14 The b0) is the same. Since the setting value of operation 1609 is the same as the setting value of operation 1601, the first light emitter of operation 1601 can be the same as the third light emitter of operation 1609.

[0169] In operation 1611, according to an embodiment, electronic device 101 (e.g., processor 120) can control a third light emitter of operation 1609 among a plurality of light emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 (e.g., Figure 11b At least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 is activated, and then the third raw data can be obtained using the image sensor 230. According to an embodiment, the electronic device 101 (e.g., processor 120) can perform operation 1505, or perform operations 1503 and 1505, on the third raw data of operation 1611.

[0170] In operation 1613, according to an embodiment, electronic device 101 (e.g., processor 120) can control a second light emitter (e.g., ...) during a fourth time period. Figure 11bAt least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 is enabled. For example, in Figure 14 During the fourth time period t4, the electronic device 101 can be based on settings for camera image processing that does not require security (e.g., Figure 14 c) controls the activation of the second optical transmitter among multiple optical transmitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112. The second optical transmitter operated by 1613 can be the same as the second optical transmitter operated by 1605.

[0171] In operation 1615, according to an embodiment, electronic device 101 (e.g., processor 120) can control the second light emitter of operation 1613 among a plurality of light emitters 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 (e.g., Figure 11b At least one of 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 is turned on, and then the fourth raw data can be obtained using the image sensor 230. According to an embodiment, the electronic device 101 (e.g., processor 120) can perform operation 1511 on the fourth raw data of operation 1615, or perform operations 1509 and operation 1511.

[0172] Those skilled in the art will understand that the embodiments described herein can be applied interchangeably within their applicable scope. For example, those skilled in the art will understand that at least some operations of the embodiments described in this disclosure may be omitted and applied, or at least some operations of the embodiments may be combined and applied.

[0173] This disclosure is not limited to the foregoing, and other unmentioned variations will be readily understood by those skilled in the art based on this disclosure.

[0174] The effects that can be obtained from this disclosure are not limited to those described above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

[0175] According to an example embodiment, an electronic device 101 may include: an image sensor 230, an image preprocessor 270 including circuitry, a memory 130 storing instructions, and at least one processor 120 including processing circuitry. When executed individually and / or jointly by the at least one processor, the instructions may cause the electronic device 101 to: acquire raw data including biometric data using the image sensor 230. When executed individually and / or jointly by the at least one processor 120, the instructions may cause the electronic device 101 to: store the raw data as a first image data in a first buffer 310. When executed individually and / or jointly by the at least one processor 120, the instructions may cause the electronic device 101 to: store a second image data, configured to be generated by applying a modification operation to the raw data using the image preprocessor 270, in a second buffer 320. When executed individually and / or jointly by the at least one processor 120, the instructions may cause the electronic device 101 to: forward the first image data stored in the first buffer 310 to a first application included in a secure area, and apply a first algorithm for biometric authentication to the first image data. When executed individually and / or jointly by at least one processor 120, the instructions can cause the electronic device 101 to: forward second image data stored in the second buffer 320 to a second application included in the non-secure area, and apply a second algorithm different from the first algorithm to the second image data.

[0176] According to an example embodiment, the instructions, when executed individually and / or jointly by at least one processor 120, may cause electronic device 101 to forward second image data stored in a second buffer 320 to a second application during a period that at least partially overlaps with a period in which first image data stored in a first buffer 310 is forwarded to a first application.

[0177] According to an example embodiment, the modification operation may include performing image processing on the original data. Image processing may include at least one of hidden line removal, blurring, or shading.

[0178] According to an example embodiment, image processing may include at least one of hidden line removal, blurring, or shading of the raw data as a whole.

[0179] According to an example embodiment, image processing may include at least one of hidden line removal, blurring, or shading processing for regions corresponding to biometric data included in the original data.

[0180] According to an example embodiment, the instructions, when executed individually and / or jointly by at least one processor 120, can cause electronic device 101 to acquire raw data when electronic device 101 is worn on a user's head. The instructions, when executed individually and / or jointly by at least one processor 120, can cause electronic device 101 to identify the location of a specified region in the raw data based on characteristics of electronic device 101. The instructions, when executed individually and / or jointly by at least one processor 120, can cause electronic device 101 to perform image processing on the region corresponding to the location of the specified region and the region corresponding to biometric data.

[0181] According to an example embodiment, when executed individually and / or jointly by at least one processor 120, the instructions can cause electronic device 101 to acquire first raw data from the raw data during a first time period using image sensor 230. When executed individually and / or jointly by at least one processor 120, the instructions can cause electronic device 101 to store at least a portion of first image data identified by performing image preprocessing on the first raw data using image preprocessor 270 in a first buffer 310. When executed individually and / or jointly by at least one processor 120, the instructions can cause electronic device 101 to acquire second raw data from the raw data during a second time period following the first time period using image sensor 230. When executed individually and / or jointly by at least one processor 120, the instructions can cause electronic device 101 to store at least a portion of second image data identified by performing image preprocessing and modification operations on the second raw data using image preprocessor 270 in a second buffer 320.

[0182] According to an example embodiment, electronic device 101 may include a plurality of optical emitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112. Instructions, when executed individually and / or jointly by at least one processor 120, may enable electronic device 101 to control, during a first time period, a first optical emitter among the plurality of optical emitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112. When executed individually and / or jointly by at least one processor 120, the instructions can cause the electronic device 101 to control the second optical transmitter 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 among a plurality of optical transmitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 to be turned on during a second time period. The number of first optical transmitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 can be less than the number of second optical transmitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112.

[0183] According to an example embodiment, when executed individually and / or jointly by at least one processor 120, the instructions can cause the electronic device 101 to control the third optical emitter 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 among a plurality of optical emitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 to be turned on during a third time period following the second time period. At least one of the third light emitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 may be different from at least one of the first light emitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112. When executed individually and / or jointly by at least one processor 120, the instructions may cause electronic device 101 to acquire third raw data from the raw data during the third time period using image sensor 230. When executed individually and / or jointly by at least one processor 120, the instructions may cause electronic device 101 to store at least a portion of the first image data identified by performing image preprocessing on the third raw data using image preprocessor 270 in the first buffer 310.

[0184] According to an example embodiment, when executed individually and / or jointly by at least one processor 120, the instructions can cause electronic device 101 to control the second light emitter 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 among a plurality of light emitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 to be turned on during a fourth time period following the third time period. When executed individually and / or jointly by at least one processor 120, the instructions can cause electronic device 101 to use image sensor 230 to acquire fourth raw data from the raw data during the fourth time period. When executed individually and / or jointly by at least one processor 120, the instructions can cause the electronic device 101 to store at least a portion of the second image data identified by performing image preprocessing and modification operations on the fourth raw data using the image preprocessor 270 in the second buffer 320.

[0185] According to an example embodiment, a method for operating an electronic device 101 may include obtaining raw data including biometric data using an image sensor 230 of the electronic device 101. The method may include storing the raw data as a first image data in a first buffer 310. The method may include storing a second image data generated by applying a modification operation to the raw data in a second buffer 320. The method may include forwarding the first image data stored in the first buffer 310 to a first application included in a secure area, and applying a first algorithm for biometric authentication to the first image data. The method may include forwarding the second image data stored in the second buffer 320 to a second application included in a non-secure area, and applying a second algorithm different from the first algorithm to the second image data.

[0186] According to an example embodiment, forwarding the second image data to the second application may include: forwarding the second image data stored in the second buffer 320 to the second application during a period that at least partially overlaps with the period during which the first image data stored in the first buffer 310 is forwarded to the first application.

[0187] According to an example embodiment, the modification operation may include performing image processing on the original data. Image processing may include at least one of hidden line removal, blurring, or shading.

[0188] According to an example embodiment, image processing may include at least one of hidden line removal, blurring, or shading of the raw data as a whole.

[0189] According to an example embodiment, image processing may include at least one of hidden line removal, blurring, or shading processing for regions corresponding to biometric data included in the original data.

[0190] According to an example embodiment, obtaining raw data may include obtaining raw data while the electronic device 101 is worn on a user's head. Modification operations may include identifying the location of a specified region in the raw data based on characteristics of the electronic device 101. Modification operations may include performing image processing on the region corresponding to the location of the specified region and the biometric data.

[0191] According to an example embodiment, obtaining raw data may include: acquiring first raw data from the raw data using image sensor 230 during a first time period. Storing the first image data in the first buffer 310 may include: storing at least a portion of the first image data identified by performing image preprocessing on the first raw data in the first buffer 310. Obtaining raw data may include: acquiring second raw data from the raw data using image sensor 230 during a second time period following the first time period. Storing the second image data in the second buffer 320 may include: storing at least a portion of the second image data identified by performing image preprocessing and modification operations on the second raw data in the second buffer 320.

[0192] According to an example embodiment, the method may include controlling a first optical transmitter 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 among a plurality of optical transmitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 to be turned on during a first time period. The method may include controlling the second optical transmitter 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 among a plurality of optical transmitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 to be turned on during a second time period. The number of first optical transmitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 can be less than the number of second optical transmitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112.

[0193] According to an example embodiment, the method may include controlling the third optical transmitter 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 among a plurality of optical transmitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 to be turned on during a third time period following the second time period. At least one of the third light emitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 may be different from at least one of the first light emitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112. The method may include acquiring third raw data from the raw data using image sensor 230 during a third time period. The method may also include storing at least a portion of the first image data identified by performing image preprocessing on the third raw data using image preprocessor 270 in a first buffer 310.

[0194] According to an example embodiment, the method may include controlling the activation of a second light emitter 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 among a plurality of light emitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 during a fourth time period following a third time period. The method may include acquiring fourth raw data from the raw data using an image sensor 230 during the fourth time period. The method may include storing at least a portion of second image data identified by performing image preprocessing and modification operations on the fourth raw data using an image preprocessor 270 in a second buffer 320.

[0195] According to an example embodiment, in a non-transitory computer-readable recording medium storing instructions, the instructions, when executed individually and / or jointly by at least one processor 120 of electronic device 101, cause electronic device 101 to perform at least one operation. This at least one operation may include acquiring raw data including biometric data using image sensor 230 of electronic device 101. The at least one operation may include storing the raw data as a first image data in a first buffer 310. The at least one operation may include storing a second image data generated by applying a modification operation to the raw data in a second buffer 320. This at least one operation may include forwarding the first image data stored in the first buffer 310 to a first application included in a secure area, and applying a first algorithm for biometric authentication to the first image data. The at least one operation may include forwarding the second image data stored in the second buffer 320 to a second application included in a non-secure area, and applying a second algorithm different from the first algorithm to the second image data.

[0196] According to an example embodiment, forwarding the second image data to the second application may include: forwarding the second image data stored in the second buffer 320 to the second application during a period that at least partially overlaps with the period during which the first image data stored in the first buffer 310 is forwarded to the first application.

[0197] According to an example embodiment, the modification operation may include performing image processing on the original data. Image processing may include at least one of hidden line removal, blurring, or shading.

[0198] According to an example embodiment, image processing may include at least one of hidden line removal, blurring, or shading of the raw data as a whole.

[0199] According to an example embodiment, image processing may include at least one of hidden line removal, blurring, or shading processing for regions corresponding to biometric data included in the original data.

[0200] According to an example embodiment, obtaining raw data may include obtaining raw data while the electronic device 101 is worn on a user's head. Modification operations may include identifying the location of a specified region in the raw data based on characteristics of the electronic device 101. Modification operations may include performing image processing on the region corresponding to the location of the specified region and the biometric data.

[0201] According to an example embodiment, obtaining raw data may include: acquiring first raw data from the raw data using image sensor 230 during a first time period. Storing the first image data in the first buffer 310 may include: storing at least a portion of the first image data identified by performing image preprocessing on the first raw data in the first buffer 310. Obtaining raw data may include: acquiring second raw data from the raw data using image sensor 230 during a second time period following the first time period. Storing the second image data in the second buffer 320 may include: storing at least a portion of the second image data identified by performing image preprocessing and modification operations on the second raw data in the second buffer 320.

[0202] According to an example embodiment, at least one operation may include controlling the first optical transmitter 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 among a plurality of optical transmitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 to be turned on during a first time period. At least one operation may include controlling the second optical transmitter 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 among a plurality of optical transmitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 to be turned on during the second time period. The number of first optical transmitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 can be less than the number of second optical transmitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112.

[0203] According to an example embodiment, at least one operation may include controlling the third optical transmitter 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 among a plurality of optical transmitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111 and 1112 to be turned on during a third time period following the second time period. At least one of the third light emitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 may be different from at least one of the first light emitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112. At least one operation may include acquiring third raw data from the raw data using image sensor 230 during a third time period. At least one operation may include storing at least a portion of the first image data identified by performing image preprocessing on the third raw data using image preprocessor 270 in the first buffer 310.

[0204] According to an example embodiment, at least one operation may include controlling the activation of a second light emitter 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 among a plurality of light emitters 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, and 1112 during a fourth time period following a third time period. At least one operation may include acquiring fourth raw data from the raw data using image sensor 230 during the fourth time period. At least one operation may include storing at least a portion of second image data identified by performing image preprocessing and modification operations on the fourth raw data using image preprocessor 270 in a second buffer 320.

[0205] The electronic device according to various embodiments of this disclosure can be one of a variety of types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. According to embodiments of this disclosure, the electronic device is not limited to the electronic devices described above.

[0206] It should be understood that the various embodiments of this disclosure and the terminology used therein are not intended to limit the technical features set forth herein to the specific embodiments, but rather to include various changes, equivalents, or substitutions of the corresponding embodiments. Regarding the description of the drawings, similar reference numerals may be used to refer to similar or related elements. It is readily understood that the singular form of a noun corresponding to an item may include one or more things unless the relevant context explicitly indicates otherwise. As used herein, each of the phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least one of A, B, or C” may include all possible combinations of the items listed together in the corresponding phrase. As used herein, terms such as “first” and “second” or “first” and “second” may be used simply to distinguish corresponding components from another component and do not limit the components in other respects (e.g., importance or order). It is readily understood that, whether or not the terms “operably” or “communically” are used, if an element (e.g., a first element) is referred to as “coupled to another element (e.g., a second element),” “coupled to another element (e.g., a second element),” “connected to another element (e.g., a second element),” or “connected to another element (e.g., a second element)”, it indicates that the element can be coupled to another element directly (e.g., wired), wirelessly, or via a third element.

[0207] As used herein, the term "module" can include units implemented in hardware, software, or firmware, and is used interchangeably with other terms such as "logic," "logic block," "part," and "circuit." A module can be a single, integral component suitable for performing one or more functions, or its smallest unit or part. For example, according to an embodiment, a module can be implemented as an application-specific integrated circuit (ASIC).

[0208] The various embodiments described herein can be implemented as software (e.g., a program) comprising one or more instructions stored in a machine-readable storage medium (e.g., an electronic device). For example, a machine's processor (e.g., a controller) can invoke and execute at least one of the instructions stored in the storage medium. This enables the machine to be operated to perform at least one function according to the invoked instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. The term "non-transitory" means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but this term does not distinguish between cases where data is semi-permanently stored in the storage medium and cases where data is temporarily stored in the storage medium.

[0209] According to embodiments, methods according to various embodiments of this disclosure can be included and provided in a computer program product. The computer program product can be traded as a commodity between a seller and a buyer. The computer program product can be distributed in the form of a machine-readable storage medium (e.g., an optical disc read-only memory (CD-ROM)) or through an app store (e.g., the Play Store). TM The computer program product may be distributed online (e.g., downloaded or uploaded) or directly between two user devices (e.g., smartphones). If distributed online, at least a portion of the computer program product may be temporarily generated or at least temporarily stored in a machine-readable storage medium, such as the memory of a manufacturer's server, an app store's server, or a relay server.

[0210] According to various embodiments, each component (e.g., a module or program) described above may include a single entity or multiple entities. Some of the multiple entities may be separately located in different components. According to various embodiments, one or more of the above-described components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform one or more functions of each of the multiple components in the same or similar manner as performed by the corresponding one of the multiple components prior to integration. According to various embodiments, operations performed by a module, program, or other component may be performed sequentially, in parallel, repeatedly, or heuristically, or may be performed in a different order or omitted, or one or more other operations may be added.

[0211] While this disclosure has been illustrated and described with reference to various exemplary embodiments, it should be understood that these exemplary embodiments are intended to be illustrative and not restrictive. Those skilled in the art will also understand that various changes in form and detail may be made without departing from the true spirit and full scope of this disclosure, including the appended claims and their equivalents. It should also be understood that any embodiment described herein may be used in conjunction with any other embodiment described herein.

Claims

1. An electronic device (101), the electronic device (101) comprising: Image sensor (230); An image preprocessor (270) includes circuitry; Memory (130), the memory (130) storing instructions; and At least one processor (120), said at least one processor (120) including processing circuitry; Wherein, when the instructions are executed individually and / or jointly by at least one processor (120), the electronic device (101) is: The image sensor (230) is used to obtain raw data including biometric data. The raw data is stored as the first image data in the first buffer (310). The second image data, generated by applying a change operation to the original data using the image preprocessor (270), is stored in the second buffer (320). The first image data stored in the first buffer (310) is forwarded to the first application included in the secure area, and a first algorithm for biometric authentication is applied to the first image data. The second image data stored in the second buffer (320) is forwarded to the second application included in the non-secure area, and a second algorithm different from the first algorithm is applied to the second image data.

2. The electronic device (101) according to claim 1, wherein, When the instructions are executed individually and / or jointly by at least one processor (120), the electronic device (101) is made to: The second image data stored in the second buffer (320) is forwarded to the second application during a period that at least partially overlaps with the period during which the first image data stored in the first buffer (310) is forwarded to the first application.

3. The electronic device (101) according to claim 1 or 2, wherein, The modification operation includes performing image processing on the original data, and The image processing includes at least one of hidden line removal, blurring, or shadowing.

4. The electronic device (101) according to any one of claims 1 to 3, wherein, The image processing includes at least one of hidden line removal, blurring, or shading of the raw data as a whole.

5. The electronic device (101) according to any one of claims 1 to 4, wherein, The image processing includes at least one of vegetative removal, blurring, or shading of the region corresponding to the biometric data included in the original data.

6. The electronic device (101) according to any one of claims 1 to 5, wherein, When the instructions are executed individually and / or jointly by at least one processor (120), the electronic device (101) is made to: The location of the designated area is identified based on the characteristics of the electronic device (101), and The image processing is performed on the region corresponding to the biometric data, wherein the region corresponding to the biometric data corresponds to the location of the designated region.

7. The electronic device (101) according to any one of claims 1 to 6, wherein, When the instructions are executed individually and / or jointly by at least one processor (120), the electronic device (101) is made to: The image sensor (230) is used to acquire the first raw data from the raw data during the first time period. At least a portion of the first image data identified by performing image preprocessing on the first raw data using the image preprocessor (270) is stored in the first buffer (310). The image sensor (230) is used to acquire second raw data from the raw data during a second time period following the first time period, and At least a portion of the second image data identified by performing the image preprocessing and the alteration operation on the second raw data using the image preprocessor (270) is stored in the second buffer (320).

8. The electronic device (101) according to any one of claims 1 to 7, wherein the electronic device (101) further comprises a plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112), the plurality of light emitters comprising light-emitting circuitry, and in, When the instructions are executed individually and / or jointly by at least one processor (120), the electronic device (101) is made to: During the first time period, the first light emitter (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) is turned on, and During the second time period, the second light emitter (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) is turned on, and The number of the first optical transmitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) is less than the number of the second optical transmitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112).

9. The electronic device (101) according to any one of claims 1 to 8, wherein, When the instructions are executed individually and / or jointly by at least one processor (120), the electronic device (101) is made to: During a third time period following the second time period, the third light emitter (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1109; 1108; 1110; 1111; 1112) is turned on, wherein at least one of the third light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1109; 1108; 1110; 1111; 1112) is different from at least one of the first light emitters. The image sensor (230) is used to obtain the third raw data in the raw data during the third time period, and At least a portion of the first image data identified by performing image preprocessing on the third raw data using the image preprocessor (270) is stored in the first buffer (310).

10. The electronic device (101) according to any one of claims 1 to 9, wherein, When the instructions are executed individually and / or jointly by at least one processor (120), the electronic device (101) is made to: During the fourth period following the third period, the second light emitter (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) among the plurality of light emitters (1100; 1101; 1102; 1103; 1104; 1105; 1106; 1107; 1108; 1109; 1110; 1111; 1112) is turned on. The image sensor (230) is used to obtain the fourth raw data in the raw data during the fourth time period, and At least a portion of the second image data identified by performing image preprocessing and the alteration operation on the fourth raw data using the image preprocessor (270) is stored in the second buffer (320).

11. A method of operating an electronic device (101), the method comprising: Raw data, including biometric data, is obtained using the image sensor (230) of the electronic device (101); The raw data is stored as the first image data in the first buffer (310); The second image data generated by applying a change operation to the original data is stored in the second buffer (320); The first image data stored in the first buffer (310) is forwarded to the first application included in the secure area, and the first algorithm for biometric authentication is applied to the first image data; as well as The second image data stored in the second buffer (320) is forwarded to the second application included in the non-secure area, and a second algorithm different from the first algorithm is applied to the second image data.

12. The method according to claim 11, in, Forwarding the second image data to the second application includes: The second image data stored in the second buffer (320) is forwarded to the second application during a period that at least partially overlaps with the period during which the first image data stored in the first buffer (310) is forwarded to the first application.

13. The method according to claim 11 or 12, in, The modification operation includes performing image processing on the original data, and The image processing includes at least one of hidden line removal, blurring, or shadow line processing.

14. The method according to any one of claims 11 to 13, in, The image processing includes at least one of hidden line removal, blurring, or shadow line processing applied to the entire original data.

15. A non-transitory computer-readable recording medium storing instructions that, when executed alone and / or jointly by at least one processor (120) of an electronic device (101), cause the electronic device (101) to perform at least one operation including: Raw data, including biometric data, is obtained using the image sensor (230) of the electronic device (101); The raw data is stored as the first image data in the first buffer (310); The second image data generated by applying a change operation to the original data is stored in the second buffer (320); The first image data stored in the first buffer (310) is forwarded to the first application included in the secure area, and the first algorithm for biometric authentication is applied to the first image data; as well as The second image data stored in the second buffer (320) is forwarded to the second application included in the non-secure area, and a second algorithm different from the first algorithm is applied to the second image data.

Images