Electronic device including image sensor, operating method thereof, and computer-readable recording medium

Abstract

An electronic device according to the present disclosure may be configured to: control a first image sensor to crop image data constructed on the basis of information acquired from first sensor pixels corresponding to a first region of interest (RoI) among a plurality of sensor pixels and to output the cropped image data to at least one processor; receive a touch input via a touchscreen; determine, on the basis of the touch input, a second RoI differing from the first RoI in at least one of position and area; and control the first image sensor to crop image data constructed on the basis of information acquired from second sensor pixels corresponding to the second RoI among the plurality of sensor pixels and to output the cropped image data to the at least one processor. Various other embodiments are possible.

Description

Electronic device including an image sensor, method of operation thereof, and computer-readable recording medium

[0001] The present disclosure relates to an electronic device including an image sensor, a method of operating the same, and a computer-readable recording medium storing a computer program for performing the method of operating.

[0002] Various types of electronic devices utilizing electrical and electronic technology or information technology are being developed and distributed. For example, there are portable electronic devices with various functions, such as smartphones, tablet PCs, or wearable devices (e.g., smartwatches, smart rings, smart glasses). Electronic devices may be equipped with cameras that include image sensors. Electronic devices can acquire image data through shooting operations using the camera.

[0003] When a user holds an electronic device including a camera in their hand and photographs a subject, the field of view (FoV), which is the range of the scene included in the image captured by the camera, may shake. The image quality of the generated image data may degrade due to shaking of the field of view. The camera may include an image stabilizer to compensate for shaking caused by the movement of the electronic device. Image stabilization performed by the image stabilizer may include, for example, optical image stabilization (OIS), electronic image stabilization (EIS), or digital image stabilization (DIS).

[0004] Optical image stabilization may include an action in which, when movement occurs in an electronic device while the camera module is acquiring image data, the image stabilizer moves the camera's optical elements (e.g., lens or image sensor) in a direction that compensates for the movement of the electronic device. Electronic image stabilization or digital image stabilization may include a method of compensating for the movement of the electronic device through electronic methods or digital image processing. For example, electronic image stabilization or digital image stabilization may include an action in which the camera module acquires an image by cropping a portion of the image data that can be acquired through the image sensor, and moves the position of the cropped area in a direction that compensates for the movement of the electronic device.

[0005] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art in relation to the present disclosure.

[0006] In one embodiment, the electronic device may include a display, a first camera, at least one processor, and memory. The display may include a touchscreen. The first camera may include a first image sensor. The first image sensor may include a plurality of sensor pixels arranged along a plurality of pixel lines. The memory may store at least one computer program including computer-executable instructions. The first image sensor may include a microlens array, a color filter array, and a light receiving unit. The microlens array may include a plurality of microlenses. The color filter array may include a plurality of color channels. The light receiving unit may include a plurality of light receiving elements corresponding to at least some of the plurality of sensor pixels. The plurality of light receiving elements may constitute the plurality of sensor pixels. A sensor pixel may refer to a unit in which the image sensor can detect data. A single light receiving element may constitute a single sensor pixel. However, it is not limited thereto. A single sensor pixel may be composed of a plurality of light receiving elements. The present disclosure is described on the premise that one light-receiving element corresponds to one sensor pixel, but is not limited thereto. The plurality of light-receiving elements may be arranged to correspond to one of the plurality of micro-lenses. The instructions may be executed individually or collectively by the at least one processor to control the first image sensor so that the electronic device crops image data configured based on information obtained from first sensor pixels corresponding to a first region of interest (RoI) among the plurality of sensor pixels and outputs it to the at least one processor.The above commands may be executed individually or collectively by the at least one processor to enable the electronic device to receive touch input through the touchscreen. The above commands may be executed individually or collectively by the at least one processor to enable the electronic device to determine a second region of interest in which at least one of the location or area differs from the first region of interest based on the touch input. The above commands may be executed individually or collectively by the at least one processor to enable the electronic device to control the first image sensor to crop image data configured based on information obtained from second sensor pixels corresponding to the second region of interest among the plurality of sensor pixels and output it to the at least one processor.

[0007] In one embodiment, a method of operating an electronic device including a first image sensor may include controlling the first image sensor to crop and output image data configured based on information obtained from an area corresponding to a first region of interest within a light receiving portion of the first image sensor. The method may include receiving a touch input through a touchscreen of the electronic device. The method may include determining a second region of interest in which at least one of the location or area differs from the first region of interest based on the touch input. The method may include controlling the first image sensor to crop and output image data configured based on information obtained from an area corresponding to the second region of interest within the light receiving portion.

[0008] In one embodiment, a computer-readable non-transient recording medium may record a computer program comprising computer-executable instructions. The instructions may cause the first image sensor to perform an operation of cropping and outputting image data configured based on information obtained from an area corresponding to a first region of interest within a light-receiving portion of the first image sensor when the electronic device including a first camera including a first image sensor is executed. The instructions may cause the electronic device to perform an operation of receiving a touch input through a touchscreen of the electronic device when the electronic device is executed. The instructions may cause the electronic device to perform an operation of determining a second region of interest in which at least one of the location or area differs from the first region of interest based on the touch input when the electronic device is executed. The instructions may cause the electronic device to perform an operation of cropping and outputting image data configured based on information obtained from an area corresponding to the second region of interest within a light-receiving portion when the electronic device is executed.

[0009] FIG. 1 is a block diagram of an electronic device in a network environment according to one embodiment of the present disclosure.

[0010] FIG. 2 is a block diagram illustrating a camera module according to one embodiment of the present disclosure.

[0011] FIG. 3 is a block diagram illustrating the configuration of an electronic device according to one embodiment.

[0012] FIG. 4 is a diagram illustrating the configuration of an image sensor according to one embodiment.

[0013] FIG. 5 is a diagram illustrating examples of patterns of a light-receiving element, a microlens, and a color filter array arranged in an image sensor according to one embodiment.

[0014] FIG. 6 is a circuit diagram illustrating an example of a circuit for outputting the values ​​of pixels included in an image sensor according to one embodiment.

[0015] FIG. 7 is a diagram illustrating an example of a color pattern of pixels included in an image obtained through an image sensor according to one embodiment.

[0016] FIG. 8 is a diagram illustrating an example of a color pattern of pixels included in an image obtained through an image sensor according to one embodiment.

[0017] FIG. 9 is a diagram illustrating an example of a color pattern of pixels included in an image obtained through an image sensor according to one embodiment.

[0018] FIG. 10 is a diagram illustrating an example of an image provided by an electronic device according to one embodiment based on a region of interest.

[0019] FIG. 11 is a drawing illustrating an example of a pixel line and a region of interest for an image sensor according to one embodiment.

[0020] FIG. 12 is a flowchart illustrating the process of an electronic device according to one embodiment capturing an image through a camera.

[0021] FIG. 13 is a block diagram illustrating the configuration of an electronic device for capturing images according to one embodiment.

[0022] FIG. 14 is a flowchart illustrating a process for determining whether an electronic device according to one embodiment will change the shooting mode.

[0023] FIG. 15 illustrates the range in which an electronic device according to one embodiment can move a region of interest within an area capable of being captured through light-receiving elements.

[0024] FIG. 16 illustrates an example of a user interface displayed through a touchscreen by an electronic device according to one embodiment.

[0025] FIG. 17 illustrates an example for explaining whether a region of interest to be changed according to one embodiment is within the limits according to the hardware of the image sensor.

[0026] FIG. 18 illustrates an example in which an electronic device according to one embodiment determines a region of interest based on user input including panning input.

[0027] FIG. 19 illustrates an example in which an electronic device according to one embodiment determines a region of interest based on user input including pinch input.

[0028] FIG. 20 is a flowchart illustrating a process in which an electronic device according to one embodiment determines a region of interest according to a shooting mode.

[0029] FIG. 21 is a flowchart illustrating the process of an electronic device according to one embodiment capturing an image in a second shooting mode.

[0030] FIG. 22 illustrates an example in which an electronic device according to one embodiment determines a region of interest based on the location of an object of interest.

[0031] FIG. 23 illustrates an example in which an electronic device according to one embodiment determines a region of interest based on user input including the position of an object of interest and panning input.

[0032] FIG. 24 illustrates an example in which an electronic device according to one embodiment determines a region of interest based on user input including the location of an object of interest and pinch input.

[0033] FIG. 25 is a flowchart illustrating a process in which an electronic device including a first camera and a second camera according to one embodiment displays a preview image.

[0034] FIG. 26 illustrates an example of an area where an image sensor outputs data while an electronic device according to one embodiment is fixed.

[0035] FIG. 27 illustrates an example of an area where an image sensor outputs data when an electronic device according to one embodiment moves to a location where the region of interest is included in the output area of ​​the image sensor.

[0036] FIG. 28 illustrates an example of an area where an image sensor outputs data when an electronic device according to one embodiment moves the region of interest to a location outside the output area of ​​the image sensor.

[0037] FIG. 29 illustrates an example in which an electronic device according to one embodiment acquires image data through a second camera when the region of interest moves to a location outside the field of view of the first camera.

[0038] FIG. 30 is a flowchart illustrating a process in which an electronic device according to one embodiment reads out pixel lines based on the location of an object of interest.

[0039] FIG. 31 illustrates an example in which an electronic device according to one embodiment reads pixel lines based on the location of an object of interest.

[0040] FIG. 32 is a flowchart illustrating a process in which an electronic device according to one embodiment controls a region of interest to acquire image data based on touch input.

[0041] If the user wishes to change the range of the scene captured in the video through the camera, the user may tilt or move the electronic device containing the camera to change the direction in which the camera lens is facing. The electronic device can change the range of the scene included in the video data by utilizing the image stabilizer included in the camera module, but in this case, the performance of the image stabilization function may be sacrificed. When shooting video with a high zoom magnification, the captured image may move significantly even if the direction of the camera lens changes by a small angle. In this case, it may be difficult for the user to adjust the range of the scene to be captured by directly tilting the electronic device.

[0042] The electronic device, the method of operation thereof, and the recording medium having a computer program according to the present disclosure may be for manipulating the shooting range according to the conditions for capturing an image.

[0043] The technical problems to be solved in this document are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure pertains.

[0044] Hereinafter, embodiments are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the contents of this disclosure. However, the disclosed embodiments may be implemented in various different forms and are not limited to the embodiments described herein.

[0045] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0046] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0047] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0048] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0049] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0050] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0051] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0052] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0053] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0054] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0055] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0056] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0057] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0058] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0059] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0060] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0061] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0062] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0063] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0064] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0065] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0066] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In one embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0067] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0068] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0069] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0070] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0071] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0072] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0073] FIG. 2 is a block diagram (200) illustrating a camera module (180) according to various embodiments. Referring to FIG. 2, the camera module (180) may include a lens assembly (210), a flash (220), an image sensor (230), an image stabilizer (240), a memory (250) (e.g., a buffer memory), or an image signal processor (260). The lens assembly (210) may collect light emitted from a subject that is the target of image capture. The lens assembly (210) may include one or more lenses. According to one embodiment, the camera module (180) may include a plurality of lens assemblies (210). In this case, the camera module (180) may form, for example, a dual camera, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies (210) may have the same lens properties (e.g., angle of view, focal length, autofocus, f-number, or optical zoom), or at least one lens assembly may have one or more lens properties different from the lens properties of other lens assemblies. The lens assemblies (210) may include, for example, a wide-angle lens or a telephoto lens.

[0074] A flash (220) may emit light used to enhance light emitted or reflected from a subject. According to one embodiment, the flash (220) may include one or more light-emitting diodes (e.g., RGB (red-green-blue) LED, white LED, infrared LED, or ultraviolet LED), or a xenon lamp. An image sensor (230) may acquire an image corresponding to the subject by converting light emitted or reflected from the subject and transmitted through a lens assembly (210) into an electrical signal. According to one embodiment, the image sensor (230) may include one image sensor selected from image sensors with different properties, such as an RGB sensor, a BW (black and white) sensor, an IR sensor, or a UV sensor, a plurality of image sensors having the same properties, or a plurality of image sensors having different properties. Each image sensor included in the image sensor (230) can be implemented using, for example, a CCD (charged coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor.

[0075] The image stabilizer (240) may move at least one lens or image sensor (230) included in the lens assembly (210) in a specific direction or control the operational characteristics of the image sensor (230) (e.g., adjusting read-out timing, etc.) in response to the movement of the camera module (180) or the electronic device (101) containing it. This allows for compensating for at least some of the negative effects caused by the movement on the image being captured. According to one embodiment, the image stabilizer (240) may detect such movement of the camera module (180) or the electronic device (101) using a gyroscope sensor (not shown) or an accelerometer sensor (not shown) placed inside or outside the camera module (180). According to one embodiment, the image stabilizer (240) may be implemented as, for example, an optical image stabilizer. The memory (250) may temporarily store at least a portion of the image acquired through the image sensor (230) for the next image processing operation. For example, if image acquisition by the shutter is delayed or multiple images are acquired at high speed, the acquired original image (e.g., a Bayer-patterned image or a high-resolution image) is stored in the memory (250), and the corresponding copy image (e.g., a low-resolution image) can be previewed through the display module (160). Subsequently, when a specified condition is satisfied (e.g., user input or system command), at least a portion of the original image stored in the memory (250) may be acquired and processed by, for example, an image signal processor (260). According to one embodiment, the memory (250) may be configured as at least a portion of the memory (130) or as a separate memory that operates independently thereof.

[0076] The image signal processor (260) can perform one or more image processing operations on an image obtained through the image sensor (230) or an image stored in memory (250). The above one or more image processing methods may include, for example, depth map generation, 3D modeling, panorama generation, feature point extraction, image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softing). Additionally or generally, the image signal processor (260) may perform control (e.g., exposure time control, or readout timing control, etc.) over at least one of the components included in the camera module (180) (e.g., image sensor (230)). The image processed by the image signal processor (260) may be stored back in memory (250) for further processing or provided to an external component of the camera module (180) (e.g., memory (130), display module (160), electronic device (102), electronic device (104), or server (108)). According to one embodiment, the image signal processor (260) is at least part of the processor (120). It may be configured as a separate processor that operates independently of the processor (120). If the image signal processor (260) is configured as a separate processor from the processor (120), at least one image processed by the image signal processor (260) may be displayed through the display module (160) as is or after additional image processing by the processor (120).

[0077] According to one embodiment, the electronic device (101) may include a plurality of camera modules (180) each having different attributes or functions. In this case, for example, at least one of the plurality of camera modules (180) may be a wide-angle camera and at least another may be a telephoto camera. Similarly, at least one of the plurality of camera modules (180) may be a front camera and at least another may be a rear camera.

[0078] In the present disclosure, a sensor pixel may refer to a unit in which an image sensor can detect data. A single light-receiving element may constitute a single sensor pixel. However, it is not limited thereto. A single sensor pixel may be composed of a plurality of light-receiving elements. Although the present disclosure is described on the premise that a single light-receiving element corresponds to a single sensor pixel, it is not limited thereto.

[0079] FIG. 3 is a block diagram illustrating the configuration of an electronic device (101) according to one embodiment.

[0080] In one embodiment, the electronic device (101) may include at least one processor (320) (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2), memory (330) (e.g., memory (130) of FIG. 1, memory (250) of FIG. 2), and at least one camera (381, 383) (e.g., camera module (180) of FIG. 1 and 2). The at least one camera (380) may include a first camera (381) including a first image sensor (382). The at least one camera may further include a second camera (383) including a second image sensor (384), but the at least one camera may be composed of a single camera. The at least one camera may include three or more cameras. Memory (330) may store a computer program containing computer-executable instructions that at least one processor executes individually or collectively to cause the electronic device to operate. In the present disclosure, the operation performed by the electronic device (101) may be understood as being performed by at least one processor executing instructions stored in memory to perform operations or by controlling components of the electronic device (101). In one embodiment, at least one processor (320) may include at least one of a central processing unit (CPU), an image signal processor (ISP) (e.g., the image signal processor (260) of FIG. 2), a graphical processing unit (GPU), or a neural processing unit (NPU). For example, at least one processor (320) may include an application processor configured to perform a plurality of functions.At least one processor (320) can acquire image data based on an image frame containing information read from at least one of the first image sensor (383) or the second image sensor (384).

[0081] In one embodiment, at least one processor (320) can control the operation of at least one of the first image sensor (382) or the second image sensor (384). The electronic device (101) can acquire an image frame based on pixel values ​​obtained from pixels placed in a portion of a plurality of sensor pixels included in the first image sensor (382). The portion of the area where the sensor pixels for acquiring the image frame are placed may be referred to as a region of interest (RoI). At least one processor (320) can control the image sensor to read pixel values ​​from a region including the region of interest for acquiring the image frame. For example, at least one processor (320) can transmit a control signal related to the area where the image sensor reads pixel values ​​to the first image sensor (382) based on control communication (e.g., I2C (inter-integrated circuit) communication, I3C (improved inter-integrated circuit) communication). The first image sensor (382) can output pixel values ​​read from sensor pixels placed in an area including a region of interest within a pixel array included in the first image sensor (382). For example, the first image sensor (382) can transmit image data through an interface (e.g., MIPI (mobile industry processor interface)) connected to at least one processor (320). The at least one processor (320) can obtain an enlarged image corresponding to the region of interest based on the pixel values ​​output from the first image sensor (382).

[0082] In one embodiment, the electronic device (101) may further include a display (360) (e.g., the display module (160) of FIG. 1). At least one processor (320) may control the display (360) to display the acquired image. If the display (360) includes a touchscreen, the electronic device (101) may receive touch inputs corresponding to the location of the displayed screen. The at least one camera may further include a second camera (383) including a second image sensor (384). At least one processor (320) may acquire image data through at least one of the first camera (381) or the second camera (383). At least one processor (320) may control the display (360) to display a preview screen based on the image acquired through at least one of the first camera (381) or the second camera (383). In one embodiment, the first camera (381) supports a first field of view (FoV), and the second camera (383) may support a second FoV different from the first FoV. For example, the first camera may include a wide-angle camera, and the second camera may include an ultra-wide-angle camera that supports a FoV including a scene of an area formed at a wider angle than the first camera. However, it is not limited thereto.

[0083] FIG. 4 is a diagram illustrating the configuration of an image sensor according to one embodiment (e.g., the image sensor (230) of FIG. 2, the first image sensor (382) of FIG. 3, and the second image sensor (384)).

[0084] In one embodiment, the image sensor may include a micro lens array (MLA) (411), a color filter array (CFA) (413), a light receiving unit (415), and a processing unit (417).

[0085] In one embodiment, the micro-lens array (411) may be configured such that a beam of light (421) that passes through a lens unit (e.g., lens assembly (210) of FIG. 2) to form an image on an image sensor is collected by a light receiving element of the light receiving unit (415). The beams of light (423) that pass through the micro-lens array (411) may pass through a color filter array (413), at least some wavelengths other than the band corresponding to a specific color may be blocked. The beams of light (425) that pass through the color filter array (413) may be detected by a light receiving element (e.g., a photodiode) of the light receiving unit (415). The light receiving unit (415) may include a light receiving element that generates a charge and converts it into an electrical signal upon receiving light, and a circuit that selectively reads out the charge of the light receiving element. Between the light receiving unit (415) and the operation unit (417), a circuit for digitizing the signal read from the light receiving unit (415) or reducing noise may be arranged.

[0086] In one embodiment, the microlens array (411) may be arranged to correspond to at least one light-receiving element. For example, when viewed from the direction in which the light beam (421) is incident, the area in which a single microlens included in the microlens array (411) is arranged may overlap at least partially with the area in which a plurality of light-receiving elements are arranged. The microlenses included in the microlens array (411) may be arranged on a color filter of a different color channel from an adjacent microlens, but a plurality of microlenses corresponding to a color filter of the same color channel may also be arranged adjacent to each other. The arrangement between the microlens array (411), the color filter array (413), and the light-receiving unit (415) may be configured differently depending on the type of image sensor.

[0087] In one embodiment, the calculation unit (417) can perform a calculation to process electrical data (427) output from the light receiving unit (415). The calculation unit (417) can output the acquired data based on the calculation result. The output of the calculation unit (417) can be the output of an image sensor (e.g., the image sensor (230) of FIG. 2), the first image sensor (382) of FIG. 3, and the second image sensor (384). In one embodiment, the calculation unit (417) can perform an operation to calibrate the data read from the calculation to process the electrical data (427). For example, the operations performed by the operation unit (417) may include at least one of the following: operations to reduce deviations between pixels due to optical asymmetry or the relative position of the sensor, operations to crop some of the read pixel values, operations to reduce noise generated in the analog signal, operations to remove defects, operations to perform remosaic, or operations according to a specific application field (e.g., proximity sensor function, timing control function, HDR (high dynamic range) tone mapping function). The sensor output (429) output from the operation unit (417) may be input to at least one processor (e.g., application processor) through an interface.

[0088] In FIG. 4, the color pattern of the color filter array (413) is illustrated based on a Bayer pattern, but the color pattern of the color filter array (413) according to one embodiment is not limited to that illustrated in FIG. 4. Regions within the color filter array (413) corresponding to a plurality of adjacent microlenses may be configured to include the same color channel. For example, an image sensor including the color filter array (413) may include a structure in which the pattern illustrated in FIG. 5 is repeated.

[0089] FIG. 5 is a drawing illustrating examples of patterns for a light-receiving element (e.g., a light-receiving element included in the light-receiving part (415) of FIG. 4)), a micro-lens (e.g., a micro-lens included in the micro-lens array (411) of FIG. 4) and a color filter array (e.g., a color filter array (413) of FIG. 4) placed in an image sensor (e.g., the image sensor (230) of FIG. 2, the first image sensor (382) of FIG. 3, the second image sensor (384)) according to one embodiment.

[0090] In one embodiment, an image sensor (e.g., image sensor (230) of FIG. 2, first image sensor (382) of FIG. 3, second image sensor (384)) of an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) may be configured to output images of different resolutions depending on the operating mode in which the image sensor operates. The pattern in which the micro-lens, light-receiving element, and color filter array included in the image sensor are arranged may be configured in various ways. For example, the image sensor may include a pattern in which any one of the first sensor pattern (501), the second sensor pattern (502), or the third sensor pattern (503) is repeated. However, the patterns shown in FIG. 5 are merely examples for explaining one embodiment and are not limited thereto. The image sensor may output an image according to the operating mode based on the above pattern. Each light-receiving element may include a device (e.g., a photodiode) capable of detecting light and outputting an electrical signal. At least one light-receiving element included in the image sensor may be placed in an area corresponding to a single microlens. In FIG. 5, the area corresponding to each light-receiving element may be referred to as a sensor pixel.

[0091] According to one embodiment, in the first sensor pattern (501), four light-receiving elements (511, 512, 513, 514) arranged in a 2 x 2 array may be arranged to correspond to a single microlens (520). The microlens (520) within the first sensor pattern (501) may be placed in an area corresponding to a color filter of one color channel (e.g., green channel). Within the first sensor pattern (501), another microlens placed adjacent to the microlens (520) may be placed in an area corresponding to a color filter of a different color channel (e.g., red channel, blue channel) and a color filter of a different color channel.

[0092] According to one embodiment, in the second sensor pattern (502), nine light-receiving elements (551, 552, 553, 554, 555, 556, 557, 558, 559) arranged in a 3 x 3 array may be arranged to correspond to one microlens (560). The microlens (560) within the second sensor pattern (502) may be placed in an area corresponding to a color filter of one color channel (e.g., green channel) of a color filter. Within the second sensor pattern (502), another microlens placed adjacent to the microlens (560) may be placed in an area corresponding to a color filter of a different color channel (e.g., red channel, blue channel) than the color filter of the color channel corresponding to the area where the microlens (560) is placed.

[0093] According to one embodiment, in the third sensor pattern (503), four microlenses (591, 592, 593, 594) arranged in a 2 x 2 array may be placed in an area corresponding to a color filter of the same color channel (e.g., green channel). The light receiving elements may include four light receiving elements arranged in a 2 x 2 array and placed to correspond to one microlens. For example, in the third sensor pattern (503), four light receiving elements (571, 572, 573, 574) may be placed to correspond to the microlens (591). Four light receiving elements (575, 576, 577, 578) may be placed to correspond to the microlens (592). Four light receiving elements (579, 580, 581, 582) may be placed to correspond to the microlens (593). Four light-receiving elements (583, 584, 585, 586) can be arranged to correspond to the micro lens (594).

[0094] In one embodiment, the pattern of the image sensor is not limited to the patterns shown in FIG. 5. For example, the pattern of the image sensor may be replaced with other patterns that can operate by changing the resolution.

[0095] FIG. 6 is a circuit diagram illustrating an example of a circuit for outputting the values ​​of pixels included in an image sensor (e.g., the image sensor (230) of FIG. 2, the first image sensor (382) of FIG. 3, and the second image sensor (384)) according to one embodiment.

[0096] In one embodiment, the pixel circuit constituting the sensor pixels included in the image sensor may include a plurality of at least one photodiode (611, 612, 613, 614, 615, 616, 617, 618). For example, four photodiodes (611, 612, 613, 614) having a 2 x 2 array arranged to correspond to one microlens may correspond to light-receiving elements (511, 512, 513, 514) included in the first pattern (501) of FIG. 5. For example, four photodiodes (611, 612, 613, 614) having a 2 x 2 array arranged to correspond to one microlens may correspond to light-receiving elements (571, 572, 573, 574) included in the third pattern (503) of FIG. 5. For example, four photodiodes (615, 616, 617, 618) having a 2 x 2 array arranged to correspond to a single microlens may correspond to light-receiving elements (575, 576, 577, 578) included in the third pattern (503) of FIG. 5. However, FIG. 6 is merely an example of a pixel circuit for explaining one embodiment, and the configuration of the pixel circuit is not limited to that shown in FIG. 6. In one embodiment, five or more photodiodes may be arranged to correspond to a single microlens. For example, nine photodiodes having a 3 x 3 array may be arranged to correspond to a single microlens.

[0097] Referring to FIG. 6, four photodiodes (e.g., first photodiode (611), second photodiode (612), third photodiode (613) and fourth photodiode (614)) arranged to correspond to one microlens and four photodiodes (e.g., fifth photodiode (615), sixth photodiode (616), seventh photodiode (617) and eighth photodiode (618)) arranged to correspond to another microlens may be connected to a floating diffusion node (630). Switches (621, 622, 623, 624, 625, 626, 627, 628) may be connected between each of the photodiodes (611, 612, 613, 614, 615, 616, 617, 618) and the floating node (630). For example, the switches (621, 622, 623, 624, 625, 626, 627, 628) may include transistors (e.g., transfer gates).

[0098] According to one embodiment, charge can be accumulated in the photodiodes (611, 612, 613, 614, 615, 616, 617, 618) during the exposure time. While charge is being accumulated, the switch can be kept off so that the photodiode and the floating node (630) are not connected. When the switch is turned on, the photodiode is connected to the floating node (630), and the accumulated charge can move to the floating node (630). For example, when the first switch (621) is turned on, the charge accumulated in the first photodiode (611) can move to the floating node (630). The charge stored in the floating node (630) can be read out through the source follower (SF) (650) and output as an electrical signal. An image sensor (e.g., image sensor (230) of FIG. 2, first image sensor (382), second image sensor (384) of FIG. 3)) can acquire analog data corresponding to the charge transferred to the floating node (630). For example, the analog data may include information about the amount of charge accumulated in at least one photodiode during the exposure time.

[0099] In one embodiment, the line selector (SEL) (660) may be controlled to an on or off state to output analog data for the selected line. The line selector (660) may be controlled to an on state to read out the value of a sensor pixel connected to a floating node (630) according to the read order.

[0100] In one embodiment, the image sensor can acquire analog data corresponding to the pixel values ​​of raw image data. For example, the image sensor can acquire analog data corresponding to light intensity data acquired through at least one of the photodiodes (611, 612, 613, 614, 615, 616, 617, 618) by controlling switches (621, 622, 623, 624, 625, 626, 627, 628). When the image sensor turns on the first switch (621), the image sensor can acquire analog data based on the light intensity data acquired through the first photodiode (611). The image sensor can acquire pixel values ​​by converting the acquired analog data into digital through an analog-digital converter (ADC).

[0101] In one embodiment, after acquiring analog data, the image sensor may perform a reset operation to remove the charge accumulated in the floating node (630) by turning on a reset switch (e.g., a reset transistor (RST) (670). The image sensor may read out the output of each of the photodiodes (611, 612, 613, 614, 615, 616, 617, 618) as a single pixel value by performing the reset operation with a time delay while sequentially turning on the switches (621, 622, 623, 624, 625, 626, 627, 628) one by one. After performing the reset operation by turning on the reset transistor (670), the image sensor may turn on the first switch (621) to transfer the charge accumulated by the first photodiode (611) to the floating node (630). The first photo When the charge output by the diode (611) is accumulated in the floating node (630) and the line selector (660) is turned on, the image sensor can read out a voltage value corresponding to the charge accumulated by the first photodiode (611) through the source follower (650). The image sensor can perform a reset operation to remove the charge accumulated in the floating node (630) by turning on the reset transistor (670). Afterward, the image sensor can turn on the second switch (622) to transfer the charge accumulated by the second photodiode (612) to the floating node (630). When the charge output by the second photodiode (612) is accumulated in the floating node (630) and the line selector (660) is turned on, the image sensor can read out a voltage value corresponding to the charge accumulated by the second photodiode (612) through the source follower (650). Afterwards, the image sensor can perform a reset operation to remove the charge accumulated in the floating node by turning on the reset transistor (670).Afterward, the image sensor can turn on the third switch (623) to transfer the charge accumulated by the third photodiode (613) to the floating node (630). When the charge output by the third photodiode (613) is accumulated in the floating node (630) and the line selector (660) is turned on, the image sensor can read out a voltage value corresponding to the charge accumulated by the third photodiode (613) through the source follower (650). Afterward, the image sensor can turn on the reset transistor (670) to perform a reset operation to remove the charge accumulated in the floating node (630). Afterward, the image sensor can turn on the fourth switch (624) to transfer the charge accumulated by the fourth photodiode (614) to the floating node (630). When the charge output by the fourth photodiode (614) is accumulated in the floating node (630) and the line selector (660) is turned on, the image sensor can read out a voltage value corresponding to the charge accumulated by the fourth photodiode (614) through the source follower (650). Afterwards, the image sensor can turn on the reset transistor (670) to perform a reset operation to remove the charge accumulated in the floating node (630). Afterwards, the image sensor can turn on the fifth switch (625) to transfer the charge accumulated by the fifth photodiode (615) to the floating node (630). When the charge output by the fifth photodiode (615) is accumulated in the floating node (630) and the line selector (660) is turned on, the image sensor can read out a voltage value corresponding to the charge accumulated by the fifth photodiode (615) through the source follower (650). Afterwards, the image sensor can turn on the reset transistor (670) to perform a reset operation to remove the charge accumulated in the floating node (630). Afterwards, the image sensor can turn on the sixth switch (626) to transfer the charge accumulated by the sixth photodiode (616) to the floating node (630).When the charge output by the sixth photodiode (616) is accumulated in the floating node (630) and the line selector (660) is turned on, the image sensor can read out a voltage value corresponding to the charge accumulated by the sixth photodiode (616) through the source follower (650). Afterwards, the image sensor can turn on the reset transistor (670) to perform a reset operation to remove the charge accumulated in the floating node (630). Afterwards, the image sensor can turn on the seventh switch (627) to transfer the charge accumulated by the seventh photodiode (617) to the floating node (630). When the charge output by the seventh photodiode (617) is accumulated in the floating node (630) and the line selector (660) is turned on, the image sensor can read out a voltage value corresponding to the charge accumulated by the seventh photodiode (617) through the source follower (650). Afterwards, the image sensor can turn on the reset transistor (670) to perform a reset operation to remove the charge accumulated in the floating node (630). Afterwards, the image sensor can turn on the eighth switch (628) to transfer the charge accumulated by the eighth photodiode (618) to the floating node (630). When the charge output by the eighth photodiode (618) is accumulated in the floating node (630) and the line selector (660) is turned on, the image sensor can read out a voltage value corresponding to the charge accumulated by the eighth photodiode (618) through the source follower (650). Afterwards, the image sensor can turn on the reset transistor (670) to perform a reset operation to remove the charge accumulated in the floating node (630). For example, when operating based on a high-pixel mode that generates a high-pixel image, the image sensor can read one pixel value from each of the photodiodes (611, 612, 613, 614, 615, 616, 617, 618).

[0102] In one embodiment, when the switches (621, 622, 623, 624, 625, 626, 627, 628) are turned on together, the charge accumulated in the photodiodes (611, 612, 613, 614, 615, 616, 617, 618) can be moved to the floating node (630). In this case, by analogously summing the light amounts corresponding to the eight photodiodes (611, 612, 613, 614, 615, 616, 617, 618), the image sensor can obtain light amount data corresponding to two microlenses (e.g., the first microlens (591) and the second microlens (592) of FIG. 5). The image sensor may acquire light quantity data corresponding to similarly adjacent microlenses (e.g., the third microlens (593) and the fourth microlens (594) of FIG. 5) and then sum the light quantity data corresponding to two microlenses (e.g., the first microlens (591) and the second microlens (592) of FIG. 5) to produce a value corresponding to one color channel as a single pixel value. For example, when operating based on a low-pixel mode that generates an image of lower pixels than a high-pixel mode, the image sensor may produce a value by summing the charges accumulated in the photodiodes corresponding to four microlenses (e.g., the first microlens (591), the second microlens (592), the third microlens (593), and the fourth microlens (594) of FIG. 5) as a single pixel value. Here, the pixel values ​​produced may have a color order according to a Bayer pattern.

[0103] In one embodiment, when the second switch (622), the fourth switch (624), the sixth switch (626) and the eighth switch (628) are off and the first switch (621), the third switch (623), the fifth switch (625) and the seventh switch (627) are turned on, the image sensor can obtain pixel values ​​for pixels placed on the first side among the pixels corresponding to the microlens (e.g., the first photodiode (611), the third photodiode (613), the fifth photodiode (615), and the seventh photodiode (617) of FIG. 6) (hereinafter referred to as 'left pixels'). When the first switch (621), the third switch (623), the fifth switch (625), and the seventh switch (627) are off, and the second switch (622), the fourth switch (624), the sixth switch (626), and the eighth switch (628) are turned on, the image sensor can acquire pixel values ​​for pixels located on the second side, which is in a direction different from the first side among the pixels corresponding to the micro-lens (e.g., the second photodiode (612), the fourth photodiode (614), the sixth photodiode (616), and the eighth photodiode (618) of FIG. 6) (hereinafter referred to as 'right pixels'). An electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) can acquire phase difference information based on the correlation between the left pixel value and the right pixel value. However, this is for the purpose of explaining an example, and the method of acquiring the left pixel value and the right pixel value is not limited thereto. For example, an electronic device according to one embodiment may output luminance values ​​corresponding to the first photodiode (611) and the third photodiode (613) and luminance values ​​corresponding to the fifth photodiode (615) and the seventh photodiode (617) as individual left pixel values.

[0104] In one embodiment, when the color pattern of the output image frame is different from the Bayer pattern, the electronic device can perform a remosaic operation that converts the output image data into a Bayer pattern that can be processed by an image signal processor.

[0105] The pixel circuit illustrated in FIG. 6 is an example illustrating one embodiment, and the pixel circuit included in the image sensor according to one embodiment may include a circuit different from the circuit illustrated in FIG. 6.

[0106] FIG. 7 is a diagram illustrating an example of a color pattern of pixels included in an image obtained through an image sensor according to one embodiment (e.g., the image sensor (230) of FIG. 2, the first image sensor (382) of FIG. 3, and the second image sensor (384)).

[0107] In one embodiment, an image sensor of an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) may output an image having a first raw image pattern (710) corresponding to a Bayer pattern by performing a binning operation that merges the outputs of sensor pixels in a 2 x 2 array (e.g., light receiving elements (511, 512, 513, 514) of FIG. 5) containing information for the same color channel within a first sensor pattern (501). In this case, the image having the first raw image pattern (710) may have a resolution of 1 / 4 relative to the number of sensor pixels of the image sensor. For example, from light receiving elements in an 8 x 8 array included in the first sensor pattern (501) shown in FIG. 5, an image having a first raw image pattern (710) having a resolution of 4 x 4 may be output.

[0108] In one embodiment, an image sensor of an electronic device may acquire an image having a second raw image pattern (720) in which each sensor pixel corresponds to the pixels of the image. The electronic device may acquire an image having a Bayer pattern (725) by performing a remosaic operation that converts the color order on the image having the second raw image pattern (720). In the present disclosure, the remosaic operation may be performed on an image acquired from the image sensor by at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3), but it may be understood that the image sensor may output the result of performing the remosaic operation. An image acquired based on the second raw image pattern (720) (e.g., an image having a Bayer pattern (725)) may have a resolution four times higher than an image acquired based on the first raw image pattern (710).

[0109] FIG. 8 is a diagram illustrating an example of a color pattern of pixels included in an image obtained through an image sensor according to one embodiment (e.g., the image sensor (230) of FIG. 2, the first image sensor (382) of FIG. 3, and the second image sensor (384)).

[0110] In one embodiment, an image sensor of an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) may output an image having a first raw image pattern (810) corresponding to a Bayer pattern by performing a binning operation that merges the outputs of sensor pixels in a 3 x 3 array containing information for the same color channel within a second sensor pattern (502) (e.g., light receiving elements (551, 552, 553, 554, 555, 556, 557, 558, 559) of FIG. 5). In this case, the image having the first raw image pattern (810) may have a resolution of 1 / 9 relative to the number of sensor pixels of the image sensor. For example, from the light receiving elements of a 6 x 6 array included in the second sensor pattern (502) shown in FIG. 5, an image having a first raw image pattern (810) having a resolution of 2 x 2 can be output.

[0111] In one embodiment, an image sensor of an electronic device may acquire an image having a second raw image pattern (820). The electronic device may acquire an image having a Bayer pattern (825) by performing a remosaic operation on the image having the second raw image pattern (820). The image acquired based on the second raw image pattern (820) (e.g., an image having a Bayer pattern (825)) may have a resolution nine times higher than that of the image acquired based on the first raw image pattern (810).

[0112] FIG. 9 is a drawing illustrating an example of a color pattern of pixels included in an image obtained through an image sensor according to one embodiment (e.g., the image sensor (230) of FIG. 2, the first image sensor (382) of FIG. 3, and the second image sensor (384)).

[0113] In one embodiment, an image sensor of an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) may output an image having a first raw image pattern (910) corresponding to a Bayer pattern by performing a binning operation that merges sensor pixels in a 4 x 4 array containing information for the same color channel within a third sensor pattern (503). The image having the first raw image pattern (910) may have a resolution of 1 / 16 relative to the number of sensor pixels of the image sensor.

[0114] In one embodiment, the electronic device may obtain an image having a second raw image pattern (920) by performing a binning operation that merges the outputs of sensor pixels in a 2 x 2 array. The electronic device may obtain an image having a Bayer pattern (925) by performing a remosaic operation on the image having the second raw image pattern (920). The image obtained based on the second raw image pattern (920) (e.g., an image having a Bayer pattern (925)) may have a resolution four times higher than the image obtained based on the first raw image pattern (910).

[0115] In one embodiment, the electronic device may acquire an image having a third raw image pattern (930) in which each sensor pixel corresponds to the pixels of the image. The electronic device may acquire an image having a Bayer pattern (935) by performing a remosaic operation on the image having the third raw image pattern (930). The image acquired based on the third raw image pattern (930) (e.g., an image having the Bayer pattern (935)) may have a resolution 16 times higher than the image acquired based on the first raw image pattern (910).

[0116] FIG. 10 is a drawing illustrating an example of an image provided by an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) based on a region of interest.

[0117] In one embodiment, the electronic device may determine a first region of interest (1010) within an area (1000) where an image can be acquired through an image sensor (e.g., image sensor (230) of FIG. 2, first image sensor (382) of FIG. 3, second image sensor (384)). For example, the electronic device may determine the region of interest (1010) based on at least one of a zoom magnification set for image capture or the position of a subject (1015). For example, the electronic device may determine the first region of interest (1010) based on the position of the subject (1015) based on a touch input selecting the subject (1015).

[0118] In one embodiment, the electronic device may acquire an image (1030) corresponding to a first region of interest (1010). For example, the electronic device (101) may control the image sensor (e.g., image sensor (230) of FIG. 2, first image sensor (382) of FIG. 3, second image sensor (384)) to crop and output pixel values ​​corresponding to the first region of interest (1010) among pixel values ​​acquired from at least some of the plurality of pixel lines included in the image sensor (e.g., image sensor (230) of FIG. 2, first image sensor (382) of FIG. 3, second image sensor (384)). The electronic device may acquire an image (1030) by scaling up the cropped image data. The electronic device (101) may display the image (1030) through a display as a preview image. The electronic device (101) can store image data (e.g., at least one of a still image file or a video file) including an image (1030) in a memory (e.g., memory (130) of FIG. 1, memory (250) of FIG. 2, memory (330) of FIG. 3). The electronic device (101) can reduce the degree of image quality degradation caused by digital zoom operation by acquiring the image (1030) through an operation mode of an image sensor capable of acquiring high-resolution image data (e.g., a mode for acquiring an image having the Bayer pattern (725) of FIG. 7, the Bayer pattern (825) of FIG. 8, the Bayer pattern (925) of FIG. 9, or the Bayer pattern (935) of FIG. 9 through a remosaic operation).

[0119] In one embodiment, the electronic device (101) can track the location of an object (1015). The electronic device (101) can identify an object determined to be identical to the object (1015) within a subsequent image taken after the image (1000). For example, if the location of the object (1015) moves (1025) within the subsequent image, the electronic device (101) can determine the area corresponding to the moved location as a second region of interest (1020). By acquiring an image based on the second region of interest (1020), the electronic device (101) can ensure that the object (1015) does not disappear from the image even if the object (1015) moves within the camera's field of view.

[0120] FIG. 11 is a drawing illustrating examples of pixel lines and regions of interest for an image sensor according to one embodiment (e.g., the image sensor (230) of FIG. 2, the first image sensor (382) of FIG. 3, and the second image sensor (384)). FIG. 11 is described with the configuration of the first image sensor (382), but the second image sensor (384) may also include the same configuration.

[0121] In one embodiment, the first image sensor (382) may include a plurality of sensor pixels (PX). The sensor pixels (PX) may be arranged in a first axis direction and a second axis direction. Each of the sensor pixels may include at least one light-receiving element (e.g., a photodiode). For example, the first axis direction may mean the +y-axis direction. For example, the second axis direction may mean the +x-axis direction perpendicular to the first axis direction. In the present disclosure, the resolution of the image sensor may correspond to the number of sensor pixels included in the image sensor. For example, if a number of sensor pixels (8 in the case of FIG. 11) are arranged in the x-axis direction and b number of sensor pixels (8 in the case of FIG. 11) are arranged in the y-axis direction, the resolution of the image sensor may be referred to as axb.

[0122] An image sensor according to one embodiment may include a plurality of pixel lines (1111, 1112, 1113, 1114, 1115, 1116, 1117, 1118). In the present disclosure, a pixel line may refer to a group of pixels arranged in one direction. Referring to FIG. 11, a pixel line may refer to pixels arranged in a second axis direction. An electronic device may read pixel values ​​for at least some of the sensor pixels included in the pixel line in parallel. In the present disclosure, "reading a pixel line" may mean reading information about pixel values ​​(or the pixel values ​​themselves) from the sensor pixels included in the pixel line. In the present disclosure, data obtained by reading pixel lines may be referred to as image data included in an image frame. In the present disclosure, "reading an image frame" may mean including an operation of reading pixel lines to obtain image data.

[0123] In one embodiment, the image sensor can read pixel lines, at least a portion of which is included in the region of interest (1120). In the present disclosure, "at least one pixel line associated with the region of interest" may mean at least one pixel line in which at least a portion of the pixels included in the pixel line are located within the region of interest. The region of interest (1120) may include at least a portion of the area in which sensor pixels (PX) are arranged within the image sensor. For example, referring to FIG. 10, the region of interest (1120) may include an area in which sensor pixels are arranged for capturing an image corresponding to the region of interest (1010) of FIG. 10. Referring to FIG. 11, when the region of interest (1120) determined for acquiring an image frame is determined, the image sensor can read pixel lines (1114, 1115, 1116). For example, when the reading direction is forward, the image sensor may read out pixel line (1114) and then sequentially read out pixel line (1115) and pixel line (1116) placed in the -y-axis direction. For example, when the reading direction is reverse, the image sensor may read out pixel line (1116) and then sequentially read out pixel line (1115) and pixel line (1114) placed in the +y-axis direction. However, it is not limited thereto. For example, the image sensor may read out additional pixel lines (e.g., pixel lines (1112, 1113, 1117, 1118)) in addition to pixel lines (1114, 1115, 1116) containing pixels corresponding to the region of interest (1120).

[0124] FIG. 12 is a flowchart (1200) illustrating a process in which an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) according to one embodiment captures an image through a camera (e.g., the camera module (180) of FIG. 1 and 2, the first camera (381) of FIG. 3, the second camera (383)).

[0125] In one embodiment, the operations illustrated in the flowchart (1200) may be performed while the electronic device captures an image. For example, the electronic device may perform the operations illustrated in the flowchart (1200) while displaying a live preview image or capturing a video based on an image stream output through at least one camera.

[0126] According to one embodiment, in operation 1210, the electronic device may acquire motion information regarding the movement of the electronic device. For example, the electronic device may acquire motion information based on an electrical signal or data value corresponding to a state detected through a motion sensor included in the electronic device (e.g., sensor module (176) of FIG. 1). For example, the electronic device may acquire motion information based on a motion vector containing information regarding the movement of pixels from an image captured through a camera. However, the method of acquiring motion information is not limited to the examples described above.

[0127] According to one embodiment, in operation 1220, the electronic device may acquire image data based on motion information in a first shooting mode. In the present disclosure, "shooting mode" may mean a method by which the electronic device determines a region of interest (RoI) for capturing an image. For example, in the first shooting mode, the electronic device may acquire image data based on a region of interest at a location that can offset the movement of the electronic device to implement digital image stabilization. In the first shooting mode, the electronic device may not consider user input when determining the region of interest.

[0128] According to one embodiment, in operation 1230, the electronic device can determine whether the video recording has ended. For example, the electronic device can determine whether the camera application running to record the video has ended. If the video recording has ended, the electronic device can terminate the process illustrated in the flowchart (1200).

[0129] According to one embodiment, if the shooting of the image has not ended, in operation 1240, the electronic device may determine whether at least one shooting mode change condition regarding whether to change the shooting mode from the first shooting mode is satisfied. For example, the electronic device may determine whether the shooting mode change condition is satisfied based on at least one of whether the location of the region of interest is located within the range that can be shot through the camera, whether the range in which the region of interest can be moved is greater than or equal to a threshold, or whether user input to change the shooting mode is received. If the shooting mode change condition is not satisfied, the electronic device may perform operations 1210 to 1220 to acquire image data based on the first shooting mode. The electronic device may display a preview image through a display or store the image in memory based on the image in which the scene corresponding to the region of interest is captured.

[0130] According to one embodiment, when a shooting mode change condition is satisfied, the electronic device may receive a touch input detected through a touchscreen in operation 1250. The touch input may include an input for controlling a region of interest. For example, the touch input may include at least one of a panning input or a pinch in / out input. A panning input may include a touch input in which one touch position moves from one position to another. A pinch in input may include touch inputs in which two or more touch inputs move in a direction toward each other. A pinch out input may include inputs in which two or more touch inputs move in a direction toward each other. The touch input may be replaced by other types of user input (e.g., physical button input, voice input).

[0131] In operation 1260, the electronic device may acquire image data based on the touch input received in the second shooting mode in response to the detection of a touch input in operation 1250. For example, the electronic device may determine at least one of the location or area of ​​a region of interest within an area that can be captured through a camera based on the touch input. The electronic device may display a preview image through a display or store the image in memory based on the captured image of a scene corresponding to the determined region of interest. In the second shooting mode, the electronic device may limit or disable the image stabilization function, but is not limited thereto. In the second shooting mode, if movement of the electronic device is detected, the electronic device may determine the region of interest by considering the touch input and movement information to stabilize the image being captured. In the second shooting mode, if the captured image (e.g., preview image) contains an object of interest being tracked, the electronic device may determine the region of interest based on the location of the object of interest and the touch input.

[0132] In one embodiment, an image sensor included in at least one camera (e.g., image sensor (230) of FIG. 2, first image sensor (382), second image sensor (384) of FIG. 3) may support a plurality of operating modes for outputting raw image data of different resolutions. For example, the image sensor may output raw image data including an image pattern according to the operating mode among the image patterns shown in FIG. 7 to 9 (first raw image pattern (710), second raw image pattern (720) of FIG. 7, first raw image pattern (810), second raw image pattern (820) of FIG. 8, first raw image pattern (910), second raw image pattern (920), third raw image pattern (930) of FIG. 9). While the electronic device is operating in a second shooting mode, the image sensor may output image data including an image pattern for providing high resolution (e.g., the second raw image pattern (720) of FIG. 7, the second raw image pattern (820) of FIG. 8, the second raw image pattern (920) of FIG. 9, and the third raw image pattern (930)) to provide an image corresponding to a high zoom magnification.

[0133] According to one embodiment, in operation 1270, the electronic device can determine whether the video recording has ended. For example, the electronic device can determine whether the camera application running to record the video has ended. If the video recording has ended, the electronic device can terminate the process illustrated in the flowchart (1200).

[0134] According to one embodiment, if the shooting of an image has not ended, in operation 1280, the electronic device may determine whether a shooting mode change condition regarding whether to change the shooting mode from the second shooting mode is satisfied. For example, if the range in which the area of ​​interest can be moved as the zoom magnification decreases is below a threshold, the electronic device may determine that a shooting mode change condition for changing the shooting mode from the second shooting mode is satisfied. For example, if a specified user input (e.g., a double tap input in which a touch input is entered twice at the same or similar location within a specified time interval) is entered, the electronic device may determine that a shooting mode change condition for changing the shooting mode from the second shooting mode is satisfied. If it is determined in operation 1280 that the shooting mode change condition is satisfied, the electronic device may perform operation 1210 to acquire image data based on the first shooting mode. If it is determined in operation 1280 that the shooting mode change condition is not satisfied, the electronic device may perform operation 1250 to acquire image data based on the second shooting mode.

[0135] FIG. 13 is a block diagram illustrating the configuration of an electronic device for capturing images according to one embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3).

[0136] In one embodiment, the electronic device may include at least one camera (380) (e.g., camera module (180) of FIG. 1 and 2, first camera (381) and second camera (383) of FIG. 3), a display (360) (e.g., display module (160) of FIG. 1), a composition controller (1310), a mode controller (1320), and a motion sensor (1376) (e.g., sensor module (176) of FIG. 1). At least one camera (380) may include a lens assembly (210) and an image sensor (230). The display (360) may include a touchscreen (1360). The composition controller (1310) and the mode controller (1320) may be implemented using at least one processor (e.g., processor (120) of FIG. 1, processor (320) of FIG. 3) and a memory containing a computer program (e.g., memory (130) of FIG. 1, memory (250) of FIG. 2, memory (330) of FIG. 3). The lens assembly (210) may include a lens and an actuator for controlling the position of the lens.

[0137] In one embodiment, the mode controller (1320) can obtain information about the range in which the region of interest can be moved from the image sensor (230). The mode controller (1320) can obtain information about the region of interest determined by the region of interest determining unit. The mode controller (1320) can determine whether the conditions for changing to a second shooting mode are satisfied based on the information about the range in which the region of interest can be moved and the information about the region of interest. The touchscreen (1360) can receive user input for changing the shooting mode. The mode controller (1320) can determine whether user input is detected through the touchscreen. The mode controller (1320) can identify whether there is an object of interest to be tracked within the image (e.g., preview image) obtained through the image sensor (230). The mode controller can obtain information about the movement of the electronic device from the motion sensor (1376). The mode controller (1320) can determine the shooting mode based on at least one of whether the condition for changing to the second shooting mode is satisfied, whether user input is detected, whether an object of interest is identified, or information about movement. The mode controller (1320) can transmit information about the determined shooting mode to the area of ​​interest determination unit (1313) of the composition controller (1310).

[0138] In one embodiment, the composition controller (1310) can control a configuration (e.g., location or size of a region of interest) for capturing an image based on touch input received through the touchscreen (1360) in a second shooting mode. The composition controller (1310) may include a region of interest implementation unit (1311) and a region of interest determination unit (1313). The region of interest determination unit (1313) can determine a region of interest based on information about the shooting mode received from the mode controller (1320). For example, if the information about the shooting mode indicates a first shooting mode, the region of interest determination unit (1313) can determine a region of interest based on motion information obtained from the motion sensor (1376). For example, if the information about the shooting mode indicates a second shooting mode, the region of interest determination unit (1313) can determine a region of interest based on user input received through the touchscreen (1360). The region of interest implementation unit (1311) can receive information about the region of interest from the region of interest determination unit (1313). For example, the region of interest implementation unit (1311) can receive address information indicating the region of interest from the region of interest determination unit (1313). If it is impossible to obtain an image corresponding to the region of interest determined by the region of interest determination unit (1313), the region of interest implementation unit (1311) can transmit information notifying the region of interest determination unit (1313) that it is impossible to implement an image corresponding to the region of interest. The region of interest implementation unit (1311) can perform an operation to implement an image corresponding to the region of interest determined by the region of interest determination unit (1313). The region of interest implementation unit (1311) can determine the position of the lens based on information obtained from the Hall sensor included in the lens assembly (210).Based on information about the region of interest and the position of the lens, the region of interest implementation unit (1311) can control the image sensor (230) so that the image sensor outputs raw image data including pixel values ​​included in the region of interest. The region of interest implementation unit (1311) can obtain image data corresponding to the region of interest from the raw image data obtained from the image sensor (230). For example, the region of interest implementation unit (1311) can obtain image data by cropping an area corresponding to the region of interest from the raw image data. The region of interest implementation unit (1311) can transmit the obtained image data to at least one of the touchscreen (1360) or the mode controller (1320). The touchscreen (1360) can display a screen including a preview image based on the image data.

[0139] FIG. 14 is a flowchart illustrating a process (1240) for determining whether an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) according to one embodiment will change the shooting mode.

[0140] According to one embodiment, in operation 1410, the electronic device may receive user input related to the zoom magnification. For example, the electronic device may receive touch input to select the zoom magnification through a touchscreen. The electronic device may determine the zoom magnification based on the user input.

[0141] According to one embodiment, in operation 1420, the electronic device may determine whether the zoom magnification determined in operation 1410 is greater than or equal to a first threshold. As the zoom magnification increases, the electronic device may acquire an image based on pixel values ​​acquired within a narrower region of interest. Thus, as the zoom magnification increases, the range of movement of the region of interest may increase. The first threshold may correspond to a zoom magnification such that the ratio of the range of movement of the region of interest to the size of the region of interest is greater than or equal to a specific value. If the determined zoom magnification is less than the first threshold, the electronic device may perform operation 1210 of FIG. 12 to acquire image data based on a first shooting mode.

[0142] According to one embodiment, when the determined zoom magnification is greater than or equal to a first threshold, in operation 1430, the electronic device may display a user interface for selecting a second shooting mode through a touchscreen (e.g., the display module (160) of FIG. 1, the display (360) of FIG. 3, the touchscreen (1360) of FIG. 13). In operation 1440, the electronic device may determine whether user input for selecting the second shooting mode is received through the user interface. If user input is not received, the electronic device may perform operation 1210 of FIG. 12 to acquire image data based on the first shooting mode. Based on the receipt of user input, the electronic device may perform operation 1250 of FIG. 12 to acquire image data based on the second shooting mode.

[0143] In one embodiment, at least one of operation 1430 or operation 1440 may be omitted. For example, based on the fact that the zoom magnification in operation 1420 is greater than or equal to a first threshold, the electronic device may perform operation 1250 of FIG. 12 without displaying a separate user interface or user input. For example, if the zoom magnification in operation 1420 is greater than or equal to a first threshold, the electronic device may perform operation 1250 of FIG. 12 based on the detection of a gesture specified in operation 1440 (e.g., touch input gesture, motion gesture). For example, if the zoom magnification in operation 1420 is greater than or equal to a first threshold, the electronic device may display a user interface related to a second shooting mode and perform operation 1250 of FIG. 12 without separate user input.

[0144] FIG. 15 illustrates a range in which an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) can move a region of interest (1510) within a region (1500) that can be captured through light receiving elements.

[0145] In one embodiment, the electronic device may determine that the mode change condition for entering a second shooting mode is satisfied when the range in which the region of interest (1510) can be moved within the shooting area (1500) is sufficiently wide. The electronic device may determine that the mode change condition is satisfied when the ratio of the range in which the region of interest can be moved in the horizontal or vertical direction to the size of the region of interest in the horizontal or vertical direction is greater than or equal to a specific value. For example, the electronic device may determine that the mode change condition is satisfied when the following mathematical formula 1 is satisfied.

[0146]

[0147] In the above mathematical formula 1, th1 may be a value determined to determine whether the mode change condition is satisfied. Alternatively, for example, the electronic device may determine that the mode change condition is satisfied when the following mathematical formula 2 is satisfied.

[0148]

[0149] In the above mathematical formula 2, th2 may be a value determined to determine whether the mode change condition is satisfied.

[0150] FIG. 16 illustrates an example of a user interface displayed by an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) according to one embodiment through a touchscreen (e.g., the display module (160) of FIG. 1, the display (360) of FIG. 3, the touchscreen (1360) of FIG. 13).

[0151] In one embodiment, the electronic device may control a touchscreen of the electronic device (e.g., the display module (160) of FIG. 1, the display (360) of FIG. 3, the touchscreen (1360) of FIG. 13) to display a screen (1610) including a preview image acquired through a camera. The electronic device may receive a touch input (1615) corresponding to a specified gesture on the screen (1610) displayed on the touchscreen. For example, the touch input (1615) may include a long press input in which the touch input is maintained for a specified time or longer at a stationary position within the preview image. However, the type of touch input (1615) is not limited thereto. For example, the touch input (1615) may include a double tap input in which the touch input is repeated multiple times within a specified time. The electronic device may switch the shooting mode to a second shooting mode in response to the touch input (1615).

[0152] In one embodiment, the electronic device may display a screen (1620) including a user interface object (1625) (e.g., an icon, a button) for switching shooting modes along with a preview image acquired through a camera. The electronic device may switch the shooting mode to a second shooting mode in response to a user input selecting the user interface object (1625).

[0153] FIG. 17 illustrates an example for explaining whether a region of interest to be changed according to one embodiment is within the limits according to the hardware of an image sensor (e.g., image sensor (230) of FIG. 2, first image sensor (382) of FIG. 3, second image sensor (384) of FIG. 13, image sensor (230)).

[0154] In one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) may determine a region of interest (1710) within a hardware limit range (1700) that can be captured through an image sensor. The electronic device may acquire an image frame containing image data based on pixel values ​​within the determined region of interest (1710). The electronic device may determine a region of interest for acquiring the next image frame. The hardware limit range (1700) may refer to a range of scenes that can be captured using components of the electronic device (e.g., an image sensor, an optical component, a processor).

[0155] In one embodiment, when a first region of interest (1720) within the hardware limit range (1700) of the image sensor is determined as the region of interest for acquiring the next image frame, the electronic device may acquire the image frame based on information acquired from sensor pixels corresponding to the first region of interest (1720). When a second region of interest (1730) is determined to be outside the hardware limit range (1700) of the region of interest for acquiring the next image frame, the electronic device may control the composition for capturing the image so that the electronic device captures the image based on the region of interest within the hardware limit range (1700). For example, the image sensor of the electronic device may ignore the capture command based on the second region of interest (1730) outside the hardware limit range (1700) and acquire the next image frame based on the previous region of interest (1710). For example, the region of interest determining unit (1313) of the electronic device may move the position by the amount that the second region of interest (1730) is outside the hardware limit range (1700). For example, in the second shooting mode, if the region of interest of the next image frame determined based on user input cannot be determined within the hardware limit range (1700), the electronic device may exit the second shooting mode and return to the first shooting mode.

[0156] FIG. 18 illustrates an example in which an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) determines a region of interest based on user input including panning input.

[0157] In one embodiment, the electronic device may acquire a preview image (1810) based on information obtained from light-receiving elements corresponding to a first region of interest (1821) set within a shooting area (1820) through an image sensor (e.g., image sensor (230) of FIG. 2, first image sensor (382) of FIG. 3, second image sensor (384) of FIG. 13, image sensor (230) of FIG. 13). The electronic device may display the preview image (1810) through a touchscreen (e.g., display module (160) of FIG. 1, display (360) of FIG. 3, touchscreen (1360) of FIG. 13). In one embodiment, the electronic device may display the preview image (1810) and an image corresponding to the entire shooting area (1820) together. For example, the electronic device may display the preview image (1810) as a main image, while displaying an image corresponding to the entire shooting area (1820) as an auxiliary image in a small area. For example, the electronic device may display an image corresponding to the entire shooting area (1820) as a main image, while displaying a preview image (1810) as an auxiliary image in a small area.

[0158] In one embodiment, the electronic device may detect a panning input including a touch input moving from one point (1811) to another point (1812) via a touchscreen. The electronic device may determine the coordinates of a second region of interest (1841) for acquiring the next image frame based on the amount of movement (1813) of the panning input and the coordinates of a first region of interest (1821), which is a region of interest for the previous image frame. For example, the electronic device may determine the coordinates of the second region of interest (1841) based on the following Equation 3. In the present disclosure, the coordinates of the region of interest may correspond to the position where a light-receiving element is placed within a light-receiving unit (e.g., the light-receiving unit (415) of FIG. 4) in which light-receiving elements are arranged.

[0159]

[0160] In one embodiment, the electronic device can display a preview image (1830) obtained based on information obtained from light receiving elements corresponding to a second region of interest (1841) through a touchscreen.

[0161] FIG. 19 illustrates an example in which an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) determines a region of interest based on user input including pinch input.

[0162] In one embodiment, the electronic device can obtain a preview image (1910) based on information obtained from light receiving elements corresponding to a first region of interest (1921) set within a shooting area (1920) through an image sensor (e.g., image sensor (230) of FIG. 2, first image sensor (382) of FIG. 3, second image sensor (384), image sensor (230) of FIG. 13).

[0163] In one embodiment, the electronic device may detect a plurality of touch inputs, including a first touch input and a second touch input detected at a first touch position (1911) and a second touch position (1912) through a touchscreen. The electronic device may detect a pinch input in which the distance (1913) between the plurality of touch inputs decreases or increases. For example, the electronic device may detect that the first touch input moves from the first touch position (1911) to the third touch position (1931), and that the second touch input moves to the fourth touch position (1932). In response to the detection of the pinch input, the electronic device may adjust the zoom magnification for the image to be provided. The electronic device may determine the zoom magnification based on the distance (1913) between the touch inputs and the distance (1933) between the changed touch inputs. For example, if the zoom magnification before the change is 1.0x, the electronic device may determine the zoom magnification based on the following Equation 4.

[0164]

[0165] In one embodiment, the electronic device may determine the location of a second region of interest (1941) for capturing the next image frame in response to a pinch input. The electronic device may determine the coordinates of the second region of interest (1941) based on a representative location for the locations of the touch inputs. For example, the electronic device may determine the coordinates indicating the second region of interest (1941) based on the center of gravity of the touch inputs. For example, the center of gravity of the touch inputs may be determined based on the following Equation 5.

[0166]

[0167] For example, the coordinates indicating the second region of interest (1941) can be determined based on the following mathematical formula 6.

[0168]

[0169] In one embodiment, the electronic device can display a preview image (1930) obtained based on information obtained from light receiving elements corresponding to a second region of interest (1941) through a touchscreen.

[0170] FIG. 20 is a flowchart (2000) illustrating a process in which an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) according to one embodiment determines a region of interest according to a shooting mode.

[0171] According to one embodiment, in operation 2010, the electronic device may acquire image data based on a first shooting mode. In operation 2020, the electronic device may determine whether a shooting mode change condition for determining whether to change from the first shooting mode to a second shooting mode is satisfied. For example, the electronic device may determine that the shooting mode change condition is satisfied when a condition for entering the second shooting mode is satisfied and a user input to enter the second shooting mode is received. For example, the condition for entering the second shooting mode may include, for example, a case where the ratio of the range of movement of the region of interest to the size of the region of interest is greater than or equal to a threshold.

[0172] According to one embodiment, based on the determination that the shooting mode change condition is satisfied, in operation 2030, the electronic device may identify whether an object of interest exists within a previously captured image. For example, the electronic device may extract feature points from the image and determine that an object of interest exists if feature points corresponding to features of a defined category exist. For example, the electronic device may identify the object of interest based on the location of a touch input selecting an area where the object is located within a preview image. Based on the identification of the object of interest, in operation 2040, the electronic device may determine an area of ​​interest based on the location of the identified object. For example, the electronic device may determine the location and size of the area of ​​interest to include the area where the identified object exists.

[0173] According to one embodiment, based on the fact that the object of interest is not identified, in operation 2050, the electronic device may determine whether the magnitude of motion information regarding the movement of the electronic device is less than or equal to a second threshold. The second threshold may be a value determined to determine whether the position of the electronic device is substantially fixed, such as when the electronic device is placed on a stand, for example. If the magnitude of the motion information is less than or equal to the second threshold, there may be a low need for the electronic device to perform a digital image stabilization (or electronic image stabilization) function based on the motion information. If the magnitude of the motion information is less than or equal to the second threshold, in operation 2060, the electronic device may determine a region of interest based on a touch input. For example, the electronic device may determine a region of interest based on the touch input illustrated in FIG. 18 or FIG. 19. In operation 2060, the electronic device may be configured to limit or disable the digital image stabilization (or electronic image stabilization) function.

[0174] According to one embodiment, when the magnitude of the motion information exceeds a second threshold, the electronic device can determine a region of interest based on the motion information and touch input. For example, the electronic device can determine a second region of interest as a location moved from the location of the region of interest determined based on the touch input to a location that offsets the movement of the electronic device based on the motion information.

[0175] In one embodiment, operations 2030 to 2070 may be performed while the electronic device captures an image based on a second shooting mode. For example, operations 2030 to 2070 may be performed in operation 1260 of FIG. 12. However, the order in which operations 2030 to 2070 are performed is not limited thereto.

[0176] FIG. 21 is a flowchart (2100) illustrating a process in which an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) according to one embodiment captures an image in a second shooting mode.

[0177] In one embodiment, the operations illustrated in the flowchart (2100) may be performed while the electronic device captures an image based on a second shooting mode. For example, the operations illustrated in the flowchart (2100) may be performed in operations 1250 to 1260 of FIG. 12. FIG. 21 illustrates an example of a process for capturing an image within a second shooting mode, and the operations for capturing an image in a second shooting mode are not limited to the operations illustrated in FIG. 21.

[0178] According to one embodiment, in operation 2110, the electronic device may enter a second shooting mode to capture an image. In operation 2120, the electronic device may receive user input. For example, the electronic device may receive a touch input (e.g., a touch input shown in FIG. 18 or FIG. 19) for determining a region of interest through a touchscreen (e.g., a display module (160) of FIG. 1, a display (360) of FIG. 3, a touchscreen (1360) of FIG. 13). In operation 2130, the electronic device may determine whether the received touch input includes multiple touch inputs.

[0179] According to one embodiment, based on the determination that the touch input is a single touch input, in operation 2140, the electronic device may determine whether the location where the touch input is detected is moving. If the location where the touch input is detected is moving, in operation 2141, the electronic device may move the location of the region of interest in response to the movement of the location where the touch input is detected. The electronic device may acquire an image frame based on the moved region of interest and, in operation 2145, standby for one frame until performing an operation to acquire the next frame. Afterward, the electronic device may again perform operations including operation 2120 to acquire the next image frame.

[0180] According to one embodiment, if the position where the touch input is detected in operation 2140 does not move, in operation 2143, it can be determined whether the touch input has been released. If the state in which the touch input is detected is maintained, the electronic device can perform operation 2145. If the touch input has been released, in operation 2150, the electronic device can activate a standby timer. In operation 2151, the electronic device can determine whether an additional touch input has been detected. Based on the detection of an additional touch input within a third threshold from the point in time when the touch input was released in operation 2143, the electronic device can terminate the second shooting mode. If no additional touch input has been detected, in operation 2153, the electronic device can determine whether the standby time has exceeded a predetermined third threshold based on the activated standby timer. If the standby time has not exceeded the predetermined third threshold, the electronic device can perform operation 2151 after waiting for one frame in operation 2155. If the waiting time exceeds a set third threshold, the electronic device can initialize the waiting timer and perform operation 2120.

[0181] According to one embodiment, when a plurality of touch inputs are detected, in operation 2160, the electronic device may determine whether a pinch input has been detected. Based on the detection of a pinch input, in operation 2161, the electronic device may perform an operation to control the zoom magnification to a value corresponding to the pinch input. In operation 2163, the electronic device may wait for one frame, and then in operation 2165, determine whether the touch input has been released. If the touch input is maintained, the electronic device may perform operation 2160. If the touch input is released, the electronic device may perform operation 2120. If the touch input is released in operation 2165, the electronic device may perform operations 2150 to 2155.

[0182] FIG. 22 illustrates an example in which an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) determines a region of interest based on the location of an object of interest. For example, the example illustrated in FIG. 22 can be performed in operation 2040 of FIG. 20.

[0183] In one embodiment, the electronic device may acquire a preview image (2210) based on information obtained from light-receiving elements corresponding to a first region of interest (2221) set within a shooting area (2220) through an image sensor (e.g., image sensor (230) of FIG. 2, first image sensor (382) of FIG. 3, second image sensor (384) of FIG. 13, image sensor (230) of FIG. 13). The electronic device may display the preview image (2210) through a touchscreen (e.g., display module (160) of FIG. 1, display (360) of FIG. 3, touchscreen (1360) of FIG. 13). The electronic device may detect a subject (2211) within the preview image (2210). The electronic device may identify the amount of movement (2213) of the subject (2211) within the preview image (2210). The electronic device can determine the coordinates of a second region of interest (2241) for acquiring the next image frame based on the amount of movement (2213) of the subject (2211) and the coordinates of a first region of interest (2221), which is a region of interest for the previous image frame. For example, the electronic device can determine the coordinates of the second region of interest (2241) based on the following mathematical formula 7.

[0184]

[0185] In one embodiment, the electronic device may display a preview image (2230) obtained based on information obtained from light receiving elements corresponding to a second region of interest (2241) through a touchscreen. The electronic device may determine the second region of interest (2241) such that the position of the subject (2211) in the preview image (2210) and the position of the subject (2231) in the preview image (2230) are maintained.

[0186] FIG. 23 illustrates an example in which an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) determines a region of interest based on user input including the location of an object of interest and panning input. For example, the example illustrated in FIG. 23 can be performed in operation 2040 of FIG. 20.

[0187] In one embodiment, the electronic device may acquire a preview image (2310) based on information obtained from light-receiving elements corresponding to a first region of interest (2321) set within a shooting area (2320) capable of capturing a scene through an image sensor (e.g., image sensor (230) of FIG. 2, first image sensor (382) of FIG. 3, second image sensor (384) of FIG. 13, image sensor (230) of FIG. 13). The electronic device may display the preview image (2310) through a touchscreen (e.g., display module (160) of FIG. 1, display (360) of FIG. 3, touchscreen (1360) of FIG. 13). The electronic device may detect a subject (2311) within the preview image (2310). The electronic device may identify the amount of movement (2313) of the subject (2311) within the preview image (2310). The electronic device can further detect panning input (2312) through a touchscreen. The electronic device can determine the coordinates of a second region of interest (2341) based on the amount of movement (2313) of the subject (2311) and the amount of movement of the coordinates of the touch input included in the panning input (2312). For example, the electronic device can determine the coordinates of the second region of interest (2341) based on the following mathematical formula 8.

[0188]

[0189] In one embodiment, the electronic device may display a preview image (2330) obtained based on information obtained from light receiving elements corresponding to a second region of interest (2341) through a touchscreen. The electronic device may determine the second region of interest (2341) such that the subject (2331) appears at a position moved according to the amount of movement of the panning input (2312) of the subject (2311) within the preview image (2310).

[0190] FIG. 24 illustrates an example in which an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) determines a region of interest based on user input including the location of an object of interest and pinch input. For example, the example illustrated in FIG. 24 can be performed in operation 2040 of FIG. 20.

[0191] In one embodiment, the electronic device may acquire a preview image (2410) based on information obtained from light-receiving elements corresponding to a first region of interest (2421) set within a shooting area (2420) capable of capturing a scene through an image sensor (e.g., image sensor (230) of FIG. 2, first image sensor (382) of FIG. 3, second image sensor (384) of FIG. 13, image sensor (230) of FIG. 13). The electronic device may display the preview image (2410) through a touchscreen (e.g., display module (160) of FIG. 1, display (360) of FIG. 3, touchscreen (1360) of FIG. 13). The electronic device may detect a subject (2411) within the preview image (2410). The electronic device may identify the amount of movement (2413) of the subject (2411) within the preview image (2410). The electronic device may further detect pinch inputs (2412) including a first touch input and a second touch input through a touchscreen. The electronic device may determine a zoom magnification based on a change in the distance between the touch inputs included in the pinch input (2412). For example, if the zoom magnification before the change is 1.0x, the electronic device may adjust the zoom magnification based on the above Equation 4.

[0192] In one embodiment, the electronic device may determine a second region of interest (2441) based on the amount of movement (2413) of the subject (2411) and pinch input (2412). For example, coordinates indicating the location of the second region of interest (2441) may be determined based on the following Equation 9.

[0193]

[0194] In one embodiment, the electronic device may display a preview image (2430) obtained based on information obtained from light receiving elements corresponding to a second region of interest (2441) through a touchscreen. The electronic device may determine the second region of interest (2441) such that the zoom magnification is changed from the preview image (2410) and the subject (2413) appears in the preview image (2430) at a position moved from the position of the subject (2411) in the preview image (2410) according to the movement of the center of gravity of the touch inputs.

[0195] FIG. 25 is a flowchart (2500) illustrating a process in which an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) including a first camera (e.g., the first camera (381) of FIG. 3) and a second camera (e.g., the second camera (383) of FIG. 3) according to one embodiment displays a preview image.

[0196] According to one embodiment, in operation 2510, the electronic device may determine whether the magnitude of motion information regarding the movement of the electronic device is less than a fourth threshold. The fourth threshold may be a value determined to determine whether the position of the electronic device is substantially fixed, for example, when the electronic device is placed on a stand. If the magnitude of the motion information is less than the fourth threshold, in operation 2520, the electronic device may obtain image data by reading pixel values ​​from first pixel lines (e.g., first pixel lines (2630) of FIG. 26) included in the first image sensor of the first camera (e.g., first image sensor (382) of FIG. 1). The first pixel lines may refer to pixel lines including sensor pixels included in the region of interest. In operation 2580, the electronic device can display a preview image corresponding to the region of interest through a display (e.g., the display module (160) of FIG. 1, the display (360) of FIG. 3, the display (360) of FIG. 13) based on the image data acquired in operation 2520.

[0197] According to one embodiment, if the magnitude of the motion information is greater than or equal to a fourth threshold, in operation 2530, the electronic device may determine whether the magnitude of the motion information is less than a fifth threshold. The fifth threshold may be a value determined to determine, for example, when a user is holding the electronic device and standing. If the magnitude of the motion information is less than the fifth threshold, in operation 2540, the electronic device may obtain image data by reading pixel values ​​from the second pixel lines of the first image sensor (e.g., the second pixel lines (2730) of FIG. 27). The second pixel lines include pixel lines containing sensor pixels included in the region of interest, and may include a greater number of pixel lines than the first pixel lines. According to one embodiment, in operation 2540, the electronic device may read pixel values ​​from a wider range of pixel lines as the magnitude of the motion increases, but is not limited thereto. In operation 2580, the electronic device can display a preview image corresponding to the region of interest through a display based on the image data acquired in operation 2540.

[0198] According to one embodiment, if the magnitude of the motion information is greater than or equal to a fifth threshold, the electronic device in operation 2550 may determine whether the magnitude of the motion information is less than a sixth threshold. The sixth threshold may be a value determined, for example, to determine whether the location of the region of interest determined by the movement of the electronic device can be determined within a range that can be captured through the first camera (e.g., field of view (FoV)). If the magnitude of the motion information is less than the sixth threshold, the electronic device in operation 2560 may acquire image data by changing the operation mode of the first image sensor. For example, the electronic device may change the operation mode of the first image sensor from a first operation mode to a second operation mode in operation 2560. The first operation mode may mean an operation mode in which the first image sensor outputs pixel values ​​obtained from sensor pixels in at least a portion of the first image sensor based on a raw image pattern having high resolution (e.g., the second raw image pattern (720) of FIG. 7, the second raw image pattern (820) of FIG. 8, the second raw image pattern (920) of FIG. 9, or the third raw image pattern (930)). The second operation mode may mean an operation mode in which the first image sensor outputs pixel values ​​based on a raw image pattern having lower resolution than the first operation mode by binning the output of a plurality of sensor pixels (e.g., the first raw image pattern (710) of FIG. 7, the first raw image pattern (810) of FIG. 8, the first raw image pattern (910) of FIG. 9). In operation 2560, the first image sensor may read pixel values ​​from the entire pixel lines (e.g., the entire pixel lines (2831) of FIG. 28). In operation 2580, the electronic device can display a preview image corresponding to the region of interest through a display based on the image data acquired in operation 2560.

[0199] According to one embodiment, when the magnitude of the motion information is greater than or equal to a sixth threshold, in operation 2570, the electronic device may acquire image data (e.g., image data (2940) of FIG. 29) based on the second camera. The second camera may support a field of view formed at an angle wider than the field of view of the first camera. For example, the first camera may include a wide-angle camera, and the second camera may include an ultra-wide-angle camera. In operation 2580, the electronic device may display a preview image corresponding to the region of interest through a display based on the image data acquired in operation 2570.

[0200] The process illustrated in FIG. 25 can be performed multiple times while an image stream is output from an image sensor to display a preview image acquired through a camera or to capture a video.

[0201] FIG. 26 illustrates an example of an area where a first image sensor (e.g., image sensor (230) of FIG. 2, first image sensor (381) of FIG. 3, image sensor (230) of FIG. 13) outputs data while an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) according to one embodiment is fixed.

[0202] In one embodiment, the electronic device may determine an area of ​​interest (2610) within a captureable area (2600) of the first image sensor to acquire image data. Based on the determination that the electronic device is in a fixed state, the electronic device may control the first image sensor to read pixel values ​​from first pixel lines (2630) placed in an area corresponding to the width of the area of ​​interest (2610) (e.g., width (2620) in a direction perpendicular to the direction in which the pixel lines are arranged). The first image sensor may output raw image data cropped from the pixel values ​​read from the first pixel lines (2630) that are included in the area of ​​interest (2610) to a processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, area of ​​interest implementation unit (1311) of FIG. 13). In FIG. 26, the first image sensor can operate based on a first operation mode that outputs raw image data based on a raw image pattern that supports high resolution.

[0203] FIG. 27 illustrates an example of an area where an image sensor outputs data when an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) moves to a location that is included in the output area of ​​an image sensor (e.g., the image sensor (230) of FIG. 2, the first image sensor (381) of FIG. 3, the image sensor (230) of FIG. 13).

[0204] In one embodiment, the electronic device may acquire an image frame containing image data obtained from sensor pixels included in a first region of interest (2710) within a captureable area (2700) of the first image sensor. The first image sensor may read pixel values ​​from second pixel lines (2730) placed in an area including the width (2720) of the first region of interest (2710) set to read the previous frame. The first image sensor may output pixel values ​​included in an output area (2715) including the first region of interest (2710) among the pixel values ​​read from the second pixel lines (2730) as raw image data to a processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, region of interest implementation unit (1311) of FIG. 13).

[0205] In one embodiment, the electronic device may determine a second region of interest (2711) within the captureable area (2700) of the first image sensor. For example, the electronic device may determine the second region of interest (2711) by performing an image stabilization function based on motion information. For example, the electronic device may determine the second region of interest (2711) by receiving user input that adjusts the region of interest. For example, the electronic device may determine the second region of interest (2711) by tracking the position of a subject. If the second region of interest (2711) is included within the output area (2715), the electronic device may obtain image data by cropping the portion of the output area (2715) corresponding to the second region of interest (2711). After the position of the second region of interest (2711) is stabilized, the first image sensor may read pixel values ​​from the third pixel lines (2731) based on the position of the second region of interest (2711). The first image sensor can crop and output an output area (2716) including a second region of interest (2711) among pixel values ​​read from the third pixel lines (2731). In FIG. 27, the first image sensor can operate based on a first operation mode that outputs raw image data based on a raw image pattern that supports high resolution.

[0206] FIG. 28 illustrates an example of an area where an image sensor outputs data when an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) moves to a location outside the output area of ​​an image sensor (e.g., the image sensor (230) of FIG. 2, the first image sensor (381) of FIG. 3, the image sensor (230) of FIG. 13).

[0207] In one embodiment, the electronic device may acquire an image frame containing image data obtained from a sensor pixel included in a first region of interest (2810) within a captureable area (2800) of the first image sensor. Here, the first image sensor may operate based on a first operation mode that outputs raw image data based on a raw image pattern that supports high resolution. The first image sensor may read pixel values ​​from second pixel lines (2830) placed in an area including the width (2820) of the first region of interest (2810). The first image sensor may output pixel values ​​included in an output area (2815) including the first region of interest (2810) among the pixel values ​​read from the second pixel lines (2830) as raw image data to a processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, region of interest implementation unit (1311) of FIG. 13).

[0208] In one embodiment, the electronic device may determine a second region of interest (2811) that is contained within the captureable area (2800) of the first image sensor and extends at least partially beyond the output area (2815). For example, the electronic device may determine the second region of interest (2811) by performing an image stabilization function based on motion information. For example, the electronic device may determine the second region of interest (2811) by receiving user input that adjusts the region of interest. For example, the electronic device may determine the second region of interest (2811) by tracking the position of a subject.

[0209] In one embodiment, to acquire image data for a second region of interest (2811), the first image sensor may read pixel values ​​from all pixel lines (2831) included within a captureable area (2800). Here, the first image sensor may operate based on a second operation mode that outputs raw image data based on a raw image pattern that supports a lower resolution than the first operation mode. The first image sensor may output the read pixel values ​​to a processor. The electronic device may acquire image data by cropping the area corresponding to the second region of interest (2811) among the pixel values ​​output by the first image sensor based on the second operation mode.

[0210] In one embodiment, when the position of the second region of interest (2811) is stabilized (e.g., when the magnitude of motion information is reduced to a reference value or less), the first image sensor can read pixel values ​​from the third pixel lines (2832) based on the position of the second region of interest (2811). The first image sensor can crop and output an output area (2816) containing the second region of interest (2811) among the pixel values ​​read from the third pixel lines (2832). The first image sensor can operate based on a first operation mode that outputs raw image data based on a raw image pattern supporting high resolution. The electronic device can obtain image data by cropping the second region of interest (2811) from the output area (2816).

[0211] FIG. 29 illustrates an example in which an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) acquires image data through a second camera (e.g., the camera module (180) of FIG. 1 and 2, the first camera (381) of FIG. 3, the camera (380) of FIG. 13)) when the region of interest moves to a location outside the field of view of the first camera (e.g., the camera module (180) of FIG. 1 and 2, the second camera (383) of FIG. 3, the camera (380) of FIG. 13).

[0212] In one embodiment, the electronic device may acquire an image frame containing image data acquired from sensor pixels included in a first region of interest (2910) within a captureable area (2900) of a first image sensor (e.g., image sensor (230) of FIG. 2, first image sensor (381) of FIG. 3, image sensor (230) of FIG. 13). Here, the first image sensor may operate based on a first operation mode that outputs raw image data based on a raw image pattern that supports high resolution. The first image sensor may read pixel values ​​from second pixel lines (2930) placed in an area including the width (2920) of the first region of interest (2910). The first image sensor can output pixel values ​​included in an output area (2915) including a first region of interest (2910) among pixel values ​​read from the second pixel lines (2930) as raw image data to a processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, region of interest implementation unit (1311) of FIG. 13).

[0213] In one embodiment, the electronic device may determine a second region of interest (2911) that extends at least partially beyond the captureable area (2900) of the first image sensor. For example, the electronic device may determine the second region of interest (2911) by performing an image stabilization function based on motion information. For example, the electronic device may determine the second region of interest (2911) by receiving user input that adjusts the region of interest. For example, the electronic device may determine the second region of interest (2911) by tracking the position of a subject.

[0214] In one embodiment, to acquire image data for a second region of interest (2911), the electronic device may activate a second camera that supports a field of view formed at an angle wider than the field of view of the first camera. The electronic device may control a second image sensor of the second camera (e.g., image sensor (230) of FIG. 2, second image sensor (383) of FIG. 3, image sensor (230) of FIG. 13) to output raw image data including a shootable range (2940) of the second camera. The electronic device may acquire image data by cropping the second region of interest (2911) within the raw image data output from the second image sensor. The electronic device may control components of the electronic device so that the first camera can shoot a moved range (2901) including the second region of interest (2911). For example, the electronic device may control an optical image stabilizer of the first camera to move the position of the lens of the first camera or the first image sensor. The electronic device can enable the first image sensor to capture the range (2901) in which it has moved.

[0215] In one embodiment, when the position of the second region of interest (2911) is stabilized (e.g., when the magnitude of the motion information is reduced to a reference value or less), the first image sensor can read pixel values ​​from the third pixel lines (2931) based on the position of the second region of interest (2911) within the moved range (2901). The first image sensor can crop the output area (2916) containing the second region of interest (2911) among the pixel values ​​read from the third pixel lines (2931) and output it to a processor. The first image sensor can operate based on a first operation mode. The electronic device can obtain image data by cropping the second region of interest (2911) from the output area (2916).

[0216] FIG. 30 is a flowchart (3000) illustrating a process in which an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) reads pixel lines based on the location of an object of interest.

[0217] According to one embodiment, in operation 3010, the electronic device can identify an object of interest within image data acquired through a first camera (e.g., camera module (180) of FIG. 1 and 2, first camera (381) of FIG. 3, camera (380) of FIG. 13). For example, the electronic device can extract feature points from a preview image displayed on a display (e.g., display module (160) of FIG. 1, display (360) of FIG. 3, display (360) of FIG. 13), and determine that an object of interest exists if there are feature points corresponding to features of a defined category. For example, the electronic device may identify an object of interest based on the location of a touch input that selects an area where an object is located within the preview image.

[0218] According to one embodiment, in operation 3020, the electronic device can determine whether the position of the object of interest has deviated from a specified range. For example, the electronic device can determine whether the position of the object of interest has deviated from an output area output from the first image sensor of the first camera (e.g., the image sensor (230) of FIG. 2, the first image sensor (381) of FIG. 3, the image sensor (230) of FIG. 13).

[0219] According to one embodiment, when the location of an object of interest is within a specified range, in operation 3030, the electronic device may control a first image sensor (e.g., image sensor (230) of FIG. 2, first image sensor (381) of FIG. 3, image sensor (230) of FIG. 13) to read out first pixel lines (e.g., pixel lines (3110) of FIG. 31) to acquire image data.

[0220] According to one embodiment, when the location of the object of interest deviates from a specified range, in operation 3040, the electronic device may control the first image sensor to read out a greater number of second pixel lines (e.g., the second pixel lines (3111) of FIG. 31) than the first pixel lines to acquire image data.

[0221] According to one embodiment, in operation 3050, the electronic device may determine whether the operation of capturing an image based on the location of an object of interest has ended. For example, the electronic device may determine whether the shooting mode tracking the object of interest has ended or whether the camera application has ended. If the operation of capturing an image based on the location of the object of interest has not ended, the electronic device may perform operation 3010 again. If the operation of capturing an image has ended, the electronic device may terminate the process.

[0222] FIG. 31 illustrates an example in which an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) reads pixel lines based on the location of an object of interest.

[0223] In one embodiment, an image sensor (e.g., the image sensor (230) of FIG. 2, the first image sensor (381) of FIG. 3, and the image sensor (230) of FIG. 13) can read pixel values ​​from first pixel lines (3110) within a captureable area (3100). The image sensor can output pixel values ​​included in a first output area (3120) among the pixel values ​​read from the first pixel lines (3110). An electronic device can acquire image data by cropping a first region of interest (3130) among the pixel values ​​included in the first output area (3120). The electronic device can identify and track the location of a subject (3140) within the image data.

[0224] In one embodiment, the electronic device can identify that a subject moves to a position (3141) outside the output area (3120) output from the image sensor. Based on identifying that the subject moves out of the output area (3120), the electronic device can control the image sensor to read pixel values ​​from a number of second pixel lines (3111) greater than the first pixel lines (3110) to include the position (3141) of the subject. The image sensor can crop and output the second output area (3121) among the pixel values ​​read from the second pixel lines (3111).

[0225] In one embodiment, the electronic device may determine a second region of interest (3131) to include the position (3141) of a subject within a second output area (3121) output from an image sensor. The electronic device may obtain image data by cropping the second region of interest (3131) within the second output area (3121). When the position (3141) of a subject is included in the image data corresponding to the second region of interest (3131), the electronic device may control the image sensor to read pixel values ​​from a number of third pixel lines (3112) that is smaller than the number of second pixel lines (3111). The image sensor may crop and output the third output area (3122) among the pixel values ​​read from the third pixel lines (3112).

[0226] FIG. 32 is a flowchart (3200) illustrating a process in which an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) controls a region of interest to acquire image data based on touch input.

[0227] According to one embodiment, in operation 3210, the electronic device may control the image sensor of the electronic device (e.g., the first image sensor (381) of FIG. 3) based on a first region of interest. The electronic device may control the image sensor to crop and output image data configured based on information (e.g., pixel values) obtained from the first sensor pixels corresponding to the first region of interest. The image data output from the image sensor may be transmitted to the processor of the electronic device (e.g., the processor (120) of FIG. 1, the image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, the composition controller (1310) of FIG. 13, the mode controller (1320)).

[0228] According to one embodiment, in operation 3220, the electronic device may receive touch input through a touchscreen (e.g., the display module (160) of FIG. 1, the display (360) of FIG. 3, the display (360) of FIG. 13). For example, the electronic device may receive touch input while displaying a preview image corresponding to image data acquired based on a first region of interest.

[0229] According to one embodiment, in operation 3230, the electronic device may determine a second region of interest in which at least one of the location or area differs from a first region of interest based on a received touch input. For example, the electronic device may determine the second region of interest based on a method for determining the second region of interest based on touch inputs presented in FIGS. 18 to 24.

[0230] According to one embodiment, in operation 3240, the electronic device can control the image sensor based on a determined second region of interest. The electronic device can control the image sensor to crop and output image data configured based on information obtained from second sensor pixels corresponding to the second region of interest. The image data output from the image sensor can be transmitted to the processor of the electronic device.

[0231] In one embodiment, when the image sensor of an electronic device supports a first output mode for reading image data having a first resolution and a second output mode for reading image data having a second resolution higher than the first resolution, the process illustrated in the flowchart (3200) of FIG. 32 can be performed while the image sensor operates based on the second output mode.

[0232] In one embodiment, the process illustrated in the flowchart (3200) of FIG. 32 may be performed in the second shooting mode illustrated in FIG. 12, provided, but not limited thereto. For example, the process illustrated in the flowchart (3200) of FIG. 32 may be performed regardless of the shooting mode. For example, the process illustrated in the flowchart (3200) of FIG. 32 may be performed to determine the region of interest by further considering at least one of the movement of the electronic device or the movement of the subject.

[0233] In one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) may include a display (e.g., display module (160) of FIG. 1, display (360) of FIG. 3, display (360) of FIG. 13), a first camera (e.g., camera module (180) of FIG. 1 and 2, first camera (381) of FIG. 3, camera (380) of FIG. 13), at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) and a memory (e.g., memory (130) of FIG. 1, memory (250) of FIG. 2, memory (330) of FIG. 3). The above display (e.g., display module (160) of FIG. 1, display (360) of FIG. 3, display (360) of FIG. 13)) may include a touchscreen (e.g., touchscreen (1360) of FIG. 13). The above first camera (e.g., camera module (180) of FIG. 1 and 2, first camera (381) of FIG. 3, camera (380) of FIG. 13)) may include a first image sensor (e.g., first image sensor (381) of FIG. 3). The first image sensor (e.g., the first image sensor (381) of FIG. 3) may include a plurality of sensor pixels (e.g., sensor pixels (PX) of FIG. 11) arranged along a plurality of pixel lines (e.g., pixel lines (1111, 1112, 1113, 1114, 1115, 1116, 1117, 1118) of FIG. 11. The memory (e.g., memory (130) of FIG. 1, memory (250) of FIG. 2, memory (330) of FIG. 3)) may store at least one computer program including computer-executable instructions. The first image sensor (e.g., the first image sensor (381) of FIG. 3) may include a micro-lens array (e.g., the micro-lens array (411) of FIG. 4), a color filter array (the color filter array (413) of FIG. 4), and a light receiving unit (e.g., the light receiving unit (415) of FIG. 4).A microlens array (e.g., the microlens array (411) of FIG. 4) may include a plurality of microlenses. A color filter array (the color filter array (413) of FIG. 4) may include a plurality of color channels. The light receiving unit (e.g., the light receiving unit (415) of FIG. 4) may include a plurality of light receiving elements corresponding to at least some of the plurality of sensor pixels. The plurality of light receiving elements may be arranged to correspond to one of the microlenses. The plurality of light receiving elements may constitute the plurality of sensor pixels. The above commands are executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) so that the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) crops image data configured based on information obtained from first sensor pixels corresponding to a first region of interest (RoI) among a plurality of sensor pixels and outputs it to the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)), thereby the first image sensor (e.g., first image of FIG. 3). It can be made to control the sensor (381).The above commands may be executed individually or collectively by at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) to receive touch input through the touchscreen (e.g., touchscreen (1360) of FIG. 13). The above commands may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) to determine a second region of interest in which at least one of the location or area differs from the first region of interest based on the touch input.The above commands may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to control the first image sensor (e.g., first image sensor (381) of FIG. 3) so that the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) crops image data configured based on information obtained from second sensor pixels corresponding to the second region of interest among the plurality of sensor pixels and outputs it to the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)).

[0234] In one embodiment, the instructions may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) to acquire movement information regarding the movement of the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3). The above instructions may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to determine a third region of interest (RoI) corresponding to a location that compensates for the movement of the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) based on the movement information in a first shooting mode.The above commands may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) to acquire image data from at least some of the plurality of sensor pixels of the first image sensor (e.g., first image sensor (381) of FIG. 3) corresponding to the third region of interest in the first shooting mode. The above commands may be executed individually or collectively by at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to cause the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) to change the first shooting mode to a second shooting mode based on whether a specified condition is satisfied in the first shooting mode. The above commands may be executed individually or collectively by at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) to receive touch input through the touchscreen (e.g., touchscreen (1360) of FIG. 13)) in the second shooting mode.The above commands may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) to determine the second region of interest based at least partially on the touch input regardless of the motion information in the second shooting mode. The above commands may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) to acquire image data corresponding to the second region of interest from the first image sensor (e.g., first image sensor (381) of FIG. 3) in the second shooting mode.

[0235] In one embodiment, the instructions may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) to receive user input related to zoom magnification through the touchscreen (e.g., touchscreen (1360) of FIG. 13). The above commands may be executed individually or collectively by at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) to determine whether the zoom magnification corresponding to the user input is greater than or equal to a first threshold. The above commands may be executed individually or collectively by at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) to display a user interface that can select the second shooting mode through the display (e.g., display module (160) of FIG. 1, display (360) of FIG. 3, display (360) of FIG. 13)) based on the fact that the zoom magnification is above a threshold.The above commands may be executed individually or collectively by at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to allow the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) to determine that the specified condition is satisfied based on receiving an input selecting the second shooting mode through the user interface.

[0236] In one embodiment, the first image sensor (e.g., the first image sensor (381) of FIG. 3) may be configured to read out image data having a first resolution from an area of ​​the light receiving unit (e.g., the light receiving unit (415) of FIG. 4) in a first output mode. The first image sensor (e.g., the first image sensor (381) of FIG. 3) may be configured to read out image data having a second resolution higher than the first resolution from an area of ​​the light receiving unit (e.g., the light receiving unit (415) of FIG. 4) in a second output mode. The first image sensor (e.g., the first image sensor (381) of FIG. 3) may be configured to output image data based on the second output mode while the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) is operating based on the second shooting mode.

[0237] In one embodiment, the first image sensor (e.g., the first image sensor (381) of FIG. 3) may be configured to output a single pixel value by binning the outputs of two or more light-receiving elements that are arranged adjacent to each other and correspond to one color channel or one microlens in the first output mode.

[0238] In one embodiment, the first image sensor (e.g., the first image sensor (381) of FIG. 3) may be configured to output one pixel value corresponding to one light receiving element in the second output mode.

[0239] In one embodiment, the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) may further include a motion sensor for detecting the motion information. The instructions may be executed individually or collectively by the at least one processor (e.g., the processor (120) of FIG. 1, the image signal processor (260) of FIG. 2, the at least one processor (320) of FIG. 3, the composition controller (1310) of FIG. 13, the mode controller (1320)) to enable the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3)) to determine whether the magnitude of the motion information detected through the motion sensor is greater than or equal to a second threshold. The above commands may be executed individually or collectively by at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) to determine the second region of interest based on the touch input when the magnitude of the motion information is less than the second threshold in the second shooting mode. The above commands may be executed individually or collectively by at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) to determine the second region of interest based on the motion information and the touch input when the magnitude of the motion information is greater than or equal to the second threshold in the second shooting mode.

[0240] In one embodiment, the instructions may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) to display a preview image through the display (e.g., display module (160) of FIG. 1, display (360) of FIG. 3, display (360) of FIG. 13)) based on an image acquired through the first image sensor (e.g., first image sensor (381) of FIG. 3). The above commands may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) to identify whether there is an object to be tracked within the preview image. The above commands may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) to determine the second region of interest based on the location of the object and the touch input in the second shooting mode, based on the electronic device identifying that the object to be tracked exists.

[0241] In one embodiment, the instructions may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) to read pixel values ​​from the first pixel lines including the second region of interest among the plurality of pixel lines (e.g., pixel lines (1111, 1112, 1113, 1114, 1115, 1116, 1117, 1118) of FIG. 11) in the second shooting mode based on the fact that the size of the motion information corresponds to the first size. The above commands may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) to read pixel values ​​from a number of second pixel lines (e.g., pixel lines (1111, 1112, 1113, 1114, 1115, 1116, 1117, 1118) of FIG. 11) that is greater than the number of first pixel lines, based on the fact that the size of the motion information corresponds to a second size that is larger than the first size in the second shooting mode.

[0242] In one embodiment, the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) may further include a second camera (e.g., the camera module (180) of FIG. 1 and 2, the second camera (383) of FIG. 3, the camera (380) of FIG. 13). The second camera (e.g., the camera module (180) of FIG. 1 and 2, the second camera (383) of FIG. 3, the camera (380) of FIG. 13)) may support a second field of view corresponding to an angle wider than the angle of the first field of view supported by the first camera (e.g., the camera module (180) of FIG. 1 and 2, the first camera (381) of FIG. 3, the camera (380) of FIG. 13). The second camera (e.g., camera module (180) of FIG. 1 and 2, second camera (383) of FIG. 3, camera (380) of FIG. 13) may include a second image sensor (e.g., second image sensor (383) of FIG. 3). The above commands may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) to acquire an image from the second camera (e.g., camera module (180) of FIG. 1 and 2, second camera (383) of FIG. 3, camera (380) of FIG. 13)) based on the fact that the size of the motion information corresponds to a third size larger than the second size in the second shooting mode.The above commands may be executed individually or collectively by the at least one processor (e.g., processor (120) of FIG. 1, image signal processor (260) of FIG. 2, at least one processor (320) of FIG. 3, composition controller (1310) of FIG. 13, mode controller (1320)) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3)) to display a preview image through the display (e.g., display module (160) of FIG. 1, display (360) of FIG. 3, display (360) of FIG. 13)) based on an image acquired from the second camera (e.g., camera module (180) of FIG. 1 and 2, second camera (383) of FIG. 3, camera (380) of FIG. 13).

[0243] In one embodiment, a method of operating an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) including a first camera (e.g., camera module (180) of FIG. 1 and 2, first camera (381) of FIG. 3, camera (380) of FIG. 13) including a first image sensor (e.g., first image sensor (381) of FIG. 3)) may include an operation of controlling the first image sensor (e.g., first image sensor (381) of FIG. 3) to crop and output image data configured based on information obtained from an area corresponding to a first region of interest within a light receiving part of the first image sensor (e.g., first image sensor (381) of FIG. 3). The method may include an operation of receiving a touch input through a touchscreen of the electronic device (e.g., touchscreen (1360) of FIG. 13). The above method may include an operation of determining a second region of interest in which at least one of the location or area differs from the first region of interest based on the touch input. The above method may include an operation of controlling a first image sensor to crop and output image data configured based on information obtained from an area corresponding to the second region of interest within the light receiving unit.

[0244] In one embodiment, the method may include an operation of acquiring motion information regarding the movement of the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3). The method may include, in a first shooting mode, an operation of determining a third region of interest corresponding to a position that compensates for the movement of the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) based on the motion information. The method may include, in the first shooting mode, an operation of acquiring image data from at least a portion corresponding to the third region of interest within a light receiving part (e.g., the light receiving part (415) of FIG. 4) of the first image sensor (e.g., the first image sensor (381) of FIG. 3). The method may include, in the first shooting mode, an operation of changing the first shooting mode to a second shooting mode based on whether a specified condition is satisfied. The operation of receiving the touch input and the operation of determining the second region of interest may be performed in the second shooting mode.

[0245] In one embodiment, the operation of changing the first shooting mode to the second shooting mode may include receiving user input related to the zoom magnification through the touchscreen (e.g., the touchscreen (1360) of FIG. 13). The operation of changing the first shooting mode to the second shooting mode may include determining whether the zoom magnification corresponding to the user input is greater than or equal to a first threshold. Based on the fact that the zoom magnification is greater than or equal to the threshold, the operation may include displaying a user interface that allows selecting the second shooting mode through the touchscreen (e.g., the touchscreen (1360) of FIG. 13). The operation of changing the first shooting mode to the second shooting mode may include determining that the specified condition is satisfied based on receiving an input selecting the second shooting mode through the user interface.

[0246] In one embodiment, the operation of determining the second region of interest may include determining whether the magnitude of the motion information regarding the movement of the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) is greater than or equal to a second threshold. The operation of determining the second region of interest may include determining the second region of interest based on the touch input when the magnitude of the motion information is greater than or equal to the second threshold. The operation of determining the second region of interest may include determining the second region of interest based on the motion information and the touch input when the magnitude of the motion information is greater than or equal to the second threshold.

[0247] In one embodiment, the method may include an operation of displaying a preview image through the touchscreen (e.g., the touchscreen (1360) of FIG. 13) based on an image acquired through the first image sensor (e.g., the first image sensor (381) of FIG. 3). The method may further include an operation of identifying whether there is an object to be tracked within the preview image. The method may include an operation of determining the second region of interest based on the location of the object and the touch input in the second shooting mode, based on the identification that the object to be tracked exists.

[0248] In one embodiment, the operation of acquiring image data corresponding to the second region of interest may include, in the second shooting mode, an operation of reading pixel values ​​from first pixel lines including the second region of interest among a plurality of pixel lines (e.g., pixel lines of FIG. 11 (1111, 1112, 1113, 1114, 1115, 1116, 1117, 1118)) of the first image sensor (e.g., the first image sensor (381) of FIG. 3) based on the fact that the size of the motion information corresponds to the first size. The operation of acquiring image data corresponding to the second region of interest may include, in the second shooting mode, an operation of reading pixel values ​​from a number of second pixel lines greater than the first pixel lines among the plurality of pixel lines (e.g., the pixel lines of FIG. 11 (1111, 1112, 1113, 1114, 1115, 1116, 1117, 1118)), based on the fact that the size of the motion information corresponds to a second size larger than the first size.

[0249] In one embodiment, the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) may further include a second camera (e.g., the camera module (180) of FIG. 1 and 2, the first camera (381) of FIG. 3, the camera (380) of FIG. 13)) which includes a second field of view corresponding to an angle wider than the angle of the first field of view supported by the first camera (e.g., the camera module (180) of FIG. 1 and 2, the second camera (383) of FIG. 3, the camera (380) of FIG. 13)) which includes a second image sensor (e.g., the second image sensor (383) of FIG. 3). The above method may further include an operation of acquiring an image from the second camera (e.g., camera module (180) of FIG. 1 and 2, second camera (383) of FIG. 3, camera (380) of FIG. 13) based on the fact that the size of the motion information in the second shooting mode corresponds to a third size larger than the second size. The above method may further include an operation of displaying a preview image through the touchscreen (e.g., touchscreen (1360) of FIG. 13) based on the image acquired from the second camera (e.g., camera module (180) of FIG. 1 and 2, second camera (383) of FIG. 3, camera (380) of FIG. 13).

[0250] In one embodiment, a computer-readable non-transient recording medium may have a computer program recorded thereon that includes computer-executable instructions. The instructions may cause the electronic device (e.g., the electronic device of FIG. 1 (101), the electronic device of FIG. 3 (101)) including a first camera (e.g., the camera module (180) of FIG. 1 and 2, the first camera (381) of FIG. 3, the camera (380) of FIG. 13)) to perform an operation to control the first image sensor (e.g., the first image sensor (381) of FIG. 3) to crop and output image data configured based on information obtained from an area corresponding to a first region of interest within the light receiving part of the first image sensor (e.g., the first image sensor (381) of FIG. 3)) when the electronic device (e.g., the electronic device of FIG. 1 (101), the electronic device of FIG. 3 (101)) is executed. The above commands may cause the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) to perform an operation of receiving touch input through the touchscreen of the electronic device (e.g., the touchscreen (1360) of FIG. 13) when the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3)) is executed. The above commands may cause the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) to perform an operation of determining a second region of interest in which at least one of the location or area is different for the first region of interest based on the touch input.The above commands may enable the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) to perform an operation to control the first image sensor (e.g., the first image sensor (381) of FIG. 3) so that, when the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) is executed, the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3)) crops and outputs image data configured based on information obtained from an area corresponding to the second area of ​​interest within the light receiving unit.

[0251] The above commands may cause the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) to perform an operation of acquiring movement information regarding the movement of the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) when the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3)) is executed. The above commands may cause the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) to perform an operation of determining a third region of interest corresponding to a position that compensates for the movement of the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) based on the movement information when the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3)) is executed in a first shooting mode. The above commands may cause the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) to perform an operation of acquiring image data from at least a portion corresponding to the third region of interest within the light receiving part (e.g., the light receiving part (415) of FIG. 4) of the first image sensor (e.g., the first image sensor (381) of FIG. 3)) when the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3)) is executed when the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3)) performs an operation of changing the first shooting mode to a second shooting mode based on whether a specified condition is satisfied in the first shooting mode. The operation of receiving the touch input and the operation of determining the second region of interest may be performed in the second shooting mode.

[0252] In one embodiment, the operation of changing the first shooting mode to the second shooting mode may include receiving user input related to the zoom magnification through the touchscreen (e.g., the touchscreen (1360) of FIG. 13). The operation of changing the first shooting mode to the second shooting mode may include determining whether the zoom magnification corresponding to the user input is greater than or equal to a first threshold. The operation of changing the first shooting mode to the second shooting mode may include displaying a user interface that allows selecting the second shooting mode through the touchscreen (e.g., the touchscreen (1360) of FIG. 13) based on the fact that the zoom magnification is greater than or equal to the threshold. The operation of changing the first shooting mode to the second shooting mode may include determining that the specified condition is satisfied based on the fact that an input selecting the second shooting mode is received through the user interface.

[0253] In one embodiment, the operation of determining the second region of interest may include determining whether the magnitude of the motion information regarding the movement of the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (101) of FIG. 3) is greater than or equal to a second threshold. The operation of determining the second region of interest may include determining the second region of interest based on the touch input when the magnitude of the motion information is greater than or equal to the second threshold. The operation of determining the second region of interest may include determining the second region of interest based on the motion information and the touch input when the magnitude of the motion information is greater than or equal to the second threshold.

[0254] In one embodiment, the commands may cause the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3) to perform an operation of displaying a preview image through the touchscreen (e.g., the touchscreen (1360) of FIG. 13) based on an image obtained through the first image sensor (e.g., the first image sensor (381) of FIG. 3). The commands may further include an operation of identifying whether there is an object to be tracked within the preview image when the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (101) of FIG. 3)) is executed. The operation of determining the second region of interest may include, based on identifying that the object to be tracked exists, determining the second region of interest based on the location of the object and the touch input in the second shooting mode.

[0255] In one embodiment, the operation of acquiring image data corresponding to the second region of interest may include, in the second shooting mode, an operation of reading pixel values ​​from first pixel lines including the second region of interest among a plurality of pixel lines (e.g., pixel lines of FIG. 11 (1111, 1112, 1113, 1114, 1115, 1116, 1117, 1118)) of the first image sensor (e.g., the first image sensor (381) of FIG. 3) based on the fact that the size of the motion information corresponds to the first size. The operation of acquiring image data corresponding to the second region of interest may include, in the second shooting mode, an operation of reading pixel values ​​from a number of second pixel lines greater than the first pixel lines among the plurality of pixel lines (e.g., the pixel lines of FIG. 11 (1111, 1112, 1113, 1114, 1115, 1116, 1117, 1118)), based on the fact that the size of the motion information corresponds to a second size larger than the first size.

[0256] In the present disclosure, an electronic device, a method of operation thereof, and a recording medium having a computer program may be provided, which enables a user to precisely control the shooting range in a situation where the movement of the scene being shot is large compared to the angle at which the camera's direction changes (e.g., when shooting a scene at a distance by applying a high zoom magnification).

[0257] In the present disclosure, an electronic device capable of controlling the range of a scene being captured even when the electronic device is physically fixed, a method of operation thereof, and a recording medium having a computer program are provided.

[0258] In the present disclosure, an electronic device, a method of operation thereof, and a recording medium recording a computer program may be provided, which prevents loss in terms of user experience in situations where a function to control the range of a scene being filmed based on user input is unnecessary.

[0259] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description of the present disclosure.

[0260] Methods according to the embodiments described in the claims or specification may be implemented in the form of hardware, software, or a combination of hardware and software.

[0261] When implemented in software, a computer-readable storage medium may be provided for storing one or more programs (software modules). One or more programs stored in the computer-readable storage medium are configured for execution by one or more processors within an electronic device. One or more programs include instructions that cause the electronic device to execute methods according to the claims or embodiments described in the specification of this disclosure.

[0262] In the present disclosure, the function or operation performed by an electronic device may be performed by one or more processors executing one or more instructions stored in memory. The function or operation of the electronic device mentioned in the present disclosure may be performed by a single processor executing one or more instructions, or by a combination of multiple processors executing one or more instructions. A processor mentioned in the present disclosure is understood to include a circuit for performing operations or controlling other components of the electronic device. For example, the one or more processors may include a central processing unit (CPU), a micro-processor unit (MPU), an application processor (AP), a communication processor (CP), a neural processing unit (NPU), a system on chip (SoC), or an integrated circuit (IC) configured to execute one or more instructions. The one or more processors may be configured to perform the operation of the electronic device described above.

[0263] In the present disclosure, a program (software module, software) may be stored in a random access memory, a non-volatile memory including flash memory, a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic disc storage device, a compact disc-ROM (CD-ROM), digital versatile discs (DVDs), or other forms of optical storage devices, or a magnetic cassette. Alternatively, it may be stored in a memory composed of some or all of these. The memory may be composed of a single storage medium or a combination of multiple storage media. The one or more instructions may be stored in a single storage medium or distributed across multiple storage media.

[0264] Additionally, the above program may be stored on an attachable storage device that can be accessed via a communication network such as the Internet, Intranet, LAN (local area network), WLAN (wide LAN), or SAN (storage area network), or a combination thereof. Such a storage device may be connected to a device performing an embodiment of the present disclosure through an external port. Additionally, a separate storage device on a communication network may be connected to a device performing an embodiment of the present disclosure.

[0265] In the specific embodiments of the present disclosure described above, the components included in the disclosure are expressed in a singular or plural form according to the specific embodiments presented. However, the singular or plural expression is selected to suit the situation presented for convenience of explanation, and the present disclosure is not limited to singular or plural components; even if a component is expressed in the plural form, it may be composed of a singular form, and even if a component is expressed in the singular form, it may be composed of a plural form.

[0266] Additionally, in the present disclosure, terms such as “part,” “module,” etc. may be a hardware component, such as a processor or circuit, and / or a software component executed by a hardware component, such as a processor.

[0267] "Parts" and "modules" may be implemented by a program that is stored on an addressable storage medium and can be executed by a processor. For example, "parts" and "modules" may be implemented by components such as software components, object-oriented software components, class components, and task components, as well as by processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables.

[0268] The specific embodiments described in this disclosure are merely examples and do not limit the scope of this disclosure in any way. For the sake of brevity, descriptions of prior electronic configurations, control systems, software, and other functional aspects of said systems may be omitted.

[0269] Additionally, in the present disclosure, “comprising at least one of a, b, or c” may mean “comprising only a, comprising only b, comprising only c, or comprising a combination of two or more (comprising a and b, comprising b and c, comprising a and c, or comprising all of a, b, and c).”

[0270] Meanwhile, although specific embodiments have been described in the detailed description of the present disclosure, it is understood that various modifications are possible within the scope of the present disclosure. Therefore, the scope of the present disclosure should not be limited to the described embodiments, but should be defined by the claims set forth below as well as equivalents thereof.

[0271] In the present disclosure, the term “if” will be understood, depending on the context, to mean “when, upon,” “in response to a decision,” or “in response to a detection.” Similarly, “when decided to,” or “when [mentioned condition or event] is detected” will be understood, optionally, to mean “when decided,” or “in response to a decision,” “when [mentioned condition or event] is detected,” or “in response to a detection.”

[0272] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit (or processing circuit) may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

[0273] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or instruct the processing unit independently or collectively. Software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0274] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may continuously store a program executable by a computer, or temporarily store it for execution or download. Additionally, the medium may be various recording or storage means in the form of a single or several hardware combined, and may not be limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Additionally, other examples of media may include recording or storage media managed by an app store that distributes applications or a site or server that supplies or distributes various other software.

[0275] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

Claims

1. In an electronic device, A display including a touchscreen; A first camera comprising a first image sensor including a plurality of sensor pixels arranged along a plurality of pixel lines; At least one processor; and It includes memory for storing at least one computer program that includes computer-executable instructions, and The first image sensor above is: Microlens array including multiple microlenses, A color filter array including multiple color channels, and It includes a light receiving unit comprising a plurality of light receiving elements arranged to correspond to one of the plurality of micro lenses and constituting the sensor pixels, and The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Control the first image sensor to crop image data configured based on information obtained from first sensor pixels corresponding to a first region of interest (RoI) among the plurality of sensor pixels and output it to the at least one processor, and Receive touch input through the above touchscreen, and Based on the above touch input, a second region of interest is determined in which at least one of the location or area is different relative to the first region of interest, and An electronic device that controls the first image sensor to crop image data configured based on information obtained from second sensor pixels corresponding to the second region of interest among the plurality of sensor pixels and output it to the at least one processor.

2. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Acquiring movement information regarding the movement of the above electronic device, and In the first shooting mode, a third region of interest corresponding to a position that compensates for the movement of the electronic device based on the movement information is determined, and In the first shooting mode, image data is acquired from at least some of the plurality of sensor pixels of the first image sensor that correspond to the third region of interest, and In the above first shooting mode, the first shooting mode is changed to a second shooting mode based on whether a specified condition is satisfied, and In the above second shooting mode, the touch input is received, and In the second shooting mode above, the second region of interest is determined based at least partially on the touch input regardless of the motion information, and An electronic device that acquires image data corresponding to the second region of interest from the first image sensor in the second shooting mode.

3. In Claim 2, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Receiving user input related to zoom magnification through the above touchscreen, and It determines whether the zoom magnification corresponding to the above user input is greater than or equal to a first threshold, and Based on the fact that the above zoom magnification is greater than or equal to a threshold, a user interface is displayed that allows selecting the second shooting mode through the display, and An electronic device that determines that the specified condition is satisfied based on receiving an input selecting the second shooting mode through the user interface.

4. In Claim 2, The first image sensor above is: In the first output mode, image data having a first resolution is read out from one area of ​​the light receiving unit, and In the second output mode, image data having a second resolution higher than the first resolution is read from one area of ​​the light receiving unit, and An electronic device configured to output image data based on the second output mode while the electronic device operates based on the second shooting mode.

5. In Claim 4, An electronic device configured such that, in the first output mode, the first image sensor outputs a single pixel value by binning the outputs of two or more light-receiving elements arranged adjacent to each other and corresponding to one color channel or one microlens.

6. In Claim 4, The first image sensor is an electronic device configured to output one pixel value corresponding to one light-receiving element in the second output mode.

7. In Claim 2, It further includes a motion sensor that detects the above-mentioned movement information, and The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Determining whether the magnitude of the motion information detected through the motion sensor is greater than or equal to a second threshold, In the second shooting mode above, if the magnitude of the motion information is less than the second threshold, the second region of interest is determined based on the touch input, and An electronic device that determines the second region of interest based on the motion information and the touch input when the magnitude of the motion information in the second shooting mode is greater than or equal to the second threshold.

8. In Claim 2, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Based on the image obtained through the first image sensor, a preview image is displayed through the display, and Identify whether an object to be tracked exists within the above preview video, and An electronic device that determines the second region of interest based on the location of the object and the touch input in the second shooting mode, based on the identification that the object to be tracked exists.

9. In Claim 2, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: In the second shooting mode above, pixel values ​​are read from first pixel lines including the second region of interest among the plurality of pixel lines based on the fact that the size of the motion information corresponds to the first size, and An electronic device that, in the second shooting mode, reads pixel values ​​from a number of second pixel lines greater than the number of first pixel lines among a plurality of pixel lines, based on the fact that the size of the motion information corresponds to a second size larger than the first size.

10. In claim 9, the second camera further comprises a second image sensor that supports a second field of view corresponding to an angle wider than the angle of the first field of view supported by the first camera, and The above instructions are executed individually or collectively by the at least one processor, and the electronic device: In the second shooting mode above, an image is acquired from the second camera based on the fact that the size of the motion information corresponds to a third size larger than the second size, and An electronic device that displays a preview image through the display based on an image obtained from the second camera.

11. A method of operating an electronic device including a first camera including a first image sensor, wherein An operation to control the first image sensor to crop and output image data configured based on information obtained from a region corresponding to a first region of interest within the light receiving part of the first image sensor; The operation of receiving touch input through the touchscreen of the above electronic device; An operation of determining a second region of interest in which at least one of the location or area is different for the first region of interest based on the touch input; and A method comprising controlling a first image sensor to crop and output image data configured based on information obtained from a region corresponding to the second region of interest within the light receiving unit.

12. In Claim 11, An operation to acquire movement information regarding the movement of the above electronic device; In a first shooting mode, an operation of determining a third region of interest corresponding to a position that compensates for the movement of the electronic device based on the movement information; In the first shooting mode, an operation of acquiring image data from at least a portion corresponding to the third region of interest within the light receiving portion of the first image sensor; and The above first shooting mode further includes an operation of changing the first shooting mode to a second shooting mode based on whether a specified condition is satisfied. A method in which the operation of receiving the touch input and the operation of determining the second region of interest are performed in the second shooting mode.

13. In Claim 12, The operation of changing the above-mentioned first shooting mode to the above-mentioned second shooting mode is: The operation of receiving user input related to zoom magnification through the above touchscreen, An operation to determine whether the zoom magnification corresponding to the above user input is greater than or equal to a first threshold, An operation of displaying a user interface that allows selecting the second shooting mode through the touchscreen based on the fact that the above zoom magnification is greater than or equal to a threshold, and A method comprising determining that the specified condition is satisfied based on receiving an input selecting the second shooting mode through the user interface.

14. In Claim 12, The operation of determining the second region of interest above is, An operation to determine whether the magnitude of the movement information regarding the movement of the electronic device is greater than or equal to a second threshold, When the magnitude of the above motion information is greater than or equal to a second threshold, the operation of determining the second region of interest based on the touch input, and A method comprising determining the second region of interest based on the motion information and the touch input when the magnitude of the motion information is greater than or equal to a second threshold.

15. In a computer-readable non-transient recording medium, when an electronic device including a first camera including a first image sensor is executed, the electronic device: An operation to control the first image sensor to crop and output image data configured based on information obtained from a region corresponding to a first region of interest within the light receiving part of the first image sensor; The operation of receiving touch input through the touchscreen of the above electronic device; An operation of determining a second region of interest in which at least one of the location or area is different for the first region of interest based on the touch input; and A recording medium having a computer program that records a method for controlling a first image sensor to crop and output image data configured based on information obtained from a region corresponding to the second region of interest within the light receiving unit.

Images