Data reading method of image sensor and image sensor thereof

Abstract

An electronic device according to various embodiments may include an image sensor and an image signal processor. The image sensor may include a pixel array including a plurality of pixels for detecting light, a row driver selecting a row of driven pixels from among the plurality of pixels within the pixel array, and a column driver selecting a column of driven pixels from among the plurality of pixels in the pixel array. The image sensor reads pixel values based on a specified pixel pattern by operation of the row driver and the column driver, and outputs image data acquired based on the read pixel values to the image signal processor. The pixels included in the pixel pattern may be arranged in a plurality of rows and a plurality of columns in the pixel array.

Description

Technical Field

[0001] The present disclosure relates to a method for reading out data of an image sensor and an image sensor thereof. Background Art

[0002] The image sensor reads out pixel values ​​from pixels for detecting light to create image data. According to a scheme of driving a shutter, a method of reading out pixel values ​​may be classified into a rolling shutter scheme and a global shutter scheme.

[0003] The rolling shutter scheme may refer to a scheme that sequentially repeats the following operations: resetting pixels in an image sensor for configuring image data in row units, exposing the pixels, and then reading out pixel values ​​of pixels included in the corresponding row in row units. The global shutter scheme may refer to a scheme that resets all pixels in an array, exposes the pixels, and then sequentially reads out pixel values ​​of pixels belonging to one row in row units. Both the rolling shutter scheme and the global shutter scheme are schemes in which an image sensor reads out pixel values ​​in row units.

[0004] The foregoing information may be provided as related art to assist in understanding the present disclosure. No claim or determination is made as to whether any of the foregoing is applicable as prior art related to the present disclosure. Summary of the invention

[0005] Solution to the problem The electronic device according to the embodiment may include an image sensor and an image signal processor. The image sensor may include: a pixel array including a plurality of pixels configured to detect light, a row driver configured to select a row of pixels to be driven from a plurality of pixels in the pixel array, and a column driver configured to select a column of pixels to be driven from a plurality of pixels in the pixel array. The image sensor may be configured to read out pixel values ​​based on a specified pixel pattern through the operation of the row driver and the column driver, and output image data acquired based on the pixel values ​​to the image signal processor. The specified pixel pattern may include pixels in a plurality of rows and a plurality of columns in the pixel array.

[0006] A method of operating an electronic device including an image sensor including a pixel array, the pixel array including a plurality of pixels, the method may include: reading out pixel values ​​based on a pixel pattern, the pixel pattern specifying rows and columns of pixels to be driven among pixels arranged into a plurality of rows and a plurality of columns in the pixel array. The method may include creating image data based on the readout pixel values. The method may include: outputting the image data to an exterior of the image sensor.

[0007] The image sensor according to the embodiment may include: a pixel array including a plurality of pixels configured to detect light; a row driver driving the plurality of pixels in units of rows; and a column driver driving the plurality of pixels in units of columns. The image sensor may be configured to read out pixel values ​​based on a pixel pattern specified by the operation of the row driver and the column driver, and output image data acquired based on the read-out pixel values ​​to the outside of the image sensor. The pixels included in the pixel pattern may be arranged to a plurality of rows and a plurality of columns of the pixel array. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram illustrating an electronic device in a network environment according to various embodiments; Figure 2 is a block diagram illustrating a camera module according to various embodiments; Figure 3 An example of the structure of an image sensor according to an embodiment is shown; Figure 4 is a block diagram showing a structure of an electronic device including an image sensor according to an embodiment; Figure 5 The operation of an image sensor reading out pixel values ​​and creating image data according to an embodiment is shown; Figure 6 is a diagram for explaining an operation of reading out pixel values ​​over time in an image sensor according to an embodiment; Figure 7 shows a structure of an image sensor for outputting image data according to an embodiment; and Figure 8 is a flowchart illustrating a process of operating an image sensor according to an embodiment. DETAILED DESCRIPTION

[0009] An image sensor (e.g., a complementary metal oxide semiconductor (CMOS) image sensor) can read out pixel values ​​of pixels included in the image sensor in units of lines (e.g., horizontal rows). In order to make the image acquired by the image sensor have uniform quality in the entire area, the environment in which the image sensor is driven should be constant when each line is read out. For example, when the voltage level used to drive the sensor when reading out the pixel value included in the first line is different from the voltage level used to drive the sensor when reading out the pixel value included in another row or when noise is included in the power source, the pixel value may have different offsets. In particular, since a high gain value is applied to improve the signal brightness in a low illumination environment, the offset of the pixel value can be significantly amplified. Therefore, the noise generated in units of lines may be amplified in the image due to the offset signal. Even if the noise generated in units of lines is post-processed in an image signal processor, the noise may be recognized as a horizontal pattern and may therefore be enhanced by image processing, which may result in degradation of the quality of the image finally acquired.

[0010] Therefore, there may be a need for an image sensor capable of acquiring image data such that noise is not generated in a row or a column of the image data created by the image sensor, and a method of operating the same.

[0011] Advantages obtained in the present disclosure are not limited to the aforementioned advantages, and other advantages not mentioned herein may be clearly understood from the following description by those skilled in the art to which the present disclosure pertains.

[0012] Figure 1 1 is a block diagram illustrating an electronic device 101 in a network environment 100 according to various embodiments. Figure 1, the electronic device 101 in the network environment 100 may communicate with the electronic device 102 via the first network 198 (e.g., a short-range wireless communication network), or communicate with at least one of the electronic device 104 or the server 108 via the second network 199 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 via the server 108. According to an embodiment, the electronic device 101 may include a processor 120, a memory 130, an input module 150, a sound output module 155, a display module 160, an audio module 170, a sensor module 176, an interface 177, a connection terminal 178, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a user identification module (SIM) 196, or an antenna module 197. In some embodiments, at least one of the above components (e.g., the connection terminal 178) may be omitted from the electronic device 101, or one or more other components may be added to the electronic device 101. In some embodiments, some of the above-described components (eg, sensor module 176, camera module 180, or antenna module 197) may be implemented as a single integrated component (eg, display module 160).

[0013] The processor 120 may run, for example, software (e.g., program 140) to control at least one other component (e.g., hardware component or software component) of the electronic device 101 connected to the processor 120, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, the processor 120 may store a command or data received from another component (e.g., sensor module 176 or communication module 190) in the volatile memory 132, process the command or data stored in the volatile memory 132, and store the resultant data in the non-volatile memory 134. According to an embodiment, the processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)) or an auxiliary processor 123 (e.g., a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is independent of or combined with the main processor 121 in operation. For example, when the electronic device 101 includes a main processor 121 and an auxiliary processor 123, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or adapted to be dedicated to a specific function. The auxiliary processor 123 may be implemented separately from the main processor 121, or as part of the main processor 121.

[0014] When the main processor 121 is in an inactive (e.g., sleep) state, the auxiliary processor 123 (rather than the main processor 121) may control at least some of the functions or states related to at least one component among the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190), or when the main processor 121 is in an active state (e.g., running an application), the auxiliary processor 123 may control at least some of the functions or states related to at least one component among the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) together with the main processor 121. According to an embodiment, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., a camera module 180 or a communication module 190) that is functionally related to the auxiliary processor 123. According to an embodiment, the auxiliary processor 123 (e.g., a neural processing unit) may include a hardware structure dedicated to artificial intelligence model processing. The artificial intelligence model may be generated through machine learning. For example, such learning may be performed by the electronic device 101 where the artificial intelligence is executed or via a separate server (e.g., server 108). The learning algorithm may include, but is not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, for example. The artificial intelligence model may include multiple artificial neural network layers. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q network or a combination of two or more thereof, but is not limited thereto. Additionally or optionally, the artificial intelligence model may include a software structure in addition to a hardware structure.

[0015] The memory 130 may store various data used by at least one component of the electronic device 101 (e.g., the processor 120 or the sensor module 176). The various data may include, for example, software (e.g., the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a nonvolatile memory 134.

[0016] The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system (OS) 142 , middleware 144 , or applications 146 .

[0017] The input module 150 may receive commands or data to be used by other components (e.g., the processor 120) of the electronic device 101 from outside (e.g., a user) of the electronic device 101. The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus).

[0018] The sound output module 155 can output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records. The receiver can be used to receive incoming calls. Depending on the embodiment, the receiver can be implemented as a separate part from the speaker, or as part of the speaker.

[0019] The display module 160 may visually provide information to the outside (eg, user) of the electronic device 101. The display device 160 may include, for example, a display, a holographic device, or a projector and a control circuit for controlling a corresponding one of the display, the holographic device, and the projector.

[0020] According to an embodiment, the display module 160 may include a touch sensor adapted to detect a touch or a pressure sensor adapted to measure the intensity of a force caused by a touch.

[0021] The audio module 170 may convert sound into an electrical signal, or vice versa. According to an embodiment, the audio module 170 may obtain sound via the input module 150, or output sound via the sound output module 155 or an earphone of an external electronic device (e.g., electronic device 102) directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0022] The sensor module 176 may detect an operating state (e.g., power or temperature) of the electronic device 101 or an environmental state (e.g., a state of a user) outside the electronic device 101, and then generate an electrical signal or a data value corresponding to the detected state. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0023] The interface 177 may support one or more specific protocols to be used to connect the electronic device 101 directly (e.g., wired) or wirelessly with an external electronic device (e.g., the electronic device 102). According to an embodiment, the interface 177 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0024] The connection end 178 may include a connector, wherein the electronic device 101 can be physically connected to an external electronic device (e.g., the electronic device 102) via the connector. According to an embodiment, the connection end 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0025] The haptic module 179 may convert the electric signal into mechanical stimulation (eg, vibration or motion) or electric stimulation that can be recognized by the user via his sense of touch or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electric stimulator.

[0026] The camera module 180 may capture still images or moving images. According to an embodiment, the camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0027] The power management module 188 may manage power supply to the electronic device 101. According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0028] The battery 189 may power at least one component of the electronic device 101. According to an embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0029] The communication module 190 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. The communication module 190 may include one or more communication processors capable of operating independently from the processor 120 (e.g., an application processor (AP)) and supporting direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules can communicate with an external electronic device via a first network 198 (e.g., a short-range communication network such as Bluetooth, Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a traditional cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules may be implemented as a single component (e.g., a single chip), or may be implemented as multiple components (e.g., multiple chips) separated from each other. The wireless communication module 192 may identify and authenticate the electronic device 101 in a communication network (such as the first network 198 or the second network 199) using user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the user identification module 196.

[0030] The wireless communication module 192 can support 5G networks after 4G networks and next-generation communication technologies (e.g., new radio (NR) access technology). NR access technology can support enhanced mobile broadband (eMBB), massive machine type communication (mMTC), or ultra-reliable low-latency communication (URLLC). The wireless communication module 192 can support high-frequency bands (e.g., millimeter wave bands) to achieve, for example, high data transmission rates. The wireless communication module 192 can support various technologies for ensuring performance on high-frequency bands, such as, for example, beamforming, massive multiple-input multiple-output (massive MIMO), full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. The wireless communication module 192 can support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., a second network 199). According to an embodiment, the wireless communication module 192 may support a peak data rate for implementing eMBB (e.g., 20 Gbps or greater), loss coverage for implementing mMTC (e.g., 164 dB or less), or U-plane delay for implementing URLLC (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less for a round trip).

[0031] The antenna module 197 may transmit or receive a signal or power to or from the outside of the electronic device 101 (e.g., an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a radiation element formed of a conductive material or a conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, the antenna module 197 may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as the first network 198 or the second network 199) may be selected from the plurality of antennas by, for example, the communication module 190 (e.g., the wireless communication module 192). A signal or power may then be transmitted or received between the communication module 190 and the external electronic device via the selected at least one antenna. According to an embodiment, another component (e.g., a radio frequency integrated circuit (RFIC)) other than the radiation element may be additionally formed as a part of the antenna module 197.

[0032] According to various embodiments, the antenna module 197 may form a millimeter wave antenna module. According to an embodiment, the millimeter wave antenna module may include a printed circuit board, a radio frequency integrated circuit (RFIC), and a plurality of antennas (e.g., array antennas), wherein the RFIC is disposed on a first surface (e.g., bottom surface) of the printed circuit board, or adjacent to the first surface and capable of supporting a designated high frequency band (e.g., millimeter wave band), and the plurality of antennas are disposed on a second surface (e.g., top surface or side surface) of the printed circuit board, or adjacent to the second surface and capable of transmitting or receiving signals of the designated high frequency band.

[0033] At least some of the above components may be connected to each other via an inter-peripheral communication scheme (e.g., a bus, a general purpose input output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) and communicatively transfer signals (e.g., commands or data) therebetween.

[0034] According to an embodiment, a command or data may be sent or received between the electronic device 101 and the external electronic device 104 via the server 108 connected to the second network 199. Each of the electronic device 102 or the electronic device 104 may be a device of the same type as the electronic device 101, or a device of a different type from the electronic device 101. According to an embodiment, all or some operations to be executed in the electronic device 101 may be executed in one or more of the external electronic device 102, the external electronic device 104, or the server 108. For example, if the electronic device 101 should automatically execute a function or service or should execute a function or service in response to a request from a user or another device, the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service instead of executing the function or service, or the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service in addition to executing the function or service. The one or more external electronic devices receiving the request may execute at least part of the requested function or service, or execute another function or another service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may provide the result as at least a partial reply to the request with or without further processing the result. To this end, cloud computing technology, distributed computing technology, mobile edge computing (MEC) technology, or client-server computing technology, for example, may be used. The electronic device 101 may use, for example, distributed computing or mobile edge computing to provide ultra-low latency services. In an embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. The server 108 may be an intelligent server using machine learning and / or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology or IoT-related technologies.

[0035] The electronic device according to various embodiments may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. According to an embodiment of the present disclosure, the electronic device is not limited to those electronic devices described above.

[0036] It should be understood that the various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, but include various changes, equivalent forms or alternative forms for the corresponding embodiments. For the description of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that the nouns in the singular form corresponding to the term may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include any one or all possible combinations of the items listed together with the corresponding one of the multiple phrases. As used herein, terms such as "1st" and "2nd" or "first" and "second" may be used to simply distinguish the corresponding component from another component, and do not limit the component in other aspects (e.g., importance or order). It will be understood that if an element (e.g., a first element) is referred to as being “coupled with another element (e.g., the second element)”, “coupled to another element (e.g., the second element)”, “connected with another element (e.g., the second element)”, or “connected to another element (e.g., the second element)” with or without the terms “operably” or “communicatively” being used, it means that the element may be directly (e.g., wired) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0037] As used in connection with various embodiments of the present disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "portion," or "circuit"). A module may be a single integrated component adapted to perform one or more functions or a minimum unit or portion of the single integrated component. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0038] The various embodiments described herein may be implemented as software (e.g., program 140) including one or more instructions stored in a storage medium (e.g., internal memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). For example, under the control of a processor, a processor (e.g., processor 120) of the machine (e.g., electronic device 101) may call at least one of the one or more instructions stored in the storage medium and execute the at least one instruction with or without the use of one or more other components. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Among them, the term "non-transitory" only means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but the term does not distinguish between data being semi-permanently stored in the storage medium and data being temporarily stored in the storage medium.

[0039] According to an embodiment, the method according to various embodiments of the present disclosure may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be released in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be downloaded via an application store (e.g., Play Store TM ) the computer program product may be published (e.g., downloaded or uploaded) online, or the computer program product may be distributed (e.g., downloaded or uploaded) directly between two user devices (e.g., smart phones). If published online, at least part of the computer program product may be temporarily generated, or at least part of the computer program product may be at least temporarily stored in a machine-readable storage medium (such as a memory of a manufacturer's server, a server of an application store, or a forwarding server).

[0040] According to various embodiments, each of the above-mentioned components (e.g., a module or a program) may include a single entity or multiple entities, and some of the multiple entities may be separately arranged in different components. According to various embodiments, one or more of the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., a module or a program) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each of the multiple components in the same or similar manner as a corresponding one of the multiple components performing one or more functions before integration. According to various embodiments, the operations performed by a module, a program or another component may be performed sequentially, in parallel, repeatedly or in a heuristic manner, or one or more of the operations may be run or omitted in a different order, or one or more other operations may be added.

[0041] Figure 2 2 is a block diagram showing a camera module 180 according to various embodiments. Figure 2 , the camera module 180 may include a lens assembly 210, a flash 220, an image sensor 230, an image stabilizer 240, a memory 250 (e.g., a buffer memory), or an image signal processor 260. The lens assembly 210 may collect light emitted or reflected from an object whose image is to be captured. The lens assembly 210 may include one or more lenses. According to an embodiment, the camera module 180 may include a plurality of lens assemblies 210. In this case, the camera module 180 may form, for example, a dual camera, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies 210 may have the same lens properties (e.g., viewing angle, focal length, autofocus, f-number, or optical zoom), or at least one lens assembly may have one or more lens properties different from those of another lens assembly. The lens assembly 210 may include, for example, a wide-angle lens or a telephoto lens.

[0042] The flash 220 may emit light, wherein the emitted light is used to enhance light reflected from an object. According to an embodiment, the flash 220 may include one or more light emitting diodes (LEDs) (e.g., red, green, blue (RGB) LEDs, white LEDs, infrared (IR) LEDs, or ultraviolet (UV) LEDs) or xenon lamps. The image sensor 230 may acquire an image corresponding to an object by converting light emitted or reflected from an object and transmitted through the lens assembly 210 into an electrical signal. According to an embodiment, the image sensor 230 may include one image sensor selected from a plurality of image sensors having different properties (e.g., an RGB sensor, a black and white (BW) sensor, an IR sensor, or a UV sensor), a plurality of image sensors having the same properties, or a plurality of image sensors having different properties. Each image sensor included in the image sensor 230 may be implemented using, for example, a charge coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor.

[0043] The image stabilizer 240 may move the image sensor 230 or at least one lens included in the lens assembly 210 in a specific direction, or control an operational property of the image sensor 230 (e.g., adjust the readout timing) in response to the movement of the camera module 180 or the electronic device 101 including the camera module 180. In this way, at least a portion of the negative effects (e.g., image blur) generated due to the movement of the image being captured is allowed to be compensated. According to an embodiment, the image stabilizer 240 may sense such movement of the camera module 180 or the electronic device 101 using a gyroscope sensor (not shown) or an acceleration sensor (not shown) arranged inside or outside the camera module 180. According to an embodiment, the image stabilizer 240 may be implemented as, for example, an optical image stabilizer. The memory 250 may at least temporarily store at least a portion of the image acquired via the image sensor 230 for subsequent image processing tasks. For example, if multiple images are captured quickly or image capture is delayed due to shutter lag, the acquired original image (e.g., Bayer pattern image, high-resolution image) may be stored in the memory 250, and its corresponding copy image (e.g., low-resolution image) may be previewed via the display module 160. Then, if a specified condition is satisfied (e.g., by a user's input or a system command), at least a portion of the original image stored in the memory 250 may be acquired and processed by, for example, the image signal processor 260. According to an embodiment, the memory 250 may be configured as at least a portion of the memory 130, or the memory 250 may be configured as a separate memory that operates independently of the memory 130.

[0044] The image signal processor 260 may perform one or more image processing on an image acquired via the image sensor 230 or an image stored in the memory 250. The one or more image processing may include, for example, depth map generation, three-dimensional (3D) modeling, panorama generation, feature point extraction, image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Additionally or alternatively, the image signal processor 260 may perform control (e.g., exposure time control or readout timing control) on at least one of the components included in the camera module 180 (e.g., the image sensor 230). The image processed by the image signal processor 260 may be stored back to the memory 250 for further processing, or the image may be provided to an external component (e.g., the memory 130, the display module 160, the electronic device 102, the electronic device 104, or the server 108) outside the camera module 180. According to an embodiment, the image signal processor 260 may be configured as at least a part of the processor 120, or the image signal processor 260 may be configured as a separate processor that operates independently of the processor 120. If the image signal processor 260 is configured as a separate processor from the processor 120, at least one image processed by the image signal processor 260 may be displayed as it is by the processor 120 via the display module 160, or may be displayed after being further processed.

[0045] According to an embodiment, the electronic device 101 may include a plurality of camera modules 180 having different properties or functions. In this case, at least one camera module 180 of the plurality of camera modules 180 may form, for example, a wide-angle camera, and at least another camera module 180 of the plurality of camera modules 180 may form a telephoto camera. Similarly, at least one camera module 180 of the plurality of camera modules 180 may form, for example, a front camera, and at least another camera module 180 of the plurality of camera modules 180 may form a rear camera.

[0046] Figure 3 An example of the structure of the image sensor 230 according to the embodiment is shown.

[0047] The image sensor 230 according to the embodiment may include a pixel array 310 in which a plurality of pixels for detecting light are arranged, a row driver 320, and a column driver 330. The row driver 320 may select at least one row in the pixel array arranged in the pixel array 310. The row driver 320 may include a circuit for driving pixels arranged to the selected at least one row. The circuit for driving pixels arranged to the at least one row may include a row selection switch coupled between the pixels of a row and the row driver 320. The row driver 320 may control the row selection switch to drive the pixels coupled to the row selection switch. The column driver 330 may select at least one column in the pixel array arranged in the pixel array 310. The column driver 330 may include a circuit for driving pixels in a column selected from the pixels of the row driven by the row driver 320. The circuit for driving the pixels of the selected column may include a column selection switch controlled by the column driver 330. The column selection switch may be coupled to the column driver 330 and each pixel of a column. The column driver 330 may control the column selection switch to operate the pixels of the selected column.

[0048] In an embodiment, the image sensor 230 may read out pixel values ​​by operating the row driver 320 and the column driver 330 based on a specified pixel pattern. The pixel pattern may be configured so that the pixels whose pixel values ​​are to be read out are not arranged to a row. In order that the pixels read out by a single readout operation are not arranged to a row in an image, the image sensor 230 may perform a readout operation based on a pixel pattern for allowing pixels included in a plurality of rows to be read out. Alternatively, in order that the readout pixels are not arranged to adjacent positions in the same row or column in an image, the image sensor 230 may arrange the pixels included in the pixel pattern to a plurality of rows and a plurality of columns in the pixel array 310.

[0049] The readout operation may be performed based on a pixel pattern consisting of pixels arranged to non-adjacent columns, so that even if noise is generated in some of the readout operations, the noise does not appear in the form of lines in the image data 340 output based on the readout pixel values.

[0050] For example, refer to Figure 3 , when the pixel values ​​are read out in units of rows as in the rolling shutter scheme, the image sensor 230 may read out the pixel values ​​of the pixels included in the first pixel group 311 arranged to one column by one readout operation. Thereafter, the image sensor 230 may read out the pixel values ​​of the pixels included in the second pixel group 312 arranged to another column by another readout operation. When noise (for example, an offset due to the sensor voltage) is generated when the second pixel group 312 is read out, a horizontal line may appear on the image data 340. When the noise is provided in the form of a horizontal line, even if a post-processing operation is performed on the image data 340, the image quality may not be sufficiently improved.

[0051] On the contrary, refer to Figure 3 According to an embodiment, the image sensor 230 may read out the pixel values ​​of the pixels included in the third pixel group 313 included in the first array pattern through one readout operation. The image sensor 230 may read out the pixel values ​​of the pixels included in the fourth pixel group 314 included in the second array pattern through another readout operation. Even if noise is generated when the pixel values ​​of the pixels included in the fourth pixel group 314 are read out, the noise may be included at a low level in the pixels around the pixels generating the noise, or the noise may not be included. Therefore, the noise included in the image data 340 may have low visibility. In addition, when the image data 340 is corrected by a post-processing operation, the noise may be removed more effectively. The group of pixels included in the pixel pattern and activated may be a predetermined pattern. For example, the image sensor 230 may read out the pixel values ​​based on the pixel pattern configured when the image sensor 230 is manufactured. The pixel pattern according to the embodiment may be specified by a user, or may be a programmable pattern. Alternatively, the pixel pattern may be determined according to the shooting environment. Alternatively, the pixel pattern may include pixels randomly selected when shooting an image. For example, when selecting to include the pixels of the first column in the pixel pattern among the pixels of the first row in the pixel array, the pixels of the second column can be selected so that the pixels arranged to the remaining rows except the first row are included in the pixel pattern. However, the present disclosure is not limited to this. For example, with respect to a single pixel pattern, the row of pixels read out in the second column can be randomly selected, regardless of the row of pixels to be read out from the first column. That is, the pixels included in at least some columns of the pixel array can be arranged to the same row. However, in this case, the pixels included in different pixel patterns can be selected to be non-overlapping.

[0052] For example, the pixels included in the pixel pattern may be selected by the processor 120 or the image signal processor 420. For example, the pixels included in the pixel pattern may be selected by the row driver 320 and the column driver 330. For example, the pixel pattern may be a pixel pattern stored in the memory 410.

[0053] In an embodiment, the image sensor 230 may output image data 340 based on pixel values ​​acquired through a plurality of readout operations. For example, the image sensor 230 may store the pixel values ​​acquired through the readout operation in a line buffer memory or a memory included in the image sensor 230. The image sensor 230 may output the image data 340 created by combining the pixel values ​​stored in the line buffer to the outside of the image sensor 230 through an interface (e.g., a mobile industry processor interface (MIPI)). For example, the image sensor 230 may output the image data 340 to an image signal processor (e.g., Figure 2 image signal processor 260).

[0054] Figure 4 is a block diagram showing a structure of an electronic device 101 including an image sensor 230 according to an embodiment. Figure 5 The operation of the image sensor 230 reading out pixel values ​​and creating image data according to an embodiment is shown.

[0055] The electronic device 101 according to the embodiment may include a camera module 180 including an image sensor 230 and an image signal processor 420 (eg, Figure 2 The image sensor 230 according to the embodiment may include a pixel array 310 on which a plurality of pixels are arranged, a row driver 320, a column driver 330, and a memory 410. The image sensor 230 may drive the pixels selected in the pixel array 310 by the row driver 320 and the column driver 330 to read out the pixel values ​​of the pixels belonging to the pixel group corresponding to the pixel pattern. The image sensor 230 may store the read-out pixel values ​​in the memory 410. The memory 410 may be a buffer memory for temporarily storing the pixel values ​​to be read out until the pixel values ​​are output as image data. The memory 410 according to the embodiment may include information about the pixel pattern.

[0056] The image sensor 230 according to the embodiment may be configured to acquire image data by restoring an array pattern based on pixel values ​​stored in the memory 410 so as to correspond to the pixel array 310. The image sensor 230 may output the acquired image data to the image signal processor 420.

[0057] For example, refer to Figure 5 , Figure 5The process of creating image data using a pixel array 500 composed of a 4×4 Bayer pattern is shown. The image sensor 230 may acquire pixel values ​​of a first pixel group 511 corresponding to a first pattern through a first readout operation. The acquired pixel values ​​of the first pixel group 511 may be stored in the memory 410. The image sensor 230 may acquire pixel values ​​of a second pixel group 512 corresponding to a second pattern through a second readout operation. The acquired pixel values ​​of the second pixel group 512 may be stored in the memory 410. The image sensor 230 may acquire pixel values ​​of a third pixel group 513 corresponding to a third pattern through a third readout operation. The acquired pixel values ​​of the third pixel group 513 may be stored in the memory 410. The image sensor 230 may acquire pixel values ​​of a fourth pixel group 514 corresponding to a fourth pattern through a fourth readout operation. The acquired pixel values ​​of the fourth pixel group 514 may be stored in the memory 410. Pixels in pixel array 500 may be restored by combining pixels included in first pixel group 511, second pixel group 512, third pixel group 513, and fourth pixel group 514 stored in memory 410. Thus, image sensor 230 may create image data 520 based on pixel values ​​stored in memory 410.

[0058] Since the memory 410 should generally be included in this structure to read out pixel values, even if the image sensor 230 according to the embodiment reads out pixel values ​​based on the array pattern and creates image data, there is no problem of increasing the size of the image sensor 230 to have the memory 410.

[0059] The image signal processor 420 according to the embodiment may process one or more images of the image data. For example, the image signal processor 420 may reduce the noise generated in the image data by interpolating the pixel values ​​of the pixels generating the noise using the pixel values ​​of the adjacent pixels. The pixels whose pixel values ​​are read out by a single readout operation may be arranged so as not to overlap with each other. Since the pixel values ​​of the adjacent pixels of the pixel generating the noise may not include the noise, the visibility of the noise generated in the image may be reduced by interpolation.

[0060] The electronic device 101 according to an embodiment may include at least one processor 120 operably coupled to the camera module 180. The at least one processor 120 may control the camera module 180 to select a pixel pattern through which the image sensor 230 reads out an image according to a shooting condition when the electronic device 101 shoots an image through the camera module 180. The shooting condition may include an operating state of the electronic device 101 (camera module 180) and / or a condition of the surrounding environment of the electronic device 101 when shooting an image. The operating state of the electronic device 101 may relate to, for example, a shooting mode configured to shoot an image, an internal temperature of the electronic device 101, or power supplied to the electronic device 101.

[0061] The condition of the surrounding environment of the electronic device 101 may relate to, for example, the illumination around the electronic device 101. The electronic device 101 according to an embodiment may further include an illumination sensor 476 (eg, Figure 1 The at least one processor 120 may acquire information related to the illuminance value through the illuminance sensor 476. However, the present disclosure is not limited thereto. For example, the at least one processor 120 may acquire information related to the illuminance value from information about light detected by the image sensor 230.

[0062] According to the embodiment, at least one processor 120 of the electronic device 101 may determine whether to apply the pixel pattern used by the image sensor 230 to read out the pixel value based on the shooting conditions, or may determine the type of the pixel pattern. For example, based on the acquired illuminance value, when the illuminance value is greater than or equal to the threshold value, the at least one processor 120 may control the camera module 180 so that the image sensor 230 reads out the pixel value in a single row unit (for example, Figure 3 Based on the acquired illuminance value, when the illuminance value is less than the threshold value, at least one processor 120 may control the camera module 180 to use a predetermined pixel pattern (e.g., Figure 3 The pixel values ​​are read out in units of the third pixel group 313 and the fourth pixel group 314 .

[0063] According to an embodiment, at least one processor 120 or an image signal processor 420 may analyze the image data output from the image sensor 230. At least one processor 120 or an image signal processor 420 may determine a pattern based on a result of analyzing the image data. At least one processor 120 or an image signal processor 420 may be configured to update information about a pixel pattern stored in the memory 410 based on the determined pattern. The image sensor 230 may read out a pixel value based on the updated pixel pattern.

[0064] Figure 6 2 is a diagram for explaining an operation of reading out pixel values ​​over time by the image sensor 230 according to the embodiment.

[0065] The image sensor 230 according to the embodiment may perform a first readout operation 611. The image sensor 230 may reset the pixels of each row through the row driver 320, and may continue to perform exposure of the reset pixels. The image sensor 230 according to the embodiment may sequentially reset the pixels of each row and may start exposure. The image sensor 230 may selectively drive the pixels of the columns from which the pixel values ​​are to be read out from each row through the column driver 330 while driving each row. For example, during the first readout operation 611, the image sensor 230 may drive the pixels of the third column Col.3 of the pixels of the first row, the pixels of the fourth column Col.4 of the pixels of the second row, the pixels of the second column Col.2 of the pixels of the third row, and the pixels of the first column Col.1 of the pixels of the fourth row. The image sensor 230 may read out the pixel values ​​of the first pixel group 511 including the driven pixels.

[0066] In an embodiment, the image sensor 230 may reset the pixels of each row through the row driver 320, and may continue to perform exposure of the reset pixels. During the second readout operation 612, the image sensor 230 may selectively drive the pixels of the columns from which the pixel values ​​are to be read out from each row through the column driver 330 while driving each row. For example, during the second readout operation 612, the image sensor 230 may selectively drive the pixels of the second column Col.2 and the fourth column Col.4 of the pixels of the first row, the pixels of the third column Col.3 of the pixels of the second row, and the pixels of the first column Col.1 of the pixels of the third row. The image sensor 230 may read out the pixel values ​​of the second pixel group 512 including the driven pixels.

[0067] In an embodiment, there may be a time difference between the timing of reading out the first pixel group 511 and the timing of reading out the second pixel group 512. The time difference between the pixel groups may correspond to a value obtained by multiplying the number of lines of pixels to be read out in the sensor array by the readout time required to read out one line, and dividing the result by the number of pixel groups. Figure 4 The image signal processor 420 of the image sensor may correct the image data to reduce image distortion that occurs due to a difference in timing of reading out pixel values ​​of pixels included in the image data. Since the time difference between the timings of reading out the pixels included in the image data is based on the image sensor (e.g., Figures 2 to 4 The image sensor 230) is configured with predetermined values, so the image signal processor (eg, Figure 4 The image signal processor 420 may be configured to correct time differences between pixels in the image data.

[0068] Figure 7 A structure of an image sensor 230 for outputting image data according to an embodiment is shown.

[0069] In an embodiment, the image sensor 230 may include imaging elements 700 arranged in a two-dimensional grid pattern on an imaging surface. The imaging element 700 may include a photoelectric conversion element to form a pixel that outputs an electrical signal when receiving light. A control signal may be input to each of the imaging elements 700 through a horizontal control line and a vertical control line.

[0070] In an embodiment, the vertical control line may be coupled to the column driver 330. The horizontal control line may be coupled to the row driver 320. The row driver 320 and the column driver 330 may control the operation of the imaging element 700 based on the exposure control signal. For example, the row driver 320 and the column driver 330 may be coupled to a timing generator that receives an input of the exposure control signal from the outside.

[0071] The output pixel signal output from each imaging element 700 may be configured to pass through a switch for performing selection in units of columns. The pixel signal output from the pixel of the selected column may be input to a correlated double sampling (CDS) circuit 710. Reset noise may be suppressed in the CDS circuit 710 and input to an analog-to-digital converter (ADC) circuit 720. The analog signal may be converted into a digital signal in the ADC circuit 720, and the image data 340 may be output to the outside of the image sensor 230.

[0072] Figure 8 is a flowchart illustrating a process of operating the image sensor 230 according to an embodiment.

[0073] In operation 810, the image sensor 230 according to the embodiment may read out pixel values ​​based on a pixel pattern. The pixel pattern for reading out pixel values ​​may include a pixel array so that even if a signal containing noise is detected in some readout operations, the signal containing noise does not appear in the image data in the form of a line (e.g., a horizontal line). The image sensor 230 according to the embodiment may repeat the operation of reading out pixel values ​​multiple times based on different patterns to obtain pixel values ​​of the sensor array. In operation 810, the image sensor 230 may perform an operation of storing the readout pixel values ​​in the memory 410 included in the image sensor 230.

[0074] According to an embodiment, in operation 810, the image sensor 230 may determine a scheme for reading out pixel values ​​based on the illuminance value. For example, the image sensor 230 may read out pixel values ​​in units of a single row when the illuminance value is greater than or equal to a threshold value. The image sensor may read out pixel values ​​in units of a pixel pattern when the illuminance value is less than a threshold value.

[0075] In operation 820, the image sensor 230 according to the embodiment may create image data by arranging the read pixel values ​​into a pattern of an initial sensor array. The image sensor 230 according to the embodiment may acquire image data based on the data stored in the memory in the image sensor 230 in operation 810.

[0076] In operation 830, the image sensor 230 according to the embodiment may output the created image sensor to the outside of the image sensor 230. For example, the image sensor 230 may output the image data to the image signal processor 420.

[0077] The output image data may be corrected by interpolating pixel values ​​included in the image data to reduce noise included in the image data.

[0078] According to an electronic device (eg, Figure 1 or Figure 4 The electronic device 101 may include an image sensor (eg, Figure 2 , Figure 3 or Figure 4 image sensor 230) and an image signal processor (e.g., Figure 2 The image signal processor 260, Figure 4 image signal processor 420). Image sensor (e.g., Figure 2 , Figure 3 or Figure 4 The image sensor 230 may include: a pixel array (eg, Figure 3 or Figure 4 The pixel array 310, Figure 5 A pixel array 500 comprising a plurality of pixels for detecting light; a row driver (eg, Figure 3 , Figure 4 or Figure 7 row driver 320), from the pixel array (e.g., Figure 3 or Figure 4 The pixel array 310) selects a row of pixels to be driven from among a plurality of pixels in the pixel array 310; a column driver ( Figure 3 , Figure 4 or Figure 7 Column driver 330), from the pixel array (e.g., Figure 3 or Figure 4 The pixel array 310, Figure 5 A pixel array 500 of an image sensor (e.g., Figure 2 , Figure 3 or Figure 4 The image sensor 230 may be configured to be connected to the image sensor 230 through a row driver (eg, Figure 3 , Figure 4 or Figure 7 The row driver 320) and column driver operate based on a specified pixel pattern (e.g., Figure 5 511, 512, 513 and 514) read out pixel values; and send to an image signal processor (eg, Figure 2 The image signal processor 260, Figure 4 The image signal processor 240 outputs image data based on the read pixel values. Including in the pixel pattern (for example, Figure 5 The pixels in the pixel patterns 511, 512, 513 and 514) may be arranged into a pixel array (eg, Figure 3 or Figure 4 The pixel array 310, Figure 5 Multiple rows and multiple columns in the pixel array 500).

[0079] According to an embodiment, an image sensor (e.g., Figure 2 , Figure 3 or Figure 4 The image sensor 230 may also include a memory (eg, Figure 4 memory 410). Image sensor (e.g., Figure 2 , Figure 3 or Figure 4 The image sensor 230 may be configured to store the read pixel values ​​in a memory (eg, Figure 4 410), and based on the information stored in the memory (e.g., Figure 4 The image data is acquired by restoring the array pattern of pixel values ​​in the memory 410).

[0080] According to an embodiment, an image signal processor (eg, Figure 2 The image signal processor 260, Figure 4 The image signal processor 420 may be configured to acquire image data corrected by interpolating pixel values ​​included in the image data.

[0081] According to an electronic device (eg, Figure 1 or Figure 4 The electronic device 101 may further include an illumination sensor (eg, Figure 4 illuminance sensor 476) and operably coupled to an illuminance sensor (e.g., Figure 4 illumination sensor 476) and an image sensor (e.g., Figure 2 , Figure 3 or Figure 4 image sensor 230) and at least one processor (e.g., Figure 1 or Figure 4 At least one processor (e.g., Figure 1 or Figure 4 The processor 120 may be configured to detect the presence of an illuminance sensor (eg, Figure 4 The illumination sensor 476 of the image sensor 476 obtains the illumination value and controls the image sensor (eg, Figure 2 , Figure 3 or Figure 4 230), so that the image sensor (eg, Figure 2 , Figure 3 or Figure 4 The image sensor 230 determines a pixel pattern (eg, Figure 5 Pixel pattern 511, 512, 513 or 514).

[0082] According to an embodiment, at least one processor (e.g. Figure 1 or Figure 4 The processor 120 may be configured to control the image sensor (eg, Figure 2 , Figure 3 or Figure 4 ), so that when the illumination value is greater than or equal to the threshold, the image sensor (eg, Figure 2 , Figure 3 or Figure 4 The image sensor 230 of FIG. 1 reads out pixel values ​​in units of a single line, and controls the image sensor (eg, Figure 2 , Figure 3 or Figure 4 ), so that when the illumination value is less than the threshold, the image sensor (eg, Figure 2 , Figure 3 or Figure 4 The image sensor 230) has a pixel pattern (e.g., Figure 5 The pixel value is read out in units of pixel pattern 511, 512, 513 or 514).

[0083] According to an embodiment, an image sensor (e.g., Figure 2 , Figure 3 or Figure 4 The image sensor 230 may include a row driver coupled to the image sensor 230 (eg, Figure 3 , Figure 4 or Figure 7 The row driver 320) is connected to the pixel array (e.g., Figure 3 or Figure 4 The pixel array 310, Figure 5 A row select switch between pixels in a row of the pixel array 500), and a row select switch coupled to a column driver and a pixel array (eg, Figure 3 or Figure 4 The pixel array 310, Figure 5A plurality of column selection switches for each pixel in a column of a pixel array 500 of an image sensor (e.g., Figure 2 , Figure 3 or Figure 4 The image sensor 230 may be configured such that based on a pixel pattern (eg, Figure 5 Pixel patterns 511, 512, 513 and 514), a row driver (eg, Figure 3 , Figure 4 or Figure 7 A row driver 320) controls the row selection switches, and a column driver controls each of the plurality of column selection switches.

[0084] According to an embodiment, a pixel pattern (e.g., Figure 5 The pixel patterns 511, 512, 513 and 514) may be included in the pixel pattern (eg, Figure 5 A group of pixels selected from the pixel patterns 511, 512, 513 and 514).

[0085] According to an embodiment, an image sensor (e.g., Figure 2 , Figure 3 or Figure 4 The image sensor 230 may also include a memory (eg, Figure 4 410 ), which includes information about the pixel pattern (e.g., Figure 5 The pixel patterns 511, 512, 513 and 514) are provided.

[0086] According to an embodiment, at least one processor (e.g. Figure 1 or Figure 4 The processor 120 of the embodiment may be configured to analyze the image data and determine the pattern based on the result of analyzing the image data. Figure 1 or Figure 4 The processor 120 may be configured to store data stored in the memory (eg, Figure 4 410 ) in the memory 410 ) about the pixel pattern (eg, Figure 5 The information of the pixel patterns 511, 512, 513 and 514) is updated to the determined pattern.

[0087] According to an embodiment, the pixel pattern (eg, Figure 5 The pixels in the pixel pattern 511, 512, 513 and 514) may be a pixel array (eg, Figure 3 or Figure 4 The pixel array 310, Figure 5 Pixels that are not adjacent to each other in the pixel array 500).

[0088] One operation has an image sensor (e.g., Figure 2, Figure 3 or Figure 4 An electronic device (e.g., an image sensor 230) Figure 1 or Figure 4 The method of the electronic device 101), wherein the image sensor includes a pixel array (eg, Figure 3 or Figure 4 The pixel array 310, Figure 5 The pixel array 500 includes a plurality of pixels, and the method may include: based on a pixel pattern (eg, Figure 5 The method may include creating image data based on the read pixel values. The method may include outputting the image data to an image sensor (e.g., Figure 2 , 3 or 4 outside of the image sensor 230).

[0089] According to an embodiment, the reading step may include storing the read pixel values ​​in an image sensor (eg, Figure 2 , Figure 3 or Figure 4 The image sensor 230 includes a memory (eg, Figure 4 The step of creating the image data may include creating the image data based on the image data stored in the memory (e.g., Figure 4 The image data is acquired by restoring the array pattern of pixel values ​​in the memory 410).

[0090] The method according to the embodiment may further include acquiring image data corrected by interpolating pixel values ​​stored in the image data.

[0091] The method according to an embodiment may further include acquiring an illumination value. The reading step may include: when the illumination value is greater than or equal to a threshold, the image sensor (e.g., Figure 2 , Figure 3 or Figure 4 The image sensor 230 of FIG. 2 reads out pixel values ​​in units of a single line, and when the illumination value is less than a threshold value, the image sensor (eg, Figure 2 , Figure 3 or Figure 4 The image sensor 230) has a pixel pattern (e.g., Figure 5 The pixel value is read out in units of pixel patterns 511, 512, 513 and 514).

[0092] According to an embodiment, the pixel pattern (eg, Figure 5The pixels in the pixel pattern 511, 512, 513 and 514) may be a pixel array (eg, Figure 3 or Figure 4 The pixel array 310, Figure 5 Pixels that are not adjacent to each other in the pixel array 500).

[0093] According to an image sensor of an embodiment (eg, Figure 2 , Figure 3 or Figure 4 The image sensor 230 may include: a pixel array including a plurality of pixels for detecting light (eg, Figure 3 or Figure 4 The pixel array 310, Figure 5 A pixel array 500 of FIG. 5 ), a row driver (eg, Figure 3 , Figure 4 or Figure 7 A row driver 320 for driving a plurality of pixels in columns, and a column driver for driving a plurality of pixels in columns. Figure 2 , Figure 3 or Figure 4 The image sensor 230 may be configured based on a row driver (e.g., Figure 3 , Figure 4 or Figure 7 The row driver 320) and column driver operate to specify the pixel pattern (e.g., Figure 5 511, 512, 513, and 514) to read out pixel values, and output image data acquired based on the read pixel values ​​to an image sensor (eg, Figure 2 , Figure 3 or Figure 4 The image sensor 230 includes a pixel pattern (eg, Figure 5 The pixels in the pixel patterns 511, 512, 513 and 514) may be arranged into a pixel array (eg, Figure 3 or Figure 4 The pixel array 310, Figure 5 Multiple rows and multiple columns in the pixel array 500).

[0094] According to an image sensor of an embodiment (eg, Figure 2 , Figure 3 or Figure 4 The image sensor 230 may also include a memory (eg, Figure 4 memory 410). Image sensor (e.g., Figure 2 , Figure 3 or Figure 4 The image sensor 230 may be configured to store the read pixel values ​​in a memory (eg, Figure 4410), and based on the information stored in the memory (e.g., Figure 4 The image data is acquired by restoring the array pattern of pixel values ​​in the memory 410).

[0095] According to an image sensor of an embodiment (eg, Figure 2 , Figure 3 or Figure 4 The image sensor 230 may further include a row driver (eg, Figure 3 , Figure 4 or Figure 7 The row driver 320) is connected to the pixel array (e.g., Figure 3 or Figure 4 The pixel array 310, Figure 5 A row select switch between pixels in a row of the pixel array 500), and a row select switch coupled to a column driver and a pixel array (eg, Figure 3 or Figure 4 The pixel array 310, Figure 5 A plurality of column selection switches for each pixel in a column of a pixel array 500 of an image sensor (e.g., Figure 2 , Figure 3 or Figure 4 The image sensor 230 may be configured such that based on a pixel pattern (eg, Figure 5 Pixel patterns 511, 512, 513 and 514, a row driver (eg Figure 3 , Figure 4 or Figure 7 A row driver 320) controls the row selection switches, and a column driver controls each of the plurality of column selection switches.

[0096] According to an embodiment, a pixel pattern (e.g., Figure 5 The pixel patterns 511, 512, 513 and 514) may be included in the pixel pattern (eg, Figure 5 A group of pixels selected from the pixel patterns 511, 512, 513 and 514).

[0097] According to an embodiment, the pixel pattern (eg, Figure 5 The pixels in the pixel pattern 511, 512, 513 and 514) may include a pixel array (eg, Figure 3 or Figure 4 The pixel array 310, Figure 5 Pixels that are not adjacent to each other in the pixel array 500).

[0098] According to an embodiment of the present disclosure, an electronic device and an operating method thereof can be provided, which includes an image sensor capable of reducing the visibility of generated noise because the noise caused by the operating environment that varies according to the timing of reading out pixel values ​​is generated in units of pixels (for example, in units of lines) without having a pattern.

[0099] According to an embodiment of the present disclosure, an electronic device including an image sensor capable of acquiring image data and an operating method thereof can be provided, because noise caused by an operating environment that varies according to the timing of reading out pixel values ​​is generated in units of pixels, thereby acquiring image data to easily detect and correct noise.

[0100] Advantages obtained in the present disclosure are not limited to the aforementioned advantages, and other advantages not mentioned herein may be clearly understood from the following description by those skilled in the art to which the present disclosure pertains.

[0101] The methods according to the embodiments disclosed in the claims and / or the specification of the present disclosure may be implemented in hardware, software, or a combination of both.

[0102] When implemented in software, a computer-readable recording medium for storing one or more programs (i.e., software modules) may be provided. The one or more programs stored in the computer-readable recording medium are configured to be executed by one or more processors in an electronic device. The one or more programs include instructions for allowing the electronic device to execute the method according to the embodiments disclosed in the claims and / or specification of the present disclosure.

[0103] The program (i.e., software module or software) may be stored in a random access memory, a non-volatile memory including a flash memory, a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic disk storage device, a compact disk-ROM (CD-ROM), a digital versatile disk (DVD) or other forms of optical storage devices, and a magnetic tape cassette. Alternatively, the program may be stored in a memory configured with a combination of all or some of these storage media. In addition, the configured memory may be plural in number.

[0104] In addition, the program can be stored in an attachable storage device that can access the electronic device through a communication network such as the Internet, an intranet, a local area network (LAN), a wide LAN (WLAN), or a storage area network (SAN), or a communication network constituted by combining the networks. The storage device can access the device for executing the embodiments of the present disclosure via an external port. In addition, an additional storage device on the communication network can access the device for executing the embodiments of the present disclosure.

[0105] In the foregoing specific embodiments of the present disclosure, the components included in the present disclosure are expressed in singular or plural form according to the specific embodiments proposed herein. However, singular or plural expression is appropriately selected for the situation proposed for ease of explanation, so the various embodiments of the present disclosure are not limited to single or multiple components. Therefore, components expressed in plural form can also be expressed in singular form, or vice versa.

[0106] In addition, in the present disclosure, the terms “unit”, “module” and the like may be a hardware component such as a processor or a circuit and / or a software component executed by a hardware component such as a processor.

[0107] "Units" and "modules" may be implemented by programs stored in addressable storage media and executable by processors. For example, "units" and "modules" may be implemented by software components, object-oriented software components, class components and task components, processes, functions, properties, procedures, subroutines, program code segments, drivers, firmware, microcodes, circuits, data, databases, data structures, tables, arrays, and variables.

[0108] The specific implementation described in this disclosure is only one example and does not limit the scope of this disclosure in any way. For the sake of brevity of the specification, descriptions of conventional electronic components, control systems, software and other functional aspects of the system may be omitted.

[0109] In addition, in the present disclosure, "including at least one of a, b or c" may mean "including only a", "including only b", "including only c", "including a and b", "including b and c", "including a and c" or "including a, b and c".

[0110] Although the present disclosure has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims. Therefore, the scope of the present disclosure is not limited by its detailed description but by the appended claims, and all differences within the equivalents of the scope will be construed as being included in the present disclosure.

Claims

1. An electronic device, comprising: Image sensor; as well as Image Signal Processor, Wherein, the image sensor comprises: a pixel array comprising a plurality of pixels configured to detect light; a row driver configured to select a row of pixels to be driven from among the plurality of pixels in the pixel array; and a column driver configured to select a column of pixels to be driven from the plurality of pixels in the pixel array, Wherein, the image sensor is configured as: By operating the row driver and the column driver, reading out pixel values ​​based on a designated pixel pattern; and outputting image data acquired based on the pixel value to the image signal processor, The specified pixel pattern includes pixels located in a plurality of rows and a plurality of columns in the pixel array.

2. The electronic device according to claim 1, wherein: The image sensor further includes a memory, Wherein, the image sensor is configured as: storing the read pixel value in the memory; and The image data is acquired by generating an array pattern based on pixel values ​​stored in the memory.

3. The electronic device according to claim 1, wherein: The image signal processor is configured to acquire image data corrected by interpolating pixel values ​​included in the image data.

4. The electronic device according to claim 1, further comprising: Illuminance sensor; as well as at least one processor operably coupled to the illumination sensor and the image sensor, Wherein, the at least one processor is configured to: Acquiring an illuminance value by means of the illuminance sensor; and The image sensor is controlled so that the image sensor determines the pixel pattern based on the illumination value.

5. The electronic device according to claim 3, wherein: The at least one processor is configured to: Based on the illumination value being greater than or equal to a threshold, controlling the image sensor to read out pixel values ​​in units of a single line; as well as Based on the illumination value being less than the threshold, the image sensor is controlled to read out pixel values ​​in units of the pixel pattern.

6. The electronic device according to claim 1, wherein: The image sensor comprises: a row select switch coupled between the row driver and pixels of a row in the pixel array; and a plurality of column selection switches coupled to the column driver and to respective pixels of a column in the pixel array, Wherein, based on the pixel pattern, the row driver is configured to control the row selection switch, and the column driver is configured to control each of the plurality of column selection switches.

7. The electronic device according to claim 1, wherein: The pixel pattern includes a group of pixels selected in the pixel pattern.

8. The electronic device according to claim 1, wherein: The image sensor also includes a memory including information about the pixel pattern.

9. The electronic device according to claim 8, wherein: The at least one processor is configured to: analyzing the image data; determining a pattern based on a result of analyzing the image data; as well as The information about the pixel pattern stored in the memory is updated to the determined pattern.

10. The electronic device according to claim 1, wherein: The pixels included in the pixel pattern are pixels that are not adjacent to each other in the pixel array.

11. A method of operating an electronic device including an image sensor, the image sensor including a pixel array, the pixel array including a plurality of pixels, the method comprising: reading out pixel values ​​based on a pixel pattern that specifies rows and columns of pixels to be driven among pixels located in a plurality of rows and a plurality of columns in the pixel array; Creating image data based on the read pixel values; as well as The image data is output to the outside of the image sensor.

12. The method according to claim 11, in, The step of reading out includes storing the read-out pixel value in a memory included in the image sensor, The step of creating the image data includes acquiring the image data by restoring an array pattern based on pixel values ​​stored in the memory.

13. The method according to claim 11, further comprising: Image data corrected by interpolating pixel values ​​included in the image data is acquired.

14. The method according to claim 11, further comprising: Acquire an illuminance value, wherein the reading step includes: when the illuminance value is greater than or equal to a threshold value, the image sensor reads out the pixel value in units of a single line, and when the illuminance value is less than the threshold value, the image sensor reads out the pixel value in units of the pixel pattern.

15. The method according to claim 11, wherein: The pixels included in the pixel pattern are pixels that are not adjacent to each other in the pixel array.

Images