Electronic device and method for operating the same

Abstract

The present disclosure relates to an electronic device and a method for operating the electronic device. The electronic device includes: an image sensor for acquiring a pixel value of a first pixel sensed during a first exposure time and a pixel value of a second pixel sensed during a second exposure time longer than the first exposure time; and a controller for outputting an output image acquired based on the pixel value of the first pixel and a corrected saturated pixel value obtained by correcting the pixel value of the saturated pixel using the pixel value of at least one first pixel among the first pixels that is closest to the position of the saturated pixel having a pixel value exceeding a threshold value among the second pixels.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the priority of Korean Patent Application No. 10-2021-0106150 filed in the Korean Intellectual Property Office on August 11, 2021, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to an electronic device, and more particularly to an electronic device and an operating method thereof. Background Art

[0004] Recently, with the development of computer and communication industries, the demand for image sensors has increased in various electronic devices including smartphones, digital cameras, game consoles, the Internet of Things, robots, surveillance cameras, medical cameras, autonomous vehicles, etc.

[0005] Generally, an image sensor uses an array structure in which red pixels, green pixels, and blue pixels are arranged. Recently, research has been conducted to replace green pixels with yellow pixels or the like in order to obtain an image with reduced noise in a low-light environment where the amount of light is insufficient.

[0006] However, in the case of yellow pixels with excellent sensitivity, there is a problem that the pixel value quickly reaches the maximum limit in a lighting environment with sufficient light. This may cause the color of the image reproduced by the combination of red pixels, blue pixels and yellow pixels to become inaccurate. Summary of the invention

[0007] According to one aspect of the present disclosure, an electronic device may be provided, comprising: an image sensor configured to acquire a pixel value of a first pixel sensed during a first exposure time and a pixel value of a second pixel sensed during a second exposure time longer than the first exposure time; and a controller configured to output an output image obtained based on the pixel value of the first pixel and a corrected saturated pixel value, the corrected saturated pixel value being obtained by correcting the pixel value of the saturated pixel using the pixel value of at least one first pixel among the first pixels that is closest to a position of a saturated pixel among the second pixels having a pixel value exceeding a threshold.

[0008] According to another aspect of the present disclosure, a method for operating an electronic device may be provided, the method comprising: acquiring pixel values ​​of a plurality of pixels, the plurality of pixels comprising a first pixel sensed during a first exposure time and a second pixel sensed during a second exposure time longer than the first exposure time; correcting the pixel value of the first pixel based on a ratio of the first exposure time to the second exposure time; detecting a saturated pixel among the second pixels having a pixel value exceeding a threshold; correcting the saturated pixel value of the saturated pixel by using a corrected first pixel value of at least one first pixel among the first pixels that is closest to the saturated pixel; and outputting an output image based on the corrected first pixel value of the first pixel and the corrected saturated pixel value. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Examples of embodiments will be described more fully hereinafter with reference to the accompanying drawings; however, they may be embodied in different forms and should not be construed as limited to the embodiments set forth herein.

[0010] In the accompanying drawings, dimensions may be exaggerated for clarity of illustration. It should be understood that when an element is referred to as being "between" two elements, it may be the only element between the two elements, or one or more intervening elements may also be present. The same reference numerals always represent the same elements.

[0011] Figure 1 is a diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0012] Figure 2 is a diagram showing an image sensor according to an embodiment of the present disclosure.

[0013] Figure 3 is a diagram showing a controller according to an embodiment of the present disclosure.

[0014] Figure 4 is a diagram showing an image according to an embodiment of the present disclosure.

[0015] Figure 5A and Figure 5B is a diagram showing images to which different exposure times are applied according to an embodiment of the present disclosure.

[0016] Fig. 6A and Figure 6B is a diagram illustrating a method of correcting a first pixel according to an embodiment of the present disclosure.

[0017] Fig. 7A and Figure 7B is a diagram illustrating a method of correcting a second pixel according to an embodiment of the present disclosure.

[0018] Fig. 8A and Figure 8Bis a diagram illustrating a color conversion method according to an embodiment of the present disclosure.

[0019] Fig. 9A and Fig. 9B is a diagram showing a ratio of a first pixel to a second pixel according to an embodiment of the present disclosure.

[0020] Fig.10 is a diagram illustrating an operating method of an electronic device according to an embodiment of the present disclosure.

[0021] Fig.11 is a diagram showing an implementation example of an electronic device. DETAILED DESCRIPTION

[0022] The specific structural or functional description disclosed herein is only for the purpose of describing the embodiments according to the concept of the present disclosure. The embodiments according to the concept of the present disclosure can be implemented in various forms and should not be interpreted as being limited to the embodiments set forth herein.

[0023] The embodiment provides an electronic device capable of improving color reproducibility of an image while reducing noise, and an operating method of the electronic device.

[0024] Figure 1 is a diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0025] Reference Figure 1 According to the embodiment of the present disclosure, the electronic device 10 can acquire an image. In addition, the electronic device 10 can store an output image obtained by processing the image, display the output image, or output the output image to an external device. According to the embodiment of the present disclosure, the electronic device 10 can output the output image to the host 20 according to the request of the host 20.

[0026] In one embodiment, the electronic device 10 may be implemented in the form of a package module, a component, etc. The electronic device 10 may be installed in a host 20. The host 20 may be implemented as various electronic devices. For example, the host 20 may be implemented as a digital camera, a mobile device, a smart phone, a personal computer (PC), a tablet PC, a notebook computer, a personal digital assistant (PDA), an enterprise digital assistant (EDA), a portable multimedia player (PMP), a wearable device, a black box, a robot, an autonomous driving vehicle, etc.

[0027] In another embodiment, the electronic device 10 may be implemented as an electronic device separated from the host 20, and may be a camera, a digital camera, a video camera, a closed-circuit television (CCTV), a webcam, a security camera, an industrial visual camera, a mobile device, a smart phone, a personal computer (PC), a tablet PC, a notebook computer, a personal digital assistant (PDA), an enterprise digital assistant (EDA), a portable multimedia player (PMP), a wearable device, a black box, a robot, an autonomous vehicle, a vehicle visual camera, a set-top box, a game console, an electronic dictionary, an e-book reader, a desktop computer, a server, an MP3 player, an intelligent medical device, a television, a digital video disk (DVD) player, a speaker, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave oven, a washing machine, an air purifier, a smart mirror, a smart window, an electronic key, an electronic photo frame, a digital advertising board, a security control panel, etc. The wearable device may be a smart watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, a head mounted device (HMD), a skin pad, an electronic tattoo, or a bio-implantable circuit, etc.

[0028] The electronic device 10 may include an image sensor 100 and a controller 200 .

[0029] The image sensor 100 can acquire an image. Specifically, when a command for controlling the image sensor 100 to acquire an image is received from the host 20 or the controller 200, the image sensor 100 can acquire an image. To this end, the image sensor 100 can be implemented as a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor. In addition, the image sensor 100 can transmit the acquired image to the controller 200.

[0030] The image sensor 100 may acquire pixel data. The pixel data may include at least one of a position, a color, a pixel value, and an exposure time of a pixel. The pixel data may be respectively mapped to pixels included in an image.

[0031] The position of a pixel may indicate the position where a corresponding pixel among a plurality of pixels is arranged. A plurality of pixels may be distinguished from each other according to their positions. A pixel at a corresponding pixel may be selected by an address indicating the position of the pixel.

[0032] The color of a pixel may represent the color of light with respect to the corresponding pixel. For example, the color of each of the plurality of pixels may be one of red, blue, and yellow. In addition, a pixel having a red color may be simply designated as a red pixel, a pixel having a blue color may be simply designated as a blue pixel, and a pixel having a yellow color may be simply designated as a yellow pixel.

[0033] The pixel value may represent the brightness of light. For example, as the pixel value becomes larger, the brightness of light is brighter. The pixel value may have a value within the range of data bits allocated according to the resolution of the image sensor 100. For example, when the data bit is 8 bits, the pixel value may have an integer value included in the range of 0 to 255. The pixel value may have a value less than 0 as a minimum limit or a value exceeding 255 as a maximum limit.

[0034] The exposure time of a pixel may indicate the time during which the corresponding pixel is exposed to light. Generally, when the exposure time is reduced, a sufficient amount of light is not incident, and thus, noise may occur in the pixel. In addition, when the exposure time is increased, an amount of light equal to or greater than a limit value is incident, and thus, it is likely that the pixel will be saturated.

[0035] In the present disclosure, pixels with various colors can be used to replace green pixels. For example, the pixels replacing the green pixels can be yellow pixels. In another example, the pixels replacing the green pixels can be white pixels. In another example, the pixels replacing the green pixels can be modified to pixels with various colors, such as cyan pixels. The pixels replacing the green pixels can have a color with a higher sensitivity than the green pixels. Therefore, an image with reduced noise in a low lighting environment can be acquired.

[0036] In the present disclosure, a separate exposure time can be set for each pixel. That is, each pixel may include a pixel value obtained in an exposure time unit set for each pixel. In particular, the first exposure time and the second exposure time can be set according to the position of a yellow pixel (white pixel, etc.) with high sensitivity. The second exposure time may be a time longer than the first exposure time. For example, the first exposure time may be 1ms and the second exposure time may be 4ms. However, this is only an embodiment, and the exposure time may be changed in various ways. Therefore, color reproducibility can be improved by restoring saturated pixels.

[0037] The image sensor 100 may acquire pixel values ​​of a plurality of pixels. The plurality of pixels may include a first pixel and a second pixel. The first pixel may be a pixel among the plurality of pixels to which a first exposure time is set. The second pixel may be a pixel among the plurality of pixels to which a second exposure time is set. That is, the image sensor 100 may acquire a pixel value of a first pixel sensed for a first exposure time and a pixel value of a second pixel sensed for a second exposure time. The second exposure time may be an exposure time longer than the first exposure time. In addition, the first pixel and the second pixel may be pixels of the same color. For example, the first pixel may be a yellow pixel among the plurality of pixels to which a first exposure time is set, and the second pixel may be a yellow pixel among the plurality of pixels to which a second exposure time is set.

[0038] The controller 200 may control the overall operation of the electronic device 10. For example, the controller 200 may control the image sensor 100 to adjust the exposure time or perform an operation of acquiring an image. In another example, the controller 200 may store data in the memory 300 or access data stored in the memory 300.

[0039] The controller 200 may output an output image acquired based on a corrected saturated pixel value and a pixel value of a first pixel, the corrected saturated pixel value being obtained by correcting the pixel value of the saturated pixel using the pixel value of at least one first pixel among the first pixels that is closest to the saturated pixel. A saturated pixel may be referred to as a saturated pixel. The corrected saturated pixel value may be referred to as a corrected saturated pixel value. A saturated pixel may be a second pixel among the second pixels that has a pixel value exceeding a threshold value (or a reference value). Each of the first pixels may be a pixel that senses a pixel value during a first exposure time, and each of the second pixels may be a pixel that senses a pixel value during a second exposure time. The second exposure time may be a time longer than the first exposure time.

[0040] Specifically, the controller 200 may correct the pixel value of the first pixel based on the ratio of the first exposure time to the second exposure time. The controller 200 may detect a saturated pixel having a pixel value exceeding a threshold value among the second pixels. The controller 200 may correct the pixel value of the saturated pixel by using the corrected first pixel value of the first pixel closest to the position of the saturated pixel among the first pixels. The controller 200 may output an output image based on the corrected first pixel value of the first pixel and the corrected saturated pixel value of the saturated pixel.

[0041] In one embodiment, the electronic device 10 may further include a memory 300. The memory 300 may be implemented as a non-volatile memory device. For example, the memory 300 may be configured as various non-volatile memory devices, such as a read-only memory (ROM) that can only perform data reading, a one-time programmable (OTP) memory that can only perform one write, an erasable and programmable ROM (EPROM) that can erase and write stored data, a NAND flash memory, and a NOR flash memory.

[0042] The memory 300 may store a color conversion matrix. The color conversion matrix represents a matrix for converting a pixel value of a yellow pixel (white pixel, etc.) into a pixel value of a green pixel.

[0043] According to the present disclosure, in one embodiment, an electronic device and an operating method thereof can be provided that can improve the color reproducibility of an image while reducing image noise. Hereinafter, the electronic device and the operating method of the electronic device will be described in more detail with reference to the accompanying drawings.

[0044] Figure 2is a diagram showing an image sensor according to an embodiment of the present disclosure.

[0045] Reference Figure 2 , the image sensor 100 may include an optical lens LS, a pixel array 110 , a row decoder 120 , a timing generator 130 , a signal converter 140 , and an output buffer 150 .

[0046] The optical lens LS may refract light reflected from an object. The light refracted by the optical lens LS may travel toward the pixel array 110. That is, the optical lens LS may refract incident light to each of the pixels of the pixel array 110. The optical lens LS may be one of a lens or a plurality of lenses arranged on a path of travel of the light. In addition, the optical lens LS may include a microlens assembly. The object may include at least one of various elements such as an object, an animal, a person, and a background existing outside.

[0047] Pixel array 110 may generate an electrical signal representing the intensity or amount of light to which it is exposed.

[0048] To this end, the pixel array 110 may include a plurality of units. The plurality of units may be arranged in a row direction and a column direction. The plurality of units included in the pixel array 110 may correspond to a plurality of pixels included in an image. The plurality of units may be units arranged in a physical area, and the plurality of pixels may be pixels arranged in a digital area. The unit and the pixel may have a corresponding relationship at the same arrangement position. For example, a pixel (x, y) of an image may correspond to a unit (x, y) of the pixel array 110. Here, x and y are natural numbers. In addition, the pixel value of the pixel (x, y) of the image may be proportional to the amount of light to which the unit (x, y) of the pixel array 110 is exposed. The color of the pixel (x, y) of the image may be the same color as the color of the color filter included in the unit (x, y) of the pixel array 110.

[0049] Each cell may include a color filter and a sensing circuit. The color filter may be disposed on top of the sensing circuit. During the exposure time, light may pass through the color filter and then reach the sensing circuit. Depending on the color of the color filter, the cell may be designated as a red cell, a blue cell, a yellow cell, etc.

[0050] The sensing circuit may include a light sensing element. When light is incident, the light sensing element may generate an electrical signal by using the photoelectric effect. For example, the light sensing element may be implemented as various semiconductor elements such as a pn junction photodiode, a positive-intrinsic-negative (PIN) photodiode, an avalanche photodiode (APD), and a phototransistor.

[0051] The color filter may be one of a red color filter, a yellow color filter, and a blue color filter. The red color filter may allow light representing red to be transmitted through by filtering incident light. The yellow color filter may allow light representing yellow to be transmitted through by filtering incident light. The blue color filter may allow light representing blue to be transmitted through by filtering incident light. However, this is only one embodiment, and the yellow color filter may be modified differently to a white color filter, a cyan color filter, etc.

[0052] Multiple color filters can be laid out in a repeating pattern. This will refer to Figure 4 Give a description.

[0053] The row decoder 120 may select a cell located on a row corresponding to the address in response to the address and the control signal output from the timing generator 130. In addition, the timing generator 130 may set an exposure time of each cell. In one embodiment, the timing generator 130 sets the exposure time of each cell under the control of the controller 200. For example, when receiving a control command CMD from the controller 200, the timing generator 130 may change the exposure time of the first pixel to the first exposure time, and change the exposure time of the second pixel to the second exposure time.

[0054] The signal converter 140 may acquire a pixel value of each of the plurality of pixels based on each of the signals output from the pixel array 110. The signal converter 140 may transfer the pixel values ​​corresponding to the pixels of the selected row to the output buffer 150 under the control of the timing generator 130.

[0055] The output buffer 150 may store pixel values ​​of pixels sequentially transferred from the signal converter 140 and output an image Img including the pixel values ​​of the pixels.

[0056] In addition, the pixel value of the pixel can be obtained based on the amount of light incident on the unit corresponding to the pixel during the exposure time set for the pixel. For example, the pixel value of the pixel can be proportional to the amount of charge accumulated by the light exposed during the exposure time set for the pixel.

[0057] In a specific example, the cell may generate a current having a level corresponding to the intensity of incident light passing through the sensing circuit. The cell may accumulate charge through the current generated during the exposure time. The charge may be accumulated in the sensing circuit. The cell may generate a voltage having a level proportional to the amount of accumulated charge passing through the sensing circuit. The level of the voltage generated in the cell may represent the pixel value of a pixel having the same arrangement position as the cell.

[0058] In addition, when the amount of charge accumulated in a cell reaches a maximum limit, no charge is accumulated even when the corresponding cell is exposed to light. The pixel value of a pixel having the same arrangement position as the corresponding cell may converge to the maximum limit. Such a pixel may be defined as a saturated pixel. When a saturated pixel exists among the pixels contained in the image, the color reproducibility of the image may deteriorate. In one embodiment, the electronic device 10 may determine a pixel having a pixel value exceeding a threshold as a saturated pixel. The threshold may be less than or equal to the limit, and the threshold may be preset.

[0059] In one embodiment, the image sensor 100 may perform black level compensation. Specifically, the image sensor 100 may compensate for the pixel value corresponding to the dark current for all pixels. The dark current is independent of light and may be a current generated due to temperature, etc. That is, this is because the pixel value corresponding to the dark current may be added to the pixel value of all pixels. The image Img output from the image sensor 100 may be an image Img in which the pixel value corresponding to the dark current is compensated.

[0060] For example, the image sensor 100 may subtract the pixel value corresponding to the dark current from the pixel value of each of the plurality of pixels. In one example, the pixel value corresponding to the dark current may be 64. In addition, depending on the structure, material, etc. of the image sensor 100, the pixel value corresponding to the dark current may have different values.

[0061] The dark current may be measured in various ways. For example, the image sensor 100 may include a black cell. The black cell may include a black color filter for blocking light, and a sensing circuit formed on the bottom of the black color filter. The dark current may be a current measured in the sensing circuit of the black cell. In another example, the dark current may be a current measured in the sensing circuit of the red cell, the blue cell, the yellow cell, etc. in a darkroom environment where light is blocked.

[0062] Figure 3 is a diagram showing a controller according to an embodiment of the present disclosure.

[0063] Reference Figure 3 , the controller 200 may acquire an image Img from the image sensor 100. The image Img may include a plurality of pixels. The image Img may include a plurality of pixel values ​​of the plurality of pixels. The plurality of pixels may include a first pixel and a second pixel.

[0064] The color of the first pixel may be equal to the color of the second pixel. In one embodiment, the colors of the first pixel and the second pixel may be yellow or white. That is, the first pixel and the second pixel may both be yellow pixels. Alternatively, the first pixel and the second pixel may both be white pixels.

[0065] A first exposure time may be set for a first pixel. Each of the first pixels may have a pixel value acquired during the first exposure time. A second exposure time may be set for a second pixel. Each of the second pixels may have a pixel value acquired during the second exposure time. The second exposure time may be a time longer than the first exposure time.

[0066] In one embodiment, a plurality of pixels may be arranged such that a pattern according to pixel color is repeated for each grid area in the image Img. That is, the first pixels may be arranged to have the same distance. That is, the first pixels may be arranged at the same distance in the image Img.

[0067] According to an embodiment of the present disclosure, the ratio of the number of first pixels to the number of second pixels may be 1:3. In addition, in another embodiment, the ratio of the number of first pixels to the number of second pixels may be 1:7. This is because since the first exposure time is relatively short, noise may appear in the pixel, and since the second exposure time is relatively long, the pixel may be saturated. Therefore, it is necessary to control the ratio of the number of pixels for which the first exposure time is set to the number of pixels for which the second exposure time is set.

[0068] In one embodiment, the second exposure time can be set for other pixels except the first pixel and the second pixel among the plurality of pixels. That is, each of the other pixels can have a pixel value acquired during the second exposure time. However, this is only one embodiment, and different exposure times can be set for other pixels. In addition, the color of each of the other pixels can be red or blue. That is, each of the other pixels can be a red pixel or a blue pixel. Each of the other pixels can be a red pixel or a blue pixel with a ratio of 1:1.

[0069] The controller 200 may acquire a corrected image Img' obtained by correcting the image Img based on the first exposure time and the second exposure time. The corrected image Img' is obtained by correcting the pixel values ​​of the selected pixels among the plurality of pixels included in the image Img. That is, the corrected image Img' may include the corrected pixel values ​​of the selected pixels among the plurality of pixels included in the image Img.

[0070] Specifically, the controller 200 can correct the pixel value of the first pixel based on the ratio of the first exposure time and the second exposure time. In addition, the controller 200 can detect a saturated pixel having a pixel value exceeding a threshold value among the second pixels. In addition, the controller 200 can correct the pixel value of the saturated pixel by using the corrected first pixel value of the first pixel closest to the position of the saturated pixel among the first pixels.

[0071] The controller 200 may output the output image Out_Img based on the corrected image Img′. Specifically, the controller 200 may output the output image Out_Img based on the corrected first pixel value of the first pixel and the corrected saturated pixel value of the saturated pixel.

[0072] In certain embodiments, the controller 200 may include an exposure time controller 210 , a signal processor 220 , and a color converter 230 .

[0073] The exposure time controller 210 can set (or adjust) the exposure time of at least one pixel among the multiple pixels. Specifically, the exposure time controller 210 can set the exposure time of the selected pixel. That is, the exposure time controller 210 can change the exposure time of the selected pixel. For example, the exposure time controller 210 can provide the address of the selected pixel, the exposure time and the control command for setting the exposure time to the image sensor 100. In response to this, the timing generator 130 of the image sensor 100 can set the exposure time of the pixel selected by the address based on the address and the control command CMD. In addition, the image sensor 100 can obtain the pixel value of the pixel according to the exposure time set for the pixel. The image sensor 100 can output an image Img including the pixel values ​​of multiple pixels.

[0074] In one embodiment, the exposure time controller 210 may send a control command CMD to the image sensor 100 to set the exposure time of the first pixel to the first exposure time and the exposure time of the second pixel to the second exposure time. The first exposure time may be shorter than the second exposure time.

[0075] The exposure time controller 210 may transmit exposure time information ET to the signal processor 220. The exposure time information ET may be information indicating an exposure time set for each of pixels included in the image Img.

[0076] The signal processor 220 may acquire the image Img. For example, the signal processor 220 may receive the image Img from the image sensor 100.

[0077] The signal processor 220 may acquire exposure time information ET. For example, the signal processor 220 may receive the exposure time information ET from the exposure time controller 210. In another example, the signal processor 220 may receive the exposure time information ET from the image sensor 100. The exposure time information ET may include an exposure time set for each of a plurality of pixels. The exposure time may be one of a first exposure time and a second exposure time longer than the first exposure time.

[0078] The signal processor 220 may acquire a corrected image Img' based on the exposure time information ET and the image Img. The corrected image Img' may be an image obtained by correcting the pixel value of the first pixel and the saturated pixel value of the saturated pixel included in the image Img. That is, the signal processor 220 may acquire a corrected image Img' including the corrected first pixel value and the corrected saturated pixel value. In one embodiment, the image Img and the corrected image Img' may include pixel values ​​regarding yellow, pixel values ​​regarding red, and pixel values ​​regarding blue. In another embodiment, the image Img and the corrected image Img' may include pixel values ​​regarding white, pixel values ​​regarding red, and pixel values ​​regarding blue.

[0079] Specifically, the signal processor 220 may correct the pixel value of the first pixel by using the ratio of the first exposure time set for the first pixel to the second exposure time set for the second pixel. That is, the signal processor 220 may acquire the corrected first pixel value obtained by correcting the pixel value of the first pixel. For example, the ratio of the first exposure time to the second exposure time may be a value obtained by dividing the first exposure time by the second exposure time. The signal processor 220 may correct the pixel value of the first pixel by multiplying the pixel value of the first pixel by the ratio of the first exposure time to the second exposure time.

[0080] In addition, the signal processor 220 may compare each of the pixel values ​​of the second pixel with a threshold value, and detect the second pixel having a pixel value exceeding the threshold value among the second pixels as a saturated pixel. That is, a saturated pixel may represent a pixel whose pixel value exceeds the threshold value, because the saturated pixel has a long exposure time and has a saturated brightness corresponding to the pixel value. The threshold value may be a value predetermined based on the maximum limit value within the range of the data bits assigned as the pixel value. For example, the threshold value may be a value corresponding to 90%, 95%, etc. of the maximum limit value. The word "predetermined" used here with respect to a parameter (such as a predetermined value, distance, constant value, row or column) means that the value of the parameter is determined before the parameter is used in a process or algorithm. For some embodiments, the value of the parameter is determined before the process or algorithm starts. In other embodiments, the value of the parameter is determined during the process or algorithm but before the parameter is used in the process or algorithm.

[0081] In addition, the signal processor 220 can correct the pixel value of the saturated pixel by using the corrected first pixel value of at least one nearest first pixel (or at least one nearest first pixel) among the first pixels that is closest to the saturated pixel. That is, the signal processor 220 can obtain the corrected saturated pixel value by using the corrected first pixel value corresponding to at least one first pixel closest to the saturated pixel among the corrected first pixel values. The saturated pixel and the first pixel may be pixels of the same color. The saturated pixel may be a pixel for which the second exposure time is set, and the first pixel may be a pixel for which the first exposure time is set.

[0082] Specifically, the signal processor 220 may select a first pixel having the same color as the saturated pixel and for which the first exposure time is set among the plurality of pixels included in the image. The signal processor 220 may calculate the distance value between the position of each selected first pixel and the position of the saturated pixel, and identify at least one pixel having the minimum distance value among the calculated distance values ​​as the nearest first pixel. That is, at least one nearest first pixel may represent at least one first pixel located at the closest distance to the position of the saturated pixel among the first pixels for which the first exposure time is set.

[0083] The signal processor 220 may correct the pixel value of the saturated pixel by using at least one nearest first pixel. The number of the at least one nearest first pixel may be one or more. When the number of the at least one nearest first pixel is 1, the signal processor 220 may correct the pixel value of the saturated pixel to be equal to the corrected first pixel value of the nearest first pixel. That is, the signal processor 220 may replace the pixel value of the saturated pixel with the pixel value of the first pixel at the position closest to the saturated pixel.

[0084] In one embodiment, when the number of at least one nearest first pixel is plural, the signal processor 220 may correct the pixel value of the saturated pixel so that it becomes an average value or a median value of the corrected first pixel values ​​of the plurality of nearest first pixels. However, this is merely one embodiment, and the pixel value of the saturated pixel may be corrected to various pixel values ​​based on the corrected first pixel value of at least one first pixel located within a predetermined distance from the position of the saturated pixel.

[0085] The signal processor 220 may acquire a corrected image Img', which includes pixel values ​​of other pixels except the first pixel and the saturated pixel, the corrected first pixel value of the first pixel, and the corrected saturated pixel value of the saturated pixel. That is, the corrected image Img' may be an image obtained by correcting the pixel values ​​of some pixels in the image Img. The above-mentioned some pixels may include the first pixel and the saturated pixel. The color layout of the pixels of the image Img and the corrected image Img' may be maintained in the same manner.

[0086] The signal processor 220 may transfer the corrected image Img′ to the color converter 230 .

[0087] The color converter 230 may acquire an output image Out_Img by applying a color conversion matrix (CCM) 310 to the corrected image Img'. The color conversion matrix 310 may represent a matrix for acquiring an output image Out_Img obtained by converting the color of a pixel included in the corrected image Img' into another color. The output image Out_Img may include a pixel value regarding green, a pixel value regarding red, and a pixel value regarding blue. The color of the pixel included in the output image Out_Img may be equal to the color of the pixel constituting the display.

[0088] The color converter 230 can generate an output image Out_Img including a green pixel corresponding to the position of the first pixel and the second pixel by applying the color conversion matrix 310 to a plurality of pixels included in the corrected image Img'. That is, the color converter 230 can obtain the output image Out_Img by matrix calculation of the pixel value about one of yellow and white, the pixel value about red, and the pixel value about blue with the color conversion matrix.

[0089] In an embodiment, the electronic device 10 may further include a memory 300. The memory 300 may store a color conversion matrix 310. When receiving the corrected image Img' from the signal processor 220, the color converter 230 may request the color conversion matrix 310 from the memory 300. The color converter 230 may receive a signal CCM representing the color conversion matrix 310 from the memory 300. However, this is merely an embodiment, and the color conversion matrix 310 may be stored in a memory included in the controller 200.

[0090] As described above, according to the present disclosure, in one embodiment, an electronic device and an operating method of the electronic device can be provided that can improve the color reproducibility of an image while reducing noise of the image. Hereinafter, embodiments will be described with reference to the accompanying drawings.

[0091] Figure 4is a diagram showing an image according to an embodiment of the present disclosure.

[0092] Reference Figure 4 According to (1) and (2), the image sensor 100 according to the embodiment of the present disclosure can acquire images 410 and 420 including a plurality of pixels. Figure 4 Some pixels are represented in the images 410 and 420 of (1) and (2), and the images 410 and 420 may also include other pixels that repeat the same arrangement.

[0093] The image 410 or 420 may include a plurality of pixels. The plurality of pixels may be arranged in a column direction and a row direction. The plurality of pixels may be distinguished from each other according to their colors.

[0094] The plurality of pixels may be arranged such that a pattern according to pixel colors is repeated for each grid area 415 or 425. Specifically, the image 410 or 420 may include a plurality of grid areas 415 or 425. The grid area 415 or 425 may include a plurality of pixels arranged according to a pattern.

[0095] In an embodiment, the pattern may be a Bayer pattern. The Bayer pattern may be a pattern in which pixels of the same color are arranged at positions located in a diagonal direction and pixels of different colors are arranged at other positions.

[0096] In one embodiment, if Figure 4 As shown in (1), the grid area 415 may include a first yellow pixel Ya, a red pixel R, a blue pixel B, and a second yellow pixel Yb arranged in 2×2. The first yellow pixel Ya and the second yellow pixel Yb having the same yellow color may be arranged at positions (1,1) and (2,2) along the diagonal direction. The red pixel R and the blue pixel B having different colors may be arranged at positions (1,2) and (2,1) along the diagonal direction. The image 410 may have a pattern in which the grid area 415 repeats in the column direction and the row direction.

[0097] In one embodiment, if Figure 4 As shown in (2), the grid area 425 may include a first white pixel Wa, a red pixel R, a blue pixel B, and a second white pixel Wb arranged in 2×2. The first white pixel Wa and the second white pixel Wb having the same white color may be arranged at positions (1,1) and (2,2) along the diagonal direction. The red pixel R and the blue pixel B having different colors may be arranged at positions (1,2) and (2,1) along the diagonal direction. The image 420 may have a pattern that the grid area 425 repeats in the column direction and the row direction.

[0098] In addition, the layout and colors of the plurality of pixels included in the image 410 or 420 may be determined by the layout and colors of the plurality of cells included in the pixel array 110. For example, the layout and colors of the plurality of pixels may be equal to the layout and colors of the plurality of cells.

[0099] In the following, for the convenience of description, it is assumed and described that the image Img is similar to Figure 4 The image 410 shown in (1) includes a first yellow pixel Ya, a red pixel R, a blue pixel B, and a second yellow pixel Yb, and has the layout pattern of the image 410.

[0100] Figure 5A and Figure 5B is a diagram showing images to which different exposure times are applied according to an embodiment of the present disclosure.

[0101] Reference Figure 5A and Figure 5B , the controller 200 may acquire an image 510 through the image sensor 100. The image 510 may include a plurality of pixels. The plurality of pixels may include a red pixel R, a blue pixel B, and yellow pixels Y1 and Y2. A separate exposure time may be set for each of the plurality of pixels.

[0102] The yellow pixels Y1 and Y2 may be divided into a first pixel Y1 for which a first exposure time t1 is set and a second pixel Y2 for which a second exposure time t2 is set. The second exposure time t2 may be set for the red pixel R and the blue pixel B.

[0103] In one embodiment, the first pixels Y1 included in the image 510 may be arranged to have the same distance. The distance may indicate the length (or value) between the positions (or coordinates) of two pixels. The ratio of the number of first pixels Y1 to the number of second pixels Y2 may be 1:3 (or 2:6).

[0104] Reference Figure 5A (1), in one embodiment, the image 510 may include a plurality of grid areas 520. The plurality of grid areas 520 may have the same pixel layout. For example, pixels arranged at the same position in each grid area 520 may be pixels having the same color and having the same exposure time set therefor.

[0105] Reference Figure 5A (2), the first exposure time t1 may be a time shorter than the second exposure time t2. For example, the first exposure time t1 may be a value obtained by dividing the second exposure time t2 by a predetermined constant value k. The constant value k may be a value greater than 1. In addition, the constant value k may represent a value obtained by dividing the second exposure time t2 by the first exposure time t1.

[0106] In one embodiment, reference Figure 5B , the image 510 may include a plurality of grid areas 521, 522, 523, and 524. The plurality of grid areas 521, 522, 523, and 524 may have the same pixel layout. For example, pixels arranged at the same position in each of the grid areas 521, 522, 523, and 524 may be pixels having the same color. According to the ratio of the number of first pixels Y1 to the number of second pixels Y2, the first pixel Y1 may be included in the selected grid areas 521 and 524 among the plurality of grid areas 521, 522, 523, and 524. The selected grid areas 521 and 524 may be arranged to have the same distance.

[0107] Fig. 6A and Figure 6B is a diagram illustrating a method of correcting a first pixel according to an embodiment of the present disclosure.

[0108] Reference Fig. 6A and Figure 6B , the controller 200 may correct the first pixel Y1 included in the image 610 and acquire a corrected image 620 including the corrected first pixel Y1 ′.

[0109] Specifically, the controller 200 may correct the pixel value PV_Y1 of the first pixel Y1 included in the image 610 based on the ratio of the first exposure time t1 and the second exposure time t2. The ratio of the first exposure time t1 and the second exposure time t2 may represent a constant value obtained by dividing the first exposure time t1 by the second exposure time t2.

[0110] The controller 200 may acquire a corrected first pixel value PV_Y1' of the corrected pixel Y1' as a result obtained by correcting the pixel value PV_Y1 of the first pixel Y1 included in the image 610. For example, the controller 200 may acquire a corrected first pixel value PV_Y1' of the first pixel 621 at the same position (1, 1) as the first pixel 611 at the position (1, 1) included in the image 610 as a result obtained by correcting the pixel value PV_Y1 of the first pixel 611.

[0111] The controller 200 may acquire a corrected first pixel value PV_Y1' of the corrected first pixel Y1' as a result obtained by multiplying the pixel value PV_Y1 of the first pixel Y1 included in the image 610 by a constant value k. The controller 200 may acquire a corrected image 620 including the corrected first pixel value PV_Y1' of the corrected first pixel Y1'. In addition, the controller 200 may correct the pixel values ​​of saturated pixels among the pixel values ​​PV_Y2 of the second pixels Y2 included in the image 610. Each of the second pixels Y2 may be a pixel to which the second exposure time t2 is set. The saturated pixels may be corrected using a method different from that of the first pixels.

[0112] Fig. 7A and Figure 7B is a diagram illustrating a method of correcting a second pixel according to an embodiment of the present disclosure.

[0113] Reference Fig. 7A and Figure 7B , the controller 200 may detect a saturated pixel having a pixel value exceeding a threshold value among a plurality of pixels included in the image 710. Fig. 7A As shown in (1) and (2), the image 710 may include a corrected first pixel value PV_Y1' of the corrected first pixel Y1' and a second pixel value PV_Y2 of the second pixel Y2. The pixel value PV_Y2 of the second pixel Y2 may be uncorrected. It is assumed that the pixels 711 and 721 located at (2, 2) among the second pixels Y2 are saturated pixels having pixel values ​​exceeding a threshold value.

[0114] The controller 200 may correct the saturated pixels included in the image 710 and acquire the corrected image 720 including the corrected saturated pixel value PV_Y2' of the corrected saturated pixel Y2'. For example, the controller 200 may acquire the corrected saturated pixel value PV_Y2' of the corrected saturated pixel Y2' as a result obtained by correcting the pixel value PV_Y2 of the saturated pixel included in the image 710. The saturated pixel and the corrected saturated pixel Y2' may be pixels 711, 721 located at the same position.

[0115] Reference Fig. 7A and Figure 7B , the controller 200 may correct the pixel value PV_Y2 of the saturated pixel by using the corrected first pixel value PV_Y1 ′ of the first pixel 723 or 725 closest to the saturated pixel among the corrected first pixels Y1 ′. Figure 7BThe arrow in (2) may indicate that the corrected saturated pixel value PV_Y2' is obtained according to an operation using the corrected first pixel value PV_Y1'. In one embodiment, the above operation may be to calculate an average value or a median value. The saturated pixel and the first pixel may be pixels 711, 721 of the same color. The saturated pixel may be a pixel for which the second exposure time t2 is set, and the first pixel may be a pixel for which the first exposure time is set.

[0116] In one embodiment, the first pixels 723 and 725 located at (1, 1) and (3, 3) of the position (2, 2) closest to the saturated pixel among the corrected first pixels Y1' are identified as the nearest first pixels. For example, when the number of the nearest first pixels is multiple, the controller 200 can obtain the average value or median value of the corrected first pixel values ​​PV_Y1' of the multiple first pixels 723 and 725 as the corrected saturated pixel value PV_Y2'.

[0117] Fig. 8A and Figure 8B is a diagram illustrating a color conversion method according to an embodiment of the present disclosure.

[0118] Reference Fig. 8A and Figure 8B , the controller 200 may generate an output image 830 by applying the color conversion matrix 310 to the corrected image 810. For example, the controller 200 may acquire the output image 830 through the corrected image 810 and the color conversion matrix 310. That is, the output image 830 may be obtained by converting the colors of some pixels included in the corrected image 810.

[0119] The color conversion matrix 310 may include parameters C11 to C44 corresponding to predetermined rows and predetermined columns. The number and values ​​of the parameters C11 to C44 included in the color conversion matrix 310 may be changed according to the colors of the pixels included in the corrected image 810 and the colors of the pixels included in the output image 830.

[0120] As a result obtained by multiplying the color conversion matrix 310 by the pixel values ​​Ya_PV_in, R_PV_in, B_PV_in, and Yb_PV_in of the pixels Ya, R, B, and Yb included in the unit area 815 of the corrected image 810, the controller 200 can acquire the pixel values ​​Ga_PV_out, R_PV_out, B_PV_out, and Gb_PV_out of the pixels Ga, R, B, and Gb included in the unit area 835 of the output image 830. The controller 200 can acquire the pixel values ​​Ga_PV_out, R_PV_out, B_PV_out, and Gb_PV_out of the pixels Ga, R, B, and Gb representing the converted colors for each unit area 835 of the output image 830 through the calculation result obtained by applying the color conversion matrix 310 to each unit area 815 of the corrected image 810.

[0121] The corrected image 810 may include unit regions 815 repeatedly arranged in the row direction and the column direction. The unit region 815 may include a first yellow pixel Ya, a red pixel R, a blue pixel B, and a second yellow pixel Yb.

[0122] The output image 830 may include unit areas 835 repeatedly arranged in the row direction and the column direction. The unit area 835 may include a first green pixel Ga, a red pixel R, a blue pixel B, and a second green pixel Gb. That is, the yellow pixels Ya and Yb of the corrected image 810 may be converted into green pixels Ga and Gb of the output image 830.

[0123] Fig. 9A and Fig. 9B is a diagram showing a ratio of a first pixel to a second pixel according to an embodiment of the present disclosure.

[0124] Reference Fig. 9A and Fig. 9B , the controller 200 may acquire an image 910 through the image sensor 100. The image 910 may include a plurality of pixels. The plurality of pixels may include a red pixel R, a blue pixel B, and yellow pixels Y1 and Y2. A separate exposure time may be set for each of the plurality of pixels. The yellow pixels Y1 and Y2 may be divided into a first pixel Y1 for which a first exposure time t1 is set and a second pixel Y2 for which a second exposure time t2 is set. The second exposure time t2 may be set for the red pixel R and the blue pixel B.

[0125] In one embodiment, the ratio of the number of the first pixels Y1 to the number of the second pixels Y2 may be 1:7.

[0126] Reference Fig. 9A(1), in one embodiment, the image 910 may include a plurality of grid areas 920. The plurality of grid areas 920 may have the same pixel layout. For example, pixels arranged at the same position in each grid area 920 may be pixels having the same color and having the same exposure time set therefor. The grid areas 920 having the same pixel layout may be repeatedly arranged in the row direction and the column direction.

[0127] Reference Fig. 9A (2), the first exposure time t1 may be shorter than the second exposure time t2.

[0128] Reference Fig. 9B , the image 910 may include a plurality of grid areas 921, 922, 923, and 924. The plurality of grid areas 921, 922, 923, and 924 may have the same pixel layout. For example, pixels arranged at the same position in each of the grid areas 921, 922, 923, and 924 may be pixels having the same color. According to the ratio of the number of first pixels Y1 to the number of second pixels Y2, the first pixel Y1 may be included in the selected grid area 921 among the plurality of grid areas 921, 922, 923, and 924. The plurality of grid areas 921, 922, 923, and 924 may be repeatedly arranged alternately in the row direction and the column direction.

[0129] Fig.10 is a diagram illustrating an operating method of an electronic device according to an embodiment of the present disclosure.

[0130] Reference Fig.10 , the operating method of the electronic device 10 may include: step S1010, acquiring pixel values ​​of a plurality of pixels, the plurality of pixels including a first pixel sensed during a first exposure time and a second pixel sensed during a second exposure time longer than the first exposure time; step S1020, correcting the pixel value of the first pixel based on a ratio of the first exposure time to the second exposure time; step S1030, detecting a saturated pixel having a pixel value exceeding a threshold value among the second pixels; step S1040, correcting the saturated pixel value of the saturated pixel by using a corrected first pixel value of at least one first pixel among the first pixels that is closest to the saturated pixel; and step S1050, outputting an output image based on the corrected saturated pixel value of the saturated pixel.

[0131] Specifically, pixel values ​​of a plurality of pixels including a first pixel and a second pixel may be obtained (S1010). A first exposure time may be set for the first pixel. A second exposure time may be set for the second pixel. The second exposure time may be a time longer than the first exposure time. To this end, the operating method according to an embodiment of the present disclosure may further include the step of adjusting the first exposure time of the first pixel and the second exposure time of the second pixel. In one embodiment, the second pixel may have the same color as the first pixel.

[0132] In an embodiment, the step of acquiring pixel values ​​of the plurality of pixels may include acquiring pixel values ​​corresponding to the intensity of light incident during an exposure time set for each of the plurality of pixels.

[0133] In one embodiment, the colors of the first pixel and the second pixel may be yellow or white. However, this is only one embodiment, and the colors of the first pixel and the second pixel may be changed to various colors, such as cyan.

[0134] In one embodiment, a plurality of pixels may be arranged in a plurality of grid areas included in an image so that pixels of the same color are located at the same position in each grid area. That is, a plurality of pixels may be arranged so that a pattern according to pixel color is repeated for each grid area. An image may include a plurality of grid areas arranged in a matrix form. Each of the plurality of grid areas may be an area of ​​the same size. For example, each grid area may include pixels arranged in 2×2. The first pixels included in the image may be arranged to have the same distance.

[0135] In one embodiment, the ratio of the number of first pixels to the number of second pixels may be 1:3. For example, when one grid area includes pixels arranged in 2×2, the one grid area may include one red pixel, one blue pixel, and two yellow pixels. Eight yellow pixels may be included in an area obtained by adding the 2×2 grid areas. The number of first pixels as yellow pixels for which the first exposure time is set may be 2, and the number of second pixels as yellow pixels for which the second exposure time is set may be 6.

[0136] In one embodiment, the ratio of the number of first pixels to the number of second pixels may be 1:7. Eight yellow pixels may be included in an area obtained by adding 2×2 grid areas. The number of first pixels as yellow pixels for which the first exposure time is set may be 1, and the number of second pixels as yellow pixels for which the second exposure time is set may be 7.

[0137] In addition, the pixel value of the first pixel may be corrected based on the ratio of the first exposure time to the second exposure time (S1020). For example, the ratio of the first exposure time to the second exposure time may be a value k obtained by dividing the second exposure time by the first exposure time. The corrected first pixel value of the first pixel may be obtained by multiplying the pixel value of the first pixel by the ratio of the first exposure time to the second exposure time. That is, the pixel value of the first pixel may be corrected by a value calculated by multiplying each pixel value of the first pixel by the calculation of the ratio of the first exposure time to the second exposure time.

[0138] In addition, a saturated pixel having a pixel value exceeding a threshold value among the second pixels may be detected (S1030). That is, by a calculation that compares the pixel value of the second pixel with the threshold value, the second pixel having a pixel value exceeding the threshold value among the second pixels may be detected as a saturated pixel. The threshold value is a predetermined value relative to the maximum value of the pixel value and may be stored in the controller 200 or the memory 300. For example, if the maximum value of the pixel value is 255, the threshold value may be one of various values ​​including 230, 240, 250, etc.

[0139] Also, the saturated pixel value of the saturated pixel may be corrected by using the corrected first pixel value of the first pixel closest to the position of the saturated pixel among the first pixels ( S1040 ).

[0140] In one embodiment, an average value or a median value of the corrected first pixel value of at least one first pixel may be corrected to a saturated pixel value. In a specific embodiment, when the number of first pixels closest to the position of the saturated pixel among the first pixels is 1, the pixel value of the saturated pixel may be corrected to become the same pixel value as the corrected first pixel value of the first pixel. In another embodiment, when the number of first pixels closest to the position of the saturated pixel among the first pixels is plural, the pixel value of the saturated pixel may be corrected to become an average value or a median value of the corrected first pixel values ​​of each of the plural first pixels.

[0141] Also, an output image may be output based on the corrected first pixel value of the first pixel and the corrected saturated pixel value of the saturated pixel ( S1050 ).

[0142] In one embodiment, a corrected image may be acquired, the corrected image including pixel values ​​of other pixels except the first pixel and the saturated pixel among the pixel values ​​of the plurality of pixels, the corrected first pixel value of the first pixel, and the corrected saturated pixel value. A second exposure time may be set for the other pixels. Each of the other pixels may have a red color or a blue color.

[0143] Furthermore, an output image including pixel values ​​regarding green, pixel values ​​regarding red, and pixel values ​​regarding blue can be acquired through matrix calculation of the corrected image and the color conversion matrix.

[0144] To this end, a color conversion matrix for converting a pixel value related to one of yellow and white into a pixel value related to green may be stored. The color conversion matrix may be stored in the memory 300.

[0145] Fig.11 is a diagram showing an implementation example of an electronic device.

[0146] Reference Fig.11 , the electronic device 10 may be implemented as a computing system 2000. The computing system 2000 may include an image sensor 2010, a processor 2020, a storage device 2030, a memory device 2040, an input / output (I / O) device 2050, and a display device 2060. Fig.11 Although not shown in the figure, the computing system 2000 may further include a port capable of communicating with the storage device 2030, the memory device 2040, the I / O device 2050, the display device 2060, etc. or with an external device.

[0147] The image sensor 2010 may acquire an image Img including a plurality of pixels to which exposure times are individually applied. The image sensor 2010 may acquire a corrected image Img' obtained by correcting the pixel values ​​of the first pixel and the saturated pixel included in the image Img. The image sensor 2010 may generate an output image Out_Img obtained by applying a color conversion matrix to the corrected image Img', and output the output image Out_Img. The image sensor 2010 may be connected to the processor 2020 via an address bus, a control bus, and a data bus, or a communication line different therefrom, to perform communication.

[0148] The image sensor 2010 may be implemented using various types of packages. For example, at least some components of the image sensor 2010 may be implemented using packages such as package on package (PoP), ball grid array (BGA), chip scale package (CSP), plastic lead chip carrier (PLCC), plastic dual in-line package (PDIP), a die in a waffle package, a die in wafer form, a chip on board (COB), a ceramic dual in-line package (CERDIP), a plastic metric quad flat package (MQFP), a thin quad flat package (TQFP), a small outline integrated circuit (SOIC), a shrink small outline package (SSOP), a thin small outline package (TSOP), a system-in-package (SIP), a multi-chip package (MCP), a wafer-level manufacturing package (WFP), or a wafer-level processing stack package (WSP). In some embodiments, the image sensor 2010 may be integrated with the processor 2020 in one chip, or the image sensor 2010 and the processor 2020 may be integrated in different chips.

[0149] The processor 2020 may control the overall operation of the computing system 2000. The processor 2020 may control the display device 2060 to display the output image Out_Img. The processor 2020 may store the output image Out_Img in the storage device 2030.

[0150] The processor 2020 may perform specific calculations or tasks. According to an embodiment of the present disclosure, the processor 2020 may include at least one of a central processing unit (CPU), an application processing unit (APU), a graphics processing unit (GPU), and the like.

[0151] The processor 2020 may be connected to the storage device 2030, the memory device 2040, and the I / O device 2050 through an address bus, a control bus, and a data bus to perform communication. According to an embodiment of the present disclosure, the processor 2020 may also be connected to an expansion bus such as a peripheral component interconnect (PCI) bus.

[0152] The storage device 2030 may store data of the output image Out_Img, etc. The data stored in the storage device 2030 may be saved not only when the computing system 2000 is driven but also when the computing system 2000 is not driven. For example, the storage device 2030 may be configured with at least one of all types of nonvolatile storage devices such as a flash memory device, a solid state drive (SSD), a hard disk drive (HDD), and an optical disk.

[0153] The storage device 2040 may store data of the output image Out_Img, etc. The storage device 2040 may temporarily store data to be processed by the processor 2020 or temporarily store data processed by the processor 2020. The data stored in the storage device 2040 may be saved only when the computing system 2000 is driven. Alternatively, the data stored in the storage device 2040 may be saved not only when the computing system 2000 is driven but also when the computing system 2000 is not driven. For example, the storage device 2040 may include a volatile storage device (such as a dynamic random access memory (DRAM) and a static random access memory (SRAM)) and a non-volatile storage device (such as an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), and a flash memory device).

[0154] The I / O device 2050 may include an input device and an output device. An input device is a device capable of inputting a user command through interaction, and may be implemented as, for example, a keyboard, a keypad, a mouse, a microphone, etc. An output device is a device capable of outputting data, and may be implemented as a printer, a speaker, etc.

[0155] The display device 2060 is a device for visually outputting data of the output image Out_Img, etc. To this end, the display device 2060 can be implemented with various types of displays, such as a liquid crystal display (LCD) that uses a separate backlight unit (e.g., a light emitting diode (LED), etc.) as a light source to control the molecular arrangement of liquid crystals, thereby adjusting the degree of light emitted from the backlight unit to be transmitted through the liquid crystals (brightness of light or intensity of light), and a display that uses a self-luminous element (e.g., a small LED with a size of 100 to 200 μm, a micro LED with a size of 100 μm or less, an organic LED (OLED), a quantum dot LED (QLED), etc.) as a light source. The display device 2060 can emit red, green, and blue light corresponding to the output image Out_Img to the outside.

[0156] According to the present disclosure, in one embodiment, an electronic device capable of improving color reproducibility of an image while reducing noise and an operating method of the electronic device may be provided.

[0157] Although the present disclosure has been shown and described with reference to certain examples of the embodiments of the present disclosure, it will be understood by those skilled in the art that various changes may be made in form and details without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents. Therefore, the scope of the present disclosure should not be limited to the examples of the above-mentioned embodiments, but should be determined not only by the appended claims but also by their equivalents.

[0158] In the above-described embodiments, all steps may be selectively performed, or some steps may be omitted. In each embodiment, the steps do not have to be performed according to the described order, and may be rearranged. The embodiments disclosed in this specification and the accompanying drawings are merely examples that are helpful for understanding the present disclosure, and the present disclosure is not limited thereto. That is, it should be understood by those skilled in the art that various modifications may be made based on the technical scope of the present disclosure.

[0159] In addition, examples of embodiments of the present disclosure have been described in the drawings and the specification. Although specific terms are used herein, these terms are only used to explain the embodiments of the present disclosure. Therefore, the present disclosure is not limited to the above-described embodiments, and many changes within the spirit and scope of the present disclosure are possible. It is obvious to those skilled in the art that, in addition to the embodiments disclosed herein, various modifications may also be made based on the technical scope of the present disclosure.

Claims

1. An electronic device, comprising: an image sensor that acquires a pixel value of a first pixel sensed during a first exposure time and a pixel value of a second pixel sensed during a second exposure time longer than the first exposure time; a controller that outputs an output image acquired based on the pixel value of the first pixel and a corrected saturated pixel value obtained by correcting the pixel value of the saturated pixel using the pixel value of at least one first pixel among the first pixels that is closest to a position of a saturated pixel having a pixel value exceeding a threshold value among the second pixels; as well as a memory storing a color conversion matrix for converting at least one of the pixel value of the first pixel and the pixel value of the second pixel into a pixel value corresponding to green, Wherein, each of the pixel value of the first pixel and the pixel value of the second pixel corresponds to one of yellow and white.

2. The electronic device according to claim 1, wherein: The controller includes a signal processor, wherein: acquiring a corrected first pixel value obtained by correcting the pixel value of the first pixel using a ratio of the first exposure time to the second exposure time; acquiring the corrected saturated pixel value by using a corrected first pixel value corresponding to the at least one first pixel among the corrected first pixel values; and acquiring a corrected image including the corrected first pixel value and the corrected saturated pixel value, and The sensitivity of the first pixel and the second pixel is higher than the sensitivity of green.

3. The electronic device according to claim 2, wherein: The signal processor obtains one of an average value and a median value of first pixel values ​​corresponding to the at least one first pixel as the corrected saturated pixel value.

4. The electronic device according to claim 2, wherein: The plurality of pixel values ​​included in the corrected image include a pixel value corresponding to one of yellow and white, a pixel value corresponding to red, and a pixel value corresponding to blue.

5. The electronic device according to claim 4, wherein: The controller includes a color converter that obtains the output image through matrix calculation of the pixel value corresponding to one of the yellow and white colors, the pixel value corresponding to the red color, and the pixel value corresponding to the blue color with the color conversion matrix.

6. The electronic device according to claim 1, wherein: The output image includes pixel values ​​corresponding to green, pixel values ​​corresponding to red, and pixel values ​​corresponding to blue.

7. The electronic device according to claim 1, wherein: Each of the pixel value of the first pixel and the pixel value of the second pixel is a pixel value corresponding to one of yellow and white.

8. The electronic device according to claim 1, wherein: A ratio of the number of the first pixels to the number of the second pixels is one of 1:3 and 1:

7.

9. The electronic device according to claim 1, wherein: The controller includes an exposure time controller that sends a command for adjusting the first exposure time of the first pixel and the second exposure time of the second pixel to the image sensor.

10. A method of operating an electronic device, the method comprising: acquiring pixel values ​​of a plurality of pixels, the plurality of pixels including a first pixel sensed during a first exposure time and a second pixel sensed during a second exposure time longer than the first exposure time; correcting the pixel value of the first pixel based on a ratio of the first exposure time to the second exposure time; detecting a saturated pixel having a pixel value exceeding a threshold value among the second pixels; correcting a saturated pixel value of the saturated pixel by using a corrected first pixel value of at least one first pixel among the first pixels that is closest to a position of the saturated pixel; storing a color conversion matrix, the color conversion matrix being used to convert at least one of the pixel value of the first pixel and the pixel value of the second pixel into a pixel value corresponding to green; as well as outputting an output image based on the corrected first pixel value and the corrected saturated pixel value of the first pixel, Wherein, each of the pixel value of the first pixel and the pixel value of the second pixel corresponds to one of yellow and white.

11. The method according to claim 10, wherein: In correcting the saturated pixel value, one of an average value and a median value of the corrected first pixel values ​​of the at least one first pixel is corrected to the saturated pixel value, and The sensitivity of the first pixel and the second pixel is higher than the sensitivity of green.

12. The method according to claim 10, wherein: Outputting the output image includes: acquiring a corrected image, the corrected image including pixel values ​​of other pixels except the first pixel and the saturated pixel among the pixel values ​​of the plurality of pixels, a corrected first pixel value of the first pixel, and the corrected saturated pixel value; and The output image including the pixel value corresponding to the green color, the pixel value corresponding to the red color, and the pixel value corresponding to the blue color is acquired through matrix calculation of the corrected image and the color conversion matrix.

13. The method according to claim 10, wherein: In correcting the pixel value of the first pixel, the pixel value of the first pixel is corrected to a value calculated by multiplying each of the pixel values ​​of the first pixel by a calculation of a ratio of the first exposure time to the second exposure time.

14. The method according to claim 10, further comprising: A second pixel having a pixel value exceeding the threshold value among the second pixels is detected as the saturated pixel through calculation comparing the pixel value of the second pixel with the threshold value.

15. The method according to claim 10, further comprising: At least one first pixel having a minimum distance value among the distance values ​​of the saturated pixel and the first pixel is detected as the at least one first pixel.

16. The method according to claim 10, further comprising: The first exposure time of the first pixel and the second exposure time of the second pixel are adjusted.

Images