A method for removing a bad pixel from a pixel image produced by an image sensor, an image sensor that uses the method, and an application processor that uses the method.

Abstract

Image sensor that includes: an image generation unit (105) configured to generate a pixel image according to the intensity of the incident light; and a bad pixel processing unit (200) configured to detect a bad pixel and output a compensated pixel image, the bad pixel processing unit (200) comprises: a bad pixel detection unit (220) configured to compare coordinates of a first pixel with pre-stored (S10) coordinates of a representative pixel representing at least one bad pixel, to determine whether the representative pixel is contained in a kernel, wherein the kernel has the first pixel at a center of the kernel, and to determine whether the first pixel is a bad pixel if the representative pixel is contained in the kernel; and a bad pixel compensation unit (230) configured to compensate the first pixel by means of a second pixel in the kernel when the first pixel is determined to be a bad pixel, and to output the compensated pixel image, wherein the bad pixel detection unit (220) outputs the first pixel as the compensated pixel image without performing bad pixel compensation if the representative pixel is not contained in the kernel.

Description

TECHNICAL AREA

[0001] Embodiments of the present inventive concept relate to a method for removing a bad pixel from a pixel image captured by an image sensor, and in particular an image sensor using the method and an application processor using the method. DISCUSSION OF THE STATE OF THE TECHNOLOGY

[0002] A pixel array in an image sensor can comprise a large number of pixels that convert a captured image of an object into electrical signals. This large number of pixels can include defective pixels that do not function correctly.

[0003] Post-image processing to identify and compensate for bad pixels would improve quality and performance.

[0004] US 2013 / 0162871 A1 relates to a method for correcting image defects in a color filter array (CFA). The method describes the correction of image defects prior to color interpolation. Both isolated defective pixels (singles) and adjacent defective pixels (couplets) are detected and corrected using a two-stage ring and singlet corrector. The ring corrector detects a defective pixel within a ring of pixels surrounding a central pixel. The singlet corrector detects and corrects the central pixel and removes a pair of pixels if the ring corrector is enabled, while with the ring corrector disabled, the singlet corrector corrects only individual defects. The correction takes into account whether the image is locally flat or textured and automatically adapts to recording conditions such as analog gain. A peak-and-valley detector prevents overcorrection at true signal peaks. SUMMARY

[0005] The invention is defined by the subject matter of the independent claims. Preferred embodiments are defined by the dependent claims.

[0006] According to one embodiment of the present inventive concept, a method for removing a bad pixel from a pixel image is provided. The method comprises determining whether a representative pixel, which represents at least one bad pixel, is contained in a kernel, determining whether a first pixel is a bad pixel if the representative pixel is contained in the kernel, and compensating the first pixel with a second pixel in the kernel if it is determined that the first pixel is a bad pixel. The kernel has the first pixel at its center.

[0007] The procedure may further include adjusting an initial pixel range of the kernel if the representative pixel is contained in the kernel.

[0008] Determining whether the representative pixel is contained in the kernel can involve comparing the coordinates of the first pixel with the coordinates of the representative pixel, calculating a horizontal and a vertical offset based on the comparison result, and determining whether the horizontal and vertical offsets are within a predetermined value corresponding to a kernel size. The horizontal offset can be a distance in a first direction between the first pixel and the representative pixel. The vertical offset can be a distance in a second direction perpendicular to the first direction between the first pixel and the representative pixel.

[0009] The process can also include storing information about the representative pixel.

[0010] The information about the representative pixel can include the coordinates of the representative pixel, or the number of bad pixels next to the representative pixel.

[0011] The kernel size can be 5x5.

[0012] According to an exemplary embodiment of the present inventive concept, an image sensor is provided. The image sensor can comprise an image generation unit and a bad-pixel processing unit. The image generation unit is configured to generate a pixel image according to the intensity of the incident light. The bad-pixel processing unit is configured to detect a bad pixel and output a compensated pixel image. The bad-pixel processing unit comprises a bad-pixel detection unit and a bad-pixel compensation unit. The bad-pixel detection unit is configured to determine whether a representative pixel, which represents at least one bad pixel, is contained in a kernel and to determine whether a first pixel is a bad pixel if the representative pixel is contained in the kernel.The bad pixel compensation unit is configured to compensate for the first pixel using a second pixel in the kernel if the first pixel is determined to be a bad pixel. The kernel has the first pixel located at its center.

[0013] The bad pixel detection unit can adjust an initial pixel range of the kernel if the representative pixel is contained within the kernel.

[0014] The bad pixel detection unit can compare the coordinates of the first pixel with the coordinates of the representative pixel, calculate a horizontal and a vertical offset based on the comparison result, and determine whether the horizontal and vertical offsets are within a predetermined value, corresponding to a kernel size. The horizontal offset can be a distance in a first direction between the first pixel and the representative pixel. The vertical offset can be a distance in a second direction perpendicular to the first direction between the first pixel and the representative pixel.

[0015] The bad pixel processing unit may also include a bad pixel memory configured to store information about the representative pixel.

[0016] The information about the representative pixel can include the coordinates of the representative pixel, or the number of bad pixels next to the representative pixel.

[0017] The kernel size can be 5x5.

[0018] The bad pixel detection unit can output a first pixel as the compensated pixel image if the representative pixel is contained in the kernel.

[0019] The image generation unit can comprise a pixel array, a readout block, and a control unit. The pixel array can include a multitude of pixels, each configured to generate an electrical signal that varies depending on the intensity of the incident light. The readout block can be configured to convert the electrical signal into the pixel image in a digital format. The control unit can be configured to control the pixel array and the readout block.

[0020] According to an exemplary embodiment of the present inventive concept, an application processor is provided. The application processor comprises a camera interface and an image signal processor. The camera interface is configured to receive a pixel image. The image signal processor is configured to process the pixel image to generate image data. The image signal processor comprises a bad pixel detection unit and a bad pixel compensation unit. The bad pixel detection unit is configured to determine whether a representative pixel, which represents at least one bad pixel, is contained in a kernel and to determine whether a first pixel is a bad pixel if the representative pixel is contained in the kernel.The bad pixel compensation unit is configured to compensate for the first pixel using a second pixel in the kernel if the first pixel is determined to be a bad pixel. The kernel has the first pixel located at its center.

[0021] The bad pixel detection unit can adjust an initial pixel range of the kernel if the representative pixel is contained within the kernel.

[0022] The bad pixel detection unit can compare the coordinates of the first pixel with the coordinates of the representative pixel, calculate a horizontal and a vertical offset based on the comparison result, and determine whether the horizontal and vertical offsets are within a predetermined value, corresponding to a kernel size. The horizontal offset can be a distance in a first direction between the first pixel and the representative pixel. The vertical offset can be a distance in a second direction perpendicular to the first direction between the first pixel and the representative pixel.

[0023] The image signal processor may also include a bad pixel memory configured to store information about the representative pixel.

[0024] The information about the representative pixel can include the coordinates of the representative pixel, or the number of bad pixels next to the representative pixel.

[0025] The kernel size can be 5x5.

[0026] The bad pixel detection unit can output a first pixel as the compensated pixel image if the representative pixel is contained in the kernel.

[0027] According to one embodiment of the present inventive concept, a method for processing a bad pixel from a pixel image is provided. The method comprises detecting a bad pixel using the coordinates of a representative pixel that represents at least one bad pixel, and compensating for a first pixel by means of a second pixel in a kernel if the first pixel is a bad pixel. The kernel has the first pixel at its center. Detecting a bad pixel comprises determining whether the representative pixel is contained in the kernel, determining whether the first pixel is a bad pixel if the representative pixel is contained in the kernel, and determining that the first pixel is a bad pixel if a brightness value of the first pixel is not within a first pixel range. The first pixel range is determined based on an average brightness of the kernel.

[0028] The detection process can also include reducing the first pixel area of ​​the kernel if the representative pixel is contained within the kernel.

[0029] Detecting the bad pixel may also include outputting the first pixel as a compensated pixel image if the representative pixel is not included in the kernel.

[0030] The detection of the bad pixel may further include outputting the first pixel as a compensated pixel image if the brightness value of the first pixel is within the first pixel range in the kernel.

[0031] The second pixel can surround the first pixel in the kernel and have essentially the same color as the first pixel. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The above and other features and advantages of the inventive ideas will become clearer through the detailed description of exemplary embodiments thereof, with reference to the attached drawings, in which the following applies: Fig. Figure 1 is a block diagram of a data processing system with an image sensor, according to an exemplary embodiment of the inventive concept; Fig. Figure 2 is a block diagram of a data processing system according to an embodiment of the present inventive concept; Fig. Figure 3 is a block diagram of a bad pixel processing unit, as shown in Fig. 1 and Fig. 2, according to an exemplary embodiment of the present inventive concept; Fig. Figure 4 is a diagram of a pixel image being fed into the bad pixel processing unit. Fig. 3 is entered, according to an embodiment of the present inventive concept; Fig. Figure 5 is a diagram to explain the operation of the bad pixel detection unit. Fig. 3, according to an embodiment of the present inventive concept. Fig. Figure 6 is a diagram illustrating a procedure for generating representative pixel information stored in a bad pixel memory. Fig. 3 are stored, according to one embodiment of the present inventive concept. Fig. 7 is a diagram for comparing data stored in a low-pixel memory. Fig. 3 are stored, on a case-by-case basis, according to one embodiment of the present inventive concept. and Fig. Figure 8 is a flowchart of the operation of a bad pixel processing unit. Fig. 3, according to an embodiment of the present inventive concept. DETAILED DESCRIPTION OF THE EXECUTION FORMS

[0033] The present inventive concept will now be explained in more detail with reference to the accompanying drawings, which show embodiments of the inventive ideas. However, the present inventive concept can be embodied in many different forms without departing from the spirit and scope of the present inventive concept and should not be interpreted as limiting it to the embodiments described below. In the drawings, the size and relative dimensions of the layers and areas are exaggerated for clarity. The same reference numerals refer to the same elements throughout the description and in all figures.

[0034] The term “and / or” used herein includes any combination of one or more of the related listed items and can be shortened by “ / ”.

[0035] The singular forms “ein(e)” and “der / die / das” used herein are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0036] Fig. Figure 1 is a block diagram of an image processing system 10 comprising an image sensor 100 according to an embodiment of the present inventive concept. The image processing system 10 may comprise the image sensor 100, a digital signal processor (DSP) 300, a display unit 400, and a lens 450. The image sensor 100 may comprise an image generation unit 105 and a bad pixel processing unit 200.

[0037] The image generation unit 105 can generate a pixel image PI corresponding to the intensity of light passing through the lens 450. The image generation unit 105 can comprise a pixel array 110, a readout block 120, and a control unit 130.

[0038] The pixel array 110 can comprise a plurality of pixels, each of which accumulates photocharges generated in response to light coming from an object 350 through the lens 450, producing a pixel signal corresponding to the photocharges. The pixels can be arranged in a matrix of n rows and m columns, where n and m are integers of at least 1.

[0039] Each pixel contains a multitude of transistors and a multitude of photoelectric converter elements. Each of the photoelectric converter elements can be a photodiode, a pinned photodiode, or similar. The pixel array 110 captures light using the multitude of photoelectric converter elements and converts the light into electrical signals, thus generating the pixel signals. The pixels can be described as image pixels in the sense that they generate a signal corresponding to a captured image.

[0040] The read block 120 can remove noise from a pixel signal (for example, reset noise) generated by each pixel and perform analog-to-digital conversion of the removed pixel signal. The read block 120 can temporarily store the pixel image PI, which corresponds to the pixel signal in a digital format after conversion, amplify the pixel image PI, and output the amplified pixel image PI. The read block 120 can include an analog-to-digital converter (ADC) that removes the noise and performs the analog-to-digital conversion, a memory (for example, static random-access memory (SRAM)) that temporarily stores the pixel image PI, and a buffer that amplifies and outputs the pixel image PI.

[0041] The control unit 130 can generate a variety of control signals for controlling the operations of the pixel array 110 and the readout block 120. The control unit 130 can include a row driver 140, a column driver 150, a clock generator 160, and a control register block 170.

[0042] The row driver 140 operates the pixel array 110 by means of a single row. For example, pixels in a row can be provided with the same control signal. The row driver 140 can decode a control signal output by the clock generator 160 and provide control signals for the pixel array 110.

[0043] The column driver 150 can generate a variety of control signals according to the control of the clock generator 160 to control the operation of the read block 120. The clock generator 160 can apply a control signal to the row driver 140 and the column driver 150 to control the operation or timing of the row driver 140 and the column driver 150. The clock generator 160 can generate the control signal or a clock signal to be applied to the row driver 140 and the column driver 150 by means of a control signal and a clock signal received from an external device.

[0044] The control register block 170 operates according to the control of a camera control unit 310 and can store or buffer the control signal and the clock signal. Furthermore, the control register block 170 can control the image generation unit 105 and the bad pixel processing unit 200 of the image sensor 100.

[0045] The bad pixel processing unit 200 can receive the pixel image PI, detect bad pixels within the pixel image PI, compensate for the bad pixels, and output a balanced pixel image CPI. The pixel image PI can be a stream signal in which the digital values ​​of the respective pixels in the pixel array 110 are sequentially arranged. For example, if the pixel array 110 comprises pixels arranged in a matrix of ten rows and ten columns, the pixel image PI can be a stream signal in which a total of 100 digital values ​​are sequentially arranged, from a digital value corresponding to a pixel signal output by a pixel in the first row and first column to a digital value corresponding to a pixel signal output by a pixel in the tenth row and tenth column.

[0046] A bad pixel is a pixel in the PI image that outputs a different signal than the captured image. A pixel can be bad due to, for example, a fault in a transistor within the pixel.

[0047] The Bad Pixel Processing Unit 200 sequentially receives the pixels of the pixel image PI, identifies each pixel as a target pixel, recognizes which pixel is a bad pixel, and compensates for the detected bad pixel. The operation of the Bad Pixel Processing Unit 200 is described using the following example: Fig. 3 to 7 later.

[0048] The DSP 300 can generate image data by processing the compensated pixel image (CPI) output by the image sensor 100 and can output the image data to the display unit 400. The DSP 300 can include the camera control unit 310, an image signal processor (ISP) 320, and a personal computer interface (PC I / F) 330.

[0049] The camera control unit 310 controls the control register block 170. The camera control unit 310 can control the control register block 170 using an integrated IC (I2C), but the scope of the present inventive concept is not limited to this.

[0050] The ISP 320 processes the compensated pixel image (CPI) output as image data by the image sensor 100 and outputs the image data to the display unit 400 via the PC I / F 330. The ISP 320 is implemented on a chip separate from the image sensor 100. According to an exemplary embodiment of the present inventive concept, the ISP 320 and the image sensor 100 can be configured as a single chip.

[0051] The Display Unit 400 can be any device capable of outputting an image using image data generated by the DSP 300. For example, the Display Unit 400 could include a computer, a mobile phone, a smartphone, any type of image output terminal, or similar device.

[0052] Fig. Figure 2 is a block diagram of a data processing system 20, according to an embodiment of the present inventive concept. With regard to Fig. 1 and Fig. 2. The data processing system 20 can be implemented as a mobile device, such as a smartphone, a tablet personal computer (PC), a personal digital assistant (PDA), an enterprise digital assistant (EDA), a mobile internet device (MID), an e-book or the like.

[0053] The data processing system 20 can include an application processor 500, an image sensor 565, a display device 575, and a memory 585. The image sensor 565 can have the same components as the image sensor 100. Fig. 1 included, with the exception of the bad pixel processing unit 200.

[0054] The application processor 500 can include a central processing unit (CPU) 510, a read-only memory (ROM) 520, a random-access memory (RAM) 530, an ISP 540, a codec 550, a camera interface 560, a display interface 570, and a memory interface 580. The application processor 500 can be implemented as a system-on-a-chip (SoC). The components 510, 520, 530, 540, 550, 560, 570, and 580 of the application processor 500 can communicate data with each other via a bus 505.

[0055] The CPU 510 can control the entire operation of the application processor 500. The CPU 510 can process or execute programs and / or data stored in the ROM 520 and / or the RAM 530. The CPU 510 can be implemented as a multi-core processor, for example, a single computing component with two or more independent processors (or cores).

[0056] The ROM 520 can store programs and / or data that are used continuously. The ROM 520 can be constructed using erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or the like.

[0057] The RAM 530 can temporarily store programs, data, and / or instructions (for example, commands). The RAM 530 can be constructed using dynamic RAM (DRAM), SRAM, or similar technologies. The RAM 530 can temporarily store data input and / or output through interfaces 560, 570, and 580, or data generated by the Codec 550, the CPU 510, or similar devices.

[0058] The ISP 540 can perform image processing on data received from the RAM 530, the camera interface 560, the memory interface 580, or similar interfaces, and can output processed image data to the RAM 530, the display interface 570, the memory interface, or similar interfaces. The ISP 540 can be implemented as software, hardware, a combination thereof, or similar.

[0059] The ISP 540 can replace the ISP 320. Fig. 1, or can the bad pixel processing unit 200 also be from Fig. 1 include as in Fig. 2 shown. For example, the bad pixel processing unit 200 is in the image sensor 100, as shown in Fig. 1 shown, but the bad pixel processing unit 200 may be contained in the ISP 540 outside the image sensor 565, as shown in Fig. 2 shown.

[0060] The Codec 550 can encrypt or decrypt data received from the image sensor 565, or the encoded or decoded data transmitted to the display unit 575. The Codec 550 can include an encoder and a decoder.

[0061] The camera interface 560 can connect to data (e.g., the pixel image PI) input by the image sensor 565 on the outside of the application processor 500. The camera interface 570 can connect to data (e.g., image data) output to the display device 575 on the outside of the application processor 500. The display device 575 can output an image or data about an image through a display such as a liquid crystal display (LCD), an active-matrix organic light-emitting diode (AMOLED) display, or the like.

[0062] The 580 memory interface can couple data input from the 585 memory located on the outside of the 500 application processor, or data output to the 585 memory. The 585 memory can be configured with non-volatile memory, such as flash memory, resistive memory, or the like.

[0063] Fig. Figure 3 is a block diagram of a bad pixel processing unit, as shown in Fig. 1 and Fig. 2, according to an exemplary embodiment of the present inventive concept. With regard to Fig. The bad pixel processing unit 200 can comprise a bad pixel memory 210, a bad pixel acquisition unit 220, and a bad pixel compensation unit 230. The bad pixel processing unit 200 can be implemented as software, hardware, a combination thereof, or similarly.

[0064] The bad pixel memory 210 can store representative pixel information (RPI) for a representative pixel. The bad pixel memory 210 can be composed of non-volatile memory, such as ROM (e.g., EPROM, EEPROM, or similar), flash memory, or resistor memory.

[0065] A representative pixel is a pixel that is either a bad pixel or a pixel adjacent to one or more bad pixels. The representative pixel may be located at the same position as a bad pixel (for example, a first bad pixel) or may be positioned essentially in the middle of more than one bad pixel; however, the present inventive concept is not limited to this. The representative pixel information (RPI) may include the coordinates of the representative pixel and information about the number of bad pixels adjacent to the representative pixel.

[0066] The terms "kernel" and "middle pixel," which will be mentioned below, are explained first. The kernel is a group of pixels that are subjected to poor pixel detection and poor pixel compensation at a time, and the kernel can have a size of axb (where "a" is the number of pixels in a row, "b" is the number of pixels in a column, and "a" and "b" are integers of at least 1), for example, 2x2, 3x3, 5x5, 7x7, or the like. The middle pixel is a pixel located in the center of the kernel. For example, in a 2x2 kernel, the middle pixel can be a pixel at the top-left position, but is not limited to that, and in a 5x5 kernel, the central pixel can be a pixel at the intersection of the third row and third column.

[0067] The bad pixel detection unit 220 can determine whether a target pixel is a bad pixel and whether the representative pixel is included in a kernel where the target pixel is a medium pixel. A target pixel can be any pixel in the pixel image. In the case of a streaming pixel image, a first pixel is treated as the target pixel, and a second pixel adjacent to the first pixel (for example, in a raster direction, as in Fig. 4) In the pixel image PI, it is treated like the next target pixel.

[0068] The bad pixel detection unit 220 can comprise a target pixel and a kernel, where the target pixel is a center pixel. For example, if a pixel P1 from Fig. If pixel 4 is designated as a target pixel, a first kernel K1 with pixel P1 as a center pixel can be selected to perform bad pixel processing.

[0069] The Bad Pixel Detection Unit 220 can compare the coordinates of a target pixel (or the coordinates of the detected kernel) with the coordinates of a representative pixel contained in the Representative Pixel Information (RPI) and can determine whether the representative pixel is contained in the kernel.For example, the Bad Pixel Detection Unit 220 can compare the coordinates of a target pixel with the coordinates of a representative pixel, calculate a horizontal offset (for example, a difference between the X-coordinates of the target pixel and the representative pixel) and a vertical offset (for example, a difference between the Y-coordinates of the target pixel and the representative pixel), and determine whether the absolute values ​​of the horizontal and vertical offsets are equal to or less than a predetermined value, corresponding to a kernel size, to determine whether the representative pixel is contained in the kernel. For example, the predetermined value can be equal to n / 2 if the kernel size is equal to n x n (where n is an integer of at least 1).

[0070] For example, if the kernel size is 5x5 and the horizontal and vertical offsets between the coordinates of a target pixel and the coordinates of a representative pixel are +1 and -2 respectively, it can be determined that the representative pixel is in the kernel. If the horizontal and vertical offsets between the coordinates of the target pixel and the coordinates of the representative pixel are +4 and -2 respectively, it can be determined that the representative pixel is not in the kernel.

[0071] If the representative pixel is not included in the kernel, the bad pixel detection unit 220 can output the target pixel as the compensated pixel image (CPI). If the representative pixel is included in the kernel, the bad pixel detection unit 220 can determine whether the target pixel is a bad pixel. The bad pixel detection unit 220 can determine whether a value of the target pixel is within a predetermined normal pixel range (NPR) to determine whether the target pixel is a bad pixel or not; however, the present inventive concept is not limited to this embodiment. For example, the value of the target pixel can be the brightness of the target pixel, and the normal pixel range (NPR) can be determined based on the average brightness of the kernel with the target pixel as the center pixel. The higher the average brightness, the higher the average level (e.g.,the mean value between a maximum value and a minimum value) of the normal pixel area NPR and the larger the size (e.g. a difference between the maximum value and the minimum value) of the normal pixel area NPR.

[0072] If the representative pixel is contained in the kernel, or if the number of bad pixels adjacent to the representative pixel is greater than a predetermined number (for example, 2), the bad pixel detection unit 220 can adjust the normal pixel area NPR of the kernel, for example, the bad pixel detection unit 220 can decrease the normal pixel area NPR of the kernel.

[0073] If the target pixel is not within the normal pixel area (NPR), the bad pixel detection unit 220 may consider the target pixel a bad pixel and transfer it to the bad pixel compensation unit 230. If the target pixel is within the normal pixel area (NPR), the bad pixel detection unit 220 may consider the target pixel a non-bad pixel and output the target pixel as the compensated pixel image (CPI).

[0074] The bad pixel compensation unit 230 can perform compensation of the target pixel received from the bad pixel detection unit 220. The bad pixel compensation unit 230 can compensate the target pixel using surrounding pixels of the target pixel in the kernel with the target pixel as the center pixel. The compensation can be performed using an average value of the pixels that have the same color (for example, red, blue, or green) as the target pixel among the surrounding pixels, or by filtering the surrounding pixels; however, the present inventive concept is not limited to this embodiment. The bad pixel compensation unit 230 can output a compensated target pixel as the compensated pixel image (CPI).

[0075] Fig. Figure 4 is a diagram of a pixel image PI that is fed into the bad pixel processing unit 200. Fig. 3 is entered, according to an embodiment of the present inventive concept. Fig. Figure 5 is a diagram to explain the operation of the Bad Pixel Detection Unit 220. Fig. 3, according to an embodiment of the present inventive concept. Fig. Figure 6 is a diagram illustrating a procedure for generating representative pixel information RPI from a bad pixel memory 210. Fig. 3 are stored, according to one embodiment of the present inventive concept. Fig. Figure 7 is a diagram for comparing data stored in a bad pixel memory 210. Fig. 3 are stored, on a case-by-case basis, according to one embodiment of the present inventive concept.

[0076] In relation to Fig. 1 to 7, shows Fig. 4. An example of the pixel image PI. For illustrative purposes, the pixel array comprises 110 pixels arranged in a matrix of 10 rows and 10 columns. The pixel image PI is formed in a matrix of 10 rows and 10 columns, and each pixel of the pixel image PI is defined by a specific X-coordinate (for example, one from X1 to X10) and a Y-coordinate (for example, one from Y1 to Y10). For example, the coordinate of pixel P1 is (X4, Y4). Here, the number of rows and columns in the matrix are only examples, and the present inventive concept is not limited to this example.

[0077] The Bad Pixel Detection Unit 220 can receive pixels sequentially from a pixel at coordinate (X1, Y1) to a pixel at coordinate (X10, Y1) in the first row, and from a pixel at coordinate (X1, Y2) to a pixel at coordinate (X10, Y2) in the second row. For example, the Bad Pixel Detection Unit 220 can consider any of the pixels sequentially input in the raster direction from the first row to the tenth row as a target pixel and can perform bad pixel detection sequentially for each target pixel.

[0078] With reference to Fig. In the first kernel K1, pixel P1 is a target pixel, which is a center pixel of the first kernel K1. A second kernel K2 has pixel P2 as a target pixel, which is a center pixel of the second kernel K2. A third kernel K3 has a representative pixel R1 as a target pixel, which is a center pixel of the third kernel K3. It is assumed that there are two bad pixels, B1 and B2, in the pixel image PI.

[0079] With reference to Fig. In the first kernel K1, pixel P1 is designated as a target pixel, and a center pixel C1 is designated as the center pixel. If the bad pixel detection unit 220 identifies pixel P1 as the target pixel, it determines whether the representative pixel R1 is present in the first kernel K1. Since the representative pixel R1 is not present in the first kernel K1, the bad pixel detection unit 220 can output the target pixel. For example, the bad pixel detection unit 220 can output pixel P1 as a compensated pixel image (CPI) without performing bad pixel detection and bad pixel compensation. Here, bad pixel detection is an operation to determine if the target pixel is a bad pixel, and bad pixel compensation is an operation to compensate for the target pixel that was detected as a bad pixel.

[0080] The second kernel K2 has pixel P2 as a target pixel and a center pixel C2 of the second kernel K2. If the bad pixel detection unit 220 determines pixel P2 as the target pixel, it determines whether the representative pixel R1 is present in the second kernel K2. Since the representative pixel R1 is present in the second kernel K2, the bad pixel detection unit 220 performs the bad pixel detection.

[0081] The bad pixel detection unit 220 determines a normal pixel range (NPR) of pixel values ​​corresponding to pixel positions P11 to P55 (where PAB denotes the position of an intersection of an A-th row and a B-th column) in the second kernel K2 and determines whether a pixel value corresponding to position P33 of the target pixel P2 is within the normal pixel range NPR. Since the pixel value corresponding to position P33 is within the normal pixel range NPR, the bad pixel detection unit 220 cannot send the target pixel P2 to the bad pixel compensation unit 230 for bad pixel compensation and can output the compensated pixel image (CPI) to the target pixel P2.

[0082] If the representative pixel R1 is contained within the second kernel K2, or if the number of bad pixels adjacent to the representative pixel R1 is greater than a predetermined number (for example, 2), the bad pixel detection unit 220 can adjust the normal pixel area NPR. For example, the bad pixel detection unit 220 can decrease the normal pixel area NPR of the second kernel. If the representative pixel R1 is contained within the second kernel K2, or if the number of bad pixels adjacent to the representative pixel R1 is greater than a predetermined number (for example, 2), it can be understood that there is a relatively high probability that bad pixels are contained within the second kernel K2. The smaller the normal pixel area NPR, the more precise the bad pixel detection can be.Therefore, the bad pixel detection unit 220 can adjust (e.g., reduce) the normal pixel area NPR, and thus, it can be determined more accurately whether the target pixel P2 is a bad pixel or not.

[0083] The third kernel K3 has the representative pixel R1 as a target pixel and a center pixel C3 of the third kernel K3. When the bad pixel detection unit 220 determines that the representative pixel R1 is the target pixel, it determines whether the representative pixel R1 is contained in the third kernel K3. Since the representative pixel R1 is contained in the third kernel K3, the bad pixel detection unit 220 performs the bad pixel detection.

[0084] The bad pixel detection unit 220 determines the normal pixel range (NPR) of pixel values ​​corresponding to pixel positions P11 to P55 in the third kernel K3 and determines whether a pixel value corresponding to position P33 of the target pixel R1 is within the normal pixel range (NPR). Since the pixel value corresponding to position P33 is outside the normal pixel range (NPR), the bad pixel detection unit 220 can transfer the target pixel R1 to the bad pixel compensation unit 230 for bad pixel compensation.

[0085] If the representative pixel R1 is contained in the third kernel K3, or if the number of bad pixels adjacent to the representative pixel R1 is greater than a predetermined number (for example, 2), the bad pixel detection unit 220 can adjust (e.g., shrink) the normal pixel area NPR.

[0086] With reference to Fig. 6. The sensor tester 600 generates the representative pixel information RPI. The sensor tester 600 can be implemented as an electrical die sorting (EDS) tester, but the present inventive concept is not limited to this embodiment.

[0087] The Sensor Tester 600 can analyze the pixel image (PI) generated by the 100 or 565 image sensor under various conditions, such as a white level (e.g., high illuminance), a black level (e.g., low illuminance), or similar conditions, and can detect pixels at specific positions as defective. The Sensor Tester 600 can identify a representative pixel as defective based on the position of each pixel.

[0088] The representative pixel is a pixel that represents adjacent bad pixels. If a large number of bad pixels are contained in a particular kernel (for example, a kernel used by Bad Pixel Processing Unit 200), the large number of bad pixels present in the kernel can be interpreted as neighboring bad pixels.

[0089] The representative pixel is a pixel whose horizontal and vertical offset from each of the neighboring bad pixels is equal to or less than a predetermined value corresponding to a kernel size. For example, the predetermined value might be n / 2 if the kernel size is n x n. For instance, if a bad pixel is positioned as shown in Fig. As can be seen in Figure 4, one of the pixels within a specific X-coordinate range from X6 to X9 and a specific Y-coordinate range from Y6 to Y10 can be determined as the representative pixel.

[0090] The Sensor Tester 600 can generate the representative pixel information RPI, including the coordinate of the representative pixel and information about the number of bad pixels next to the representative pixel, and can store the representative pixel information RPI in the bad pixel memory 210. The information about the number of bad pixels next to the representative pixel does not have to be included in the representative pixel information RPI.

[0091] In Fig. 7. Assume that a fourth core K4 comprises two bad pixels B3 and B4 within a specific X-coordinate range from X21 to X25 and a specific Y-coordinate range from Y21 to Y25, and that a pixel at a position (X23, Y23) is designated as the representative pixel R2. The size of the data stored in the bad pixel memory 210 can vary in different cases CASE1 to CASE4. For example, each of the X and Y coordinates of a pixel is composed of 13 bits, pattern information is composed of 6 bits, and offset information is composed of 6 bits, but the present inventive concept is not limited to these.

[0092] The pattern information corresponds to a relative positional relationship formed by at least one bad pixel. For example, if there is only one bad pixel, the pattern information is expressed as "000000"; if there are two bad pixels separated by one pixel horizontally, the pattern information is expressed as "000001"; and if there are two bad pixels separated by one pixel vertically, the pattern information is expressed as "000010". The offset information includes a horizontal offset and a vertical offset of each bad pixel from a given bad pixel.

[0093] In the first case, CASE 1, according to one embodiment of the present inventive concept, initial data DATA 1, comprising only the coordinates (for example, X and Y coordinates) of the representative pixel R2, can be stored as the representative pixel information RPI in the bad pixel memory 210. For example, the capacity of the bad pixel memory 210 required to process bad pixels B3 and B4 is 26 bits, corresponding to the coordinates of the representative pixel R2. This is because, according to one embodiment of the present inventive concept, the bad pixel detection unit 220 is capable of detecting a bad pixel based solely on the coordinates of the representative pixel R2.

[0094] In the second case, CASE 2, as a comparative example, the second data DATA 2 comprises the coordinates (for example, X and Y coordinates) of bad pixel B3 and the pattern information about the relative positional relationship between bad pixels B3 and B4. For example, the capacity of the bad pixel memory 210 required to process bad pixels B3 and B4 is 32 bits, which corresponds to the coordinates of bad pixel B3 and the pattern information. This capacity is essentially 23% higher than in the first case, CASE 1.

[0095] In the third case, CASE 3, as a comparison example, the third data DATA 3 includes the coordinates of bad pixel B3 and the offset information about the horizontal offset (e.g., "000") and the vertical offset (e.g., "001") of bad pixel B4 from bad pixel B3. For example, the capacity of the bad pixel memory 210, which is needed to process bad pixels B3 and B4, is 32 bits, corresponding to the coordinates of bad pixel B3 and the offset information. This capacity is essentially 23% higher than in the first case, CASE 1.

[0096] In the fourth case, CASE 4, as a comparative example, the fourth data point, DATA 4, comprises the coordinates of bad pixel B3 and the coordinates of bad pixel B4. For example, the capacity of the bad pixel memory 210, required to process bad pixels B3 and B4, is 52 bits, corresponding to the coordinates of bad pixel B3 and the coordinates of bad pixel B4. This capacity is essentially 100% higher than in the first case, CASE 1.

[0097] With the current demand for miniaturized image sensors, the level of integration of a pixel array can be increased. Therefore, structures in which a large number of pixels share a single transistor can be used, and poor-quality pixels can be placed adjacent to each other. Additionally, the miniaturization of both image sensors and application processors allows for a reduction in the memory capacity required for poor-quality pixel processing.

[0098] Therefore, according to one embodiment of the present inventive concept, the image sensor 100 or the application processor 500 stores the coordinates of a representative pixel, representing neighboring bad pixels, in memory and performs bad pixel processing using these coordinates, thereby minimizing the memory capacity required for bad pixel processing. Furthermore, the image sensor 100 or the application processor 500 does not perform bad pixel detection or bad pixel compensation on the kernel, which does not include a representative pixel, thus reducing the amount of computational and power required for bad pixel processing.

[0099] Fig. Figure 8 is a flowchart of the operation of a bad pixel processing unit 200 from Fig. 3, according to one embodiment of the present inventive concept. With regard to Fig.1 to 8, the sensor tester can generate and store 600 representative pixel information RPI, which includes the coordinates of a representative pixel and information about the number of bad pixels adjacent to the representative pixel, in the bad pixel memory 210 in operation S10.

[0100] The bad pixel processing unit 200 can receive the pixel image PI from the image generation unit 105 of the image sensor 100 or the image sensor 565 in operation S11. The bad pixel detection unit 220 can determine whether the representative pixel is contained in a kernel with a target pixel as the center pixel in operation S12. The bad pixel detection unit 220 can compare the coordinates of the target pixel with the coordinates of the representative pixel to calculate a horizontal offset and a vertical offset, and can determine whether the representative pixel is contained in the kernel by determining whether the horizontal and vertical offsets are equal to or less than a predetermined value, corresponding to a kernel size. For example, the predetermined value can be n / 2 if the kernel size is n x n.

[0101] If the representative pixel is not included in the kernel (for example, in the case of NO) in operation S12, the bad pixel detection unit 220 can output the target pixel as a compensated pixel image (CPI) in operation S16. If the representative pixel is included in the kernel (for example, in the case of YES) in operation S12, the bad pixel detection unit 220 can determine and adjust a normal pixel area (NPR) of the kernel in operation S13. In one embodiment of the present inventive concept, if the number of bad pixels adjacent to the representative pixel is greater than a predetermined number (for example, 2), the bad pixel detection unit 220 can determine and adjust the normal pixel area (NPR) of the kernel in operation S13. In one embodiment of the present inventive concept, operation S13 can be omitted.

[0102] The bad pixel detection unit 220 can determine whether the target pixel is a bad pixel in operation S14. If the target pixel is not a bad pixel (for example, in the case of NO) in operation S14, the bad pixel detection unit 220 can output the target pixel as a compensated pixel image (CPI) in operation S16. If the target pixel is a bad pixel (for example, in the case of YES) in operation S14, the bad pixel compensation unit 230 can compensate the target pixel using pixels in the kernel in operation S15.

[0103] The bad pixel compensation unit 230 can output the compensated pixel image CPI, the target pixel that was compensated in operation S16. After the compensated pixel image CPI is output, the operation of the bad pixel processing unit 200 can be repeated until the bad pixel processing is complete for all pixels (for example, pixels corresponding to a single image). For example, the operation of the bad pixel processing unit 200 can be repeated if the bad pixel processing is not complete (for example, in the case of NO) in operation S17, and the operation of the bad pixel processing unit 200 can end when the bad pixel processing is complete for all pixels (for example, in the case of YES) in operation S17.

[0104] Furthermore, the present inventive concept can be implemented as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium is any data storage device capable of storing data that can subsequently be read by a computer system. Examples include ROM, RAM, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and the like. Additionally, the computer-readable recording medium can be distributed across a network-connected computer system, allowing the computer-readable codes to be stored and executed in a distributed manner. Moreover, functional programs, codes, and code segments for implementing the present inventive concept can be readily understood and derived by a person skilled in the art in the field to which the present inventive idea relates.

[0105] As described above, according to one embodiment of the present inventive concept, only the coordinates of a representative pixel representing neighboring bad pixels are stored in memory, and bad pixel processing is performed using these coordinates, thus minimizing the memory capacity required for bad pixel processing. Additionally, if the representative pixel is not contained in a kernel, bad pixel detection and bad pixel compensation are not performed on the kernel, thus minimizing the amount of computation and power consumption required for bad pixel processing.

[0106] While the inventive ideas have been shown and described primarily with reference to exemplary embodiments, it is understood that numerous changes to the form and details may be made herein without deviating from the spirit and scope of the following claims.

Claims

[1] Image sensor comprising: an image generation unit (105) configured to generate a pixel image according to the intensity of the incident light; and a bad pixel processing unit (200) configured to detect a bad pixel and output a compensated pixel image, the bad pixel processing unit (200) comprises: a bad pixel detection unit (220) configured to compare coordinates of a first pixel with pre-stored (S10) coordinates of a representative pixel representing at least one bad pixel, to determine whether the representative pixel is contained in a kernel, wherein the kernel has the first pixel at a center of the kernel, and to determine whether the first pixel is a bad pixel if the representative pixel is contained in the kernel; and a bad pixel compensation unit (230) configured to compensate the first pixel by means of a second pixel in the kernel when the first pixel is determined to be a bad pixel, and to output the compensated pixel image, wherein the bad pixel detection unit (220) outputs the first pixel as the compensated pixel image without performing bad pixel compensation if the representative pixel is not contained in the kernel. [2] Image sensor according to claim 1, wherein the bad pixel detection unit (220) adapts a first pixel area of ​​the kernel when the representative pixel is contained in the kernel. [3] Image sensor according to claim 1, wherein the bad pixel detection unit (220) compares coordinates of the first pixel with the coordinates of the representative pixel, calculates a horizontal offset and a vertical offset based on the comparison result, and determines whether the horizontal offset and the vertical offset are within a predetermined value corresponding to a kernel size, where the horizontal offset is a distance in a first direction between the first pixel and the representative pixel, and where the vertical offset is a distance in a second direction perpendicular to the first direction between the first pixel and the representative pixel. [4] Image sensor according to claim 1, wherein the bad pixel processing unit (200) further comprises a bad pixel memory (210) configured to store information about the representative pixel. [5] Image sensor according to claim 4, wherein the information about the representative pixel includes coordinates of the representative pixel, or the number of bad pixels adjacent to the representative pixel. [6] Image sensor according to claim 1, wherein a kernel size is equal to one of 2x2, 3x3, 5x5 and 7x7. [7] Image sensor according to claim 1, wherein the image generation unit (105) comprises: a pixel array (110) comprising a plurality of pixels, each configured to generate an electrical signal that varies depending on the intensity of the incident light; a readout block (120) configured to convert the electrical signal into the pixel image in a digital format; and a control unit (130) configured to control the pixel array (110) and the readout block (120). [8] Application processor, which includes: a camera interface (560) configured to receive a pixel image; and an image signal processor (540) configured to process the pixel image to generate image data, the image signal processor (540) comprises: a bad pixel detection unit (220) configured to compare coordinates of a first pixel with pre-stored (S10) coordinates of a representative pixel representing at least one bad pixel, to determine whether the representative pixel is contained in a kernel, wherein the kernel has the first pixel at a center of the kernel, and to determine whether the first pixel is a bad pixel if the representative pixel is contained in the kernel; and a bad pixel compensation unit (230) configured to compensate the first pixel by means of a second pixel in the kernel when the first pixel is determined to be a bad pixel, wherein the bad pixel detection unit (220) outputs the first pixel as the compensated pixel image without performing bad pixel compensation if the representative pixel is not contained in the kernel. [9] Application processor according to claim 8, wherein the bad pixel detection unit (220) adapts a first pixel area of ​​the kernel when the representative pixel is contained in the kernel. [10] Application processor according to claim 8, wherein the bad pixel detection unit (220) compares the coordinates of the first pixel with the coordinates of the representative pixel, calculates a horizontal offset and a vertical offset based on the comparison result, and determines whether the horizontal offset and the vertical offset are within a predetermined value corresponding to a kernel size, where the horizontal offset is a distance in a first direction between the first pixel and the representative pixel, and where the vertical offset is a distance in a second direction perpendicular to the first direction between the first pixel and the representative pixel. [11] Application processor according to claim 8, wherein the image signal processor (540) further comprises a bad pixel memory (210) configured to store information about the representative pixel. [12] Application processor according to claim 11, wherein the information about the representative pixel comprises coordinates of the representative pixel, or the number of bad pixels adjacent to the representative pixel. [13] Application processor according to claim 9, wherein a kernel size is equal to one of 2×2, 3×3, 5×5 and 7×7. [14] Method for processing a bad pixel contained in a pixel image, the method comprising: Detecting a bad pixel using the coordinates of a representative pixel that represents at least one bad pixel; and Compensating (S15) a first pixel by means of a second pixel in a kernel when the first pixel is a bad pixel, wherein the kernel has the first pixel in a center of the kernel, where the detection of a bad pixel includes: Comparing the coordinates of the first pixel with pre-stored (S10) coordinates of the representative pixel to determine (S12) whether the representative pixel is contained in the kernel; Determine (S13) whether the first pixel is a bad pixel if the representative pixel is included in the kernel; and Determining the first pixel as a bad pixel if a brightness value of the first pixel is not within a first pixel range, where the first pixel area is determined based on an average brightness of the kernel, and where detecting the bad pixel further includes outputting (S16) the first pixel as a compensated pixel image if the representative pixel is not contained in the kernel. [15] Method according to claim 14, wherein detecting the bad pixel further comprises reducing the first pixel area of ​​the kernel when the representative pixel is contained in the kernel. [16] Method according to claim 14, wherein detecting the bad pixel further comprises outputting the first pixel as a compensated pixel image if the brightness value of the first pixel is within the first pixel area in the kernel. [17] Method according to claim 14, wherein the second pixel surrounds the first pixel in the kernel and has a substantially the same color as the first pixel.

Images