Defective pixel correction method, and image sensing device for performing same

Abstract

An image sensing device according to the present invention comprises: a determination unit including a first determination unit and a second determination unit; and a correction unit. By using the components, small-sized pixels included in an image sensor can be corrected on a pixel-by-pixel basis.

Description

Method for correcting defective pixels and image sensing device for performing the same

[0001] The present invention relates to a technology for correcting defective pixels of an image sensor. For example, the present invention may be applied to all product families in which image sensors are used, such as smartphones, wearable devices, automobiles, aviation equipment, medical devices, drones, and robots.

[0002] An image sensor is a device that converts optical images into electrical signals. Recently, driven by advancements in the automotive, computer, medical, and telecommunications industries, the demand for image sensors in various fields such as smartphones, gaming devices, the Internet of Things (IoT), robots, and medical micro-cameras has been rapidly increasing. Consequently, there is a trend toward introducing various multi-pixel technologies to enhance pixel performance.

[0003] Image sensors using various multi-pixel technologies select special structures such as Quad (Tetra) Bayer, Nona Bayer, and RYYB structures, and place 2x2, 3x3, or more pixels under a single color filter to achieve high sensitivity and high dynamic range.

[0004] However, even if a problem occurs in only a single pixel among many, it may include defective pixels that prevent the acquisition of a normal image. Defective pixels occur during the production of the image sensor due to manufacturing processes, temperature changes, or external shocks, and examples include hot pixels, dead pixels, and stuck pixels.

[0005] If defective pixels occur, the quality of the image output from the image sensor deteriorates, causing image degradation.

[0006] The technical objective of the present invention is to provide an image sensing device capable of detecting a defective pixel among the pixels included in an image sensor and correcting it on a pixel-by-pixel basis.

[0007] Another technical objective of the present invention is to provide an image sensing device capable of correcting various types of defective pixels.

[0008] In addition, the present invention has another technical objective of providing an image sensing device capable of precisely detecting defective pixels.

[0009] An image sensing device according to one aspect of the present invention for solving the aforementioned technical problem comprises: a determination unit that determines whether a target pixel is defective by using a pixel value of a target pixel subject to defect inspection and a pixel value of adjacent pixels adjacent to the target pixel within a unit kernel composed of a plurality of pixels located on a pixel array; and a correction unit that corrects the pixel value of the target pixel to a representative value of pixels adjacent to the target pixel if the target pixel is defective, wherein the determination unit comprises: a first determination unit that determines whether the pixel value of the target pixel is included within a first defect range defined by a first lower limit and a first upper limit; and a second determination unit that determines whether, if the target pixel is included within the first defect range, the representative value of the pixel value of adjacent pixels adjacent to the target pixel is included within a second defect range defined by a second lower limit greater than the first lower limit and a second upper limit smaller than the first upper limit.

[0010] An image sensing method according to another aspect of the present invention for achieving the above-described technical problem comprises: a step of determining whether a pixel value of a target pixel subject to defect inspection within a unit kernel composed of a plurality of pixels included in a pixel array is included in a first defect range defined by a first lower limit and a first upper limit; a step of determining whether, if the target pixel is included in the first defect range, a representative value of pixel values ​​of adjacent pixels adjacent to the target pixel is included in a second defect range defined by a second lower limit greater than the first lower limit and a second upper limit smaller than the first upper limit; and a step of correcting the pixel value of the target pixel to the representative value of the adjacent pixels if the target pixel is included in the first defect range and the representative value of the adjacent pixels is included in the second defect range.

[0011] According to the present invention, the determination of whether a correction target applies to each kernel containing a plurality of pixels is made, and specifically, by repeatedly determining whether each pixel within the kernel falls within a predetermined range, the effect is to correct small pixels on a pixel-by-pixel basis.

[0012] In addition, according to the present invention, since the arbitrary value required to define the target pixel to be corrected varies depending on the characteristics and tendencies of the pixel in which the defect occurred, there is an effect that various types of defective pixels can be corrected by changing the arbitrary value according to the type of defect in the pixel and applying the correction method differently.

[0013] In addition, according to the present invention, the target pixel to be corrected is subject to correction only when it falls within a predetermined range, and adjacent pixels can also be corrected only when they fall within another predetermined range, thereby providing the effect of performing precise correction only on defective pixels that satisfy the requirements.

[0014] FIG. 1 is a drawing for explaining an image sensing device according to an embodiment of the present invention.

[0015] Figure 2 is a diagram showing a kernel window corresponding to a unit pixel and a subpixel placed inside the unit kernel window.

[0016] FIG. 3 is a diagram showing the configuration of a DPC module according to one embodiment of the present invention.

[0017] Figure 4 is a diagram showing the process of changing the inspection target inside the kernel window.

[0018] FIG. 5 is a diagram illustrating the first defect range and the second defect range.

[0019] Figure 6 is a diagram comparing the state before and after correction according to the DPC module.

[0020] FIG. 7 is a flowchart for a method to correct defective pixels according to an embodiment of the present invention.

[0021] Throughout the specification, identical reference numbers denote substantially identical components. In the following description, detailed descriptions of components and functions known in the art may be omitted if they are not related to the core components of the invention. The meanings of the terms described in this specification should be understood as follows.

[0022] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims.

[0023] The shapes, sizes, ratios, angles, numbers, etc. disclosed in the drawings for explaining embodiments of the present invention are exemplary, and therefore the present invention is not limited to the depicted details. Throughout the specification, the same reference numerals refer to the same components. Furthermore, in describing the present invention, if it is determined that a detailed description of related known technology may unnecessarily obscure the essence of the present invention, such detailed description is omitted.

[0024] Where terms such as 'comprising,' 'having,' 'consisting of,' etc. are used in this specification, other parts may be added unless 'only' is used. Where a component is expressed in the singular, it includes cases where it is included in the plural unless specifically stated otherwise.

[0025] In interpreting the components, they are interpreted to include a margin of error even in the absence of a separate explicit statement.

[0026] In the case of describing a positional relationship, for example, when the positional relationship between two parts is described using expressions such as 'on,' 'upper,' 'lower,' or 'next to,' one or more other parts may be located between the two parts unless 'immediately' or 'directly' is used.

[0027] In the case of an explanation of a temporal relationship, for example, when a temporal sequence is explained using 'after', 'following', 'next', 'before', etc., it may include cases where the sequence is not continuous unless 'immediately' or 'directly' is used.

[0028] Although terms such as "first," "second," etc. are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Accordingly, the first component mentioned below may be the second component within the technical scope of the present invention.

[0029] The term “at least one” should be understood to include all combinations that can be presented from one or more related items. For example, the meaning of “at least one of the first item, the second item and the third item” may mean not only the first item, the second item or the third item individually, but also all combinations of items that can be presented from two or more of the first item, the second item and the third item.

[0030] The features of each of the various embodiments of the present invention may be combined or combined with one another, either partially or wholly, and may technically enable various interlocking and operation. Each embodiment may be implemented independently of one another or may be implemented together in an associated relationship.

[0031] Hereinafter, embodiments of the present specification will be described in detail with reference to the attached drawings.

[0032] FIG. 1 shows a block diagram of an image sensing device (10) according to one embodiment of the present invention.

[0033] Referring to FIG. 1, the image sensing device (10) may include an image sensor (100) and an ISP (image signal processor, 200).

[0034] The image sensing device (10) may be implemented as a mobile computing device or a personal computer. The image sensing device (10) may be implemented as a smartphone, a tablet PC, a wearable device, the Internet of Things (IoT), or the Internet of Everything (IoE). Additionally, the image sensing device (10) may be a device equipped as a component in a vehicle, manufacturing equipment, and various measuring instruments.

[0035] The image sensor (100) illustrated in FIG. 1 may include a pixel array (110), a row driver (120), a timing generator (130), an analog digital converter (ADC; 140), and a defective pixel correction (DPC) module (150).

[0036] The pixel array (110) includes a plurality of unit pixels and can output a plurality of pixel signals based on incident light. The plurality of unit pixels included in the pixel array (110) can be arranged two-dimensionally.

[0037] Referring to FIG. 2(a), each unit pixel can be arranged in a 2x2 or 4x4 shape, and subpixels are arranged in a 2x2 shape for each unit pixel. For convenience of explanation, the unit pixels of the present invention are assumed to be arranged in a 2x2 shape.

[0038] In this case, each unit pixel may correspond to color filters arranged in a specific pattern. For example, each color filter may have color information corresponding to R (Red), G (Green), and B (Blue), but is not limited thereto.

[0039] Unit pixels may have a suitable pattern for sensing a color image. Specifically, color filters may be arranged in a quad-bayer pattern. For example, unit pixels may each be combined with one of an R filter, a G filter, and a B filter.

[0040] In this case, the unit pixel combined with the G filter can be named the G color pixel. The unit pixel combined with the R filter can be named the R color pixel. The unit pixel combined with the B filter can be named the B color pixel.

[0041] As shown in FIG. 2(a), each subpixel corresponding to the subcolor filters patterned as R1, R2, R3, and R4 arranged in a 2X2 shape inside the R color pixel can be defined as P0, P1, P2, and P3.

[0042] In FIG. 2(a), for convenience of explanation, the pixel array (110) is shown as containing four unit pixels, but this is just one example and the number of unit pixels constituting the pixel array (110) and the arrangement of the unit pixels can be varied.

[0043] The pixel array (110) can provide a pixel signal (PS, pixel signal) that matches one of the color values ​​of R, G, and B to the ADC (140) for each unit pixel.

[0044] Referring to FIG. 1, the row driver (120) can activate each of the plurality of pixels. The row driver (120) can drive the pixels implemented in the pixel array (110) in units of rows. The row driver (120) can generate control signals that can control the operation of the plurality of pixels included in each of the plurality of rows.

[0045] According to the control signals generated by the row driver (120), the pixel signals output from each pixel are transmitted to the ADC (140).

[0046] The timing generator (130) can control the operation of the row driver (120) and the ADC (140).

[0047] The ADC (140) can convert the pixel signal (PS) output from each unit pixel into pixel data (PD, pixel data) and transmit it to the DPC module (150).

[0048] Specifically, referring again to FIG. 1, the ADC (140) converts an analog pixel signal (PS) received from the pixel array (110) into digital pixel data (PD) and transmits it to the DPC module (150).

[0049] The maximum digital output value (P) that the ADC (140) can convert in the digital pixel data (PD) transmitted to the DPC module (150) MAX ), the lower threshold value (P) output from each subpixel (P0, P1, P2, P3). LT ), upper threshold value output from each subpixel (P UT It includes the actual values ​​of each subpixel data output from each subpixel, and the actual values ​​of each subpixel data output from each subpixel. The actual values ​​of each subpixel data output from each subpixel can be abbreviated as the 'pixel values' of each unit pixel.

[0050] Lower threshold value (P) output from each subpixel LT ) is a minimum value set to prevent the output value of each subpixel from becoming too low, and is a value that can be obtained, for example, through experiments, and is a value that can be tuned depending on the type of image sensor.

[0051] Upper threshold value (P) output from each subpixel UT ) is a maximum value set to prevent the output value of each subpixel from becoming too high, and is a value that can be obtained, for example, through experiments, and is a value that can be tuned depending on the type of image sensor.

[0052] Hereinafter, a method for correcting defective pixels according to an embodiment of the present invention will be described in detail with reference to FIGS. 2 to 4.

[0053] FIG. 2 is a diagram showing a kernel window corresponding to a unit pixel and a sub-pixel placed inside the unit kernel window, FIG. 3 is a diagram showing the configuration of a DPC (150) according to an embodiment of the present invention, and FIG. 4 is a diagram showing the process of changing the inspection target inside the kernel window.

[0054] As illustrated in FIG. 3, a DPC module (150) according to one embodiment of the present invention can correct the pixel value of a defective pixel included in a pixel array (110) using pixel data (PD) received from an ADC (140). At this time, the DPC module (150) includes a designation unit (151), a judgment unit, and a correction unit (154). In this regard, the judgment unit may include a first judgment unit (152) and a second judgment unit (153).

[0055] Referring again to FIG. 2, the designation unit (151) selects one unit pixel among the unit pixels present in FIG. 2 (a) and sets the kernel window shown in FIG. 2 (b). Specifically, the position of the kernel window to be inspected on the pixel array (110) is set arbitrarily or according to a preset standard, etc. Here, the kernel window refers to, for example, a unit for image signal processing.

[0056] As shown in FIG. 2(b), 2x2 subpixels (P0, P1, P2, P3) may exist within the set kernel window, and the set kernel window may be formed with at least one structure among Quad Bayer, Tetra Bayer, Nona Bayer, and RYYB, but the present invention is not necessarily limited thereto.

[0057] As illustrated in FIG. 3, a designation unit (151) according to one embodiment of the present invention obtains a maximum digital output value (P) from an ADC (140). MAX ), lower threshold value (P LT ), upper threshold value (P UT), the pixel value of P1, the pixel value of P2, the pixel value of P3, and the pixel value of P0 can be received.

[0058] The designation unit (151) selects one of the subpixels (P0, P1, P2, P3) existing inside the kernel window as the target pixel (P T It can be defined as ). Target pixel (P T ) can also be defined randomly, similar to the kernel window.

[0059] As shown in FIG. 4, one of the subpixels (P0, P1, P2, P3) existing inside the kernel window is the target pixel (P T If defined as ), the target pixel (P T Subpixels (P1, P2, P3) existing within the kernel window other than ) can be defined as adjacent pixels. In this case, the adjacent pixels can be composed of P0, P2, and P3, or P0, P1, and P3, or P0, P1, and P2. For the convenience of explanation, I will assume that the adjacent pixels are composed of P1, P2, and P3.

[0060] As illustrated in FIG. 3, a designation part (151) according to one embodiment of the present invention is a target pixel (P T Pixel value of ) and maximum digital output value (P MAX ) is transmitted to the first judgment unit (152), and the lower threshold value (P LT ), upper threshold value (P UT ), maximum digital output value (P MAX ), and the average value of the pixel values ​​of adjacent pixels (P1, P2, P3) (P AVG ) is transmitted to the second judgment unit (153).

[0061] Additionally, when the designation unit (151) completes the correction for all subpixels (P0, P1, P2, P3) existing within the set kernel window, it sets a kernel window on another unit pixel adjacent to the existing unit pixel. Specifically, the newly set kernel window must not include the existing subpixels (P0, P1, P2, P3).

[0062] Hereinafter, with reference to FIG. 5, a method for a judgment unit to inspect pixels using values ​​received from a designation unit (151) will be described. FIG. 5 is a diagram illustrating a first defect range and a second defect range.

[0063] As illustrated in FIG. 5, the first judgment unit (152) included in the judgment unit is a target pixel (P T It determines whether the pixel value of ) is included within the first defect range defined by the first lower limit (A1) and the first upper limit (A2).

[0064] The first lower limit (A1) and the first upper limit (A2) may vary depending on at least one of the number of subpixels included in the kernel window, the number of defective pixels included in the subpixels, occurrence characteristics and trends, and may be expressed in percentage units.

[0065] Maximum digital output value (P MAX The value obtained by multiplying ) by the first weight can be defined as the first lower limit (A1), and the target pixel (P T If the pixel value of ) is lower than the first lower limit (A1), it is considered to be included in the first defect range.

[0066] For example, when the first weight is defined as 5%, the target pixel (P T The pixel value of ) is the maximum digital output value (P MAX If the output is lower than 5% of ), the subpixel is considered to be outputting a pixel value lower than the first lower limit (A1) and is considered to be included in the first defect range.

[0067] Maximum digital output value (P MAXThe value obtained by multiplying ) by a second weight can be defined as the first upper limit (A2), and the target pixel (P T If the pixel value of ) is higher than the first upper limit (A2), it is considered to be included in the first defect range.

[0068] For example, when the first weight is defined as 95%, the target pixel (P T The pixel value of ) is the maximum digital output value (P MAX If the output is higher than 95% of ), the subpixel is considered to be outputting a pixel value higher than the first upper limit (A2) and is considered to be included within the first defect range.

[0069] The first judgment unit (152) is the target pixel (P T If the pixel value of ) is included in the first defect range, the target pixel (P) is sent to the second judgment unit (153). T Transmits the pixel value of ).

[0070] The first judgment unit (152) is the target pixel (P T If the pixel value of ) is not included in the first defect range, the target pixel (P T It is determined that no defect exists in ), and thus it is considered not to be subject to correction.

[0071] Therefore, the first judgment unit (152) determines that the designation unit (151) is a target pixel (P) existing inside the kernel window. T One pixel among the pixels adjacent to ) and (P1, P2, P3) is the target pixel (P T A first target pixel change signal (TCS 1, target change signal 1) can be transmitted to the designated unit (151) so that it can be changed to ).

[0072] The designation unit (151), which receives the first target pixel change signal (TCS 1) from the first judgment unit (152), determines whether the inspection of all subpixels existing on the pixel array (110) has been completed.

[0073] At this time, when the inspection of all subpixels existing on the pixel array (110) is completed, the designation unit (151) may stop the correction.

[0074] Additionally, if the inspection of all subpixels existing on the pixel array (110) is not completed, the designation unit (151) determines whether the inspection of all subpixels within the kernel window has been performed. Specifically, referring again to FIG. 4, the designation unit (151) can determine whether the inspection of all subpixels (P0, P1, P2, P3) within the kernel window has been performed.

[0075] When an inspection is performed on all subpixels (P0, P1, P2, P3) inside the kernel window, the designation unit (151) can set the kernel window on a unit pixel other than the existing unit pixel.

[0076] However, if inspection of all subpixels (P0, P1, P2, P3) inside the kernel window has not been performed, the designation unit (151) selects the subpixels (P0, P1, P2, P3) inside the kernel window that have not been inspected as target pixels (P T It can be decided as ).

[0077] That is, the designated part (151) is the target pixel (P T One pixel among the pixels adjacent to ) and (P1, P2, P3) is the target pixel (P T It can be changed to ). The designation unit (151) that receives the first target pixel change signal (TCS 1) can change the target pixel (P) existing inside the kernel window. T One pixel among the pixels adjacent to ) and (P1, P2, P3) is the target pixel (P T It can be changed to ).

[0078] Referring again to FIG. 3, the second judgment unit (153) included in the judgment unit is a lower threshold value (P) from the designation unit (151). LT ), upper threshold value (P UT ), maximum digital output value (PMAX ), and the average value of the pixel values ​​of adjacent pixels (P1, P2, P3) (P AVG ) is received, and the target pixel (P) from the first judgment unit (152) is received. T The pixel value of ) can be received.

[0079] Additionally, as illustrated in FIG. 5, the second judgment unit (153) can determine whether the representative value of adjacent pixels (P1, P2, P3) is included in a second defect range defined by a second lower limit (B1) and a second upper limit (B2). The representative value of adjacent pixels (P1, P2, P3) may include at least one of the average value, median value, maximum value, and minimum value of adjacent pixels (P1, P2, P3), and for convenience of explanation, the average value (P1, P2, P3) of adjacent pixels (P1, P2, P3) AVG Explained as ).

[0080] The second defect range is defined as a value between the second lower limit (B1) and the second upper limit (B2).

[0081] The second lower limit (B1) is a value higher than the first lower limit (A1) and is determined according to the following mathematical formula 1.

[0082] [Mathematical Formula 1]

[0083]

[0084] The second upper limit (B2) is a value lower than the first upper limit (A2) and is determined according to the following mathematical formula 2.

[0085] [Mathematical Formula 2]

[0086]

[0087] The second judgment unit (153) determines the average value (P1, P2, P3) of adjacent pixels (P1, P2, P3). AVG If ) is included in the second defect range, the target pixel (P) is corrected to the correction unit (154). T The average value of ) and adjacent pixels (P1, P2, P3) (P AVG Can transmit ).

[0088] The second judgment unit (153) determines the average value (P1, P2, P3) of adjacent pixels (P1, P2, P3). AVG If ) is not included in the second defect range, the target pixel (P) is used as the pixel value of adjacent pixels (P1, P2, P3). T Since it is not suitable for correcting the pixel value of ), the designated part (151) is a target pixel (P) existing inside the kernel window. T One pixel among the pixels adjacent to ) and (P1, P2, P3) is the target pixel (P T A second target pixel change signal (TCS 2) can be transmitted to enable changing to ).

[0089] The designation unit (151), which receives the second target pixel change signal (TCS 2, target change signal 2) from the second judgment unit (153), can determine whether the inspection of all subpixels existing on the pixel array (110) has been completed.

[0090] At this time, when the inspection of all subpixels existing on the pixel array (110) is completed, the designation unit (151) may stop the correction.

[0091] Additionally, if the inspection of all subpixels existing on the pixel array (110) is not completed, the designation unit (151) determines whether the inspection of all subpixels within the kernel window has been performed. Specifically, referring again to FIG. 4, the designation unit (151) can determine whether the inspection of all subpixels (P0, P1, P2, P3) within the kernel window has been performed.

[0092] When an inspection is performed on all subpixels (P0, P1, P2, P3) inside the kernel window, the designation unit (151) can set the kernel window on a unit pixel other than the existing unit pixel.

[0093] However, if inspection of all subpixels (P0, P1, P2, P3) inside the kernel window has not been performed, the designation unit (151) selects the subpixels (P0, P1, P2, P3) inside the kernel window that have not been inspected as target pixels (P T It can be decided as ).

[0094] That is, the designated part (151) is the target pixel (P T One pixel among the pixels adjacent to ) and (P1, P2, P3) is the target pixel (P T It can be changed to ). The designation unit (151) that receives the second target pixel change signal (TCS 2) can change the target pixel (P) existing inside the kernel window. T One pixel among the pixels adjacent to ) and (P1, P2, P3) is the target pixel (P T It can be changed to ).

[0095] The correction unit (154) receives the target pixel (P) from the judgment unit. T The value of ) can be corrected. Specifically, the correction unit (154) receives the target pixel (P) received from the second judgment unit (153). T The value of ) can be corrected to a representative value of adjacent pixels (P1, P2, P3). The representative value of adjacent pixels (P1, P2, P3) may include at least one of the average value, median value, maximum value, and minimum value of adjacent pixels (P1, P2, P3), and for convenience of explanation, the average value (P) of adjacent pixels (P1, P2, P3) AVG Explained as ).

[0096] In this regard, the target pixel (P) in the first judgment unit (152) T The value of ) is included in the first defect range, and the average value (P) of adjacent pixels (P1, P2, P3) in the second judgment unit (153) AVG Only when ) is included in the second defect range, the correction unit (154) receives the target pixel (P T The value of ) is the average value (P) of adjacent pixels (P1, P2, P3). AVG It can be corrected with ).

[0097] Referring again to FIG. 1, the correction unit (154) corrects the target pixel (P) that has been corrected. T Image data containing pixel values ​​of ) (I DATA Output ) to ISP(200).

[0098] An image showing the correction performed according to the correction unit (154) is shown in FIG. 6. FIG. 6 is a drawing comparing the state before and after the correction performed according to the DPC module (150).

[0099] Specifically, FIG. 6(a), which shows the state before correction, shows that defects were randomly generated in pixels corresponding to 0.1% of the total pixels, and the drawing on the left is an enlarged view of the drawing on the right. FIG. 6(b), which shows the state after correction, is a drawing after correction according to the DPC module (150).

[0100] Defect pixels may occur at random locations on the image, and their manifestation forms may also vary. For example, as shown in FIG. 6(a), they may appear as pixels with brighter brightness than surrounding pixels, such as the first to fourth defect pixels (d1, d2, d3, d4), and their locations may also be randomly determined. In this case, the manifestation forms and number of defect pixels are not limited to the example shown in FIG. 6(a).

[0101] Unless the defective pixel occurs in all subpixels within the kernel window, it is possible to correct various types of defective pixels that occur on the pixel array (110).

[0102] Referring again to FIG. 1, the ISP (200) receives image data (I) output from the DPC module (150). DATAIt can receive ) and process or handle the received image data to output the processed image data. The ISP (200) can perform image signal processing for image quality improvement by performing color correction, color conversion, and white balance.

[0103] A method for correcting defective pixels according to an embodiment of the present invention will be described below with reference to FIG. 7. FIG. 7 is a flowchart of a method for correcting defective pixels according to an embodiment of the present invention.

[0104] The designation unit (151) sets a kernel window on a unit pixel composed of multiple subpixels (S1001). Specifically, the designation unit (151) may randomly set a kernel window on a unit pixel existing on a pixel array (110), and a kernel window refers to a unit for image signal processing.

[0105] The designation unit (151) selects one of the subpixels (P0, P1, P2, P3) existing in the set kernel window as the target pixel (P T It is defined as ) (S1002). Adjacent pixels may be composed of P0, P2, and P3, or may also be composed of P0, P1, and P3 or P0, P1, and P2. For convenience of explanation, adjacent pixels are considered to be composed of P1, P2, and P3.

[0106] Afterwards, the designated part (151) is the target pixel (P T Pixel value of ) and maximum digital output value (P MAX ) is transmitted to the first judgment unit (152), and the lower threshold value (P LT ), upper threshold value (P UT ), maximum digital output value (P MAX ), and the average value of the pixel values ​​of adjacent pixels (P1, P2, P3) (P AVG ) is transmitted to the second judgment unit (153).

[0107] The first judgment unit (152) is the target pixel (P T It determines whether the pixel value of ) is included in the first defect range (S1003). Specifically, the first determination unit (152) determines whether the target pixel (P T It determines whether the pixel value of ) is included within the first defect range defined by the first lower limit (A1) and the first upper limit (A2).

[0108] In this regard, the maximum digital output value (P MAX The value obtained by multiplying ) by the first weight can be defined as the first lower limit (A1), and the target pixel (P T If the pixel value of ) is lower than the first lower limit (A1), it is considered to be included in the first defect range.

[0109] In addition, the maximum digital output value (P MAX The value obtained by multiplying ) by a second weight can be defined as the first upper limit (A2), and the target pixel (P T If the pixel value of ) is higher than the first upper limit (A2), it is considered to be included in the first defect range.

[0110] The first judgment unit (152) is the target pixel (P T If the pixel value of ) is included in the first defect range, the target pixel (P) is sent to the second judgment unit (153). T Transmits the pixel value of ).

[0111] The first judgment unit (152) is the target pixel (P T If the pixel value of ) is not included in the first defect range, the designation part (151) is a target pixel (P) existing inside the kernel window. T One pixel among the pixels adjacent to ) and (P1, P2, P3) is the target pixel (P T A first target pixel change signal (TCS 1) can be transmitted to the designated unit (151) so that it can be changed to ).

[0112] At this time, the designation unit (151), which receives the first target pixel change signal (TCS 1) from the first judgment unit (152), can determine whether the inspection of all subpixels existing on the pixel array (110) has been completed (S1006).

[0113] At this time, when the inspection of all subpixels existing on the pixel array (110) is completed, the designation unit (151) may stop the correction.

[0114] Additionally, if the inspection of all subpixels existing on the pixel array (110) is not completed, the designation unit (151) determines whether the inspection of all subpixels within the kernel window has been performed (S1007). Specifically, referring again to FIG. 4, the designation unit (151) can determine whether the inspection of all subpixels (P0, P1, P2, P3) within the kernel window has been performed.

[0115] When an inspection is performed on all subpixels (P0, P1, P2, P3) inside the kernel window, the designation unit (151) can set the kernel window on a unit pixel other than the existing unit pixel.

[0116] However, if inspection of all subpixels (P0, P1, P2, P3) inside the kernel window has not been performed, the designation unit (151) selects the subpixel (P0, P1, P2, P3) inside the kernel window that has not been corrected as the target pixel (P T It can be decided as ).

[0117] The second judgment unit (153) is the target pixel (P TIf the pixel value of ) is included in the first defect range, it can be determined whether the representative value of adjacent pixels (P1, P2, P3) is included in the second defect range defined by the second lower limit (B1) and the second upper limit (B2) (S1004). The representative value of adjacent pixels (P1, P2, P3) may include at least one of the average value, median value, maximum value, and minimum value of adjacent pixels (P1, P2, P3), and for convenience of explanation, the average value (P1, P2, P3) of adjacent pixels (P1, P2, P3) AVG Explained as ).

[0118] Specifically, the second judgment unit (153) receives a lower threshold value (P) transmitted from the designation unit (151). LT ), upper threshold value (P UT ), maximum digital output value (P MAX ), and the average value of the pixel values ​​of adjacent pixels (P1, P2, P3) (P AVG Through ), the average value (P) of the pixel values ​​of adjacent pixels (P1, P2, P3) AVG It is possible to determine whether ) is included in the second defect range.

[0119] The second defect range is defined as a value between the second lower limit (B1) and the second upper limit (B2).

[0120] The second lower limit (B1) is a value higher than the first lower limit (A1) and is determined according to the aforementioned mathematical formula 1.

[0121] The second upper limit (B2) is a value lower than the first upper limit (A2) and is determined according to the aforementioned mathematical formula 2.

[0122] The second judgment unit (153) determines the average value (P1, P2, P3) of adjacent pixels (P1, P2, P3). AVG If ) is included in the second defect range, the target pixel (P) is corrected to the correction unit (154). T The average value of ) and adjacent pixels (P1, P2, P3) (P AVG Can transmit ).

[0123] The second judgment unit (153) determines the average value (P1, P2, P3) of adjacent pixels (P1, P2, P3). AVG If ) is not included in the second defect range, the designated part (151) is a target pixel (P) existing inside the kernel window. T One pixel among the pixels adjacent to ) and (P1, P2, P3) is the target pixel (P T A second target pixel change signal (TCS 2) can be transmitted to enable changing to ).

[0124] The designation unit (151), which receives the second target pixel change signal (TCS 2) from the second judgment unit (153), can determine whether the inspection of all subpixels existing on the pixel array (110) has been completed.

[0125] At this time, when the inspection of all subpixels existing on the pixel array (110) is completed, the designation unit (151) may stop the correction.

[0126] Additionally, if the inspection of all subpixels existing on the pixel array (110) is not completed, the designation unit (151) determines whether the inspection of all subpixels within the kernel window has been performed.

[0127] The correction unit (154) receives the target pixel (P) from the judgment unit. T The value of ) can be corrected. Specifically, the correction unit (154) receives the target pixel (P) received from the second judgment unit (153). T The value of ) can be corrected to a representative value of adjacent pixels (P1, P2, P3) (S1005). The representative value of adjacent pixels (P1, P2, P3) may include at least one of the average value, median value, maximum value, and minimum value of adjacent pixels (P1, P2, P3), and for convenience of explanation, the average value (P) of adjacent pixels (P1, P2, P3) AVG Explained as ).

[0128] Afterwards, the correction unit (154) corrects the target pixel (P TImage data containing pixel values ​​of ) (I DATA Output ) to ISP(200).

[0129] Those skilled in the art to which the present invention pertains will understand that the above-described invention may be implemented in other specific forms without altering its technical concept or essential features.

[0130] Additionally, the methods described herein may be implemented at least partially using one or more computer programs or components. These components may be provided as a series of computer instructions via a computer-readable or machine-readable medium including volatile and non-volatile memory. The instructions may be provided as software or firmware and may be implemented wholly or partially in hardware configurations such as ASICs, FPGAs, DSPs, or other similar devices. The instructions may be configured to be executed by one or more processors or other hardware configurations, which perform or are capable of performing all or part of the methods and procedures disclosed herein when executing the series of computer instructions.

[0131] Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. The scope of the invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the invention.

Claims

1. A determination unit that determines whether a target pixel is defective by using the pixel value of a target pixel subject to defect inspection and the pixel values ​​of adjacent pixels adjacent to the target pixel within a unit kernel composed of a plurality of pixels located on a pixel array; and If the above target pixel is a defective pixel, the correction unit includes a correction unit that corrects the pixel value of the above target pixel to a representative value of pixels adjacent to the above target pixel. The above judgment unit is, A first determination unit for determining whether the pixel value of the above target pixel is included within a first defect range defined by a first lower limit and a first upper limit; and An image sensing device characterized by including a second determination unit that determines whether, when the target pixel is included within a first defect range, the representative value of the pixel values ​​of adjacent pixels adjacent to the target pixel is included within a second defect range defined by a second lower limit greater than the first lower limit and a second upper limit smaller than the first upper limit.

2. In Paragraph 1, The correction unit corrects the pixel value of the target pixel to the representative value of the adjacent pixels when the target pixel is included in the first defect range and the representative value of the adjacent pixels is included in the second defect range. An image sensing device in which the representative value of the adjacent pixels includes at least one value among the average value, median value, maximum value, and minimum value of the pixel values ​​of the adjacent pixels.

3. In Paragraph 1, The first lower limit is a value obtained by reflecting a first weight to the maximum pixel value of the target pixel, and the first upper limit is determined by reflecting a second weight greater than the first weight to the maximum pixel value. An image sensing device in which the first weight and the second weight vary according to at least one of the number, occurrence characteristics, and tendency of defective pixels among a plurality of pixels included in the unit kernel.

4. In Claim 1, The above second lower limit is determined according to mathematical formula 1, and the above second upper limit is determined according to mathematical formula 2, and The above mathematical formula 1 is and, the above mathematical formula 2 is And, In the above mathematical formulas 1 and 2, The second lower limit, is the second upper limit, is the lower threshold of the above target pixel output, is the upper threshold of the above target pixel output, is the average of the pixel values ​​of the aforementioned adjacent pixels, is an image sensing device representing the maximum pixel value of the above target pixel.

5. In Paragraph 1, The above-determined judgment unit is an image sensing device that changes a pixel other than the determined target pixel among the plurality of pixels into a target pixel when the target pixel is not included within a first defect range or when the representative value of the pixel values ​​of the adjacent pixels is not included in a second defect range.

6. In Paragraph 1 An image sensing device further comprising a designation unit for designating the location of a unit kernel to be inspected on a pixel array and designating a target pixel to be inspected within the unit kernel.

7. A step of determining whether the pixel value of a target pixel subject to defect inspection within a unit kernel composed of a plurality of pixels included in a pixel array is included within a first defect range defined by a first lower limit and a first upper limit; If the target pixel is included within a first defect range, a step of determining whether the representative value of the pixel values ​​of adjacent pixels adjacent to the target pixel is included within a second defect range defined by a second lower limit greater than the first lower limit and a second upper limit smaller than the first upper limit; and An image sensing method comprising the step of correcting the pixel value of the target pixel to the representative value of the adjacent pixels when the target pixel is included in the first defect range and the representative value of the adjacent pixels is included in the second defect range.

8. In Paragraph 7, The first lower limit is a value obtained by reflecting a first weight to the maximum pixel value of the target pixel, and the first upper limit is determined by reflecting a second weight greater than the first weight to the maximum pixel value. An image sensing method in which the first weight and the second weight vary according to at least one of the number, occurrence characteristics, and tendency of defective pixels among a plurality of pixels included in the unit kernel.

9. In Paragraph 7, The above second lower limit is determined according to mathematical formula 1, and the above second upper limit is determined according to mathematical formula 2, and The above mathematical formula 1 is and, the above mathematical formula 2 is And, In the above mathematical formulas 1 and 2, The second lower limit, is the second upper limit, and is the lower threshold of the target pixel output. is the upper threshold of the above target pixel output, is the average of the pixel values ​​of the aforementioned adjacent pixels, is an image sensing method representing the maximum pixel value of the above target pixel.

10. In Paragraph 7, An image sensing method further comprising the step of changing a pixel other than the determined target pixel among the plurality of pixels into a target pixel when the target pixel is not included within a first defect range or when the representative value of the pixel values ​​of the adjacent pixels is not included in a second defect range.

11. In Paragraph 7, When the correction for the plurality of pixels included within the above unit kernel is all completed, An image sensing method comprising the step of changing the inspection target to a different unit kernel among the kernels included in the pixel array.

12. In Paragraph 7, An image sensing method further comprising the step of specifying the location of a unit kernel to be inspected on a pixel array and specifying a target pixel to be inspected within the unit kernel.

Images