Apparatus and method for detecting defective pixel, and apparatus and method for correcting defective pixel

Abstract

This apparatus for detecting a defective pixel comprises: an image sensor that captures a test image and generates an image frame; a DP detection unit that detects defective pixels from pixels included in the image frame; a DP map generation unit that generates a DP map by accumulating counts of the detected defective pixels for each position of the pixels across a plurality of image frames and stores the generated DP map in a storage; a DP sorting unit that sorts the defective pixels in descending order of the accumulated counts; and a DP position storage unit that stores positions of the sorted defective pixels in the storage.

Description

Defective pixel detection device / method and defective pixel correction device / method

[0001] The present invention relates to an apparatus and method for detecting defective pixels occurring in an image sensor and correcting the detected defective pixels.

[0002] Defective pixels inevitably occur during the manufacturing of image sensors, and consequently, defects in pixel values ​​are bound to occur at fixed locations in the captured image. Therefore, in order to obtain a visually flawless image by concealing these defective pixels despite their presence, it is necessary to perform the processes of detecting these defective pixels and correcting the detected defective pixels together.

[0003] Generally, methods for correcting defective pixels are broadly divided into static correction and dynamic correction methods. The static correction method described above is performed at the factory before product shipment using black image frames and white image frames. As a result, the coordinates of black and white defective pixels are stored in a storage device, and correction is performed on the pixels located at the stored coordinates.

[0004] In contrast, the dynamic correction method detects defective pixels in real-time for each individual frame of a captured image. Because this method detects defective pixels in real-time and corrects their values ​​using surrounding pixels, it is useful in that it does not require separate pre-shipment work at the factory like the static correction method and there is no limit to the number of defective pixels that can be corrected. However, detecting defective pixels in image frames in real-time is not easy, and false positives (misidentifications) can cause degradation in the original image quality.

[0005] However, as a prerequisite for such defective pixel correction, the difficulty in detecting defective pixels lies in finding the optimal conditions under which defects occur. This is because the number of static defective pixels is limited due to hardware constraints, making it difficult to find optimal conditions within that limited range. Consequently, the complex process of searching for defective pixels by assigning an appropriate sensor gain and adjusting exposure settings accordingly can lead to inaccurate detection, particularly when there is a wide variety of sensor types, which can degrade the effectiveness of defective pixel correction.

[0006] The technical problem to be solved by the present invention is to provide a defective pixel detection device / method capable of detecting pixels with a high probability of being defective pixels in an image frame containing various types of defective pixels and noise.

[0007] Another technical objective of the present invention is to maximize the efficiency of defective pixel correction by variably setting the correction limit of defective pixels to adapt to variable hardware environments or system resources.

[0008] Another technical objective of the present invention is to utilize the static correction method and the dynamic correction method in a mutually complementary manner to maximize the advantages of both methods.

[0009] The technical problems of the present invention are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art from the description below.

[0010] A defective pixel detection device according to an embodiment of the present invention for achieving the above technical problem, comprising a processor and a memory for storing instructions executable by said processor, the device includes: an image sensor that captures a test image and generates an image frame; a DP detection unit that detects defective pixels from pixels included in said image frame; a DP map generation unit that generates a DP map by accumulating a count of said defective pixels for each pixel location in a plurality of image frames and stores said generated DP map in a storage; a DP alignment unit that aligns said defective pixels from highest to lowest order of said accumulated counts; and a DP location storage unit that stores said aligned defective pixel locations in the storage.

[0011] The above DP location storage unit stores defective pixels up to a predetermined correction limit in the storage, starting from the order of highest accumulated count among the aligned defective pixels.

[0012] The above defective pixel detection device further includes a correction limit determining unit that monitors system resources to variably determine the correction limit.

[0013] The defective pixels whose counts are accumulated in the above DP map include static defective pixels, half defective pixels, and noise.

[0014] The above DP detection unit includes: a gain control unit that changes the gain of the image frame; an exposure control unit that changes the exposure of the image frame; and a DP determination unit that determines a pixel as a defective pixel when the pixel value of a pixel included in the image frame with the changed gain and / or exposure deviates from a reference range.

[0015] The above DP determination unit determines the defective pixels within a range where the number of pixels determined to be defective pixels does not exceed a fixed number.

[0016] The above DP map generation unit includes: a memory read control unit that reads the current count of a current pixel recorded in the storage; an index update unit that increases the read current count when the DP detection unit determines that the current pixel is a defective pixel; and a memory write control unit that records the increased count for the current pixel in the storage.

[0017] The above DP map generation unit further includes a frame counter that increases a frame number identifying the number of consecutive image frames by 1, and the index update unit increases the current count when the frame number is within a range of a predetermined number of consecutive image frames.

[0018] The above DP map has a three-dimensional vector structure including the horizontal position, vertical position of a specific pixel belonging to the image frame, and the count of defective pixels for the specific pixel.

[0019] The above DP location storage unit lists the locations of the aligned defective pixels according to the scanning order of the image frames, and stores the locations of the defective pixels in the storage in the listed order.

[0020] According to another embodiment of the present invention for achieving the above technical problem, a defective pixel correction device comprising a processor and a memory for storing instructions executable by said processor, the device includes: an image sensor that captures a real-time image and generates an image frame; a DP information reading unit that reads DP information stored in storage; a system monitor that monitors system resources; a correction limit determining unit that variably determines a correction limit according to said system resources; and a DP correction unit that, within the range of said determined correction limit, compares the position of a current pixel included in said image frame with the position of a defective pixel included in said DP information, and corrects said current pixel when the two positions match each other.

[0021] The above defective pixel correction device further includes a real-time correction unit that performs real-time dynamic correction on an image frame in which defective pixels have been corrected by the DP correction unit.

[0022] The correction strength used for the above real-time dynamic correction is variably changed according to the above correction limit.

[0023] As the above correction limit increases, the correction strength used for the real-time dynamic correction decreases, and as the above correction limit decreases, the correction strength increases.

[0024] The above defective pixel correction device further includes an image output unit that outputs an image frame in which real-time dynamic correction is performed by the real-time correction unit.

[0025] According to another embodiment of the present invention for achieving the above technical problem, a method for recording defective pixels performed by said instructions in a device comprising a processor and a memory for storing instructions executable by said processor comprises: a step of capturing a test image to generate an image frame; a step of detecting defective pixels from pixels included in said image frame; a step of generating a DP map by accumulating a count of said detected defective pixels for each position of said pixel for a plurality of image frames; a step of storing said generated DP map in a storage; a step of sorting said defective pixels from highest to lowest order of said accumulated counts; and a step of storing said sorted positions of said defective pixels in said storage.

[0026] The above-mentioned storage step includes the step of storing defective pixels in the storage up to a predetermined correction limit, starting from the order of the highest accumulated count among the aligned defective pixels.

[0027] The above DP detection unit includes: a gain control unit that changes the gain of the image frame; an exposure control unit that changes the exposure of the image frame; and a DP determination unit that determines a pixel as a defective pixel when the pixel value of a pixel included in the image frame with the changed gain and / or exposure deviates from a reference range.

[0028] The step of generating the DP map comprises: reading the current count of a current pixel recorded in the storage; increasing the read current count if the current pixel is determined to be a defective pixel; and recording the increased count for the current pixel in the storage.

[0029] The above DP map has a three-dimensional vector structure including the horizontal position, vertical position of a specific pixel belonging to the image frame, and the count of defective pixels for the specific pixel.

[0030] According to the present invention, pixels with a high probability of being defective pixels can be detected preferentially in an image frame containing various types of defective pixels and noise.

[0031] In addition, according to the present invention, the correction limit for defective pixels can be variably set to maximize the efficiency of defective pixel correction within a variable hardware environment or system resources.

[0032] In addition, according to the present invention, when a static correction method and a dynamic correction method are used together, the advantages of both can be utilized to the fullest extent.

[0033] FIG. 1 is a block diagram illustrating the configuration of a defective pixel detection device according to one embodiment of the present invention.

[0034] FIG. 2 is a block diagram illustrating the configuration of a DP detection unit included in the defective pixel detection device of FIG. 1.

[0035] FIG. 3 is a block diagram illustrating the configuration of a DP map generation unit included in the defective pixel detection device of FIG. 1.

[0036] FIG. 4 is a diagram showing the structure of a DP map according to one embodiment of the present invention.

[0037] Figure 5 is a diagram illustrating a data storage format according to the structure of the DP map of Figure 4.

[0038] FIG. 6a is a diagram illustrating the distribution of pixel values ​​within an image frame at time t, and FIG. 6b is a diagram illustrating the distribution of pixel values ​​within an image frame at time t+1 after the above time.

[0039] FIG. 7 is a diagram showing the scanning order of pixels included in an image frame according to one embodiment of the present invention.

[0040] FIG. 8 is a block diagram illustrating the configuration of a defective pixel correction device that actually corrects defective pixels using DP information provided from the defective pixel detection device of FIG. 1.

[0041] Figure 9 is a graph showing allowable correction limits that vary depending on current system resources.

[0042] FIG. 10 is a drawing illustrating an example of an implementation of a computing device that realizes the defective pixel detection device of FIG. 1 or the defective pixel correction device of FIG. 8.

[0043] FIG. 11a is a flowchart illustrating a defective pixel detection method according to one embodiment of the present invention, and FIG. 11b is a flowchart illustrating a defective pixel correction method according to one embodiment of the present invention.

[0044] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same components.

[0045] Unless otherwise defined, all terms used in this specification (including technical and scientific terms) may be used in a meaning that is commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0046] The terms used herein are for describing the embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. As used herein, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the components mentioned.

[0047] Additionally, throughout this specification, "DP" is used as an expression representing a defective pixel. An embodiment of the present invention will be described in detail below with reference to the attached drawings.

[0048] FIG. 1 is a block diagram illustrating the configuration of a defective pixel detection device (100) according to one embodiment of the present invention.

[0049] The above defective pixel detection device (100) comprises at least one processor (not shown) and a memory (not shown) that stores instructions executable by the processor. The memory is a storage medium that stores results performed by the processor or data necessary for the operation of the CPU (150), and can be implemented as a volatile memory or a non-volatile memory.

[0050] As described above, the defective pixel detection device (100) is configured to include functional blocks such as an image sensor (110), a DP detection unit (120), a DP map generation unit (130), a DP alignment unit (140), and a DP location storage unit (150). Here, the operation of each functional block, excluding the image sensor (110), can be implemented by executing the instructions.

[0051] The image sensor (110) captures a test image and generates multiple image frames. Specifically, the image sensor (110) can be implemented as a photosensitive element such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal-oxide Semiconductor).

[0052] The above test image may consist of, for example, a simple black image and a white image. By capturing these black image and white image N times each with the image sensor (110), a total of 2N image frames can be obtained. Although these multiple image frames are based on the same image, noise or defective pixels that were not present in other image frames may appear, or existing noise or defective pixels may disappear, depending on changes in the external environment (temperature, humidity, dust, power changes, etc.).

[0053] In this way, the DP detection unit (120) detects defective pixels from pixels included in the plurality of image frames. Specifically, FIG. 2 is a block diagram illustrating the detailed configuration of the DP detection unit (120).

[0054] As described above, the DP detection unit (120) may be configured to include a sensor gain control unit (121), an exposure control unit (122), and a DP determination unit (123). The sensor gain control unit (121) changes the image frame, and the exposure control unit (122) changes the exposure of the image frame, thereby providing various conditions under which noise or defective pixels can be detected in the image frame.

[0055] The DP judgment unit (123) determines that a pixel is a defective pixel (candidate for defective pixels) if the pixel value of a pixel included in an image frame in which the sensor gain and / or exposure has been changed falls outside the reference range (12). Accordingly, the output from the DP judgment unit (123) will be 1-bit information.

[0056] In addition, the determination of whether it falls outside this standard range (12) can be based on how much the pixel value of the target pixel differs from the value of surrounding pixels (e.g., the average value of surrounding pixels of a certain tap size). That is, if the difference between the two is greater than a threshold, it is determined to be a defective pixel. However, since these defective pixels include not only complete defective pixels (static defective pixels) but also temporary noise and half-defect pixels, the entire group can be considered as a candidate group for defective pixels.

[0057] Here, noise refers to transient pixel values ​​that appear and disappear within the image itself, and is unrelated to dead pixels. Furthermore, a half-dead pixel refers to a pixel that appears as a defective pixel under specific conditions but functions as a normal pixel under other conditions. Additionally, such half-dead pixels can occur due to various factors, such as a pixel becoming defective only within a specific temperature range or a problem with only one of the RGB color components. Hot pixels are a representative example of these half-dead pixels. A hot pixel refers to a pixel that reacts abnormally sensitively or insensitively to light. This can occur particularly when adjusting the sensor gain in low-light conditions, or when increasing contrast or passing through the gamma correction section during image processing.

[0058] The above DP judgment unit (123) detects pixels determined to be defective pixels, but repeats this detection operation only within a range where the number does not exceed a fixed number (e.g., 2000), and then terminates the operation. Accordingly, when the DP judgment unit (123) reaches the fixed number while only a portion of the entire test image has been tested, it enables the entire test image to be tested within the fixed number by raising the threshold or changing the sensor gain, exposure, etc.

[0059] Referring again to FIG. 1, the DP map generation unit (130) generates a DP map (50 in FIG. 4) by accumulating the count of the detected defective pixels for each pixel location for a plurality of image frames. The generated DP map (50) can be temporarily stored in storage (105). Specifically, FIG. 3 is a block diagram illustrating the detailed configuration of the DP map generation unit (130).

[0060] As described above, the DP map generation unit (130) may be configured to include a memory read control unit (131), a memory write control unit (132), an index update unit (135), and a frame counter (135).

[0061] First, the memory reading control unit (131) reads the current count of the current pixel recorded in the storage (105).

[0062] The index update unit (135) increases the read current count when the current pixel is determined to be a defective pixel by the DP detection unit (120) described above.

[0063] Then, the memory recording control unit (132) records the increased count for the current pixel in the storage (105). The reading and writing method of this storage (105) can be performed using a direct memory access (DMA) method to minimize the load on the CPU.

[0064] At this time, the frame counter (133) increases the frame number identifying the number of consecutive image frames by 1. This process is performed repeatedly for a predetermined number of consecutive image frames.

[0065] Accordingly, the index update unit (135) increases the current count only while the frame number is within a predetermined number of consecutive image frames.

[0066] Meanwhile, the number of frames used in the frame counter (133) can be changed according to the sensor gain. Generally, a certain level of sensor gain is required to detect defects in hot pixels. At this time, as the sensor gain increases, it becomes difficult to distinguish between noise and defective pixels, so it is necessary to increase the number of frames.

[0067] FIG. 4 is a diagram illustrating the structure of the DP map (50) generated by the DP map generation unit (130).

[0068] As described above, the DP map (50) has a three-dimensional vector structure including the horizontal position (x axis), vertical position (y axis) of a specific pixel belonging to an image frame, and the count of defective pixels (hit index) for the specific pixel. A data storage format stored in storage (105) according to the structure of this DP map (50) is exemplified in the following FIG. 5.

[0069] In Fig. 5, the number written in each cell (corresponding to each pixel) is the count of defective pixels accumulated over multiple image frames. This count represents the probability that the corresponding pixel is a defective pixel. That is, the more times a specific pixel is judged to be a candidate for a defective pixel during the process of changing multiple image frames over time, the higher the probability that it is a defective pixel.

[0070] In Fig. 5, pixels marked in white (e.g., pixels marked 8, 9) represent a low DP probability, pixels marked in light shades (e.g., pixels marked 3, 5) represent a medium DP probability, and pixels marked in dark shades (e.g., pixels marked 0, 1) represent a high DP probability.

[0071] FIG. 6a is a diagram illustrating the distribution of pixel values ​​within an image frame at time t, and FIG. 6b is a diagram illustrating the distribution of pixel values ​​within an image frame at time t+1 after the above time. Here, the horizontal and vertical dimensions are each the size of nine pixels. When comparing FIG. 6a and FIG. 6b, pixel (42) has a peak-shaped pixel value compared to surrounding pixels, so it is highly likely to be a defective pixel; however, pixel (41) appears at time t but disappears at time t+1, so it is merely noise and highly likely not to be a defective pixel. However, determining defective pixels based on such simple comparison is not sufficient. For example, if only 5 out of 10 image frames have a peak shape, it is difficult to make a judgment, and it is also difficult to determine how to judge a pixel that reappears after the peak shape that existed in an image frame has disappeared. In particular, this problem is more pronounced in general situations where the image sensor (110) includes not only static defective pixels and noise that are relatively easy to distinguish, but also half defective pixels.

[0072] As described above, the DP map (50) according to the present invention is based on the probability that a specific pixel is a defective pixel, so even in such a situation, it is possible to detect pixels that are relatively likely to be defective pixels with priority.

[0073] Referring again to FIG. 1, the DP alignment unit (140) aligns the candidate defective pixels in the DP map (50) from highest to lowest accumulated count. Additionally, the DP location storage unit (150) stores the locations (i.e., DP information) of the aligned defective pixels in the storage (105). However, the DP location storage unit (150) may store defective pixels (final defective pixels) (from the aligned defective pixels in order of highest accumulated count) up to a predetermined "correction limit" in the storage (105). The reason for this correction limit is that, in order to store the coordinates of the defective pixels, the size of the SRAM memory within the storage (105) must also be constrained due to the logic size limit of the SoC.

[0074] These correction limits may use a fixed value, but may also be determined variably by considering system resources. Accordingly, the correction limit determination unit (180) monitors the system resources of the defective pixel detection device (100) through the system monitor (190) and determines the correction limit variably. That is, if the system resources are sufficient, the correction limit may be increased, and if the system resources are insufficient, the correction limit may be decreased.

[0075] Meanwhile, when the DP location storage unit (150) stores the location of the aligned defective pixel (i.e., DP information) in the storage (105), it follows a predetermined scanning order. At this time, a zigzag scan as shown in FIG. 7 may be used as an example of the scanning order. Such a zigzag scan increases the encoding efficiency when applying entropy encoding by allowing spatially adjacent pixels to be stored sequentially adjacently in the storage (105).

[0076] Referring to FIG. 7, the DP information stored in the storage (105) contains 5 defective pixels (DP a, DP b , DP c , DP d , DP e When ) is included, the order in which it is stored according to the scan order (20) is DP c , DP a , DP e , DP b , DP d The value stored in the actual storage (105) is the coordinate value of each defective pixel (or may be indicated as an order index).

[0077] Ultimately, the DP location storage unit (150) lists the locations of the aligned defective pixels according to the scan order (20) of the image frames, and stores the locations of the defective pixels in the storage (105) according to the listed scan order (20).

[0078] FIG. 8 is a block diagram illustrating the configuration of a defective pixel correction device (200) that actually corrects defective pixels using DP information provided from the defective pixel detection device (100) of FIG. 1.

[0079] The above defective pixel correction device (200) comprises at least one processor (not shown) and a memory (not shown) that stores instructions executable by the processor. The memory is a storage medium that stores results performed by the processor or data necessary for the operation of the CPU (150), and can be implemented as a volatile memory or a non-volatile memory.

[0080] As described above, the defective pixel correction device (200) is configured to include functional blocks such as an image sensor (210), a DP information reading unit (230), a system monitor (290), a correction limit determination unit (280), and a DP correction unit (240). Here, the operation of each functional block, excluding the image sensor (210), can be implemented by executing the instructions.

[0081] First, DP information is received from the defective pixel detection device (100) and stored in the storage (205). At this time, the DP information reading unit (230) reads the DP information stored in the storage (205). This reading of DP information follows the same order as the scan order (20) applied by the defective pixel detection device (100). Therefore, in the case of FIG. 7, the order in which defective pixels are stored is DP c , DP a , DP e , DP b , DP d Therefore, the DP information reading unit (230) reads the coordinate values ​​(or order index) of defective pixels stored in the storage (205) in the above order.

[0082] The system monitor (290) monitors the system resources of the defective pixel correction device (200). These system resources depend on the basic specifications of the defective pixel correction device (200), but may vary depending on the load of other processes currently being performed in the defective pixel correction device (200). For example, if a process that consumes a large amount of resources, such as an L1I neural network algorithm, is running within the defective pixel correction device (200), the current system resources will be significantly insufficient. Based on the current system resources provided by this system monitor (290), the correction limit (L2) is determined variably. Here, since the "correction limit (L2)" must be within the correction limit (L1) used in the aforementioned defective pixel detection device (100), the correction limit (L2) is less than or equal to the correction limit (L1).

[0083] As a specific example, the correction limit determination unit (280) can determine the correction limit (L2) according to the graph of FIG. 9. As illustrated, the correction limit determination unit (280) can increase the correction limit (L2) as the overall system resources increase. Specifically, the correction limit determination unit (280) can determine the correction limit (L2) according to a function (La) with a small slope when the system resources are insufficient compared to the threshold point (P), and according to a function (Lb) with a large slope when the system resources are sufficient compared to the threshold point (P). That is, if the system resources are below the threshold point (P), the correction limit (L2) is determined to be somewhat lower relative to the system resources, and if the system resources are above the threshold point (P), the correction limit (L2) is determined to be somewhat higher relative to the system resources. This further reduces the possibility of errors occurring during the defective pixel correction process when system resources are insufficient.

[0084] Meanwhile, the image sensor (210) captures a real-time image and generates an image frame. Specifically, the image sensor (210) can be implemented as a photosensitive element such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal-oxide Semiconductor).

[0085] The DP correction unit (240) compares the position of a current pixel included in an image frame provided from an image sensor (210) with the position of a defective pixel included in the DP information within the range of the correction limit (L2) determined by the correction limit determination unit (280). At this time, the DP correction unit (240) scans the pixels included in the image frame according to the scanning order as in FIG. 7 and determines whether the coordinates of the currently scanned pixel match the coordinates of the first defective pixel included in the DP information. If they match, the current pixel is corrected. Similarly, it determines whether the coordinates of the currently scanned pixel match the coordinates of the second defective pixel included in the DP information, and if they match, the current pixel is corrected. This process is repeated until the correction limit (L2) is reached.

[0086] However, if the correction limit (L2) is smaller than the correction limit (L1), for example, if the correction limit (L2) is 3 while the correction limit (L1) is 5 in FIG. 7, the DP correction unit (240) [corrects] the 5 defective pixels (DP a , DP b , DP c , DP d , DP e Among them, only the top 3 defective pixels with the highest cumulative count of the DP map (50) can be corrected. In this case, the DP information may include information regarding the order of the cumulative count along with the coordinates of the defective pixels. The order of the cumulative count refers to an order arranged from a high cumulative count (high DP probability) to a low cumulative count (low DP probability), provided by the DP alignment unit (140) of the aforementioned defective pixel detection device (100).

[0087] In the present invention, correction of defective pixels can be performed in various ways known in the prior art. For example, for defective pixels, a concealment method can be applied, such as adjusting the sensor gain differently compared to normal pixels or stretching surrounding normal pixels within a predetermined tap range.

[0088] The defect pixel correction by the above-described DP correction unit (240) is a so-called static correction method that detects defect pixels through testing on a test frame and corrects them according to the detected result. In contrast, dynamic correction is a method that detects and corrects defect pixels in real-time for each image frame in a real-time image. Therefore, dynamic correction has the advantage of not requiring a testing process during the product manufacturing process like static correction, and there is no limit to the number of defect pixels that can be corrected. On the other hand, system overhead caused by real-time defect pixel detection and correction, and image quality degradation due to false positives, are pointed out as disadvantages. Therefore, although dynamic correction allows the user to adjust the correction strength (filter strength), it is difficult to know exactly what level of correction strength is applied to obtain the optimal image, and one must rely on experience.

[0089] Accordingly, according to one embodiment of the present invention, a static correction method of the DP correction unit (240) and a dynamic correction method of the real-time correction unit (250) may be applied sequentially. In FIG. 8, the real-time correction unit (250) additionally performs real-time dynamic correction on an image frame in which defective pixels have been corrected by the DP correction unit (240). Here, the correction strength (S) used for the real-time dynamic correction performed by the real-time correction unit (250) is variably changed according to the correction limit (L2). For example, as the correction limit (L2) increases, the correction strength (S) decreases, and as the correction limit (L2) decreases, the correction strength (S) increases. That is, if there is a possibility that sufficient defective pixel correction cannot be performed by the static correction method when the correction limit (L2) is small, this can be compensated for by increasing the correction strength (S) of the real-time dynamic correction. On the other hand, if the above correction limit (L2) is large, it is highly likely that sufficient defective pixel correction has been performed by the static correction method, so the correction strength (S) of the real-time dynamic correction can be reduced to lower the possibility of false positives.

[0090] In this way, for an input image frame, static correction performed by the DP correction unit (240) and dynamic correction performed by the real-time correction unit (250) are performed sequentially, and then the corrected image frame is provided to the image output unit (260). The image output unit (260) can render the corrected image frame and then output it through a display.

[0091] FIG. 10 is a drawing illustrating an example of an implementation of a computing device (300) that realizes the defective pixel detection device (100) of FIG. 1 or the defective pixel correction device (200) of FIG. 8.

[0092] A computing device (300) has a bus (320), a processor (330), memory (340), storage (350), an input / output interface (310), and a network interface (360). The bus (320) is a data transmission path for the processor (330), memory (340), storage (350), input / output interface (310), and network interface (360) to transmit and receive data to and from each other. However, the method of connecting the processor (330), etc. to each other is not limited to a bus connection. The processor (330) is a computational processing unit such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), or DSP (Digital Signal Processor). The memory (340) is memory such as RAM (Random Access Memory) or ROM (Read Only Memory). The storage (350) is a storage device such as a hard disk, SSD (Solid State Drive), or memory card. Additionally, the storage (350) may be memory such as RAM or ROM.

[0093] The input / output interface (310) is an interface for connecting the computing device (300) and an input / output device. For example, a keyboard or a mouse can be connected to the input / output interface (310).

[0094] The network interface (360) is an interface for connecting the computing device (300) to an external device to communicate and transmit / receive transmission packets. The network interface (360) may be a network interface for connecting to a wired line or a network interface for connecting to a wireless line. For example, the computing device (300) may be connected to another computing device (300-1) through the network (30).

[0095] Storage (350) stores program modules that implement each function of the computing device (300). The processor (330) implements each function corresponding to the program module by executing each of these program modules. Here, when the processor (330) executes each module, it can execute the modules after reading them into memory (340).

[0096] However, the hardware configuration of the computing device (300) is not limited to the configuration shown in FIG. 10. For example, each program module may be stored in memory (340). In this case, the computing device (300) does not need to be equipped with storage (350).

[0097] In this way, the defective pixel detection device (100) of FIG. 1 or the defective pixel correction device (200) of FIG. 8 includes at least a processor (330) and a memory (340) that stores instructions executable by the processor (330), and instructions including various function blocks or steps included in the devices (100, 200) are operated by being executed by the processor (330).

[0098] FIG. 11a is a flowchart illustrating a defective pixel detection method according to an embodiment of the present invention, and FIG. 11b is a flowchart illustrating a defective pixel correction method according to an embodiment of the present invention. Each step of the above methods may be performed by said instructions in a device having a processor and a memory storing instructions executable by said processor.

[0099] First, referring to FIG. 11a, the image sensor (110) captures a test image and generates an image frame (S1).

[0100] The DP detection unit (120) detects defective pixels from pixels included in the image frame (S2).

[0101] The DP map generation unit (130) generates a DP map (50) by accumulating the count of the detected defective pixels for each pixel location for a plurality of image frames (S3), and stores the generated DP map (50) in the storage (105) (S4).

[0102] The DP alignment unit (140) aligns the defective pixels from highest to lowest accumulated count (S5).

[0103] Finally, the DP location storage unit (150) stores the location of the aligned defective pixel in the storage (105) (S6).

[0104] The above storage step (S6) may include the step of storing defective pixels in the storage (105) up to a predetermined correction limit, starting from the order of the highest accumulated count among the aligned defective pixels.

[0105] The step (S3) of generating the above DP map (50) may include the step of reading the current count of the current pixel recorded in the storage (105), the step of increasing the read current count when the current pixel is determined to be a defective pixel, and the step of recording the increased count for the current pixel in the storage (105).

[0106] Here, the DP map (50) has a three-dimensional vector structure including the horizontal position, vertical position of a specific pixel belonging to the image frame, and the count of defective pixels for the specific pixel.

[0107] Next, referring to FIG. 11b, the image sensor (210) captures a real-time image and generates an image frame (S11).

[0108] The DP information reading unit (230) reads the DP information stored in the storage (205) (S12).

[0109] The system monitor (290) monitors the current system resources of the device (200) (S13).

[0110] The correction limit determination unit (280) variably determines the correction limit according to the current system resources (S14).

[0111] The DP correction unit (250) compares the position of the current pixel included in the image frame with the position of the defective pixel included in the DP information within the range of the determined correction limit, and corrects the current pixel when the two positions match each other (S15).

[0112] The real-time correction unit (250) performs real-time dynamic correction on the image frame in which defective pixels have been corrected in the DP correction unit (240) (S16).

[0113] The image output unit (260) outputs an image frame in which real-time dynamic correction is performed by the real-time correction unit (250) (S17).

[0114] The correction strength (S) used for the real-time dynamic correction above may be variably changed according to the correction limit (L2). Specifically, as the correction limit (L2) increases, the correction strength (S) decreases, and as the correction limit (L2) decreases, the correction strength (S) increases.

[0115] Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention may be implemented in other specific forms without changing its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive.

Claims

1. A defective pixel detection device comprising a processor and a memory storing instructions executable by said processor, An image sensor that captures a test image and generates an image frame; A DP detection unit that detects defective pixels from pixels included in the image frame above; A DP map generation unit that generates a DP map by accumulating the count of the detected defective pixels for each pixel location for a plurality of image frames, and stores the generated DP map in storage; A DP alignment unit that aligns the defective pixels from highest to lowest order based on the accumulated counts; and A defective pixel detection device comprising a DP location storage unit that stores the locations of the aligned defective pixels in the storage.

2. In paragraph 1, the DP location storage unit is, A defective pixel detection device that stores defective pixels up to a predetermined correction limit, starting from the order of the highest accumulated count among the above-mentioned aligned defective pixels, in the storage.

3. In Paragraph 2, A defective pixel detection device further comprising a correction limit determining unit that monitors system resources to variably determine the correction limit.

4. In paragraph 1, the defective pixels whose counts are accumulated in the DP map are, A defective pixel detection device including static defective pixels, half defective pixels, and noise.

5. In paragraph 3, the DP sensing unit is, A gain control unit for changing the gain of the above image frame; An exposure control unit for changing the exposure of the above image frame; and A defective pixel detection device comprising a DP determination unit that determines a pixel as a defective pixel when the pixel value of a pixel included in an image frame in which the gain and / or exposure has been changed deviates from a reference range.

6. In Paragraph 5, A defective pixel detection device in which the above DP judgment unit determines the defective pixels within a range in which the number of pixels determined to be defective pixels does not exceed a fixed number.

7. In paragraph 6, the above DP map generating unit A memory read control unit that reads the current count of the current pixel recorded in the storage above; An index update unit that increases the read current count when the current pixel is determined to be a defective pixel by the DP detection unit; and A defective pixel detection device comprising a memory recording control unit that records the increased count for the current pixel in the storage.

8. In Paragraph 6, The above DP map generation unit further includes a frame counter that increases a frame number identifying the number of consecutive image frames by 1, and The above index update unit is a defective pixel detection device that increases the current count when the frame number is within a predetermined number of consecutive image frames.

9. In paragraph 1, the above DP map is, A defective pixel detection device having a three-dimensional vector structure including the horizontal position, vertical position of a specific pixel belonging to the image frame, and the count of defective pixels for the specific pixel.

10. In paragraph 1, the DP location storage unit is, A defective pixel detection device that arranges the positions of the aligned defective pixels according to the scanning order of the image frames and stores the positions of the defective pixels in the storage according to the arranged order.

11. A defective pixel correction device comprising a processor and a memory storing instructions executable by said processor, An image sensor that captures real-time images and generates image frames; A DP information reading unit that reads DP information stored in storage; System monitor that monitors system resources; A correction limit determining unit that variably determines a correction limit according to the above system resources; and A defective pixel correction device comprising a DP correction unit that, within the range of the above-determined correction limit, compares the position of a current pixel included in the image frame with the position of a defective pixel included in the DP information, and corrects the current pixel when the two positions match each other.

12. In Paragraph 11, A defective pixel correction device further comprising a real-time correction unit that performs real-time dynamic correction on an image frame in which defective pixels have been corrected in the above-mentioned DP correction unit.

13. In Paragraph 12, A defective pixel correction device in which the correction strength used for the above real-time dynamic correction is variably changed according to the above correction limit.

14. In Paragraph 12, A defective pixel correction device in which, as the correction limit increases, the correction strength used for the real-time dynamic correction decreases, and as the correction limit decreases, the correction strength increases.

15. In Paragraph 12, A defective pixel correction device further comprising an image output unit that outputs an image frame in which real-time dynamic correction is performed by the real-time correction unit.

16. A method for recording defective pixels performed by said instruction in a device comprising a processor and a memory storing instructions executable by said processor, wherein A step of capturing a test image and generating an image frame; A step of detecting defective pixels from pixels included in the image frame above; A step of generating a DP map by accumulating the count of the detected defective pixels for each pixel location for a plurality of image frames; A step of storing the generated DP map in storage; A step of arranging the defective pixels in order from highest to lowest accumulated count; and A method for recording defective pixels, comprising the step of storing the position of the aligned defective pixels in the storage.

17. In paragraph 16, the above-mentioned storing step is, A method for recording defective pixels, comprising the step of storing defective pixels in the storage according to a predetermined correction limit, starting from the order of the highest accumulated count among the above-mentioned aligned defective pixels.

18. In Clause 16, the above DP sensing unit is, A gain control unit for changing the gain of the above image frame; An exposure control unit for changing the exposure of the above image frame; and A method for recording defective pixels, comprising a DP determination unit that determines a pixel as a defective pixel when the pixel value of a pixel included in an image frame in which the gain and / or exposure has been changed deviates from a reference range.

19. In paragraph 16, the step of generating the DP map is, A step of reading the current count of the current pixel recorded in the storage above; When the above current pixel is determined to be a defective pixel, a step of increasing the read current count; and A method for recording defective pixels, comprising the step of recording the increased count for the current pixel in the storage.

20. In Paragraph 16, the above DP map is, A method for recording defective pixels having a three-dimensional vector structure including a horizontal position, a vertical position of a specific pixel belonging to the image frame, and a count of defective pixels for the specific pixel.

Images