Image dead pixel detection method, correction method, image signal processor and equipment

Abstract

The invention provides an image dead pixel detection method, a correction method, an image signal processor and imaging equipment. The method comprises the following steps: receiving a to-be-detected four-Bayer image and determining at least two candidate texture directions; the method comprises the following steps of: for each candidate texture direction of each target pixel, determining a neighborhood range corresponding to the candidate texture direction by taking the target pixel as a gravity center of gradient calculation, and calculating a gradient value in the candidate texture direction according to pixels in the neighborhood range; wherein the starting point pixel and the ending point pixel which correspond to each other in the neighborhood range are in-phase pixels; taking the candidate texture direction with the minimum gradient value as the detection texture direction of the target pixel; and finally, detecting whether the target pixel is a defective pixel according to the detection texture direction. According to the method, the texture direction detection precision of the pixel points and the detection precision of the image dead pixels can be improved, and a good foundation is laid for improving the final image quality; and any hardware cost does not need to be increased.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image bad pixel detection method, a correction method, an image signal processor and an imaging device. Background Art

[0002] Quad Bayer pattern is one of the main technologies used by image sensors to capture color images. Figure 1 , Figure 1 The following diagram schematically shows the pixel arrangement of a four-Bayer image captured by an image sensor of a four-Bayer array. The basic principle of the four-Bayer array is that the classic Bayer array divides each color into four small pixels (every four small pixels correspond to one large pixel in the classic Bayer array, such as Figure 1 However, static defects such as process deviation and material defects, as well as dynamic reasons such as long-term use, electromagnetic interference and temperature changes may cause some pixels to be lost or perform poorly, or even cause bad pixels to appear in the captured image. In related technologies, it is often necessary to first detect bad pixels in the image, and then correct the target pixels determined to be bad pixels to ensure the quality of the final image. Therefore, the accuracy of bad pixel detection directly affects the quality of the final image.

[0003] For bad pixel detection and correction of image sensors, most of the related technologies are based on the classic Bayer array (traditional Bayer) and take the target pixel P0 ( Figure 1 Marked in yellow, Figure 1 The example should be green) The general process is: first calculate the texture direction of the target pixel P0 (small pixel in the four-Bayer array) (the texture direction is calculated based on the location of the large pixel (orange frame) in the classic Bayer array corresponding to the target pixel P0), then calculate the grayscale difference between the target pixel P0 and the neighboring pixels in the texture direction to detect whether the target pixel P0 is a bad pixel, and finally perform bad pixel correction on the target pixel determined to be a bad pixel. For example, see Figure 1 and Figure 2a , Figure 2b , Figure 2c as well as Figure 2d ,in, Figure 2a It is schematically given Figure 1 The target pixel P0 (for ease of reading, P0 is marked in yellow, and the color of P0 is the same as the other three pixels of the 2×2 pixel. Figure 1 ,and Figure 2a , Figure 2b , Figure 2c as well as Figure 2d Schematic diagram of the neighboring pixel points (i.e., the surrounding adjacent pixel points) when the texture direction is 0° (green in the example); Figure 2b It is schematically given Figure 1 Schematic diagram of the neighborhood pixels of the target pixel P0 when the texture direction is 90°; Figure 2c It is schematically given Figure 1 Schematic diagram of the neighborhood pixels of the target pixel P0 when the texture direction is 135°; Figure 2d It is schematically given Figure 1 Schematic diagram of the neighboring pixels of the target pixel P0 when the texture direction is 45°. Figure 1 and Figure 2a , Figure 2b , Figure 2c as well as Figure 2d As shown, when detecting the target pixel P0, the texture direction of the target pixel P0 must first be calculated. The common practice is to use a similar Figure 2a , Figure 2b , Figure 2c as well as Figure 2d In the manner shown, the grayscale gradient values ​​in different directions around the target pixel P0 are calculated, and the direction with the smallest gradient value is the texture direction of the target pixel P0. However, due to the imbalance of signals of each pixel point in the four-Bayer array, the calculated grayscale value may have a large error, resulting in texture direction detection errors, affecting the accuracy and reliability of bad pixel detection, and ultimately causing image quality degradation.

[0004] It should be noted that the information disclosed in the background technology section of the invention is only intended to deepen the understanding of the general background technology of the invention, and should not be regarded as an admission or suggestion in any form that the information constitutes prior art already known to those skilled in the art. Summary of the invention

[0005] The purpose of the present invention is to provide an image bad pixel detection method, a correction method, an image signal processor and an imaging device. The present invention uses a target pixel as the center of gravity in each candidate texture direction and adopts the same phase gradient to calculate the gradient to determine the texture direction of the target pixel, thereby effectively reducing the influence of signal imbalance in each channel of the four-Bayer image, and can not only improve the texture direction detection accuracy of the pixel point and the detection accuracy of the image bad pixel, thereby laying a good foundation for improving the final image quality; and without increasing any hardware cost, the present invention has good applicability.

[0006] In order to achieve the above object, the present invention is implemented by the following technical scheme: a method for detecting bad pixels in an image, comprising:

[0007] receiving a quad-Bayer image to be detected and determining at least two candidate texture directions;

[0008] The following method is used to perform bad pixel detection on each target pixel in the four-Bayer image to be detected:

[0009] For each candidate texture direction of each target pixel: determine the neighborhood range corresponding to the candidate texture direction by taking the target pixel as the center of gravity of the gradient calculation, and calculate the gradient value of the candidate texture direction according to the pixels in the neighborhood range; wherein the starting point pixel and the ending point pixel corresponding to each other in the neighborhood range are in-phase pixels;

[0010] Taking the candidate texture direction with the smallest gradient value as the detected texture direction of the target pixel;

[0011] Whether the target pixel is a bad pixel is detected according to the detected texture direction.

[0012] Optionally, for each of the candidate texture directions, the starting point pixels and the ending point pixels corresponding to each other in the neighborhood range are in-phase pixels, including:

[0013] Two pixel points whose connecting line is parallel to the candidate texture direction, located at the edge of the same neighborhood range and each having the same position in the corresponding 2×2 pixels of the same color are the corresponding starting point pixel and the ending point pixel.

[0014] Optionally, for each of the candidate texture directions of each of the target pixels, calculating the gradient value of the candidate texture direction according to the pixels within the neighborhood includes:

[0015] For each of the candidate texture directions of each of the target pixels, calculating an initial gradient value in the candidate texture direction according to the pixels within the neighborhood;

[0016] A weighted pixel set is determined based on a pre-acquired gradient weight filter with the target pixel as the center, and the initial gradient value of each pixel in the weighted pixel set corresponding to the target pixel in the candidate texture direction is weighted based on the gradient weight filter to obtain the gradient value of the target pixel in the candidate texture direction.

[0017] Optionally, the detecting whether the target pixel is a bad pixel according to the detected texture direction includes:

[0018] According to the detected texture direction, a concavity and convexity detection method is used to detect whether the target pixel is a bad pixel.

[0019] Optionally, the detecting whether the target pixel is a bad pixel by using a concavity and convexity detection method according to the detected texture direction includes:

[0020] In the direction of the detected texture, a second-order difference method is used to calculate a second-order difference of the target pixel, and a first adjacent pixel second-order difference and a second adjacent pixel second-order difference of two pixels of the same color adjacent to the target pixel;

[0021] If the absolute value of the second-order difference of the target pixel is greater than a first preset value, the absolute value of the second-order difference of the first adjacent pixel is greater than a second preset value, and the absolute value of the second-order difference of the second adjacent pixel is greater than a third preset value, the target pixel is a bad pixel.

[0022] Optionally, in the direction of the detected texture, a second-order difference method is used to calculate the second-order difference of the target pixel of the target pixel, including:

[0023] In the detected texture direction, determining a first distance and a second distance between the target pixel and two adjacent pixels of the same color;

[0024] A first first-order difference is calculated according to the first distance, the pixel value of the target pixel, and the pixel value of the first same-color pixel, and a second first-order difference is calculated according to the second distance, the pixel value of the target pixel, and the pixel value of the second same-color pixel; wherein the first same-color pixel is the adjacent pixel corresponding to the calculation of the first distance, and the second same-color pixel is the adjacent pixel corresponding to the calculation of the second distance;

[0025] A target pixel second-order difference of the target pixel is calculated according to the first first-order difference, the second first-order difference, the first distance, and the second distance.

[0026] Optionally, after determining that the target pixel is a bad pixel, the image bad pixel detection method further includes:

[0027] If the target pixel second-order difference of the target pixel is less than zero, the target pixel is determined to be a bright spot;

[0028] If the target pixel second-order difference of the target pixel is greater than zero, the target pixel is determined to be a dark point.

[0029] In order to achieve the above object, the present invention further provides an image bad pixel correction method, the image bad pixel correction method comprising:

[0030] Acquire a quad-Bayer image to be corrected acquired by an image sensor;

[0031] Using any one of the above-mentioned image bad pixel detection methods to perform bad pixel detection on the four-Bayer image to be corrected;

[0032] The detected bad pixels are corrected to obtain a four-Bayer image after bad pixel correction.

[0033] In order to achieve the above-mentioned purpose, the present invention also provides an image signal processor, which includes a processor and a memory, and a computer program is stored in the memory. When the computer program is executed by the processor, the image bad pixel detection method and / or the image bad pixel correction method described in any one of the above items are implemented.

[0034] In order to achieve the above object, the present invention further provides an imaging device, which includes the above image signal processor.

[0035] Compared with the prior art, the image bad pixel detection method, correction method, image signal processor and imaging device provided by the present invention have the following advantages:

[0036] The image bad pixel detection method provided by the present invention lays a solid foundation for obtaining the accurate texture direction of the target pixel by receiving the four-Bayer image to be detected and determining at least two candidate texture directions; and for each candidate texture direction of each target pixel, the neighborhood range corresponding to the candidate texture direction is determined by taking the target pixel as the center of gravity of gradient calculation, and the gradient value on the candidate texture direction is calculated according to the pixels in the neighborhood range; wherein the starting point pixel and the ending point pixel corresponding to each other in the neighborhood range are in-phase pixels, thus it can be seen that the present invention can effectively reduce the influence of signal imbalance of each channel of the four-Bayer image by taking the target pixel as the center of gravity in each candidate texture direction and using the in-phase gradient to calculate the gradient value to determine the detection texture direction of the target pixel, and can not only improve the texture direction detection accuracy of the pixel point and the detection accuracy of the image bad pixel, thereby laying a good foundation for improving the final image quality; and no hardware cost is required to be increased, and the method has good applicability.

[0037] Furthermore, in the image bad pixel detection method provided by the present invention, the gradient value of the target pixel on each candidate texture direction is weighted according to a pre-acquired gradient weight filter, which can further improve the calculation accuracy of the gradient value of the target pixel in each candidate texture direction, thereby improving the detection accuracy of the texture direction of the target pixel.

[0038] Furthermore, the image bad pixel detection method provided by the present invention can effectively reduce the dependency when the adjacent pixels are bad pixels by using the second-order difference method to calculate the target pixel second-order difference of the target pixel in the detection texture direction, as well as the first adjacent pixel second-order difference and the second adjacent pixel second-order difference of two pixels of the same color adjacent to the target pixel, thereby improving the bad pixel detection accuracy. Furthermore, the use of the second-order difference method can also well ensure the detection accuracy while taking into account the detection efficiency.

[0039] Since the image bad pixel correction method, image signal processor and imaging device provided by the present invention belong to the same inventive concept as the image bad pixel detection method provided by the present invention, the image bad pixel correction method, image signal processor and imaging device provided by the present invention at least have all the advantages of the image bad pixel detection method provided by the present invention. For details on the beneficial effects of the image bad pixel correction method, image signal processor and imaging device provided by the present invention, please refer to the above description of the beneficial effects of the image bad pixel detection method provided by the present invention, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The pixel arrangement diagram of the original image of the image sensor with a quad-Bayer array is schematically given;

[0041] Figure 2a It is schematically given Figure 1 Schematic diagram of the neighboring pixels of the target pixel P0 when the texture direction is 0°;

[0042] Figure 2b It is schematically given Figure 1 Schematic diagram of the neighborhood pixels of the target pixel P0 when the texture direction is 90°;

[0043] Figure 2c It is schematically given Figure 1 Schematic diagram of the neighborhood pixels of the target pixel P0 when the texture direction is 135°;

[0044] Figure 2d It is schematically given Figure 1 Schematic diagram of the neighborhood pixels of the target pixel P0 when the texture direction is 45°;

[0045] Figure 3 A structural block diagram of an image signal processor provided in Embodiment 1 of the present invention;

[0046] Figure 4 A structural block diagram of an imaging device provided in Embodiment 2 of the present invention;

[0047] Figure 5 A schematic diagram of the overall process of the image bad pixel detection method provided in the third embodiment of the present invention;

[0048] Figure 6a , Figure 6b , Figure 6c , Figure 6d Schematic diagram of the neighborhood range corresponding to the target pixel located at the upper left position in one of the 2×2 pixels of the same color when the candidate texture directions are 0°, 90°, 45° and 135° respectively;

[0049] Figure 7a , Figure 7b , Figure 7c , Figure 7d Schematic diagram of the neighborhood range corresponding to the target pixel located at the upper right position in one of the 2×2 pixels of the same color when the candidate texture directions are 0°, 90°, 45° and 135° respectively;

[0050] Figure 8a , Figure 8b , Figure 8c , Figure 8d Schematic diagram of the neighborhood range corresponding to the target pixel located at the lower left position in one of the 2×2 pixels of the same color at the candidate texture directions of 0°, 90°, 45° and 135° respectively;

[0051] Figure 9a , Figure 9b , Fig.9c , Figure 9d Schematic diagram of the neighborhood range corresponding to the target pixel located at the lower right position in one of the 2×2 pixels of the same color when the candidate texture directions are 0°, 90°, 45° and 135° respectively;

[0052] Fig.10a and Fig.10b Schematic diagram of the weighted pixel set corresponding to the target pixel located at the upper left position in one of the 2×2 pixels of the same color when the gradient weight filter is 3×3 and 5×5 respectively;

[0053] Fig.11a and Fig.11b Schematic diagram of the weighted pixel set corresponding to the target pixel located at the upper right position in one of the 2×2 pixels of the same color when the gradient weight filter is 3×3 and 5×5 respectively;

[0054] Fig.12a and Figure 12b Schematic diagram of the weighted pixel set corresponding to the target pixel at the lower left position in one of the 2×2 pixels of the same color when the gradient weight filter is 3×3 and 5×5 respectively;

[0055] Fig.13a and Fig.13b Schematic diagram of the weighted pixel set corresponding to the target pixel located at the lower right position in one of the 2×2 pixels of the same color when the gradient weight filter is 3×3 and 5×5 respectively;

[0056] Fig.14a It is a schematic diagram of detecting whether one of the target pixels is a bad pixel by taking the detection texture direction as 0° as an example;

[0057] Fig.14b The following exemplary diagrams show the detection results corresponding to the target pixel P3 and the pixels P1, P2, P3 (target pixel), P4 and P5 that are adjacent to each other in the direction of the detected texture under different circumstances;

[0058] Fig.15 A schematic diagram of the overall flow of the image bad pixel correction method provided in the fourth embodiment of the present invention;

[0059] The reference numerals are as follows:

[0060] Image signal processor-100, processor-110, memory-120;

[0061] Target pixels - P0, P1, P2, P3, P4;

[0062] Image sensor-200, pixel array-210, readout circuit-220; power management unit-300, timing control unit-400, system control unit-500, output interface-600;

[0063] Application - 700. DETAILED DESCRIPTION

[0064] The following is a further detailed description of an image bad pixel detection method, correction method, image signal processor and imaging device proposed by the present invention in conjunction with the accompanying drawings. According to the following description, the advantages and features of the present invention will be clearer. It should be noted that the drawings are in a very simplified form and all use non-precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In order to make the purposes, features and advantages of the present invention more obvious and easy to understand, please refer to the accompanying drawings. It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the limiting conditions for the implementation of the present invention. Any modification of the structure, change of the proportional relationship or adjustment of the size, under the same or similar conditions as the effects and purposes that can be achieved by the present invention, should still fall within the scope of the technical content disclosed by the present invention. The specific design features of the present invention disclosed herein include, for example, specific dimensions, directions, positions and shapes, which will be determined in part by the specific environment to be applied and used. And, in the embodiments described below, sometimes the same figure mark is used in common between different drawings to represent the same part or part with the same function, and its repeated description is omitted. In this specification, similar reference numerals and letters are used to refer to similar items, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures. In addition, if the method described herein includes a series of steps, the order of these steps presented herein is not necessarily the only order in which these steps can be performed, and some of the steps described may be omitted and / or some other steps not described herein may be added to the method.

[0065] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The singular forms "a", "an" and "the" include plural objects, the term "or" is generally used in a sense including "and / or", the term "several" is generally used in a sense including "at least one", and the term "at least two" is generally used in a sense including "two or more". In addition, the terms "first", "second" and "third" are used for descriptive purposes only and are not to be understood as indicating or suggesting relative importance or implicitly indicating the number of the indicated technical features.

[0066] In order to facilitate the understanding and explanation of the present invention, the main research process of the present invention is first described as follows: Figure 2a , Figure 2b , Figure 2c , Figure 2d As described in the background art, Figure 2a , Figure 2b , Figure 2c as well as Figure 2d For example, when the texture direction of the target pixel P0 is detected, the classic Bayer pixel ( Figure 2a , Figure 2b , Figure 2c as well as Figure 2d The inventors have found through extensive investigations and studies and continuous in-depth practice that this texture direction calculation method not only fails to take into account the signal imbalance of four-Bayer pixels (for example, the target pixel P0, the four-Bayer pixels are the four small pixels in the 2×2 square where the target pixel P0 is located, such as the area marked with an orange box), which leads to a large error in the calculation of the gradient; and the mismatch between the texture gradient center of gravity and the target pixel P0 may also cause the mismatch between the calculated texture direction and the actual texture direction of the target pixel P0, resulting in texture direction detection errors, affecting the accuracy and reliability of bad pixel detection, and ultimately leading to a decrease in image quality.

[0067] Based on the above research, the core idea of ​​the present invention is to provide an image bad pixel detection method, a correction method, an image signal processor and an imaging device. The present invention determines the texture direction of the target pixel by taking the target pixel as the center of gravity in each candidate texture direction and using the same phase gradient to calculate the gradient, thereby effectively reducing the influence of signal imbalance in each channel of the four-Bayer image. It can not only improve the texture direction detection accuracy of the pixel point and the detection accuracy of the image bad pixel, thereby laying a good foundation for improving the final image quality; but also does not need to increase any hardware cost and has good applicability.

[0068] It should be noted that the image bad pixel detection method and correction method provided by the present invention can be applied to the image signal processor and imaging device provided by the present invention, and the image bad pixel detection method, correction method and image signal processor provided by the present invention can be applied to the imaging device provided by the present invention. It should be understood that the term "imaging device" or "imaging device" or other similar terms used herein include general imaging devices, such as but not limited to cameras, video cameras, mobile phones, tablet computers, learning machines, and medical imaging devices with image sensors.

[0069] In this document, Example 1 describes the image signal processor provided by the present invention, Example 2 describes the imaging device provided by the present invention, Example 3 describes the image bad pixel detection method provided by the present invention, and Example 4 describes the image bad pixel correction method provided by the present invention.

[0070] It should be understood that those skilled in the art should be able to understand that although the image signal processor is used as an example to illustrate the image bad pixel detection method and the image bad pixel correction method, this is not a limitation of the present invention. The image bad pixel detection method and the image bad pixel correction method provided by the present invention do not impose any limitation on the execution subject.

[0071] Embodiment 1

[0072] This embodiment provides an image signal processor (ISP). For example, see Figure 3 , Figure 3 FIG. 1 is a structural block diagram of the image signal processor provided in this embodiment. Figure 3As shown, the image signal processor 100 provided in this embodiment includes a processor 110 and a memory 120, and a computer program is stored on the memory 120. When the computer program is executed by the processor 110, the image bad pixel detection method and / or image bad pixel correction method provided in any embodiment of the present invention are implemented. Since the image signal processor provided in this embodiment and the image bad pixel detection method or image bad pixel correction method provided in the present invention belong to the same inventive concept, and the image bad pixel correction method provided in the present invention and the image bad pixel detection method provided in the present invention belong to the same inventive concept, therefore, the image signal processor provided in this embodiment has at least all the advantages of the image bad pixel detection method provided in the present invention. For detailed contents, please refer to the relevant description of the beneficial effects of the image bad pixel detection method below, and here, it will not be expanded one by one. In addition, please refer to the relevant embodiments of the image bad pixel detection method below for the specific contents of the image bad pixel detection method, and please refer to the relevant embodiments of the image bad pixel correction method below for the specific contents of the image bad pixel correction method. In order to avoid redundancy, it will not be expanded here.

[0073] It should be noted that the image signal processor 100 provided in this embodiment is mainly used for signal conversion between the image sensor 200 (such as a CMOS or CCD sensor) and the application (such as output to a display device), and its core function is to extract and optimize image information from the raw data output by the image sensor 200, and finally output an image that can be displayed, stored or further processed. Specifically, the memory 120 can be used to store the computer program, and the processor 110 implements various functions of the image signal processor 100 by running or executing the computer program stored in the memory 120 and calling the data stored in the memory 120, which include but are not limited to bad pixel detection and correction, noise removal, and color adjustment.

[0074] It should be noted that, as can be understood by those skilled in the art, the present invention does not impose too many limitations on the processor 110. For example, the processor 110 referred to in the present invention can be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and can also be a microcontroller unit (MCU) or other general-purpose processors. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The processor 110 is the control center of the image signal processor 100, and various interfaces and lines are used to connect the various parts of the entire image signal processor 100.

[0075] It should also be noted that, as can be understood by those skilled in the art, the memory 120 may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory 120 may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0076] In addition, computer program code for performing operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages.

[0077] It should be understood that only the part of the image signal processor 100 related to the present invention is described above. For more detailed content about the image signal processor 100 not mentioned in this article, please refer to the image signal processor-related technologies known to those skilled in the art. Due to space limitations, this article will not elaborate on this.

[0078] Embodiment 2

[0079] This embodiment provides an imaging device. For example, see Figure 4 , Figure 4 The structure block diagram of the imaging device provided in this embodiment. Figure 4 It can be seen that the imaging device provided in this embodiment includes the image signal processor 100 provided by the present invention, as well as the image sensor 200 and the application 700 (APP, Application). Since the imaging device provided in this embodiment and the image signal processor 100 provided by the present invention belong to the same inventive concept, and the image signal processor 100 provided by the present invention and the image bad pixel detection method provided by the present invention belong to the same inventive concept, the imaging device provided in this embodiment at least has all the advantages of the image bad pixel detection method provided by the present invention. For details, please refer to the relevant description of the beneficial effects of the image bad pixel detection method below, and will not be elaborated one by one here.

[0080] Please continue to see Figure 4 ,like Figure 4 As shown, the imaging device further includes a power management unit 300 (PMU, Power Management Unit), a timing control (CLK) unit 400, a system control (system control) unit 500 and an output interface (Output interface) 600; further, the image sensor 200 of the imaging device includes a pixel array 210 and a readout circuit 220. Next, in combination with Figure 4The imaging process of the imaging device is briefly described as follows: under the coordinated control of the system control unit 500 and the timing control unit 400, the pixel array 210 converts the collected original light signal into an electrical signal, the readout circuit 220 performs analog-to-digital conversion on the electrical signal to obtain original image data (such as a four-Bayer image) and sends the converted original image data to the image signal processor 100; the image signal processor 100 performs processing including but not limited to bad pixel detection and correction, noise removal, and color adjustment on the original image data and sends the processed final image data to the output interface 600, and the output interface 600 converts and adapts the final image data according to the control of the application 700 to output the image data to a display screen, a memory or other device. The power management unit 300 provides the required power for the functional modules of the imaging device such as the timing control unit 400, the system control unit 500, and the image sensor 200.

[0081] For further information, please see Figure 4 ,like Figure 4 As shown, in some preferred implementations, the processing flow of the image bad pixel detection method executed by the image signal processor 100 is roughly as follows: first, bad pixel detection is performed on the target pixel points of the received four-Bayer image to be detected. Preferably, the texture direction is first detected by using the same phase weighting, and then the bad pixel is detected by using the concavity and convexity detection method according to the determined texture direction. For more detailed information about image bad pixel detection, please refer to the relevant description of the image bad pixel detection method in this article, which will not be expanded here. Furthermore, the processing flow of the image bad pixel correction method executed by the image signal processor 100 is roughly as follows: the bad pixels detected by the image bad pixel detection method provided by the present invention are corrected to obtain the four-Bayer image after bad pixel correction. For more detailed information about image bad pixel correction, please refer to the relevant description of the image bad pixel correction method in this article, which will not be expanded here.

[0082] Embodiment 3

[0083] This embodiment provides a method for detecting bad pixels in an image. For example, see Figure 5 , Figure 5 The overall flow chart of the image bad pixel detection method provided in this embodiment is as follows. Figure 5 It can be seen that the image bad pixel detection method provided in this embodiment includes:

[0084] S100: receiving a quad-Bayer image to be detected and determining at least two candidate texture directions;

[0085] The following steps S210 and S220 are used to perform bad pixel detection on each target pixel in the to-be-detected four-Bayer image:

[0086] S210: for each candidate texture direction of each target pixel: determine a neighborhood range corresponding to the candidate texture direction by taking the target pixel as the center of gravity of gradient calculation, and calculate the gradient value of the candidate texture direction according to the pixels in the neighborhood range; wherein the starting point pixel and the ending point pixel corresponding to each other in the neighborhood range are in-phase pixels;

[0087] S220: Taking the candidate texture direction with the smallest gradient value as the detected texture direction of the target pixel;

[0088] S300: Detecting whether the target pixel is a bad pixel according to the detected texture direction.

[0089] Therefore, the image bad pixel detection method provided by the present invention lays a solid foundation for obtaining the accurate texture direction of the target pixel by receiving the four-Bayer image to be detected and determining at least two candidate texture directions; and for each candidate texture direction of each target pixel, the neighborhood range corresponding to the candidate texture direction is determined by taking the target pixel as the center of gravity of gradient calculation, and the gradient value in the candidate texture direction is calculated according to the pixels in the neighborhood range; wherein the starting point pixel and the ending point pixel corresponding to each other in the neighborhood range are in-phase pixels, thus it can be seen that the present invention can effectively reduce the influence of signal imbalance of each channel of the four-Bayer image by taking the target pixel as the center of gravity in each candidate texture direction and using the in-phase gradient to calculate the gradient value to determine the detection texture direction of the target pixel, and can not only improve the texture direction detection accuracy of the pixel point and the detection accuracy of the image bad pixel, thereby laying a good foundation for improving the final image quality; and no hardware cost is required to be increased, and the method has good applicability.

[0090] It should be noted that, as can be understood by those skilled in the art, the present invention does not impose too many restrictions on the color filter array (CFA) used in the quad-Bayer image. For example, the color filter matrix may be of RG type. The color arrangement may be any one of RGGB, BGGR, GRBG and GBGR.

[0091] Preferably, in some exemplary embodiments, there are four candidate texture directions in step S100, including directions with angles of 0°, 45°, 90° and 135° to the horizontal direction, respectively. Thus, by using the four classic directions of horizontal direction (0°), lower left-upper right direction (45°), vertical direction (90°) and upper left-lower right direction (135°) as candidate texture directions, such a setting can not only avoid too few candidate texture directions leading to too low texture direction detection accuracy, but also avoid too many candidate texture directions leading to reduced texture direction detection efficiency, so that both texture direction detection accuracy and detection efficiency can be taken into account.

[0092] It should be noted that those skilled in the art should be able to understand that the candidate texture directions herein include four directions with angles of 0°, 45°, 90° and 135° with the horizontal direction, which are only exemplary descriptions of preferred embodiments, and are not limitations of the present invention. The present invention does not impose any restrictions on the number of the candidate texture directions and the specific value of the angle between each candidate texture direction and the horizontal direction. For example, in some other embodiments, the candidate texture directions may also include but are not limited to six directions with angles of 0°, 30°, 60°, 90°, 120°, and 150° with the horizontal direction. When implementing the present invention, it should be reasonably set according to actual needs.

[0093] Preferably, in some exemplary implementations, for each of the candidate texture directions in step S210, the corresponding starting point pixels and ending point pixels in the neighborhood range are in-phase pixels, including:

[0094] Two pixel points whose connecting line is parallel to the candidate texture direction, located at the edge of the same neighborhood range and each having the same position in the corresponding 2×2 pixels of the same color are the corresponding starting point pixel and the ending point pixel.

[0095] Therefore, in the image bad pixel detection method provided by the present invention, the corresponding starting point pixel and the ending point pixel are located at the edge of the same neighborhood range, and their respective positions in the 2×2 same-color pixels are the same and the connecting line is parallel to the candidate texture direction. It can well ensure that the corresponding target pixel point is located at the center of gravity of the candidate texture direction, thereby effectively reducing the influence of signal imbalance of each channel of the four-Bayer image, thereby improving the texture direction detection accuracy of the pixel point.

[0096] For example, in some of the exemplary embodiments, see Figures 6a to 6d , Figures 7a to 7d , Figures 8a to 8d as well as Figures 9a to 9d ,in, Figure 6a , Figure 6b , Figure 6c , Figure 6d Schematic diagram of the neighborhood range corresponding to the target pixel located at the upper left position in one of the 2×2 pixels of the same color at the candidate texture directions of 0°, 90°, 45° and 135° respectively; Figure 7a , Figure 7b , Figure 7c , Figure 7d Schematic diagram of the neighborhood range corresponding to the target pixel located at the upper right position in one of the 2×2 pixels of the same color at the candidate texture directions of 0°, 90°, 45° and 135° respectively; Figure 8a , Figure 8b , Figure 8c , Figure 8d Schematic diagram of the neighborhood range corresponding to the target pixel located at the lower left position in one of the 2×2 pixels of the same color at the candidate texture directions of 0°, 90°, 45° and 135° respectively; Figure 9a , Figure 9b , Fig.9c , Figure 9d Schematic diagram of the neighborhood range corresponding to the target pixel located at the lower right position in one of the 2×2 pixels of the same color at the candidate texture directions of 0°, 90°, 45° and 135°. Specifically, Figures 6a to 6d The target pixel P1 shown, Figures 7a to 7d The target pixel P2 shown, Figures 8a to 8d The target pixel P3 and Figures 9a to 9d The target pixel P4 shown is the same 2×2 same-color pixel ( Figures 6a to 6d , Figures 7a to 7d , Figures 8a to 8d as well as Figures 9a to 9d Pixels at different positions in the image are marked with orange boxes. Figures 6a to 6d , Figures 7a to 7d , Figures 8a to 8d as well as Figures 9a to 9d It can be seen that the target pixels P1, P2, P3 and P4 are all located at the center of their corresponding neighborhood ranges. Figure 6a For example, the pair of same-phase pixels in the first row of the start point pixel located on the far left (in the direction of the diagram) and the end point pixel located on the far right (in the direction of the diagram) is located between the 2×2 same-color pixels ( Figure 6a The lower left position of the yellow dotted box in the figure, the rightmost end point pixel is located at the 2×2 same color pixels ( Figure 6a The lower left position of the target pixel P1 ( Figure 6a , Figure 6b , Figure 6c as well as Figure 6d The yellow highlight in the figure is located at the center of gravity when the candidate texture direction is 0°. Figure 6b For example, the pair of same-phase pixels in the first column of the graph, the starting point pixel at the top (in the direction of the diagram) and the ending point pixel at the bottom (in the direction of the diagram), the starting point pixel at the top is located between the 2×2 same-color pixels ( Figure 6b The lowermost end point pixel is located at the upper right position of the 2×2 pixels of the same color ( Figure 6b The target pixel P1 is located at the upper right position of the candidate texture (marked by a yellow dotted box in the figure), and the target pixel P1 is located at the center of gravity when the candidate texture direction is 90°.

[0097] It should be noted that, as can be understood by those skilled in the art, Figures 6a to 6d , Figures 7a to 7d , Figures 8a to 8d as well as Figures 9a to 9d The neighborhood range shown in is only an exemplary description of a preferred embodiment and is not a limitation of the present invention. The present invention does not set too many values ​​for the size of the neighborhood range. When implementing the present invention, the size of the neighborhood range of each candidate texture direction should be reasonably set as needed. It should be understood that in order to further ensure the accuracy of detecting the texture direction, the size of the neighborhood range of each candidate texture direction is preferably kept consistent.

[0098] Preferably, in some exemplary implementations, the step S210 of calculating the gradient value of each candidate texture direction of each target pixel according to the pixels in the neighborhood includes:

[0099] S211: For each of the candidate texture directions of each of the target pixels, calculate an initial gradient value in the candidate texture direction according to the pixels within the neighborhood;

[0100] S212: Determine a weighted pixel set based on the pre-acquired gradient weight filter with the target pixel as the center, and weight the initial gradient value of each pixel in the weighted pixel set corresponding to the target pixel in the candidate texture direction according to the gradient weight filter to obtain the gradient value of the target pixel in the candidate texture direction.

[0101] Therefore, in the image bad pixel detection method provided by the present invention, the gradient value of the target pixel on each candidate texture direction is obtained by weighting according to the pre-acquired gradient weight filter, which can further improve the calculation accuracy of the gradient value of the target pixel in each candidate texture direction, thereby improving the detection accuracy of the texture direction of the target pixel.

[0102] It should be noted that the present invention does not impose too many restrictions on the specific method of calculating the initial gradient value in the candidate texture direction according to the pixels within the neighborhood in step S212. For example, it can be adopted including but not limited to gradient calculation based on difference and gradient calculation based on convolution. For the specific calculation method of the initial gradient value of the target pixel, please refer to the adaptive understanding of the relevant content about image texture gradient known to those skilled in the art. Due to space limitations, this article will not elaborate on this.

[0103] For example, see Fig.10a , Fig.10b , Fig.11a , Fig.11b , Fig.12a , Figure 12b , Fig.13a as well as Fig.13b ,in, Fig.10a and Fig.10b Schematic diagram of the weighted pixel set corresponding to the target pixel located at the upper left position in one of the 2×2 pixels of the same color when the gradient weight filter is 3×3 and 5×5 respectively; Fig.11a and Fig.11b Schematic diagram of the weighted pixel set corresponding to the target pixel located at the upper right position in one of the 2×2 pixels of the same color when the gradient weight filter is 3×3 and 5×5 respectively; Fig.12a and Figure 12b Schematic diagram of the weighted pixel set corresponding to the target pixel at the lower left position in one of the 2×2 pixels of the same color when the gradient weight filter is 3×3 and 5×5 respectively; Fig.13a and Fig.13b Schematic diagram of the weighted pixel set corresponding to the target pixel located at the lower right position in one of the 2×2 pixels of the same color when the gradient weight filter is 3×3 and 5×5 respectively. Fig.10a , Fig.10b , Fig.11a , Fig.11b , Fig.12a , Figure 12b , Fig.13a as well as Fig.13b In the example, the target pixels P1, P2, P3 and P4 are 2×2 pixels of the same color ( Fig.10a , Fig.10b , Fig.11a , Fig.11b , Fig.12a , Figure 12b , Fig.13a as well as Fig.13b Taking the target pixel P1 as an example, when the gradient weight filter is 3×3, Fig.10a As shown, when calculating the gradient value of each candidate texture direction of the target pixel P1, Fig.10a The pixels a, b, c, d, the target pixel P1, and the pixels e, f, g, and h in the image form a weighted pixel set for calculating the candidate texture directions of the target pixel P1. The weights corresponding to the weighted pixels are as follows: Fig.10a As shown in the gradient weight filter in; when the gradient weight filter is 5×5, as shown in Fig.10b As shown, when calculating the gradient value of each candidate texture direction of the target pixel P1, Fig.10b The pixels a, b, c, d, e, f, g, h, i, j, k, l, the target pixel P1, and the pixels m, n, o, p, q, r, s, t, u, v, w, and x in the image form a weighted pixel set for calculating the candidate texture directions of the target pixel P1. The weights corresponding to the weighted pixels are as follows: Fig.10b The weight filter in is shown.

[0104] For example, Fig.10aFor example, the gradient value of the target pixel P1 can be calculated by the following formula (1):

[0105] GRAD_P1=(w_a*grad_a+……+w_h*grad_h+w_P1*grad_P1) / (w_a+

[0106] ……+w_h+w_P1)(1)

[0107] In formula (1), GRAD_P1 is the gradient value of the target pixel P1, w_a, ..., w_h, w_P1 are the filter weights of pixels a, ..., h and the target pixel P1 respectively, and grad_a, ..., grad_h and grad_P1 are the initial gradient values ​​of pixels a, ..., h and the target pixel P1 respectively.

[0108] For example, Fig.11a For example, the gradient value of the target pixel P2 can be calculated by the following formula (2):

[0109] GRAD_P2=(w_a*grad_a+……+w_h*grad_h+w_P2*grad_P2) / (w_a+

[0110] ……+w_h+w_P2)(2)

[0111] In formula (2), GRAD_P2 is the gradient value of the target pixel P2, w_a, ..., w_h, w_P2 are the filter weights of pixels a, ..., h and the target pixel P2 respectively, and grad_a, ..., grad_h and grad_P2 are the initial gradient values ​​of pixels a, ..., h and the target pixel P2 respectively.

[0112] For example, Fig.12a For example, the gradient value of the target pixel P3 can be calculated by the following formula (3):

[0113] GRAD_P3=(w_a*grad_a+……+w_h*grad_h+w_P3*grad_P3) / (w_a+

[0114] ……+w_h+w_P3)(3)

[0115] In formula (3), GRAD_P3 is the gradient value of the target pixel P3, w_a, ..., w_h, w_P3 are the filter weights of pixels a, ..., h and the target pixel P3 respectively, and grad_a, ..., grad_h and grad_P3 are the initial gradient values ​​of pixels a, ..., h and the target pixel P3 respectively.

[0116] For example, Fig.13aFor example, the gradient value of the target pixel P4 can be calculated by the following formula (4):

[0117] GRAD_P4=(w_a*grad_a+……+w_h*grad_h+w_P4*grad_P4) / (w_a+

[0118] ……+w_h+w_P4)(4)

[0119] In formula (4), GRAD_P4 is the gradient value of the target pixel P4, w_a, ..., w_h, w_P4 are the filter weights of pixels a, ..., h and the target pixel P4 respectively, and grad_a, ..., grad_h and grad_P4 are the initial gradient values ​​of pixels a, ..., h and the target pixel P4 respectively.

[0120] It should be understood that, for ease of understanding, this article describes the weighted pixel set of target pixels P1, P2, P3 and P4. Fig.10a , Fig.10b , Fig.11a , Fig.11b , Fig.12a , Figure 12b , Fig.13a as well as Fig.13b Although the same reference numerals a, b, c, d, ... are used, by comparison Fig.10a , Fig.10b , Fig.11a , Fig.11b , Fig.12a , Figure 12b , Fig.13a as well as Fig.13b It is not difficult to find that in Fig.10a , Fig.10b , Fig.11a , Fig.11b , Fig.12a , Figure 12b , Fig.13a as well as Fig.13b In the figure, even the same reference numerals do not refer to the same pixel.

[0121] It should be noted that, as can be understood by those skilled in the art, although the gradient weight filter is described herein using 3×3 pixels and 5×5 pixels as examples, it is obvious that this is only an exemplary description of a preferred embodiment and not a limitation of the present invention. The present invention does not make too many settings on the size of the gradient weight filter and the weight value of each pixel position in the gradient weight filter. When implementing the present invention, the size of the gradient weight filter and the weight of each pixel position should be reasonably set as needed.

[0122] Preferably, in some exemplary implementations, the step S300 of detecting whether the target pixel is a bad pixel according to the detected texture direction includes:

[0123] According to the detected texture direction, a concavity and convexity detection method is used to detect whether the target pixel is a bad pixel.

[0124] Therefore, the image bad pixel detection method provided by the present invention can further improve the accuracy of bad pixel detection by detecting the target pixel using the concave-convex detection method according to the detection texture direction.

[0125] Preferably, in some exemplary embodiments, the detecting whether the target pixel is a bad pixel by using a secondary differential scattered data concavity and convexity detection method according to the detected texture direction comprises:

[0126] SA1: Calculate the second-order difference of the target pixel and the first adjacent pixel second-order difference and the second adjacent pixel second-order difference of two pixels of the same color adjacent to the target pixel in the direction of the detected texture by using the second-order difference method;

[0127] SA2: If the absolute value of the second-order difference of the target pixel is greater than the first preset value, the absolute value of the second-order difference of the first adjacent pixel is greater than the second preset value, and the absolute value of the second-order difference of the second adjacent pixel is greater than the third preset value, then the target pixel is a bad pixel.

[0128] Therefore, the image bad pixel detection method provided by the present invention can effectively reduce the dependency when the adjacent pixels are bad pixels by using the second-order difference method to calculate the target pixel second-order difference of the target pixel in the detection texture direction, as well as the first adjacent pixel second-order difference and the second adjacent pixel second-order difference of two pixels of the same color adjacent to the target pixel, thereby improving the bad pixel detection accuracy. Furthermore, the use of the second-order difference method can also well ensure the detection accuracy while taking into account the detection efficiency.

[0129] It should be noted that those skilled in the art should be able to understand that the present invention does not impose too many restrictions on the specific values ​​of the first preset value, the second preset value and the third preset value. When implementing the present invention, the specific values ​​of the first preset value, the second preset value and the third preset value can be reasonably set according to actual needs.

[0130] It can be understood that if the absolute value of the second-order difference of the target pixel is not greater than the first preset value, the absolute value of the second-order difference of the first adjacent pixel is not greater than the second preset value, or the absolute value of the second-order difference of the second adjacent pixel is not greater than the third preset value, then the target pixel is not a bad pixel.

[0131] Preferably, in some exemplary embodiments, in step SA1, in the direction of the detected texture, a second-order difference method is used to calculate the second-order difference of the target pixel of the target pixel, including:

[0132] SA11: determining a first distance and a second distance between the target pixel and two adjacent pixels of the same color in the direction of the detected texture;

[0133] SA12: Calculate a first first-order difference according to the first distance, the pixel value of the target pixel, and the pixel value of the first same-color pixel, and calculate a second first-order difference according to the second distance, the pixel value of the target pixel, and the pixel value of the second same-color pixel; wherein the first same-color pixel is the adjacent pixel corresponding to the calculation of the first distance, and the second same-color pixel is the adjacent pixel corresponding to the calculation of the second distance;

[0134] SA13: Calculate a target pixel second-order difference of the target pixel according to the first first-order difference, the second first-order difference, the first distance, and the second distance.

[0135] Therefore, the image bad pixel detection method provided by the present invention uses the two pixels of the same color that are closest to the target pixel to perform second-order difference calculation, which can further improve the detection accuracy of the bad pixel.

[0136] For the calculation method of the first adjacent pixel second-order difference and the second adjacent pixel second-order difference, please refer to the above steps SA11, SA12 and SA13 for the adaptive understanding of the calculation method of the target pixel second-order difference. In order to avoid repetition, it will not be elaborated here.

[0137] Preferably, in some exemplary embodiments, after determining that the target pixel is a bad pixel in step SA2, the image bad pixel detection method further includes:

[0138] If the target pixel second-order difference of the target pixel is less than zero, the target pixel is determined to be a bright spot;

[0139] If the target pixel second-order difference of the target pixel is greater than zero, the target pixel is determined to be a dark point.

[0140] Therefore, the image bad pixel detection method provided by the present invention can not only detect whether the target pixel is a bad pixel, but also can further determine whether the target pixel is a dark pixel or a bright pixel when the target pixel is determined to be a bad pixel, thereby further improving the accuracy of bad pixel detection.

[0141] For example, see Fig.14a , Fig.14a It is a schematic diagram of detecting whether one of the target pixels is a bad pixel by taking the texture direction as 0° as an example. Fig.14aIn the figure, the target pixel is P3, the two pixels of the same color adjacent to the target pixel P4 are pixel P2 and pixel P4, the two pixels of the same color adjacent to pixel P2 are pixel P1 and target pixel P3, and the two pixels of the same color adjacent to pixel P4 are target pixel P3 and pixel P5.

[0142] First, the first distance and the second distance between the target pixel and its two adjacent pixels P2 and P4 of the same color are calculated by the following equations (5-1) and (5-2).

[0143] h1=x_p3-x_p2 (5-1)

[0144] h2=x_p4-x_p3 (5-1)

[0145] In formula (5-1) and formula (5-2), h1 is the first distance, h2 is the second distance, x_p3 is the horizontal coordinate of the target pixel P3 (as mentioned above, the texture direction is detected as 0°, that is, the horizontal direction), x_p2 is the horizontal coordinate of pixel P2, and x_p4 is the horizontal coordinate of pixel P4.

[0146] Next, the first and second first-order differences are calculated using the following equations (6-1) and (6-2):

[0147] Δy1=(y_p3-y_p2) / h1 (6-1)

[0148] Δy2=(y_p4-y_p3) / h2 (6-1)

[0149] In equations (6-1) and (6-2), Δy1 is the first first-order difference, Δy2 is the second first-order difference, h1 is the first distance, h2 is the second distance, y_p3 is the pixel value of the target pixel P3, y_p2 is the pixel value of the pixel P2, and y_p4 is the pixel value of the pixel P4.

[0150] Then, the target pixel second-order difference of the target pixel P3 is calculated by the following formula (7):

[0151] Δ2y_p3=(h1·Δy2 - h2·Δy1) / h1*h2 (7)

[0152] In formula (7), Δ2y_p3 is the target pixel second-order difference of the target pixel P3, h1 is the first distance, h2 is the second distance, Δy1 is the first first-order difference, and Δy2 is the second first-order difference.

[0153] Furthermore, a method similar to Formula (5-1), Formula (5-2), Formula (6-1), Formula (6-2) and Formula (7) is used to calculate the first adjacent pixel second-order difference Δ2y_p2 and the second adjacent pixel second-order difference Δ2y_p4 corresponding to the adjacent pixels P2 and P4 of the same color of the target pixel P3.

[0154] Finally, whether the target pixel P3 is a bad pixel can be determined by the following formula (8):

[0155] if(abs(Δ2y_p3)>Thresh1&&abs(Δ2y_p2)>Thresh2&&

[0156] abs(Δ2y_p4)> Thresh 3) then the target pixel P3 is a bad pixel (8)

[0157] In formula (8), Δ2y_p3 is the target pixel second-order difference of the target pixel P3, Δ2y_p2 is the first adjacent pixel second-order difference, Δ2y_p4 is the second adjacent pixel second-order difference, Thresh1 is the first preset value, Thresh2 is the second preset value, and Thresh3 is the third preset value.

[0158] Furthermore, if Δ2y_p3<0, the curve at the target pixel P3 is convex, and the target pixel P3 may be a bright bad pixel; if Δ2y_p3>0, the curve at the target pixel P3 is concave, and the target pixel P3 may be a dark bad pixel. The concave and convex conditions of the curves at pixels P2 and P4 can also be determined by Δ2y_p2 and Δ2y_p4. Please refer to the relevant description of the target pixel P3 for adaptive understanding, which will not be repeated.

[0159] For example, see Fig.14b , Fig.14b Example diagrams of detection results corresponding to the target pixel P3 and pixels P1, P2, P3 (target pixel), P4 and P5 that are adjacent in the direction of the detected texture under different circumstances are given as examples. Fig.14b In various cases, bad pixels are shown in red. Fig.14b In the example, the horizontal coordinates of pixels P1, P2, P3, P4 and P5 are 1, 2, 3, 4 and 5 respectively, and the first preset value, the second preset value and the third preset value are all 300.

[0160] In the first case, if the coordinates and pixel value information of pixels P1, P2, target pixel P3, and pixels P4 and P5 are: p1 (1,500), p2 (2,510), p3 (3,480), p4 (4,510) and p5 (5,490), respectively, combined with the calculation method of the second-order difference Δ2y_p3 of the target pixel, the second-order difference Δ2y_p2 of the first adjacent pixel, and the second-order difference Δ2y_p4 of the second adjacent pixel, it can be obtained that: Δ2y_p3 = 60, Δ2y_p2 = -40, Δ2y_p4 = -50. According to the above formula (8), it can be judged that the target pixel P3 is not a bad pixel.

[0161] In the second case, if the coordinates and pixel value information of pixels P1, P2, target pixel P3, and pixels P4 and P5 are: p1 (1,500), p2 (2,900), p3 (3,510), p4 (4,500) and p5 (5,505), respectively, combined with the calculation method of the second-order difference Δ2y_p3 of the target pixel, the second-order difference Δ2y_p2 of the first adjacent pixel, and the second-order difference Δ2y_p4 of the second adjacent pixel in the above text, it can be obtained that: Δ2y_p3 = 380, Δ2y_p2 = -790, Δ2y_p4 = 15. According to the above formula (8), it can be judged that the target pixel P3 is not a bad pixel.

[0162] In the third case, if the coordinates and pixel value information of pixels P1, P2, target pixel P3, and pixels P4 and P5 are respectively: p1 (1,500), p2 (2,530), p3 (3,900), p4 (4,510) and p5 (5,520), combined with the calculation method of the second-order difference Δ2y_p3 of the target pixel, the second-order difference Δ2y_p2 of the first adjacent pixel, and the second-order difference Δ2y_p4 of the second adjacent pixel, it can be obtained that in this case: Δ2y_p3 = -760, Δ2y_p2 = 340, Δ2y_p4 = 400. According to the above formula (8), it can be judged that the target pixel P3 is a bad pixel. Furthermore, since Δ2y_p3 is -760, which is less than 0, the target pixel P3 is a bright spot.

[0163] In the fourth case, if the coordinates and pixel value information of pixels P1, P2, target pixel P3, and pixels P4 and P5 are respectively: p1 (1,500), p2 (2,900), p3 (3,910), p4 (4,510) and p5 (5,520), combined with the calculation method of the second-order difference Δ2y_p3 of the target pixel, the second-order difference Δ2y_p2 of the first adjacent pixel, and the second-order difference Δ2y_p4 of the second adjacent pixel, it can be obtained that in this case: Δ2y_p3 = -410, Δ2y_p2 = -390, Δ2y_p4 = 410. According to the above formula (8), it can be judged that the target pixel P3 is a bad pixel. Furthermore, since Δ2y_p3 is -410, and -410 is less than 0, the target pixel P3 is a bright spot.

[0164] In the fifth case, if the coordinates and pixel value information of pixels P1, P2, target pixel P3, and pixels P4 and P5 are respectively: p1 (1,500), p2 (2,800), p3 (3,100), p4 (4,520) and p5 (5,500), combined with the calculation method of the second-order difference Δ2y_p3 of the target pixel, the second-order difference Δ2y_p2 of the first adjacent pixel, and the second-order difference Δ2y_p4 of the second adjacent pixel, it can be obtained that in this case: Δ2y_p3 = 1120, Δ2y_p2 = -1000, Δ2y_p4 = -420. According to the above formula (8), it can be judged that the target pixel P3 is a bad pixel. Furthermore, since Δ2y_p3 is 1120, and 1120 is greater than 0, the target pixel P3 is a dark pixel.

[0165] Embodiment 4

[0166] This embodiment provides a method for correcting bad pixels in an image. For example, see Fig.15 , which schematically shows the overall flow chart of the image bad pixel correction method provided by this embodiment. Fig.15 It can be seen that the image bad pixel correction method provided in this embodiment includes:

[0167] SB1: Obtain the quad-Bayer image to be corrected acquired by the image sensor;

[0168] SB2: performing bad pixel detection on the four-Bayer image to be corrected by using the image bad pixel detection method described in any implementation manner of the above embodiments;

[0169] SB3: Correct the detected bad pixels to obtain a four-Bayer image after bad pixel correction.

[0170] Since the image bad pixel correction method provided by the present invention and the image bad pixel detection method provided by the present invention belong to the same inventive concept, the image bad pixel correction method provided by the present invention at least has all the advantages of the image bad pixel detection method provided by the present invention. For details on the beneficial effects of the image bad pixel correction method provided by the present invention, please refer to the above description of the beneficial effects of the image bad pixel detection method provided by the present invention, and no further details will be given here.

[0171] It should be noted that those skilled in the art should be able to understand that the present invention does not impose any limitation on the specific method of bad pixel correction. For example, correction methods including but not limited to neighborhood averaging and bilateral filtering can be used to correct pixels detected as bad pixels. For more details on bad pixel correction, please refer to the technical adaptability understanding related to image bad pixel correction known in the art, which will not be elaborated in this article.

[0172] In summary, compared with the prior art, the image bad pixel detection method, correction method, image signal processor and imaging device provided by the present invention have the following advantages:

[0173] (1) The image bad pixel detection method provided by the present invention receives a four-Bayer image to be detected and determines at least two candidate texture directions, thereby laying a solid foundation for obtaining the accurate texture direction of a target pixel; and for each candidate texture direction of each target pixel, a neighborhood range corresponding to the candidate texture direction is determined by taking the target pixel as the center of gravity of gradient calculation, and a gradient value in the candidate texture direction is calculated according to the pixels in the neighborhood range; wherein the starting point pixel and the ending point pixel corresponding to each other in the neighborhood range are in-phase pixels. It can be seen that the present invention can effectively reduce the influence of signal imbalance of each channel of the four-Bayer image by taking the target pixel as the center of gravity in each candidate texture direction and using the in-phase gradient to calculate the gradient value to determine the detection texture direction of the target pixel, thereby improving the texture direction detection accuracy of the pixel point and the image bad pixel detection accuracy, thereby laying a good foundation for improving the final image quality; and there is no need to increase any hardware cost and the method has good applicability.

[0174] (2) In the image bad pixel detection method provided by the present invention, the gradient value of the target pixel on each candidate texture direction is obtained by weighting according to a pre-acquired gradient weight filter, which can further improve the calculation accuracy of the gradient value of the target pixel in each candidate texture direction, thereby improving the detection accuracy of the texture direction of the target pixel.

[0175] (3) The image bad pixel detection method provided by the present invention can effectively reduce the dependency when the adjacent pixels are bad pixels by using the second-order difference method to calculate the target pixel second-order difference of the target pixel in the detection texture direction, as well as the first adjacent pixel second-order difference and the second adjacent pixel second-order difference of two pixels of the same color adjacent to the target pixel, thereby improving the bad pixel detection accuracy. Furthermore, the use of the second-order difference method can also well ensure the detection accuracy while taking into account the detection efficiency.

[0176] It should be noted that the devices and methods disclosed in the embodiments of this article can also be implemented in other ways. The device implementation described above is only schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of this article. In this regard, each box in the flowchart or block diagram can represent a part of a module, program or code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical function, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented by a dedicated hardware-based system for performing a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0177] In addition, the functional modules in the various embodiments of this document may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0178] The above description is only a description of the preferred implementation of the image bad pixel detection method, correction method, image signal processor and imaging device provided by the present invention, and is not intended to limit the scope of the present invention. Any changes and modifications made by ordinary technicians in the field of the present invention based on the above disclosure are within the protection scope of the present invention. Obviously, technicians in this field can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. In this way, if these modifications and variations fall within the scope of the present invention and its equivalent technology, the present invention is also intended to include these modifications and variations.

Claims

1. A method for detecting bad pixels in an image, characterized in that: include: receiving a quad-Bayer image to be detected and determining at least two candidate texture directions; The following method is used to perform bad pixel detection on each target pixel in the four-Bayer image to be detected: For each candidate texture direction of each target pixel: determine the neighborhood range corresponding to the candidate texture direction by taking the target pixel as the center of gravity of the gradient calculation, and calculate the gradient value of the candidate texture direction according to the pixels in the neighborhood range; wherein the starting point pixel and the ending point pixel corresponding to each other in the neighborhood range are in-phase pixels; Taking the candidate texture direction with the smallest gradient value as the detected texture direction of the target pixel; Whether the target pixel is a bad pixel is detected according to the detected texture direction.

2. The image bad pixel detection method according to claim 1, characterized in that: For each of the candidate texture directions, the corresponding starting point pixels and ending point pixels in the neighborhood range are in-phase pixels, including: Two pixel points whose connecting line is parallel to the candidate texture direction, located at the edge of the same neighborhood range and each having the same position in the corresponding 2×2 pixels of the same color are the corresponding starting point pixel and the ending point pixel.

3. The image bad pixel detection method according to claim 1, characterized in that: For each of the candidate texture directions of each of the target pixels, calculating the gradient value of the candidate texture direction according to the pixels within the neighborhood, comprises: For each of the candidate texture directions of each of the target pixels, calculating an initial gradient value in the candidate texture direction according to the pixels within the neighborhood; A weighted pixel set is determined based on a pre-acquired gradient weight filter with the target pixel as the center, and the initial gradient value of each pixel in the weighted pixel set corresponding to the target pixel in the candidate texture direction is weighted based on the gradient weight filter to obtain the gradient value of the target pixel in the candidate texture direction.

4. The image bad pixel detection method according to claim 1, characterized in that: The detecting whether the target pixel is a bad pixel according to the detected texture direction includes: According to the detected texture direction, a concavity and convexity detection method is used to detect whether the target pixel is a bad pixel.

5. The image bad pixel detection method according to claim 4, characterized in that: The step of detecting whether the target pixel is a bad pixel by using a concave-convex detection method according to the detected texture direction includes: In the direction of the detected texture, a second-order difference method is used to calculate a second-order difference of the target pixel, and a first adjacent pixel second-order difference and a second adjacent pixel second-order difference of two pixels of the same color adjacent to the target pixel; If the absolute value of the second-order difference of the target pixel is greater than a first preset value, the absolute value of the second-order difference of the first adjacent pixel is greater than a second preset value, and the absolute value of the second-order difference of the second adjacent pixel is greater than a third preset value, the target pixel is a bad pixel.

6. The image bad pixel detection method according to claim 5, characterized in that: In the direction of the detected texture, a second-order difference method is used to calculate the second-order difference of the target pixel of the target pixel, including: In the detected texture direction, determining a first distance and a second distance between the target pixel and two adjacent pixels of the same color; A first first-order difference is calculated according to the first distance, the pixel value of the target pixel, and the pixel value of the first same-color pixel, and a second first-order difference is calculated according to the second distance, the pixel value of the target pixel, and the pixel value of the second same-color pixel; wherein the first same-color pixel is the adjacent pixel corresponding to the calculation of the first distance, and the second same-color pixel is the adjacent pixel corresponding to the calculation of the second distance; A target pixel second-order difference of the target pixel is calculated according to the first first-order difference, the second first-order difference, the first distance, and the second distance.

7. The image bad pixel detection method according to claim 5, characterized in that: After determining that the target pixel is a bad pixel, the image bad pixel detection method further includes: If the target pixel second-order difference of the target pixel is less than zero, the target pixel is determined to be a bright spot; If the target pixel second-order difference of the target pixel is greater than zero, the target pixel is determined to be a dark point.

8. A method for correcting bad pixels in an image, characterized in that: include: Acquire a quad-Bayer image to be corrected acquired by an image sensor; Using the image bad pixel detection method according to any one of claims 1 to 7 to perform bad pixel detection on the four-Bayer image to be corrected; The detected bad pixels are corrected to obtain a four-Bayer image after bad pixel correction.

9. An image signal processor, characterized in that: The invention comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the image bad pixel detection method according to any one of claims 1 to 7 and / or the image bad pixel correction method according to claim 8 are implemented.

10. An imaging device, characterized in that: Comprising the image signal processor as claimed in claim 9.

Images