Image bad point correction method and device, electronic equipment and storage medium

Abstract

This disclosure provides a method, apparatus, electronic device, and storage medium for correcting image defects. The method includes: for any pixel in the image data to be processed, determining a target data window with a preset scale centered on that pixel; determining whether the center pixel meets preset conditions based on the target data window, the preset conditions including a first condition for the pixel value of the center pixel to be satisfied and / or a second condition for the pixel values ​​of the same channel type in at least one direction corresponding to the center pixel to be satisfied; if the center pixel meets the preset conditions, determining the center pixel as a defective pixel; determining a corrected target pixel value for the center pixel based on the pixel values ​​of other pixels in the target data window; and determining corrected target image data corresponding to the image data to be processed based on the target pixel value. This method effectively balances defective pixel recall and high-contrast edge preservation, thereby improving the accuracy of defective pixel correction.

Description

Technical Field

[0001] This disclosure relates to image signal processing technology, and in particular to a method, apparatus, electronic device, and storage medium for correcting image defects. Background Technology

[0002] With the increasing application of Image Signal Processing (ISP) across various industries, higher demands are being placed on image quality. However, image data acquired by image sensors often contains dead pixels, affecting image quality. Dead pixels refer to pixels with incorrect pixel information in an image due to manufacturing defects or errors in the signal conversion process. There are generally two types of dead pixels: static dead pixels and dynamic dead pixels. Static dead pixels do not change location and are relatively easy to correct; however, dynamic dead pixels are located randomly in the image, requiring their location to be determined before correction, making dynamic dead pixel correction more difficult. Related techniques typically determine dead pixels by comparing the difference between a pixel and its neighboring pixels with a threshold. However, this method cannot effectively balance dead pixel recall and edge preservation at high-contrast edges, resulting in inaccurate dead pixel correction. Summary of the Invention

[0003] To address the aforementioned technical problems such as inaccurate image defect correction, this disclosure is proposed. Embodiments of this disclosure provide a method, apparatus, electronic device, and storage medium for correcting image defects.

[0004] According to one aspect of the present disclosure, a method for correcting image defects is provided, comprising: acquiring image data to be processed; for any pixel in the image data to be processed, determining a target data window with a preset scale centered on the pixel based on the image data to be processed; determining, based on the target data window, whether the center pixel satisfies the preset conditions, the preset conditions including a first condition and / or a second condition, wherein the first condition is a condition satisfied by the pixel value of the center pixel, and the second condition is a condition satisfied by the pixel values ​​of pixels of the same channel type in at least one direction corresponding to the center pixel in the target data window; determining, in response to the center pixel satisfying the preset conditions, that the center pixel is a defective pixel; determining a corrected target pixel value of the center pixel based on the pixel values ​​of other pixels in the target data window besides the center pixel; and determining corrected target image data corresponding to the image data to be processed based on the target pixel value.

[0005] According to another aspect of the present disclosure, an image defect correction apparatus is provided, comprising: a first acquisition module for acquiring image data to be processed; a first processing module for determining, based on the image data to be processed, a target data window with a preset scale centered on any pixel in the image data to be processed; a second processing module for determining, based on the target data window, whether the center pixel satisfies a preset condition, the preset condition including a first condition and / or a second condition, wherein the first condition is a condition satisfied by the pixel value of the center pixel, and the second condition is a condition satisfied by the pixel values ​​of at least one pixel of the same channel type corresponding to the center pixel in the target data window; a third processing module for determining, in response to the center pixel satisfying the preset condition, that the center pixel is a defective pixel; a fourth processing module for determining a corrected target pixel value of the center pixel based on the pixel values ​​of other pixels in the target data window besides the center pixel; and a fifth processing module for determining corrected target image data corresponding to the image data to be processed based on the target pixel value.

[0006] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the image defect correction method described in any of the above embodiments of the present disclosure.

[0007] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the image defect correction method according to any of the above embodiments of the present disclosure.

[0008] Based on the image defect correction method, apparatus, electronic device, and storage medium provided in the above embodiments of this disclosure, by setting a target data window with each pixel in the image data to be processed as the center pixel, and setting a first condition for the pixel value of the center pixel and a second condition for the pixel values ​​in each direction of the target data window for defect recall, it is possible to effectively balance defect recall and high contrast edge preservation, thereby improving the accuracy of defect correction and thus improving image quality.

[0009] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0010] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 This is an exemplary application scenario of the image defect correction method provided in this disclosure;

[0012] Figure 2 This is a schematic flowchart of an image defect correction method provided in an exemplary embodiment of this disclosure;

[0013] Figure 3 This is a schematic diagram of a target data window provided in an exemplary embodiment of this disclosure;

[0014] Figure 4 This is a flowchart illustrating step 203 provided in an exemplary embodiment of this disclosure;

[0015] Figure 5 This is a schematic diagram of 2*2 RGB-IR mode image data provided in an exemplary embodiment of this disclosure;

[0016] Figure 6 This is a flowchart illustrating step 2034 provided in an exemplary embodiment of this disclosure;

[0017] Figure 7 This is a flowchart illustrating step 205 provided in an exemplary embodiment of this disclosure;

[0018] Figure 8 This is a schematic diagram illustrating the principle of determining the maximum and minimum values ​​provided in an exemplary embodiment of this disclosure;

[0019] Figure 9 This is a schematic diagram illustrating the relationship between the total difference in a certain direction and the first range, provided in an exemplary embodiment of this disclosure.

[0020] Figure 10 This is a flowchart illustrating step 2035 provided in an exemplary embodiment of this disclosure;

[0021] Figure 11 This is a flowchart illustrating step 2035 provided in another exemplary embodiment of this disclosure;

[0022] Figure 12 This is a schematic diagram of a target data window with a preset size of 5*5 provided in an exemplary embodiment of this disclosure;

[0023] Figure 13This is a flowchart illustrating step 2033 provided in an exemplary embodiment of this disclosure;

[0024] Figure 14 This is a schematic diagram of a target data window with a B-channel type center pixel in 4*4 RGB-IR mode provided in an exemplary embodiment of this disclosure;

[0025] Figure 15 This is a schematic diagram of a target data window with the center pixel being of the IR channel type in 4*4 RGB-IR mode provided in an exemplary embodiment of this disclosure;

[0026] Figure 16 This is a schematic diagram of a target data window with a preset size of 5*5 and a center channel of IR channel type, provided in an exemplary embodiment of this disclosure;

[0027] Figure 17 This is a schematic diagram of a target data window with a preset size of 5*5 and a center channel of type B, provided in an exemplary embodiment of this disclosure;

[0028] Figure 18 This is a schematic diagram of a target data window with a preset size of 5*5 in the 2*2 RGB-IR mode provided in an exemplary embodiment of this disclosure;

[0029] Figure 19 This is a schematic diagram of a padding method provided by an exemplary embodiment of this disclosure;

[0030] Figure 20 This is a schematic diagram of the structure of an image defect correction device provided in an exemplary embodiment of the present disclosure;

[0031] Figure 21 This is a schematic diagram of the structure of the second processing module 503 provided in an exemplary embodiment of this disclosure;

[0032] Figure 22 This is a schematic diagram of the structure of the third determining unit 5034 provided in an exemplary embodiment of this disclosure;

[0033] Figure 23 This is a schematic diagram of the structure of the fourth processing module 505 provided in an exemplary embodiment of this disclosure;

[0034] Figure 24 This is a schematic diagram of the structure of the fourth determining unit 5035 provided in an exemplary embodiment of this disclosure;

[0035] Figure 25 This is a schematic diagram of the structure of the second determining unit 5033 provided in an exemplary embodiment of this disclosure;

[0036] Figure 26 This is a schematic diagram of the structure of an application embodiment of the electronic device disclosed herein. Detailed Implementation

[0037] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0038] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0039] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0040] It should also be understood that in the embodiments of this disclosure, "a plurality of" can refer to two or more, and "at least one" can refer to one, two or more.

[0041] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0042] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0043] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0044] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0045] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0046] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0047] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0048] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0049] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0050] This disclosure outlines

[0051] In developing this disclosure, the inventors discovered that with the increasing application of Image Signal Processing (ISP) across various industries, higher demands are being placed on image quality. However, image data acquired by image sensors often contains dead pixels, affecting image quality. Dead pixels refer to pixels with incorrect pixel information in an image due to manufacturing defects or errors in the signal conversion process. There are generally two types of dead pixels: static dead pixels and dynamic dead pixels. Static dead pixels do not change position and are relatively easy to correct. Dynamic dead pixels, however, are randomly located in the image, requiring their location to be determined before correction, making dynamic dead pixel correction more difficult. Related technologies typically determine dead pixels by comparing the difference between a pixel and its neighboring pixels with a threshold. However, this method cannot effectively balance dead pixel recall and edge preservation for high-contrast edges, resulting in inaccurate dead pixel correction.

[0052] Exemplary Overview

[0053] Figure 1 This is an exemplary application scenario of the image defect correction method provided in this disclosure.

[0054] In any scenario requiring image processing, such as autonomous driving and medical imaging, the image defect correction method disclosed herein can be used to detect and correct defective pixels in the image data acquired by the image sensor, obtaining corrected target image data and improving image quality. Specifically, preset conditions for defect detection can be set in advance. For any pixel in the image data to be processed, a target data window of a preset size is determined with that pixel as the center pixel. Based on the target data window, it is determined whether the center pixel meets the preset conditions. The preset conditions may include a first condition that the pixel value of the center pixel must meet, and a second condition that the pixel values ​​of pixels of the same channel type in at least one direction corresponding to the center pixel in the target data window must meet. The at least one direction may include at least one of the horizontal, vertical, 45° oblique, and 135° oblique directions. Based on this, defect detection not only considers the conditions that the current center pixel must meet, but also the relationship between pixels of the same channel type in at least one direction of the center pixel. Therefore, for high-contrast edges, it can effectively balance defect recall and high-contrast edge preservation, effectively improving the accuracy of defect correction and thus improving image quality.

[0055] Exemplary methods

[0056] Figure 2 This is a schematic flowchart illustrating an exemplary embodiment of the image defect correction method provided in this disclosure. This embodiment can be applied to electronic devices, such as servers or terminals, specifically, in-vehicle computing platforms for autonomous vehicles, medical image processing equipment, etc. Figure 2 As shown, it includes the following steps:

[0057] Step 201: Obtain the image data to be processed.

[0058] The image data to be processed can be image data acquired in real time by an image sensor, or image data acquired and stored in advance; there is no specific limitation.

[0059] Step 202: For any pixel in the image data to be processed, determine a target data window with a preset scale centered on that pixel based on the image data to be processed.

[0060] The preset scale can be set according to actual needs, such as 9*9, 7*7, 5*5, etc., without any specific limitation. In practical applications, the pixels in the image data to be processed can be traversed, and the corresponding target data window can be determined for each pixel to determine whether the pixel is a bad pixel.

[0061] For example, Figure 3 This is a schematic diagram of a target data window provided in an exemplary embodiment of this disclosure. In this example, the image data to be processed is taken as 4*4 RGB-IR mode image data, with a preset scale of 9*9. RGB represents the color mode, R for Red, G for Green, and B for Blue. IR (Infrared) represents infrared light. The numbers 01-89 indicate the pixel positions in the target data window. The center pixel in this target data window is the pixel at position 45, and the current pixel value is represented as G45. RGB-IR technology uses RGB-IR CFA (Color Filter Arrays) based on the Bayer format, modifying some pixels to IR pixels, while IR pixels only allow infrared light to pass through. Using RGB-IR technology, a single sensor device can simultaneously capture RGB color images and IR images, thus providing both daytime and nighttime visibility capabilities.

[0062] Step 203: Based on the target data window, determine whether the center pixel meets the preset conditions. The preset conditions include a first condition and / or a second condition. The first condition is the condition that the pixel value of the center pixel meets, and the second condition is the condition that the pixel value of at least one pixel of the same channel type in at least one direction corresponding to the center pixel in the target data window meets.

[0063] Here, the first and second conditions are pre-set conditions that must be met when identifying bad pixels. The first condition is the condition that the center pixel of the target data window must meet, and the second condition is the condition that pixels of the same channel type in at least one direction where the center pixel is located must meet. For example... Figure 3 The conditions that need to be met between the G-channel type pixels in the horizontal direction corresponding to the center pixel G45 are as follows: for example, the total difference of pixel values ​​between pixels of the same channel type in this direction needs to be greater than a certain threshold. The specific conditions can be set according to actual needs.

[0064] Step 204: In response to the center pixel meeting the preset conditions, the center pixel is determined to be a bad pixel.

[0065] Among them, if the center pixel meets the preset conditions for bad pixel detection, the center pixel can be determined to be a bad pixel and needs to be corrected.

[0066] Step 205: Based on the pixel values ​​of other pixels in the target data window besides the center pixel, determine the corrected target pixel value of the center pixel.

[0067] Since the center pixel is a defective pixel, it needs to be corrected. The correction needs to be achieved by interpolation with other pixels around the center pixel. The specific settings can be configured according to actual needs.

[0068] For example, the interpolation direction can be determined first based on the pixel values ​​of other pixels besides the center pixel, and then the target pixel value can be obtained by interpolation based on other pixel values ​​of the same channel type in the interpolation direction.

[0069] Step 206: Based on the target pixel value, determine the corrected target image data corresponding to the image data to be processed.

[0070] Specifically, after performing bad pixel correction on a single pixel, the corrected image data corresponding to that pixel can be obtained based on the target pixel value of that pixel. After traversing all pixels in the image data to be processed and completing the bad pixel detection and correction of all pixels, the corrected target image data corresponding to the image data to be processed can be obtained.

[0071] The image defect correction method provided in this embodiment sets a target data window with each pixel in the image data to be processed as the center pixel. Based on the target data window, a first condition for the pixel value of the center pixel and a second condition for the pixel values ​​in each direction of the target data window are set for defect recall. This method can effectively balance defect recall and high-contrast edge preservation, thereby improving the accuracy of defect correction and thus improving image quality.

[0072] Figure 4 This is a flowchart illustrating step 203 provided in an exemplary embodiment of this disclosure.

[0073] In an optional example, step 203 may specifically include the following steps:

[0074] Step 2031: Based on the target data window, determine the target channel type to which the center pixel belongs.

[0075] For a given center pixel, its target channel type can be any of G, R, B, or IR, depending on the specific target data window determined. For example... Figure 3 The target channel type of the center pixel 45 is G, which means that the pixel value at the center pixel in the obtained image data to be processed is the value of the G channel type.

[0076] Step 2032: Based on the target channel type, obtain the pixel values ​​of each first pixel point of the same channel type in the first preset area corresponding to the target channel type from the target data window.

[0077] The first preset area can be set according to actual needs, such as a 5*5 area centered on the center pixel. The first preset area can be the same as the preset scale of the target data window, or the first preset area can be a part of the target data window, without any specific limitation.

[0078] For example, with Figure 3 Taking G45 as an example, the first preset area can be a 5*5 area centered on G45. Pixel values ​​of nine first pixels with the same channel type as G45 are obtained from the first preset area, including G25, G34, G43, G54, G65, G56, G47, G36, and G45.

[0079] It should be noted that the number of first pixels obtained may vary depending on the target channel type. The specific number can be set according to actual needs. For example, when the center pixel is B44, the pixel values ​​of the first pixels of the same channel type can be obtained, including B22, B26, B44, B62, and B66.

[0080] Step 2033: Based on the pixel values ​​of each first pixel, determine whether the center pixel satisfies the first condition.

[0081] The rule for determining whether the center pixel meets the first condition can be set according to actual needs. For example, separate judgment rules can be set for hot spots (bright spots) and cold spots (dark spots).

[0082] For example, taking a hotspot as an example, it can be determined whether the pixel value of the center pixel is the largest among all the first pixels. If it is the largest, and the difference between the pixel value of the center pixel and the pixel value of the first pixel smaller than it is greater than a certain threshold, or the ratio of this difference to the average value of the G-channel type pixels (G_avg) is greater than another threshold, then the center pixel is determined to meet the first condition. The specific settings can be made according to actual needs. Alternatively, it can be determined whether the pixel value of the center pixel is one of the two largest (largest or second largest) among all the first pixels. If the pixel value of the center pixel is the largest or second largest among all the first pixels, and the ratio of the difference between the pixel value of the second largest first pixel and the pixel value of the third largest first pixel to G_avg is greater than a threshold, then the center pixel is determined to meet the first condition. Here, G_avg is the average value of all G-channel type pixels in the first preset area.

[0083] Step 2034: Based on the existing pixel values ​​corresponding to each pixel in the target data window, determine the difference in pixel values ​​and the dynamic threshold between pixels of the same channel type in at least one direction corresponding to the center pixel.

[0084] In this context, the existing pixel values ​​corresponding to each pixel in the target data window refer to the pixel values ​​that already exist in the image data to be processed. See [link to relevant documentation]. Figure 3 In the image data to be processed, each pixel has only one channel type pixel value, such as G45, B44, R42, IR33, etc. The center pixel corresponds to at least one direction, including at least one of the following: the horizontal direction, the vertical direction, the 45° diagonal direction, and the 135° diagonal direction. For example... Figure 3 The dynamic threshold is defined as follows: row (G41-G49 direction), column (G05-G85 direction), 45° diagonal (diagonal direction of G09-G27-G45-G63), and 135° diagonal (diagonal direction of G34-G45-G56). The difference in pixel values ​​is the difference between two adjacent or nearest pixel values, such as G43-G45 or G47-G45 in the horizontal direction. This difference can be the larger value minus the smaller value, or the absolute value of the two, such as ABS(G43-G45). ABS() is an absolute value function that ensures the difference is non-negative. The dynamic threshold can be one or one per direction, depending on the specific requirements. The dynamic threshold can be determined based on the local maxima and minima of pixel values ​​of the same channel type in each direction. This can also be set according to specific needs.

[0085] Step 2035: Based on the difference in pixel values ​​between pixels of the same channel type in at least one direction and a dynamic threshold, determine whether the center pixel satisfies the second condition.

[0086] Specifically, for the case where each direction corresponds to a dynamic threshold, the sum of the differences in pixel values ​​in that direction must be greater than the dynamic threshold for that direction. For the case where all directions correspond to a single dynamic threshold, the minimum sum of the differences in pixel values ​​for each direction must be greater than the dynamic threshold. The specific settings can be configured according to actual needs.

[0087] Step 2036: In response to the center pixel satisfying the first condition and / or the second condition, determine that the center pixel satisfies the preset condition.

[0088] When the center pixel satisfies the first condition or the second condition, or when the center pixel satisfies the first condition and the second condition, it can be determined that the center pixel satisfies the preset condition.

[0089] Optionally, to ensure the accuracy of bad pixel recall, the preset condition can be determined only when the center pixel simultaneously meets the first condition and the second condition.

[0090] This disclosure achieves accurate and effective recall of bad pixels near high-contrast edges by setting a dynamic threshold to determine whether the difference between pixel values ​​in each direction of a pixel meets the second condition, without affecting the overall image clarity.

[0091] In one optional example, the image data to be processed can be either 4x4 RGB-IR mode image data or 2x2 RGB-IR mode image data; the specific type is not limited. For example, Figure 5 This is a schematic diagram of 2*2 RGB-IR mode image data provided in an exemplary embodiment of this disclosure. For image data to be processed in different modes, corresponding first and second conditions can be set according to actual needs for bad pixel detection, such as setting corresponding preset scale, corresponding first preset region, and obtaining a corresponding number of first pixels for different target channel types, etc., which will not be elaborated further.

[0092] Figure 6 This is a flowchart illustrating step 2034 provided in an exemplary embodiment of this disclosure.

[0093] In an optional example, the dynamic threshold includes a first threshold corresponding to each direction in at least one direction; step 2034, based on the existing pixel values ​​corresponding to each pixel in the target data window, determines the difference in pixel values ​​between pixels of the same channel type corresponding to the center pixel in at least one direction, and the dynamic threshold, including:

[0094] Step 20341: Based on the existing pixel values ​​corresponding to each pixel in the target data window, determine the second pixel value of the preset backup pixel corresponding to the target channel type.

[0095] The preset backup pixels are set based on the target channel type of the center pixel in the target data window and are used for subsequent processing. Since each pixel in the target data window has only one channel type pixel value, subsequent processing requires pixel values ​​of other channel types from some pixels. Therefore, it is necessary to determine the second pixel value of these preset backup pixels based on the existing pixel values ​​in the target data window. The second pixel value is the pixel value of another channel type for its corresponding pixel, excluding the existing pixel value. For example, for a pixel currently of channel type B, the second pixel value of channel type G for that pixel is determined based on the existing pixel values ​​around it. The backup pixels required for different center pixels can be the same or different, depending on the actual needs.

[0096] For example, still taking G45 as the center pixel, the second pixel value corresponding to the preset backup pixel includes G42, G48, G15, G75, G44, G46, G35, and G55. For instance, G42 represents the G channel type pixel value of the pixel at position 42 in the target data window. It can be seen that the pixel at position 42 in the target data window already has a pixel value of R42, so G42 needs to be calculated. The specific calculation method can be obtained through interpolation, such as interpolation based on direction. Specifically, the interpolation direction can be determined first based on the existing G channel type pixel values ​​around pixel 42 (such as G41, G43, G32, and G52), and then G42 can be obtained by interpolation using the G pixel values ​​in that direction, as shown below:

[0097] Diff_horizon=ABS(G41–G43).

[0098] Diff_vertical=ABS(G32–G52).

[0099] Where horizontal represents the horizontal direction and vertical represents the vertical direction, if ABS(Diff_horizon–Diff_vertical) (ABS is an absolute value function) is less than a certain threshold (this threshold can be set according to actual needs), then point G42 is considered to have no obvious direction, and G42 = (G41+G43+G32+G52) / 4; otherwise, if Diff_horizon is greater than Diff_vertical, then G42 = (G32+G52) / 2, and if Diff_horizon is less than Diff_vertical, then G42 = (G41+G43) / 2.

[0100] In an optional example, for the second pixel values ​​of preset backup pixels adjacent to the center pixel, such as G44, G46, G35, and G55, since the interpolation operation above requires G45, and it is uncertain whether G45 is a bad pixel, to further improve accuracy, the calculation of the second pixel values ​​of these preset backup pixels can be specially considered. For example, the second pixel values ​​of these preset backup pixels can be determined by setting a pre-defined confidence rule. The confidence level can be represented by 0 and 1. For example, when the confidence level of G44 obtained by interpolation is 0, it means that it cannot participate in subsequent calculations. If the confidence level is 1, it can be used in subsequent calculations. The pre-defined confidence rule can be set according to actual needs, and this embodiment does not limit it.

[0101] Step 20342: Based on the existing pixel values ​​and the second pixel values ​​of the preset backup pixel points, determine the difference in pixel values ​​between pixels of the same channel type in at least one direction corresponding to the center pixel point, and the first threshold corresponding to each direction in at least one direction.

[0102] After determining the second pixel value of the preset backup pixel, the difference in pixel values ​​between pixels of the same channel type in at least one direction corresponding to the center pixel, and the dynamic threshold (i.e., the first threshold) corresponding to each direction can be determined by combining the existing pixel values ​​and the preset backup pixel value. Since the second pixel value of the preset backup pixel is determined, the difference in pixel values ​​between pixels of the same channel type in one direction can be determined using the nearest pixel value of the center pixel. For the first threshold in any direction, the local maxima and local minima can be determined by combining the existing pixels and the second pixel of the same type as the center pixel in that direction, and then the first threshold in that direction can be determined based on the local maxima and local minima.

[0103] For example, the difference in pixel values ​​in the horizontal direction of G45 includes ABS(G43-G45) and ABS(G47-G45), or the difference in pixel values ​​in the horizontal direction of G45 may include ABS(G44-G45) and ABS(G46-G45).

[0104] Optionally, to further improve accuracy, when using a pre-set confidence rule to calculate the second pixel value of a preset backup pixel, if the confidence of G44 is 0, or the confidence of G46 is 0, then the difference in pixel values ​​in the horizontal direction of G45 includes ABS(G43-G45) and ABS(G47-G45), instead of using G44 and G46 which have lower confidence. Otherwise, the difference in pixel values ​​in the horizontal direction of G45 may include ABS(G44-G45) and ABS(G46-G45).

[0105] This disclosure improves the accuracy of bad pixel detection by determining the second pixel value of a preset backup pixel point and using the nearest pixel value of the same channel type when determining the difference in pixel values. Furthermore, it further improves the accuracy of dynamic thresholding by using more continuous pixel values ​​of the same channel type when determining the dynamic threshold.

[0106] In one optional example, based on the existing pixel values ​​corresponding to each pixel in the target data window, the second pixel value of the preset backup pixel corresponding to the target channel type is determined, including:

[0107] A1. In response to the target channel type being type 1, type 2, or type 3, based on the existing pixel values, the second pixel value of the first part of the backup pixels is determined by interpolation operation based on direction judgment, and the second pixel value of the second part of the backup pixels is determined based on a preset confidence rule; the second pixel value of the first part of the backup pixels and the second pixel value of the second part of the backup pixels are used as the second pixel value of the preset backup pixels corresponding to the target channel type.

[0108] Among them, the first type is the G channel type, the second type is the B channel type, and the third type is the R channel type. Taking the first type G as an example, referring to the center pixel point G45 above, the second pixel values ​​corresponding to the preset spare pixels include G42, G48, G15, G75, G44, G46, G35, and G55. For the second pixel values ​​of the first part of spare pixels, G42, G48, G15, and G75, interpolation based on direction judgment can be used to determine them; while for the second part of spare pixels, G44, G46, G35, and G55... The second pixel value, located next to the center pixel, requires G45 for the interpolation operation described above. However, it's uncertain whether the current G45 is a dead pixel. Therefore, the reliability of the second pixel value determined by direction-based interpolation is also uncertain. To further improve accuracy, the second pixel value of the second set of backup pixels can be determined based on a pre-set confidence rule. Confidence can be represented by 0 and 1. For example, if the confidence of the interpolated G44 is 0, it cannot be used in subsequent calculations; if the confidence is 1, it can be used. The confidence of the second pixel value obtained from the interpolation operation can be determined based on the interpolation direction and whether the center pixel is needed for the interpolation operation in that direction. If the center pixel is needed for the interpolation operation in that direction, the interpolation result is unreliable and should be avoided in subsequent calculations. The interpolation direction can be determined based on the difference between the pixel values ​​in the horizontal and vertical directions.

[0109] A2. In response to the target channel type being the fourth type, based on the existing pixel values, interpolation operation with direction determination is used to determine the second pixel value of the preset backup pixel point corresponding to the target channel type.

[0110] The fourth type is the IR channel type. The principle of interpolation based on direction determination is explained above and will not be repeated here.

[0111] This disclosure improves the accuracy of bad pixel detection by performing a confidence-based judgment on the interpolation results of the center pixels that need to be used when calculating preset backup pixels.

[0112] In an optional example, determining the second pixel value of the second part of the backup pixels based on a preset confidence rule in A1 includes: determining the second pixel value of the second part of the backup pixels and the confidence level of the second part of the backup pixels based on the preset confidence rule; correspondingly, determining the difference in pixel values ​​between pixels of the same channel type in at least one direction corresponding to the center pixel based on the existing pixel value and the second pixel value of the preset backup pixels includes: in response to the target channel type being a first type, a second type, or a third type, determining the difference in pixel values ​​between pixels of the same channel type in at least one direction corresponding to the center pixel based on the existing pixel value, the second pixel value of the preset backup pixels, and the confidence level of the second part of the backup pixels.

[0113] Taking the calculation of G44 above as an example, it is shown as follows:

[0114] Diff_horizon=ABS(G43–G45).

[0115] Diff_vertical=ABS(G34–G54).

[0116] Let G44_w represent the credibility calculated from the value of G44. A G44_w value of 0 indicates that the value is not credible and will be avoided in subsequent calculations. A G44_w value of 1 indicates that the value is credible and will be used in subsequent calculations. If ABS(Diff_horizon–Diff_vertical) is less than the target threshold, then pixel G44 is determined to have no obvious direction, so G44_w = 0 and G44 = 0. Otherwise, if Diff_horizon is greater than Diff_vertical, the interpolation direction should be vertical. Interpolation in this vertical direction uses G34 and G54, which does not involve the center pixel G45, so it is relatively reliable. The confidence level can be determined as G44_w = 1 and G44 = (G34 + G54) / 2. If Diff_horizon is less than Diff_vertical, the interpolation direction should be horizontal. Interpolation in the horizontal direction requires G43 and G45. Since G45 is the center pixel, whether it is a bad pixel needs to be determined, and the interpolation result is unreliable. Therefore, the confidence level can be determined as G44_w = 0 and G44 = 0.

[0117] Accordingly, when determining the difference in pixel values ​​between pixels of the same channel type in at least one direction corresponding to the center pixel, when the second pixel value of the second part of the spare pixels mentioned above needs to be used, the confidence level of the second pixel value needs to be considered. When the confidence level of the second pixel value is unreliable (e.g., 0), the second pixel value should be avoided and other possible pixel values ​​should be used instead, such as using the pixel value adjacent to the second pixel value. The specific settings can be set according to actual needs.

[0118] For example, taking the aforementioned center pixel G45 as an example, the differences in pixel values ​​between this center pixel in the horizontal direction, vertical direction, 45° diagonal direction, and 135° diagonal direction are represented as follows:

[0119] Horizontal direction: If G44_w = 0 or G46_w = 0, then the difference in pixel value between pixels of type G in this direction is ABS(G43-G45) and ABS(G47-G45); if G44_w = 1 and G46_w = 1, then the difference in pixel value between pixels of type G in this direction is ABS(G44-G45) and ABS(G46-G45).

[0120] Vertical direction: If G35_w = 0 or G55_w = 0, the difference in pixel value between pixels of type G in this direction is ABS(G25-G45) and ABS(G65-G45); if G35_w = 1 and G55_w = 1, the difference in pixel value between pixels of type G in this direction is ABS(G35-G45) and ABS(G55-G45).

[0121] 45° angled: The difference in pixel value between G channel type pixels in this direction is ABS(G36-G45) and ABS(G54-G45).

[0122] 135° angled: The difference in pixel value between G channel type pixels in this direction is ABS(G34-G45) and ABS(G56-G45).

[0123] This disclosure further improves the accuracy of bad pixel detection by setting confidence levels using a second set of spare pixels.

[0124] In one optional example, Figure 7 This is a flowchart illustrating step 205 of an exemplary embodiment of the present disclosure. In this example, step 205, based on the pixel values ​​of other pixels in the target data window excluding the center pixel, determines the corrected target pixel value of the center pixel, including:

[0125] Step 2051: Based on the pixel values ​​of other pixels in the target data window that have the same channel type as the center pixel, determine the sum of the differences corresponding to each direction of the target data window.

[0126] Each direction can include at least two of the following directions from the target data window: horizontal, vertical, 45° diagonal, and 135°. The sum of differences in a direction can include the sum of pixel value differences along at least one line in that direction (e.g., at least two rows in the horizontal direction or at least two columns in the vertical direction). To ensure accuracy, the sum of differences in each direction can be determined based on other pixels in the same channel that are closer to the center pixel.

[0127] For example, taking the center pixel G45 as an example, the sum of the differences in the horizontal, vertical, 45°, and 135° directions is calculated as follows:

[0128] Main_diff_h=ABS(G34-G36)+ABS(G43-G47)+ABS(G54-G56).

[0129] Main_diff_v=ABS(G34-G54)+ABS(G25-G65)+ABS(G36-G56).

[0130] Main_diff_45=ABS(G25-G43)+ABS(G36-G54)+ABS(G47-G65).

[0131] Main_diff_135=ABS(G25–G47)+ABS(G34-G56)+ABS(G43-G65).

[0132] Where h represents the horizontal direction, v represents the vertical direction, 45 represents the 45° direction, and 135 represents the 135° direction.

[0133] Step 2052: Determine the minimum sum of differences based on the sum of all differences.

[0134] For example, taking the center pixel G45 as an example, the sum of the minimum differences is expressed as follows:

[0135] Main_diff_min=min(Main_diff_h, Main_diff_v, Main_diff_45, Main_diff_135).

[0136] Step 2053: Determine the interpolation direction corresponding to the center pixel based on the minimum sum of differences.

[0137] For example, the direction of Main_diff_min is determined as the interpolation direction corresponding to the center pixel.

[0138] Step 2054: Determine the target pixel value based on the interpolation direction.

[0139] For example, taking the center pixel G45 mentioned above as an example, the target pixel value is represented as follows:

[0140] If Main_diff_min equals Main_diff_h, then the corrected target pixel value correct_pixel = (G43 + G47) / 2.

[0141] If Main_diff_min equals Main_diff_v, then correct_pixel = (G25 + G65) / 2.

[0142] If Main_diff_min equals Main_diff_45, then correct_pixel = (G36 + G54) / 2.

[0143] If Main_diff_min equals Main_diff_135, then correct_pixel = (G34 + G56) / 2.

[0144] In correcting bad pixels, this disclosure determines the sum of differences in each direction based on the pixel values ​​of other pixels in the same channel surrounding the center pixel. The direction with the smallest sum of differences is used as the interpolation direction to determine the target pixel value of the center pixel after correction, thereby further improving the accuracy of the correction result.

[0145] In an optional example, step 2054, determining the target pixel value based on the interpolation direction, includes: in response to the target channel type being a second type or a third type, determining the target pixel value based on the interpolation direction, other pixel values ​​of the same channel type in the interpolation direction, a second pixel value of the same channel type in the interpolation direction, and the confidence level corresponding to the second pixel value; and in response to the target channel type being a first type or a fourth type, determining the target pixel value based on the interpolation direction and other pixel values ​​of the same channel type in the interpolation direction.

[0146] The first type is the G-channel type, the second type is the B-channel type, the third type is the R-channel type, and the fourth type is the IR-channel type. (See also...) Figure 3 In the image data to be processed, there are relatively few pixels of B-channel and R-channel types. To further improve accuracy, the determination of the target pixel value can be achieved by combining preset spare pixels.

[0147] For example, taking pixel B44 as the center, it is represented as follows:

[0148] If Main_diff_min equals Main_diff_h, then correct_pixel = (B42 + B46) / 2 (when the default confidence level of the spare pixel is B42_w = 1 and B46_w = 1), or correct_pixel = (B22 + B26 + B62 + B66) / 4 (when the default confidence level of the spare pixel is B42_w = 0 or B46_w = 0).

[0149] If Main_diff_min equals Main_diff_v, then correct_pixel = (B24 + B64) / 2 (when B24_w = 1 and B64_w = 1), or correct_pixel = (B22 + B26 + B62 + B66) / 4 (when B24_w = 0 or B64_w = 0).

[0150] If Main_diff_min equals Main_diff_45, then correct_pixel = (B26 + B62) / 2.

[0151] If Main_diff_min equals Main_diff_135, then correct_pixel = (B22 + B66) / 2.

[0152] In an optional example, step 20342, which determines the first threshold corresponding to each direction in at least one direction corresponding to the center pixel based on the existing pixel value and the second pixel value of the preset spare pixel, includes:

[0153] B1. Based on the existing pixel values ​​and the second pixel values ​​of the preset backup pixel points, determine the first range corresponding to each direction in at least one direction corresponding to the center pixel point. The first range is the difference between the maximum and minimum values ​​of the pixel values ​​in the corresponding direction whose channel type is the first type. B2. Based on the first range corresponding to each direction, determine the first threshold corresponding to each direction.

[0154] The first range characterizes whether there are high-contrast regions within the target data window. A larger first range indicates higher contrast in that direction, and a smaller first range indicates lower contrast in that direction. The dynamic thresholds (i.e., the first thresholds) for each direction, determined based on the first range, can be implemented using preset rules, which can be set according to actual needs. Optionally, when preset backup pixels are needed, the availability of the second pixel value can be determined based on the confidence level of the second part of the backup pixels to further improve accuracy.

[0155] For example, taking the aforementioned center pixel G45 as an example, the first range corresponding to its horizontal, vertical, 45°, and 135° directions is calculated as follows:

[0156] If the confidence level G44_w = 1 and G46_w = 1, then GrangeH = GetGRange(G41,G42,G43,G44,G45,G46,G47,G48,G49).

[0157] If G44_w = 1 and G46_w = 0, then GrangeH = GetGRange(G41, G42, G43, G44, G45, G47, G47, G48, G49).

[0158] If G44_w = 0 and G46_w = 1, then GrangeH = GetGRange(G41,G42,G43,G43,G45,G46,G47,G48,G49).

[0159] If G44_w = 0 and G46_w = 0, then GrangeH = GetGRange(G41,G42,G43,G43,G45,G47,G47,G48,G49).

[0160] If G35_w = 1 and G55_w = 1, then GrangeV = GetGRange(G05,G15,G25,G35,G45,G55,G65,G75,G85).

[0161] If G35_w = 1 and G55_w = 0, then GrangeV = GetGRange(G05,G15,G25,G35,G45,G65,G65,G75,G85).

[0162] If G35_w = 0 and G55_w = 1, then GrangeV = GetGRange(G05,G15,G25,G25,G45,G55,G65,G75,G85).

[0163] If G35_w = 0 and G55_w = 0, then GrangeV = GetGRange(G05,G15,G25,G25,G45,G65,G65,G75,G85).

[0164] Grange45=GetGRange(G81,G72,G63,G54,G45,G36,G27,G18,G09).

[0165] Grange135=GetGRange(G01,G12,G23,G34,G45,G56,G67,G78,G89).

[0166] Where GrangeH represents the first range in the horizontal direction, GrangeV represents the first range in the vertical direction, Grange45 represents the first range in the 45° direction, Grange135 represents the first range in the 135° direction, and GetGRange() represents the range function. Taking GetGRange(G01,G12,G23,G34,G45,G56,G67,G78,G89) as an example, it means to find the difference between the maximum and minimum values ​​among G01,G12,G23,G34,G45,G56,G67,G78,G89. Specifically, the input is a series of G values ​​(e.g., G01, G12, G23, G34, G45, G56, G67, G78, G89) passing through the center pixel. These G values ​​are connected in the image data to be processed. Starting from the pixel value of the center pixel, the size of the adjacent values ​​is recursively judged to find the local maximum and local minimum values, which are used as the maximum and minimum values, respectively.

[0167] For example, Figure 8 This is a schematic diagram illustrating the principle of determining the maximum and minimum values ​​provided in an exemplary embodiment of this disclosure. Specifically, for case 1, the size of the adjacent pixels on both sides of the center pixel is calculated relative to the center pixel. When case 1 is satisfied, and both adjacent pixels are smaller than the center pixel, the larger adjacent pixel is taken as the maximum value point, and the smallest point on the other side is taken as the minimum value point. For case 2, when both adjacent pixels on both sides are larger than the center pixel, the smaller adjacent pixel is taken as the minimum value point, and the largest point on the other side is taken as the maximum value point. For cases 3 and 4, a maximum value and a minimum value are taken from both sides respectively as the maximum and minimum values. The specific principles are not elaborated further. The black dots in the diagram represent the determined maximum and minimum values, and the corresponding pixel values ​​are the maximum and minimum values, respectively.

[0168] Optionally, the determination of the first threshold can be based on the pre-obtained threshold range boundary and range range boundary.

[0169] For example, the first threshold Cur_thr is represented as follows:

[0170] G_ratio=(Grange–range_l) / (range_h–range_l).

[0171] Cur_thr=Min(Max(diff_low,(diff_high–diff_low)*G_ratio+diff_low),diff_high).

[0172] Where range_l and range_h represent the lower and upper bounds of the pre-obtained threshold range, respectively, and diff_low and diff_high represent the lower and upper bounds of the pre-obtained total difference range in the direction, respectively. These can be obtained in advance through debugging. Grange represents the first range in a certain direction. Figure 9 This is a schematic diagram illustrating the relationship between the total difference in a certain direction and the first range, provided by an exemplary embodiment of this disclosure. Here, Diff_G represents the total difference in a certain direction for channel G, Grange represents the first range, the coordinates of the lower left corner of the line are (range_l, diff_low), and the coordinates of the upper right corner are (range_h, diff_high). The area above the line represents the bad pixel region, and the area below the line represents the non-bad pixel region. The line is represented by the formula described above.

[0173] Based on the above rules, the first threshold corresponding to each direction can be determined, which are respectively represented as: Cur_thr_h, Cur_thr_v, Cur_thr_45, and Cur_thr_135, which will not be elaborated on in detail.

[0174] This disclosure uses the range (first range) to reflect whether there are high-contrast areas in the target data window. Based on the range, the threshold of the second condition is dynamically determined, thereby ensuring the detection intensity of bad pixels in low-contrast areas while taking into account the recall and sharpness preservation of bad pixels in high-contrast areas. This achieves adaptive detection intensity of bad pixels in low-contrast and high-contrast areas, further effectively improving the accuracy of bad pixel recall.

[0175] In one optional example, Figure 10 This is a flowchart illustrating step 2035 of an exemplary embodiment of the present disclosure. In this example, step 2035, determining whether the center pixel satisfies the second condition based on the difference in pixel values ​​between pixels of the same channel type in at least one direction and a dynamic threshold, includes:

[0176] Step 20351a: Determine the total difference value for each direction based on the difference of the pixel values ​​corresponding to each direction.

[0177] Each direction can include the difference between at least two pixel values. For example, the difference between pixel values ​​of G channel type pixels in the 45° diagonal direction mentioned above includes two values: ABS(G36-G45) and ABS(G54-G45). The total difference is the sum of the differences between these at least two pixel values.

[0178] For example, taking the center pixel of G45 as an example, the total difference value corresponding to each direction is expressed as follows:

[0179] If the confidence level G44_w = 0 or G46_w = 0, Diff_h = ABS(G43-G45) + ABS(G47-G45).

[0180] If the confidence level G44_w = 1 and G46_w = 1, then Diff_h = ABS(G44-G45) + ABS(G46-G45).

[0181] If the confidence level G35_w = 0 or G55_w = 0, Diff_v = ABS(G25-G45) + ABS(G65-G45).

[0182] If the confidence level G35_w = 1 and G55_w = 1, then Diff_v = ABS(G35-G45) + ABS(G55-G45).

[0183] Diff_45=ABS(G36-G45)+ABS(G54-G45).

[0184] Diff_135=ABS(G34-G45)+ABS(G56-G45).

[0185] Step 20352a: In response to the fact that the total difference value corresponding to each direction is greater than the first threshold value of the corresponding direction, it is determined that the center pixel satisfies the second condition.

[0186] Each direction corresponds to a first threshold, which is dynamically determined. Therefore, the first thresholds for different directions may be the same or different. When the total difference for each direction is greater than its first threshold (Diff_h>Cur_thr_h, and Diff_v>Cur_thr_v, and Diff_45>Cur_thr_45, and Diff_135>Cur_thr_135), the center pixel is determined to satisfy the second condition.

[0187] In one optional example, Figure 11 This is a flowchart illustrating step 2035 provided in another exemplary embodiment of this disclosure. In this example, step 2035 determines whether the center pixel satisfies the second condition based on the difference in pixel values ​​between pixels of the same channel type in at least one direction and a dynamic threshold, including:

[0188] Step 20351b: Determine the total difference value for each direction based on the difference of the pixel values ​​corresponding to each direction.

[0189] In an optional example, this example shows the case where the center pixel corresponds to a dynamic threshold. The total difference for each direction does not require a preset backup pixel. The difference in pixel values ​​and the dynamic threshold between pixels of the same channel type in each direction corresponding to the center pixel are determined only based on the existing pixel values ​​in the target data window. Then, the total difference for each direction is determined based on the difference in pixel values ​​for each direction.

[0190] For example, Figure 12 This is a schematic diagram of a target data window with a preset size of 5*5 provided in an exemplary embodiment of this disclosure. In this example, the center pixel is the pixel at position 23, with a pixel value of G23, and the total difference value corresponding to each direction is expressed as follows:

[0191] Diff_h=ABS(G25-G23)+ABS(G21-G23).

[0192] Diff_v=ABS(G03-G23)+ABS(G43-G23).

[0193] Diff_45=ABS(G14-G23)+ABS(G32-G23).

[0194] Diff_135=ABS(G12-G23)+ABS(G34-G23).

[0195] In another optional example, the specific operation of step 20351b can also refer to step 20351a above. The specific settings can be made according to actual needs, and will not be repeated here.

[0196] Step 20352b: Determine the minimum total difference based on the total difference values ​​corresponding to each direction.

[0197] Each direction has a corresponding total difference value. The minimum total difference value is the smallest total difference value among all directions. The minimum total difference value represents the difference between the center pixel and the surrounding pixels in the same channel. The larger the minimum total difference value, the greater the difference. The smaller the minimum total difference value, the smaller the difference.

[0198] For example, the minimum total difference is represented as follows:

[0199] Min_Diff=MIN(Diff_135,Diff_45,Diff_h,Diff_v).

[0200] Step 20353b: In response to the minimum total difference being greater than the dynamic threshold, determine that the center pixel satisfies the second condition.

[0201] In this example, for a center pixel, a dynamic threshold is determined. The center pixel is determined to satisfy the second condition if the minimum total difference in all directions is greater than the dynamic threshold.

[0202] In an optional example, step 2034, which determines the dynamic threshold based on the existing pixel values ​​corresponding to each pixel in the target data window, includes: determining the second range based on the existing pixel values ​​of the first channel type in the target data window, wherein the second range is the difference between the maximum and minimum values ​​among the existing pixel values ​​of the first type; and determining the dynamic threshold based on the second range.

[0203] The first type is the G-channel type. Existing pixel values ​​for all G-channel types are obtained from the target data window, and the second range is determined based on these existing pixel values. The second range characterizes whether there are high-contrast regions within the target data window.

[0204] In one alternative example, the principle for determining the second range is similar to that for the first range described above.

[0205] In one optional example, the second range is determined based on the existing pixel values ​​of the first type in the target data window, including: removing the maximum and minimum values ​​of the existing pixel values ​​of the first type in the target data window to obtain the remaining existing pixel values; and taking the difference between the maximum and minimum values ​​of the remaining existing pixel values ​​as the second range.

[0206] Specifically, to further improve accuracy, the maximum and minimum values ​​of each existing pixel value of the first type in the target data window are removed, and the difference between the maximum and minimum values ​​of the remaining existing pixel values ​​is taken as the second range.

[0207] The dynamic threshold Cur_thr, determined based on the second range value, is expressed as follows:

[0208] G_ratio=(Grange2–range_l) / (range_h–range_l).

[0209] Cur_thr=Min(Max(diff_low,(diff_high–diff_low)*G_ratio+diff_low),diff_high).

[0210] Where range_l and range_h represent the lower and upper bounds of the pre-obtained threshold range, respectively, and diff_low and diff_high represent the lower and upper bounds of the pre-obtained minimum total difference range, respectively, which can be obtained in advance through debugging. Grange2 represents the second range.

[0211] In one optional example, Figure 13This is a flowchart illustrating step 2033 provided in an exemplary embodiment of this disclosure. Determining whether the center pixel satisfies a first condition based on the pixel values ​​of each first pixel includes:

[0212] Step 20331: Sort the pixel values ​​of each first pixel to obtain the sorting result.

[0213] For example, see the above. Figure 3 The target data window has a first preset area of ​​5*5 centered on the central pixel G45. The pixel values ​​of the nine first pixels of the G channel type within this first preset area are obtained: G25, G34, G43, G54, G65, G56, G47, G36, and G45. These values ​​are then sorted according to their size, for example, in the order S0 <= S1 <= S2 <= S3 <= S4 <= S5 <= S6 <= S7 <= S8. Defective pixels must be located in the largest or smallest group after sorting. The largest group represents a preset number of pixel values ​​taken from the maximum value side, and the smallest group represents a preset number of pixel values ​​taken from the minimum value side. For example, if the largest or smallest group is 1, the defective pixel must be located at the maximum value position (S8) or the minimum value position (S0) after sorting. If the largest or smallest group is 2, the defective pixel must be located within the two largest position ranges (S8 or S7) or the two smallest position ranges (S0 or S1) after sorting.

[0214] Step 20332: In response to the fact that the pixel value of the center pixel is located in the preset position range corresponding to the target channel type in the sorting result, determine the absolute value of the first difference between the pixel value of the first pixel in the first preset position within the preset position range and the pixel value of the first pixel in the adjacent position within the preset position range.

[0215] The number of pixel values ​​that can be obtained for the first pixel may differ depending on the target channel type, thus the sorting order may also differ. A corresponding preset position range can be set for each channel type. The first preset position is the position of the pixel value of other first pixels closest to the sorting result within the preset prediction range. For example, when the preset position range is S8 and S7, the first preset position is S7, the pixel value of the first pixel adjacent to this preset position range is S6, and the absolute value of the first difference is ABS(S7-S6); when the preset position range is S8, the first... The preset position is S8, the pixel value of the first pixel adjacent to this preset position range is S7, and the absolute value of the first difference is ABS(S8-S7); similarly, when the preset position range is S0 and S1, the first preset position is S1, the pixel value of the first pixel adjacent to this preset position range is S2, and the absolute value of the first difference is ABS(S1-S2); when the preset position range is S0, the first preset position is S0, the pixel value of the first pixel adjacent to this preset position range is S1, and the absolute value of the first difference is ABS(S0-S1).

[0216] Step 20333: Determine the first mean value based on the existing pixel values ​​of the first channel type in the first preset region.

[0217] The first type is the G channel type, and the first mean value can be determined based on all existing pixel values ​​of the first type in the first preset region.

[0218] For example, Figure 3 The corresponding 5x5 area contains the pixel values ​​of all first pixels: G23, G25, G27, G34, G43, G54, G63, G65, G56, G67, G47, G36, and G45, totaling 13 pixel values. The first mean value G_avg is represented as follows:

[0219] G_avg=(G23+G25+G27+G34+G43+G54+G63+G65+G56+G67+G47+G36+G45) / 13.

[0220] Step 20334: In response to the ratio of the absolute value of the first difference to the first mean being greater than the second threshold, it is determined that the center pixel satisfies the first condition.

[0221] The ratio of the absolute value of the first difference (ABS()) to the first mean (G_avg) is expressed as ABS() / G_avg. The second threshold (thr2) can be set according to actual needs. When ABS() / G_avg > thr2, the center pixel is determined to meet the first condition. The purpose of the first mean is to make areas with higher average brightness more tolerant of the difference between adjacent pixel values ​​(absolute value of the first difference) after sorting, and within a certain range, it is not considered a bad pixel.

[0222] In one optional example, Figure 14 This is a schematic diagram of a target data window with a B-channel type center pixel in 4*4 RGB-IR mode provided by an exemplary embodiment of this disclosure. In this example, the center pixel is the pixel at position 44, and the pixel value is B44. The defect detection and correction process for this center pixel is as follows:

[0223] 1. Determine if the first condition is met:

[0224] Obtain the pixel values ​​of the five first pixels of type B within a 5x5 area of ​​the first preset region, store them in an array, and sort them as follows: S0 <= S1 <= S2 <= S3 <= S4. Defective pixels must be located within the largest or smallest group after sorting. If the smallest group is 2, and the center pixel is a cold pixel, it must be located in S0 or S1, and (S2-S1) / G_avg must be greater than the second threshold. If the smallest group is 1, and the center pixel is a cold pixel, it must be located in S0, and (S1-S0) / G_avg must be greater than the second threshold. G_avg is the average pixel value of all first pixels of type G within the first preset region. Hot spot cases are not discussed further. The second threshold can be the same or different under different conditions, and can be set according to actual needs.

[0225] 2. Determine whether the second condition is met:

[0226] 1) Interpolation calculation of the second pixel value of the preset spare pixel:

[0227] The preset backup pixels corresponding to B44 include: B42, B46, B24, B64, G62, G26, G22, G66, G42, G46, G24, and G64. The second pixel values ​​of G62, G26, G22, G66, G42, G46, G24, and G64 can all be obtained by interpolating the pixel values ​​of the G channels (top, bottom, left, and right). The specific interpolation operation is based on direction determination, which will not be elaborated here. For the four second pixel values ​​B42, B46, B24, and B64, since they are the pixel values ​​of the same channel closest to the center pixel B42, their calculation method needs to be based on a preset confidence rule. Taking B42 as an example, it is represented as follows:

[0228] Diff_horizon=ABS(B40–B44).

[0229] Diff_vertical=ABS(B22–B62).

[0230] Let B42_w represent the credibility calculated from the value of B42. B42_w = 0 means untrustworthy and will be avoided in subsequent calculations. B42_w = 1 means trustworthy and will be used in subsequent calculations.

[0231] If ABS(Diff_horizon–Diff_vertical) is less than a certain threshold (set according to actual needs), then point B42 is considered to have no obvious direction, and B42_w = 0, B42 = 0; otherwise, if Diff_horizon is greater than Diff_vertical, then B42_w = 1, B42 = (B22+B62) / 2, and if Diff_horizon is less than Diff_vertical, then B42_w = 0, B42 = 0.

[0232] 2) Calculation of the total difference in the four directions:

[0233] If B42_w = 1 and B46_w = 1, then Diff_h = ABS(B42-B44) + ABS(B46-B44).

[0234] If B42_w = 0 or B46_w = 0, then Diff_h = ABS(G43-B44) + ABS(G45-B44).

[0235] If B24_w = 1 and B64_w = 1, then Diff_v = ABS(B24-B44) + ABS(B64-B44).

[0236] If B24_w = 0 or B64_w = 0, then Diff_v = ABS(G34-B44) + ABS(G54-B44).

[0237] Diff_45=ABS(B62-B44)+ABS(B26-B44).

[0238] Diff_135=ABS(B22-B44)+ABS(B66-B44).

[0239] 3) Calculation of the first range in the four directions:

[0240] GrangeH=GetGRange(G42,G43,G44,G45,G46).

[0241] GrangeV=GetGRange(G24,G34,G44,G54,G64).

[0242] Grange45=GetGRange(G62,G53,G44,G35,G26).

[0243] Grange135=GetGRange(G22,G33,G44,G55,G66).

[0244] 4) Calculation of dynamic thresholds (first threshold) in four directions:

[0245] G_ratio=(Grange–range_l) / (range_h–range_l).

[0246] Cur_thr=Min(Max(diff_low,(diff_high–diff_low)*G_ratio+diff_low),diff_high).

[0247] Where Grange represents the first range in a certain direction. The first threshold (Cur_thr_h, Cur_thr_v, Cur_thr_45, Cur_thr_135) in the corresponding direction is determined by the first range in different directions (GrangeH, GrangeV, Grange45, Grange135).

[0248] For a bad pixel to be considered, the following conditions must be met: Diff_h > Cur_thr_h, Diff_v > Cur_thr_v, Diff_45 > Cur_thr_45, and Diff_135 > Cur_thr_135. If these conditions are met, then the second condition is satisfied.

[0249] 3. When the center pixel B44 satisfies both the first and second conditions, the center pixel is determined to be a bad pixel.

[0250] 4. Defect correction:

[0251] 1) Calculate the sum of the differences in the four directions:

[0252] Main_diff_h=(ABS(B00-B04)+ABS(B40-B44)) / 2.

[0253] Main_diff_v=(ABS(B00-B40)+ABS(B04-B44)) / 2.

[0254] Main_diff_45=ABS(B04-B40).

[0255] Main_diff_135=ABS(B00–B44).

[0256] 2) Calculate the sum of the minimum differences:

[0257] Main_diff_min=min(Main_diff_h, Main_diff_v, Main_diff_45, Main_diff_135).

[0258] 3) Determine the direction of the sum of the minimum differences as the interpolation direction.

[0259] 4) Interpolation calculation:

[0260] If Main_diff_min equals Main_diff_h, then:

[0261] When B42_w = 1 and B46_w = 1, correct_pixel = (B42 + B46) / 2.

[0262] When B42_w = 0 or B46_w = 0, correct_pixel = (B22 + B26 + B62 + B66) / 4.

[0263] If Main_diff_min equals Main_diff_v, then:

[0264] When B24_w = 1 and B64_w = 1, correct_pixel = (B24 + B64) / 2.

[0265] When B24_w = 0 or B64_w = 0, correct_pixel = (B22 + B26 + B62 + B66) / 4.

[0266] If Main_diff_min equals Main_diff_45, then correct_pixel = (B26 + B62) / 2.

[0267] If Main_diff_min equals Main_diff_135, then correct_pixel = (B22 + B66) / 2.

[0268] When the center pixel is an R channel, since the distribution of R channel type and B channel type in the image data to be processed is the same or similar, the defect detection and correction process of the center pixel of R channel type can be referred to the above B channel type, and will not be repeated here.

[0269] In one optional example, Figure 15This is a schematic diagram of a target data window with a center pixel of IR channel type in 4*4 RGB-IR mode provided by an exemplary embodiment of this disclosure. In this example, the center pixel is the pixel at position 35, and the pixel value is IR35. The defect detection and correction process for this center pixel is as follows:

[0270] 1. Determine whether the first condition is met.

[0271] The principle behind this step is similar to that of the center pixel of the aforementioned G-channel type, and will not be repeated here.

[0272] 2. Determine whether the second condition is met:

[0273] 1) Interpolation calculation of the second pixel value of the preset spare pixel:

[0274] The preset spare pixels corresponding to IR35 include: G24, G26, G44, G46, G35, G13, G15, G17, G33, G37, G53, G55, and G57. These preset spare pixels can all be obtained through interpolation based on direction determination, as described in the calculation principle of G42 mentioned above, and will not be repeated here.

[0275] 2) Calculation of the total difference in the four directions:

[0276] Diff_h=ABS(IR33-IR35)+ABS(IR37-IR35).

[0277] Diff_v=ABS(IR15-IR35)+ABS(IR55-IR35).

[0278] Diff_45=ABS(IR17-IR35)+ABS(IR53-IR35).

[0279] Diff_135=ABS(IR13-IR35)+ABS(IR57-IR35).

[0280] 3) Calculation of the first range in the four directions:

[0281] GrangeH=GetGRange(G33,G34,G35,G36,G37).

[0282] GrangeV=GetGRange(G15,G25,G35,G45,G55).

[0283] Grange45=GetGRange(G53,G44,G35,G26,G17).

[0284] Grange135=GetGRange(G13,G24,G35,G46,G57).

[0285] 4) Calculation of dynamic thresholds (first threshold) in four directions:

[0286] G_ratio=(Grange–range_l) / (range_h–range_l).

[0287] Cur_thr=Min(Max(diff_low,(diff_high–diff_low)*G_ratio+diff_low),diff_high).

[0288] Where Grange represents the first range in a certain direction. The first threshold (Cur_thr_h, Cur_thr_v, Cur_thr_45, Cur_thr_135) in the corresponding direction is determined by the first range in different directions (GrangeH, GrangeV, Grange45, Grange135).

[0289] For a bad pixel to be considered, it must satisfy the following conditions: Diff_h > Cur_thr_h, Diff_v > Cur_thr_v, Diff_45 > Cur_thr_45, and Diff_135 > Cur_thr_135. If so, the second condition is satisfied.

[0290] 3. When the center pixel B44 satisfies both the first and second conditions, the center pixel is determined to be a bad pixel.

[0291] 4. Defect correction:

[0292] 1) Calculate the sum of the differences in the four directions:

[0293] Main_diff_h=ABS(IR13-IR17)+ABS(IR33-IR37)+ABS(IR53-IR57).

[0294] Main_diff_v=ABS(IR13-IR53)+ABS(IR15-IR55)+ABS(IR17-IR57).

[0295] Main_diff_45=ABS(IR15-IR33)+ABS(IR17-IR53)+ABS(IR37-IR55).

[0296] Main_diff_135=ABS(IR15–IR37)+ABS(IR13-IR57)+ABS(IR33-IR55).

[0297] 2) Calculate the sum of the minimum differences:

[0298] Main_diff_min=min(Main_diff_h, Main_diff_v, Main_diff_45, Main_diff_135).

[0299] 3) Determine the direction of the sum of the minimum differences as the interpolation direction.

[0300] 4) Calculate the corrected target pixel value `correct_pixel` using interpolation:

[0301] If Main_diff_min equals Main_diff_h, then correct_pixel = (IR33 + IR37) / 2.

[0302] If Main_diff_min equals Main_diff_v, then correct_pixel = (IR15 + IR55) / 2.

[0303] If Main_diff_min equals Main_diff_45, then correct_pixel = (IR17 + IR53) / 2.

[0304] If Main_diff_min equals Main_diff_135, then correct_pixel = (IR13 + IR57) / 2.

[0305] In one optional example, refer to Figure 12 In the 4*4 RGB-IR mode, the target data window with a preset size of 5*5 and a center channel of type G is described below for the defect detection and correction process, where each center pixel corresponds to only one dynamic threshold:

[0306] 1. Determine whether the first condition is met.

[0307] The judgment of the first condition is consistent with the aforementioned principle, such as taking out the 5*5 area, which will not be elaborated here.

[0308] 2. Determine whether the second condition is met:

[0309] 1) Calculation of the minimum total difference Min_Diff:

[0310] The total difference for each of the four directions is calculated as follows:

[0311] Diff_h=ABS(G25-G23)+ABS(G21-G23).

[0312] Diff_v=ABS(G03-G23)+ABS(G43-G23).

[0313] Diff_45=ABS(G14-G23)+ABS(G32-G23).

[0314] Diff_135=ABS(G12-G23)+ABS(G34-G23).

[0315] The minimum total difference, Min_Diff, is calculated as follows:

[0316] Min_Diff=MIN(Diff_135,Diff_45,Diff_h,Diff_v).

[0317] 2) Calculation of the second range Grange2:

[0318] Sort all existing pixel values ​​of the G channel in the 5*5 area of ​​the target data window, remove one maximum value and one minimum value, and take the maximum value as the maximum value and the minimum value as the minimum value among the remaining existing pixel values. The difference between the maximum value and the minimum value is taken as the second range Grange2.

[0319] 3) Dynamic threshold calculation:

[0320] G_ratio=(Grange2–range_l) / (range_h–range_l).

[0321] Cur_thr=Min(Max(diff_low,(diff_high–diff_low)*G_ratio+diff_low),diff_high).

[0322] For a bad pixel to be considered, Min_Diff > Cur_thr must be satisfied to confirm that the second condition is met.

[0323] 3. When the center pixel G23 satisfies both the first and second conditions, the center pixel is determined to be a bad pixel.

[0324] 4. Defect correction:

[0325] 1) Calculate the sum of the differences in the four directions.

[0326] Main_diff_h=ABS(G12-G14)+ABS(G21-G25)+ABS(G32-G34).

[0327] Main_diff_v=ABS(G12-G32)+ABS(G03-G43)+ABS(G14-G34).

[0328] Main_diff_45=ABS(G03-G21)+ABS(G14-G32)+ABS(G25-G43).

[0329] Main_diff_135=ABS(G03–G25)+ABS(G12-G34)+ABS(G21-G43).

[0330] 2) Calculate the sum of the minimum differences:

[0331] Main_diff_min=min(Main_diff_h, Main_diff_v, Main_diff_45, Main_diff_135).

[0332] 3) Determine the direction of the sum of the minimum differences as the interpolation direction.

[0333] 4) Interpolation calculation of the corrected target pixel value correct_pixel

[0334] If Main_diff_min equals Main_diff_h, then correct_pixel = (G21 + G25) / 2.

[0335] If Main_diff_min equals Main_diff_v, then correct_pixel = (G03 + G43) / 2.

[0336] If Main_diff_min equals Main_diff_45, then correct_pixel = (G14 + G32) / 2.

[0337] If Main_diff_min equals Main_diff_135, then correct_pixel = (G12 + G34) / 2.

[0338] In one optional example, Figure 16 This is a schematic diagram of a target data window with a preset size of 5*5 and a center channel of IR channel type, provided in an exemplary embodiment of this disclosure. In this example, the center pixel is the pixel at position 33, and the pixel value is IR33. Based on this, the defect detection and correction process, which corresponds to only one dynamic threshold for each center pixel, is described below:

[0339] 1. Determine whether the first condition is met.

[0340] The judgment of the first condition is consistent with the aforementioned principle, such as taking out the 5*5 area, which will not be elaborated here.

[0341] 2. Determine whether the second condition is met:

[0342] 1) Calculation of the minimum total difference Min_Diff:

[0343] The total difference for each of the four directions is calculated as follows:

[0344] Diff_h=ABS(IR31-IR33)+ABS(IR35-IR33).

[0345] Diff_v=ABS(IR13-IR33)+ABS(IR53-IR33).

[0346] Diff_45=ABS(IR11-IR33)+ABS(IR55-IR33).

[0347] Diff_135=ABS(IR15-IR33)+ABS(IR51-IR33).

[0348] The minimum total difference, Min_Diff, is calculated as follows:

[0349] Min_Diff=MIN(Diff_135,Diff_45,Diff_h,Diff_v).

[0350] 2) Calculation of the second range Grange2:

[0351] Sort all existing pixel values ​​of the G channel in the 5*5 area of ​​the target data window, remove one maximum value and one minimum value, and take the maximum value as the maximum value and the minimum value as the minimum value among the remaining existing pixel values. The difference between the maximum value and the minimum value is taken as the second range Grange2.

[0352] 3) Dynamic threshold calculation:

[0353] G_ratio=(Grange2–range_l) / (range_h–range_l).

[0354] Cur_thr=Min(Max(diff_low,(diff_high–diff_low)*G_ratio+diff_low),diff_high).

[0355] For a bad pixel to be considered, Min_Diff > Cur_thr must be satisfied to confirm that the second condition is met.

[0356] 3. When the center pixel G23 satisfies both the first and second conditions, the center pixel is determined to be a bad pixel.

[0357] 4. Defect correction:

[0358] 1) Calculate the sum of the differences in the four directions:

[0359] Main_diff_h=ABS(IR11-IR15)+ABS(IR31-IR35)+ABS(IR51-IR55).

[0360] Main_diff_v=ABS(IR11-IR51)+ABS(IR13-IR53)+ABS(IR15-IR55).

[0361] Main_diff_45=ABS(IR13-IR31)+ABS(IR15-IR51)+ABS(IR35-IR53).

[0362] Main_diff_135=ABS(IR13–IR35)+ABS(IR11-IR55)+ABS(IR31-IR53).

[0363] 2) Calculate the sum of the minimum differences:

[0364] Main_diff_min=min(Main_diff_h, Main_diff_v, Main_diff_45, Main_diff_135).

[0365] 3) Determine the direction of the sum of the minimum differences as the interpolation direction.

[0366] 4) Calculate the corrected target pixel value `correct_pixel` using interpolation:

[0367] If Main_diff_min equals Main_diff_h, then correct_pixel = (IR31 + IR35) / 2.

[0368] If Main_diff_min equals Main_diff_v, then correct_pixel = (IR13 + IR53) / 2.

[0369] If Main_diff_min equals Main_diff_45, then correct_pixel = (IR15 + IR51) / 2.

[0370] If Main_diff_min equals Main_diff_135, then correct_pixel = (IR11 + IR55) / 2.

[0371] In one optional example, Figure 17This is a schematic diagram of a target data window with a preset size of 5*5 and a center channel of type B, provided in an exemplary embodiment of this disclosure. In this example, the center pixel is the pixel at position 22, and the pixel value is B22. Based on this, the defect detection and correction process, which corresponds to only one dynamic threshold for each center pixel, is described as follows:

[0372] 1. Determine if the first condition is met:

[0373] The judgment of the first condition is consistent with the aforementioned principle, such as taking out the 5*5 area, which will not be elaborated here.

[0374] 2. Determine whether the second condition is met:

[0375] 1) Calculation of the minimum total difference Min_Diff:

[0376] The total difference for each of the four directions is calculated as follows:

[0377] Diff_135=ABS(B00-B22)+ABS(B44-B22).

[0378] Diff_45=ABS(B04-B22)+ABS(B40-B22).

[0379] Diff_h=ABS(G21-B22)+ABS(G23-B22).

[0380] Diff_v=ABS(G12-B22)+ABS(G32-B22).

[0381] The minimum total difference, Min_Diff, is calculated as follows:

[0382] Min_diff=MIN(Diff_135,Diff_45,Diff_h,Diff_v).

[0383] 2) Calculation of the second range Grange2:

[0384] Sort all existing pixel values ​​of the G channel in the 5*5 area of ​​the target data window, remove one maximum value and one minimum value, and take the maximum value as the maximum value and the minimum value as the minimum value among the remaining existing pixel values. The difference between the maximum value and the minimum value is taken as the second range Grange2.

[0385] 3) Dynamic threshold calculation:

[0386] G_ratio=(Grange2–range_l) / (range_h–range_l).

[0387] Cur_thr=Min(Max(diff_low,(diff_high–diff_low)*G_ratio+diff_low),diff_high).

[0388] For a bad pixel to be considered, Min_Diff > Cur_thr must be satisfied to confirm that the second condition is met.

[0389] 3. When the center pixel G23 satisfies both the first and second conditions, the center pixel is determined to be a bad pixel.

[0390] 4. Defect correction:

[0391] 1) Calculate the sum of the differences in the four directions:

[0392] Main_diff_h=(ABS(B00-B04)+ABS(B40-B44)) / 2.

[0393] Main_diff_v=(ABS(B00-B40)+ABS(B04-B44)) / 2.

[0394] Main_diff_45=ABS(B04-B40).

[0395] Main_diff_135=ABS(B00–B44).

[0396] 2) Calculate the sum of the minimum differences:

[0397] Main_diff_min=min(Main_diff_h, Main_diff_v, Main_diff_45, Main_diff_135).

[0398] 3) Determine the direction of the sum of the minimum differences as the interpolation direction.

[0399] 4) Calculate the corrected target pixel value `correct_pixel` using interpolation:

[0400] If Main_diff_min equals Main_diff_h, then correct_pixel = (B00 + B04 + B40 + B44) / 4.

[0401] If Main_diff_min equals Main_diff_v, then correct_pixel = (B00 + B04 + B40 + B44) / 4.

[0402] If Main_diff_min equals Main_diff_45, then correct_pixel = (B04 + B40) / 2.

[0403] If Main_diff_min equals Main_diff_135, then correct_pixel = (B00 + B44) / 2.

[0404] In one optional example, Figure 18 This is a schematic diagram of a target data window with a preset size of 5*5 in the 2*2 RGB-IR mode provided by an exemplary embodiment of this disclosure. In this example, the center pixel is the pixel at position 22, and the pixel value is R22. Since the four channel types have the same proportion in the 2*2 RGB-IR mode, the principle of bad pixel detection and correction is the same regardless of the channel type of the center pixel. Here, we only use R22 as an example for explanation, and the process is as follows:

[0405] 1. Determine if the first condition is met:

[0406] Different first conditions can be set according to different image data modes, and there is no specific limitation. For example, in this example, the pixel values ​​(R00, R02, R04, R20, R22, R24, R40, R42, R44) of the first pixel point of the R channel type in the first preset area (in this example, the area can be set to be the same as the target data window) are obtained from the current target data window, sorted, and the first absolute difference value ABS() is determined according to the group setting and hot and cold hot spot situation in the aforementioned embodiment. The second mean value is determined based on the existing pixel values ​​in the first preset area whose channel type is the same as the center pixel point. Then, based on the ratio of the first absolute difference value and the second mean value being greater than the third threshold, it is determined that the center pixel point meets the first condition.

[0407] 2. Determine whether the second condition is met:

[0408] The second condition can be set based on pre-set reliability rules or not, depending on actual needs. Taking the condition not based on pre-set reliability rules as an example:

[0409] 1) Calculation of the minimum total difference Min_Diff:

[0410] The total difference for each of the four directions is calculated as follows:

[0411] Diff_h=ABS(R20-R22)+ABS(R24-R22).

[0412] Diff_v=ABS(R02-R22)+ABS(R42-R22).

[0413] Diff_45=ABS(R04-R22)+ABS(R40-R22).

[0414] Diff_135=ABS(R00-R22)+ABS(R44-R22).

[0415] The minimum total difference, Min_Diff, is calculated as follows:

[0416] Min_Diff=MIN(Diff_135,Diff_45,Diff_h,Diff_v).

[0417] 2) Calculation of the second range Grange2:

[0418] Sort all existing pixel values ​​of the G channel in the 5*5 area of ​​the target data window, remove one maximum value and one minimum value, and take the maximum value as the maximum value and the minimum value as the minimum value among the remaining existing pixel values. The difference between the maximum value and the minimum value is taken as the second range Grange2.

[0419] 3) Dynamic threshold calculation:

[0420] G_ratio=(Grange2–range_l) / (range_h–range_l).

[0421] Cur_thr=Min(Max(diff_low,(diff_high–diff_low)*G_ratio+diff_low),diff_high).

[0422] For a bad pixel to be considered, Min_Diff > Cur_thr must be satisfied to confirm that the second condition is met.

[0423] 3. When the center pixel G23 satisfies both the first and second conditions, the center pixel is determined to be a bad pixel.

[0424] 4. Defect correction:

[0425] 1) Calculate the sum of the differences in the four directions:

[0426] Main_diff_h=ABS(R00-R04)+ABS(R20-R24)+ABS(R40-R44).

[0427] Main_diff_v=ABS(R00-R40)+ABS(R02-R42)+ABS(R04-R44).

[0428] Main_diff_45=ABS(R02-R20)+ABS(R04-R40)+ABS(R24-R42).

[0429] Main_diff_135=ABS(R02–R24)+ABS(R00-R44)+ABS(R20-R42).

[0430] 2) Calculate the sum of the minimum differences:

[0431] Main_diff_min=min(Main_diff_h, Main_diff_v, Main_diff_45, Main_diff_135).

[0432] 3) Determine the direction of the sum of the minimum differences as the interpolation direction.

[0433] 4) Calculate the corrected target pixel value `correct_pixel` using interpolation:

[0434] If Main_diff_min equals Main_diff_h, then correct_pixel = (R20 + R24) / 2.

[0435] If Main_diff_min equals Main_diff_v, then correct_pixel = (R02 + R42) / 2.

[0436] If Main_diff_min equals Main_diff_45, then correct_pixel = (R04 + R40) / 2.

[0437] If Main_diff_min equals Main_diff_135, then correct_pixel = (R00 + R44) / 2.

[0438] It should be noted that for the outermost pixels of the image data to be processed, in order to make them the center pixels of the target data window, padding can be used to enclose the edges. For example, if the target data window size is 9*9, then padding can be 4 rows or 4 columns. The padding method can be set according to actual needs. For example, the edge portion of the image data to be processed can be copied as an extension. Figure 19This is a schematic diagram of a padding method provided by an exemplary embodiment of the present disclosure. In this example, the image data to be processed is a 9*9 area G01-G89 in the middle part of the image. If the target data window size is 5*5, then 2 rows and 2 columns of padding are required. The surrounding part of the 9*9 area G01-G89 is the padding part. 2 rows or 2 columns of the edge of the image data to be processed are copied on each side as the filling part.

[0439] The image defect correction method disclosed herein is not only applicable to defect detection and correction of image data in 4*4RGB-IR and 2*2RGB-IR modes, but also applicable to image data in other modes that can conform to similar modes, such as Bayer format image data, whose basic kernel includes (B,G,G,R). When performing defect detection, the corresponding first and second conditions can be set based on the process of this disclosure, and a corresponding target data window can be set to determine the defect based on the first and second conditions, and then defect correction can be performed based on the points around the defect. The specific principles will not be elaborated here.

[0440] Any of the image defect correction methods provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any of the image defect correction methods provided in this disclosure can be executed by a processor, such as by a processor executing any of the image defect correction methods mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0441] Exemplary device

[0442] Figure 20 This is a schematic diagram of an image defect correction apparatus provided in an exemplary embodiment of this disclosure. The apparatus of this embodiment can be used to implement corresponding method embodiments of this disclosure, such as… Figure 20 The device shown includes: a first acquisition module 501, a first processing module 502, a second processing module 503, a third processing module 504, a fourth processing module 505, and a fifth processing module 506.

[0443] A first acquisition module 501 is used to acquire image data to be processed; a first processing module 502 is used to, for any pixel in the image data to be processed acquired by the first acquisition module 501, determine a target data window with a preset scale centered on that pixel; a second processing module 503 is used to, based on the target data window determined by the first processing module 502, determine whether the center pixel satisfies a preset condition, the preset condition including a first condition and / or a second condition, wherein the first condition is a condition satisfied by the pixel value of the center pixel, and the second condition is a condition satisfied by the target data window. The conditions are met by the pixel values ​​of at least one pixel of the same channel type in the same direction corresponding to the center pixel; the third processing module 504 is used to determine the center pixel as a bad pixel in response to the second processing module 503 determining that the center pixel meets the preset conditions; the fourth processing module 505 is used to determine the corrected target pixel value of the center pixel based on the pixel values ​​of other pixels in the target data window besides the center pixel; the fifth processing module 506 is used to determine the corrected target image data corresponding to the image data to be processed based on the target pixel value determined by the fourth processing module 505.

[0444] In one optional example, Figure 21 This is a schematic diagram of the structure of the second processing module 503 provided in an exemplary embodiment of the present disclosure. In this example, the second processing module 503 includes: a first determining unit 5031, a first acquiring unit 5032, a second determining unit 5033, a third determining unit 5034, a fourth determining unit 5035, and a fifth determining unit 5036.

[0445] A first determining unit 5031 is configured to determine the target channel type to which the center pixel belongs based on the target data window; a first acquiring unit 5032 is configured to acquire, based on the target channel type, the pixel values ​​of each first pixel of the same channel type in a first preset region corresponding to the target channel type from the target data window; a second determining unit 5033 is configured to determine whether the center pixel satisfies the first condition based on the pixel values ​​of each first pixel; a third determining unit 5034 is configured to determine the difference in pixel values ​​and a dynamic threshold between pixels of the same channel type in at least one direction corresponding to the center pixel based on the existing pixel values ​​corresponding to each pixel in the target data window; a fourth determining unit 5035 is configured to determine whether the center pixel satisfies the second condition based on the difference in pixel values ​​and the dynamic threshold between pixels of the same channel type in at least one direction; and a fifth determining unit 5036 is configured to determine that the center pixel satisfies the preset condition in response to the center pixel satisfying the first condition and / or the second condition.

[0446] In one optional example, Figure 22 This is a schematic diagram of the structure of the third determining unit 5034 provided in an exemplary embodiment of this disclosure. In this example, the dynamic threshold includes a first threshold corresponding to each of the at least one direction; the third determining unit 5034 includes: a first determining subunit 50341 and a second determining subunit 50342.

[0447] The first determining subunit 50341 is used to determine the second pixel value of a preset backup pixel corresponding to the target channel type based on the existing pixel values ​​corresponding to each pixel in the target data window; the second determining subunit 50342 is used to determine the difference in pixel values ​​between pixels of the same channel type in the at least one direction corresponding to the center pixel and the first threshold corresponding to each of the at least one direction based on the existing pixel values ​​and the second pixel value of the preset backup pixel.

[0448] In an optional example, the first determining subunit 50341 is specifically configured to: in response to the target channel type being a first type, a second type, or a third type, determine the second pixel value of a first set of backup pixels using direction-based interpolation based on the existing pixel values, and determine the second pixel value of a second set of backup pixels based on a preset confidence rule; use the second pixel values ​​of the first set of backup pixels and the second pixel values ​​of the second set of backup pixels as the second pixel value of the preset backup pixel corresponding to the target channel type; and in response to the target channel type being a fourth type, determine the second pixel value of the preset backup pixel corresponding to the target channel type using direction-based interpolation based on the existing pixel values.

[0449] In an optional example, the first determining subunit 50341 is specifically used to: determine the second pixel value of the second part of the backup pixels and the confidence level of the second part of the backup pixels based on a preset confidence rule; correspondingly, the second determining subunit 50342 is specifically used to: in response to the target channel type being the first type, the second type, or the third type, determine the difference in pixel values ​​between pixels of the same channel type in the at least one direction corresponding to the center pixel based on the existing pixel value, the second pixel value of the preset backup pixels, and the confidence level of the second part of the backup pixels.

[0450] In one optional example, Figure 23This is a schematic diagram of the structure of the fourth processing module 505 provided in an exemplary embodiment of the present disclosure. In this example, the fourth processing module 505 includes: a first processing unit 5051, a second processing unit 5052, a third processing unit 5053, and a fourth processing unit 5054.

[0451] The first processing unit 5051 is used to determine the sum of differences corresponding to each direction of the target data window based on the pixel values ​​of other pixels in the target data window that have the same channel type as the center pixel; the second processing unit 5052 is used to determine the minimum sum of differences based on the sums of differences; the third processing unit 5053 is used to determine the interpolation direction corresponding to the center pixel based on the minimum sum of differences; and the fourth processing unit 5054 is used to determine the target pixel value based on the interpolation direction.

[0452] In an optional example, the fourth processing unit 5054 is specifically configured to: in response to the target channel type being the second type or the third type, determine the target pixel value based on the interpolation direction, other pixel values ​​of the same channel type in the interpolation direction, the second pixel value of the same channel type in the interpolation direction, and the confidence level corresponding to the second pixel value; and in response to the target channel type being the first type or the fourth type, determine the target pixel value based on the interpolation direction and other pixel values ​​of the same channel type in the interpolation direction.

[0453] In an optional example, the second determining subunit 50342 is specifically used to: determine a first range corresponding to each of the at least one direction corresponding to the center pixel based on the existing pixel value and the second pixel value of the preset spare pixel point, wherein the first range is the difference between the maximum and minimum values ​​of the pixel values ​​of the corresponding direction whose channel type is a first type; and determine a first threshold corresponding to each of the directions based on the first range corresponding to each direction.

[0454] In an optional example, the fourth determining unit 5035 is specifically used to: determine the total difference value corresponding to each of the directions based on the difference of the pixel values ​​corresponding to each of the directions; and determine that the center pixel satisfies the second condition in response to the fact that the total difference value corresponding to each of the directions is greater than the first threshold value of its corresponding direction.

[0455] In one optional example, Figure 24 This is a schematic diagram of the structure of the fourth determining unit 5035 provided in an exemplary embodiment of the present disclosure. In this example, the fourth determining unit 5035 includes: a first processing subunit 50351, a second processing subunit 50352 and a third processing subunit 50353.

[0456] The first processing subunit 50351 is used to determine the total difference value corresponding to each of the directions based on the difference of the pixel values ​​corresponding to each direction; the second processing subunit 50352 is used to determine the minimum total difference value based on the total difference value corresponding to each direction; the third processing subunit 50353 is used to determine that the center pixel satisfies the second condition in response to the minimum total difference value being greater than the dynamic threshold.

[0457] In an optional example, the third determining unit 5034 is specifically used to: determine a second range based on the existing pixel values ​​of the first type in the target data window, wherein the second range is the difference between the maximum and minimum values ​​among the existing pixel values ​​of the first type; and determine the dynamic threshold based on the second range.

[0458] In an optional example, the third determining unit 5034 is specifically used to: remove the maximum and minimum values ​​of each of the existing pixel values ​​of the first type in the target data window to obtain the remaining existing pixel values; and use the difference between the maximum and minimum values ​​of the remaining existing pixel values ​​as the second range.

[0459] In one optional example, Figure 25 This is a schematic diagram of the structure of the second determining unit 5033 provided in an exemplary embodiment of the present disclosure. The second determining unit 5033 includes: a fourth processing subunit 50331, a fifth processing subunit 50332, a sixth processing subunit 50333, and a seventh processing subunit 50334.

[0460] The fourth processing subunit 50331 is used to sort the pixel values ​​of each of the first pixels to obtain a sorting result; the fifth processing subunit 50332 is used to determine the absolute value of a first difference between the pixel value of the first pixel located at a first preset position in the preset position range and the pixel value of the first pixel at an adjacent position in the preset position range, in response to the pixel value of the center pixel being located in a preset position range corresponding to the target channel type in the sorting result; the sixth processing subunit 50333 is used to determine a first mean value based on the existing pixel values ​​of the first channel type in the first preset region; the seventh processing subunit 50334 is used to determine that the center pixel satisfies the first condition in response to the ratio of the absolute value of the first difference to the first mean value being greater than a second threshold.

[0461] Exemplary electronic devices

[0462] This disclosure also provides an electronic device, including: a memory for storing computer programs;

[0463] A processor is configured to execute a computer program stored in the memory, wherein when the computer program is executed, it implements the image defect correction method described in any of the above embodiments of the present disclosure.

[0464] Figure 26 This is a schematic diagram of an application embodiment of the electronic device disclosed herein. In this embodiment, the electronic device 10 includes one or more processors 11 and a memory 12.

[0465] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0466] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the methods of the various embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0467] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0468] For example, the input device 13 may be the microphone or microphone array described above, used to capture the input signal of the sound source.

[0469] In addition, the input device 13 may also include, for example, a keyboard, a mouse, etc.

[0470] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0471] Of course, for the sake of simplicity, Figure 26 Only some of the components of the electronic device 10 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0472] Exemplary computer program products and computer-readable storage media

[0473] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0474] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0475] Furthermore, embodiments of this disclosure may also be computer-readable storage media having computer program instructions stored thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0476] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0477] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0478] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0479] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0480] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0481] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0482] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0483] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for correcting image defects, comprising: Acquire the image data to be processed; For any pixel in the image data to be processed, a target data window with a preset scale centered on that pixel is determined based on the image data to be processed. Based on the target data window, it is determined whether the center pixel meets a preset condition. The preset condition includes a first condition and / or a second condition. The first condition is a condition that the pixel value of the center pixel meets, and the second condition is a condition that the pixel value of at least one pixel of the same channel type in at least one direction corresponding to the center pixel in the target data window meets. In response to the fact that the center pixel meets the preset condition, the center pixel is determined to be a bad pixel; Based on the pixel values ​​of other pixels in the target data window besides the center pixel, determine the corrected target pixel value of the center pixel; Based on the target pixel value, the corrected target image data corresponding to the image data to be processed is determined; The step of determining whether the center pixel meets the preset conditions based on the target data window includes: Based on the existing pixel values ​​corresponding to each pixel in the target data window, the difference in pixel values ​​and the dynamic threshold between pixels of the same channel type in at least one direction corresponding to the center pixel are determined. Based on the difference in pixel values ​​between pixels of the same channel type in at least one direction and the dynamic threshold, it is determined whether the center pixel satisfies the second condition; In response to the central pixel satisfying the second condition, it is determined that the central pixel satisfies the preset condition; The dynamic threshold includes a first threshold corresponding to each of the at least one direction; determining the difference in pixel values ​​and the dynamic threshold between pixels of the same channel type corresponding to the center pixel in the at least one direction based on the existing pixel values ​​corresponding to each pixel in the target data window includes: Based on the existing pixel values ​​corresponding to each pixel in the target data window, the second pixel value of the preset backup pixel corresponding to the target channel type is determined. The preset backup pixel is determined by interpolation operation based on the existing pixel values ​​using direction judgment. Based on the existing pixel value and the second pixel value of the preset spare pixel, the difference in pixel value between pixels of the same channel type in the at least one direction corresponding to the center pixel and the first threshold corresponding to each of the at least one direction are determined.

2. The method according to claim 1, wherein, The step of determining whether the center pixel meets the preset conditions based on the target data window further includes: Based on the target data window, determine the target channel type to which the center pixel belongs; Based on the target channel type, obtain the pixel values ​​of each first pixel point of the same channel type in the first preset area corresponding to the target channel type from the target data window; Based on the pixel values ​​of each of the first pixels, determine whether the center pixel satisfies the first condition; and The step of determining that the center pixel satisfies the preset condition in response to the center pixel satisfying the second condition includes: In response to the central pixel satisfying the first condition and the second condition, it is determined that the central pixel satisfies the preset condition.

3. The method according to claim 1, wherein, The step of determining the second pixel value of a preset spare pixel point corresponding to the target channel type based on the existing pixel values ​​corresponding to each pixel point in the target data window includes: In response to the target channel type being a first type, a second type, or a third type, based on the existing pixel values, the second pixel value of the first part of the backup pixels is determined by interpolation operation based on direction judgment, and the second pixel value of the second part of the backup pixels is determined based on a preset confidence rule; the second pixel values ​​of the first part of the backup pixels and the second pixel values ​​of the second part of the backup pixels are used as the second pixel values ​​of the preset backup pixels corresponding to the target channel type; In response to the target channel type being the fourth type, based on the existing pixel values, the second pixel value of the preset spare pixel point corresponding to the target channel type is determined by interpolation operation based on the direction determination.

4. The method according to claim 3, wherein, The step of determining the second pixel value of the second part of the spare pixels based on a preset confidence rule includes: The second pixel value and the confidence level of the second part of the backup pixels are determined based on the preset confidence rules. The step of determining the difference in pixel values ​​between pixels of the same channel type in at least one direction corresponding to the center pixel, based on the existing pixel value and the second pixel value of the preset spare pixel, includes: In response to the target channel type being the first type, the second type, or the third type, the difference in pixel values ​​between pixels of the same channel type in at least one direction corresponding to the center pixel is determined based on the existing pixel value, the second pixel value of the preset backup pixel, and the confidence level of the second part of the backup pixels.

5. The method according to claim 4, wherein, Determining the corrected target pixel value of the center pixel based on the pixel values ​​of other pixels in the target data window besides the center pixel includes: Based on the pixel values ​​of other pixels in the target data window that have the same channel type as the center pixel, determine the sum of the differences corresponding to each direction of the target data window; Based on the sum of all the aforementioned differences, determine the minimum sum of differences; Based on the sum of the minimum differences, the interpolation direction corresponding to the center pixel is determined; The target pixel value is determined based on the interpolation direction.

6. The method according to claim 5, wherein, Determining the target pixel value based on the interpolation direction includes: In response to the target channel type being the second type or the third type, the target pixel value is determined based on the interpolation direction, other pixel values ​​of the same channel type in the interpolation direction, the second pixel value of the same channel type in the interpolation direction, and the confidence level corresponding to the second pixel value. In response to the target channel type being either the first type or the fourth type, the target pixel value is determined based on the interpolation direction and other pixel values ​​of the same channel type in the interpolation direction.

7. The method according to any one of claims 1-6, wherein, Based on the existing pixel value and the second pixel value of the preset spare pixel, determine the first threshold corresponding to each of the at least one direction corresponding to the center pixel, including: Based on the existing pixel value and the second pixel value of the preset spare pixel, determine the first range corresponding to each of the at least one direction corresponding to the center pixel, wherein the first range is the difference between the maximum and minimum values ​​of the pixel values ​​whose channel type is the first type in the corresponding direction; Based on the first range corresponding to each of the directions, the first threshold corresponding to each of the directions is determined.

8. The method according to any one of claims 1-6, wherein, Determining whether the center pixel satisfies the second condition based on the difference in pixel values ​​between pixels of the same channel type in at least one direction and the dynamic threshold includes: Based on the difference between the pixel values ​​corresponding to each of the directions, the total difference value corresponding to each of the directions is determined; In response to the fact that the total difference value corresponding to each of the aforementioned directions is greater than the first threshold value of the corresponding direction, it is determined that the center pixel satisfies the second condition.

9. The method according to any one of claims 1-6, wherein, Determining whether the center pixel satisfies the second condition based on the difference in pixel values ​​between pixels of the same channel type in at least one direction and the dynamic threshold includes: Based on the difference between the pixel values ​​corresponding to each of the directions, the total difference value corresponding to each of the directions is determined; Based on the total difference value corresponding to each of the aforementioned directions, determine the minimum total difference value; In response to the minimum total difference being greater than the dynamic threshold, it is determined that the center pixel satisfies the second condition.

10. The method according to any one of claims 1-6, wherein, Based on the existing pixel values ​​corresponding to each pixel in the target data window, a dynamic threshold is determined, including: Based on the existing pixel values ​​of the first type in the target data window, a second range is determined, which is the difference between the maximum and minimum values ​​among the existing pixel values ​​of the first type. The dynamic threshold is determined based on the second range.

11. The method according to claim 10, wherein, Based on the existing pixel values ​​of the first channel type in the target data window, the second range is determined, including: Remove the maximum and minimum values ​​from the existing pixel values ​​of the first type in the target data window to obtain the remaining existing pixel values; The difference between the maximum and minimum values ​​among the remaining existing pixel values ​​is taken as the second range.

12. The method according to any one of claims 2-6, wherein, Determining whether the center pixel satisfies the first condition based on the pixel values ​​of each of the first pixels includes: Sort the pixel values ​​of each of the first pixels to obtain the sorting result; In response to the fact that the pixel value of the center pixel is located in a preset position range corresponding to the target channel type in the sorting result, the absolute value of the first difference between the pixel value of the first pixel in the first preset position range and the pixel value of the first pixel in the adjacent position range is determined. Based on the existing pixel values ​​of the first channel type in the first preset region, a first mean value is determined; In response to the fact that the ratio of the absolute value of the first difference to the first mean is greater than a second threshold, it is determined that the center pixel satisfies the first condition.

13. An image defect correction device, comprising: The first acquisition module is used to acquire the image data to be processed; The first processing module is used to determine, based on the image data to be processed, a target data window with a preset scale centered on any pixel in the image data to be processed. The second processing module is used to determine whether the center pixel meets a preset condition based on the target data window. The preset condition includes a first condition and / or a second condition. The first condition is a condition that the pixel value of the center pixel meets, and the second condition is a condition that the pixel value of at least one pixel of the same channel type in at least one direction corresponding to the center pixel in the target data window meets. The third processing module is used to determine that the center pixel is a bad pixel in response to the center pixel meeting the preset condition; The fourth processing module is used to determine the corrected target pixel value of the center pixel based on the pixel values ​​of other pixels in the target data window besides the center pixel. The fifth processing module is used to determine the corrected target image data corresponding to the image data to be processed based on the target pixel value; The second processing module includes: The third determining unit is used to determine the difference in pixel values ​​and the dynamic threshold between pixels of the same channel type in the at least one direction corresponding to the center pixel, based on the existing pixel values ​​corresponding to each pixel in the target data window. The fourth determining unit is used to determine whether the center pixel satisfies the second condition based on the difference in pixel values ​​between pixels of the same channel type in the at least one direction and the dynamic threshold. The fifth determining unit is configured to determine that the center pixel satisfies the preset condition in response to the center pixel satisfying the second condition; The dynamic threshold includes a first threshold corresponding to each of the at least one direction; The third determining unit includes: The first determining subunit is used to determine the second pixel value of a preset backup pixel point corresponding to the target channel type based on the existing pixel values ​​corresponding to each pixel point in the target data window. The preset backup pixel point is determined by interpolation operation based on the existing pixel values ​​using direction judgment. The second determining subunit is used to determine, based on the existing pixel value and the second pixel value of the preset spare pixel, the difference in pixel value between pixels of the same channel type in the at least one direction corresponding to the center pixel, and the first threshold corresponding to each of the at least one direction.

14. A computer-readable storage medium storing a computer program for performing the image defect correction method according to any one of claims 1-12.

15. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the image defect correction method according to any one of claims 1-12.

Images