A camera sensor bad pixel correction method, device, system and storage medium

Abstract

The present application discloses a method, device, system and storage medium for bad pixel correction of camera sensors, which are used to improve the bad pixel correction effect of camera sensors. Determine a bad pixel coordinate set; generate a neighborhood pixel search range; count the number of non-bad pixel pixels and the non-bad pixel coordinate value set of each bad pixel in the neighborhood pixel search range in the bad pixel coordinate set; perform data screening on the bad pixel coordinate set to generate a k-th level bad pixel set; calculate the spatial distance weight set of each bad pixel in the k-th level bad pixel set; calculate the gray value difference weight set of each bad pixel in the k-th level bad pixel set; generate a combined weight set; perform grayscale interpolation processing on the bad pixels in the k-th level bad pixel set on the image to be corrected; determine the k+1-th level bad pixel set from the bad pixels that have not been corrected; calculate the combined weight set on the second-level bad pixel set, and perform grayscale interpolation processing on the image to be corrected until all bad pixels are interpolated.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of display screen image correction, and in particular to a method, device, system and storage medium for correcting bad pixels of a camera sensor. Background Art

[0002] With the rapid development of industrial and Internet technologies, the industry's requirements for industrial product quality are becoming more and more refined and standardized. As far as the display panel industry is concerned, the quality of products directly affects its product competitiveness. Among them, the accuracy of product shape and size specifications is the most basic process indicator of product quality. As one of the display components of the device, the display screen is used in various high-end devices, such as mobile phones, TVs, tablets, etc. As people's requirements for screen display continue to increase, the display screen has gradually become a technologically sophisticated product.

[0003] Defect detection of display screens is a necessary process for adjustment. During the detection process, the display screen usually needs to be photographed by a special digital camera, and the captured images are used for further inspection. This type of high-precision camera sensor is selected according to the application scenarios and usage parameters of different display screens, and different types of cameras and lenses need to be matched according to the type of display screen. This type of display screen digital camera sensor consists of thousands of pixels, each of which is responsible for capturing and transmitting part of the information in the display screen image. However, due to factors such as manufacturing process defects, aging or physical damage, there are more or less pixels on the display screen camera sensor that cannot work properly, that is, bad pixels. The display screen inputs the calibrated image power, and the display screen camera sensor generates the display screen image. The bad pixels are generally bright or dark spots on the generated display screen image, and they are quite different from the surrounding points, which will seriously affect the overall quality and visual effect of each captured image. Therefore, it is very important to pre-correct these bad pixels.

[0004] Existing methods for pre-correcting bad pixels are mostly based on filtering or interpolation methods, such as first using a camera to shoot, determining the location of the bad pixel, and using the mean or median of the neighborhood pixels of these bad pixels to replace the pixel value at the bad pixel location, or using the neighborhood pixels to estimate the pixel value at the bad pixel location through interpolation. The quality of bad pixel correction for the camera sensor is evaluated based on the effect of pixel value replacement.

[0005] The above correction method can achieve good correction effect for isolated bad pixels and single background scenes, but its correction effect will be greatly affected in some scenes. For example, since filtering will lose image edge detail information and traditional interpolation methods cannot fully utilize local image information, when facing scenes with continuous bad pixels and complex backgrounds, no matter what filtering method or interpolation method is used, the correction effect of the bad pixels of the camera sensor is not ideal, resulting in a decrease in the correction effect of the bad pixels of the camera sensor. Summary of the invention

[0006] The present application discloses a method, device, system and storage medium for correcting bad pixels of a camera sensor, which are used to improve the bad pixel correction effect of the camera sensor.

[0007] The first aspect of the present application discloses a method for correcting bad pixels of a camera sensor, comprising:

[0008] Perform bad pixel calibration detection on the camera sensor to determine the bad pixel coordinate set in the camera sensor;

[0009] Taking each bad point in the bad point coordinate set as the center, set the initial search distance and generate the neighborhood pixel search range;

[0010] Count the number of non-bad pixel values ​​and the non-bad pixel coordinate value set for each bad pixel in the neighborhood pixel search range in the bad pixel coordinate set;

[0011] Set the target number threshold of normal pixels within the neighborhood pixel search range, filter the bad pixel coordinate set according to the target number threshold and the non-bad pixel pixel value set, and generate the k-th level bad pixel set. The bad pixels in the k-th level bad pixel set are the k-th layer bad pixels;

[0012] Calculate the spatial distance weight set of each bad pixel in the k-th level bad pixel set according to the non-bad pixel pixel coordinate value set;

[0013] Calculate the gray value difference weight set of each bad pixel in the k-th level bad pixel set according to the image to be corrected;

[0014] Generate a combined weight set according to the spatial distance weight set and the gray value difference weight set;

[0015] Perform grayscale interpolation processing on the bad pixels in the k-th bad pixel set on the image to be corrected according to the combined weight set;

[0016] The remaining bad pixels that have not been corrected are counted, and the k+1th level bad pixel set is determined from the bad pixels that have not been corrected according to the target number threshold and the neighborhood pixel search range. The bad pixels in the k+1th level bad pixel set are the k+1th layer bad pixels.

[0017] The combined weight set is calculated for the second-level bad pixel set, and grayscale interpolation processing is performed on the image to be corrected according to the combined weight set until all the bad pixels on the image to be corrected are interpolated.

[0018] Optionally, a spatial distance weight set of each bad pixel in the k-th level bad pixel set is calculated according to the non-bad pixel pixel coordinate value set, including:

[0019] Determine the relative distance between each bad point in the k-th level bad point set and all normal points in the neighborhood pixel search range according to the non-bad point pixel coordinate value set;

[0020] The spatial distance weights of all normal points and corresponding bad points are calculated according to the relative distances to generate a set of spatial distance weights.

[0021] Optionally, the spatial distance weights of all normal points and corresponding bad points are calculated based on the relative distances to generate a set of spatial distance weights, including:

[0022] According to the distribution pattern of non-bad points in the neighborhood pixel search range and the size of the neighborhood pixel search range, the distance weight adjustment coefficient of the non-bad points in the neighborhood pixel search range is calculated;

[0023] The spatial distance weights of all normal points and corresponding bad points are calculated according to the distance weight adjustment coefficient and relative distance of non-bad points to generate a set of spatial distance weights.

[0024] Optionally, a grayscale value difference weight set of each bad pixel in the k-th level bad pixel set is calculated according to the image to be corrected, including:

[0025] Determine the pixel values ​​of non-bad points in the neighborhood pixel search range on the image to be corrected according to the non-bad point pixel coordinate value set;

[0026] Calculate the mean value of non-bad pixel in the neighborhood pixel search range according to the pixel value of non-bad pixel in the neighborhood pixel search range;

[0027] The gray value difference weight set is calculated based on the pixel value of the non-bad pixel in the neighborhood pixel search range and the mean of the non-bad pixel.

[0028] Optionally, a gray value difference weight set is calculated based on the pixel values ​​of non-bad pixels and the mean values ​​of non-bad pixels in the neighborhood pixel search range, including:

[0029] Perform pixel attention calculation for the image to be corrected, and generate an attention value matrix for each pixel on the image to be corrected;

[0030] According to the distribution pattern of non-bad pixels within the neighborhood pixel search range and the attention value matrix, the attention coefficient of non-bad pixels within the neighborhood pixel search range is calculated;

[0031] The gray value difference weight set is calculated based on the attention coefficient of the non-bad pixel, the pixel value of the non-bad pixel in the neighborhood pixel search range, and the mean of the non-bad pixel.

[0032] Optionally, after setting a target number threshold of normal pixels within the neighborhood pixel search range, filtering the bad pixel coordinate set according to the target number threshold and the non-bad pixel value set, generating a k-th level bad pixel set, and before calculating a spatial distance weight set of each bad pixel in the k-th level bad pixel set according to the non-bad pixel coordinate value set, the method further includes:

[0033] When there is no point information in the k-th level bad pixel set, the initial search distance is updated and the neighborhood pixel search range is regenerated to expand the neighborhood pixel search range;

[0034] Re-count the number of non-bad pixel values ​​and the non-bad pixel coordinate value set for each bad pixel in the bad pixel coordinate set within the neighborhood pixel search range;

[0035] The bad pixel coordinate set is re-screened according to the target quantity threshold and the non-bad pixel pixel value set, and the points that meet the requirements are recorded in the k-th level bad pixel set.

[0036] The second aspect of the present application discloses a device for correcting bad pixels of a camera sensor, comprising:

[0037] A first determination unit is used to perform bad pixel calibration detection on the camera sensor to determine a bad pixel coordinate set in the camera sensor;

[0038] A first generating unit is used to set an initial search distance and generate a neighborhood pixel search range with each bad point in the bad point coordinate set as the center;

[0039] The first statistical unit is used to count the number of non-bad pixel values ​​and non-bad pixel coordinate value sets of each bad pixel in the bad pixel coordinate set within the neighborhood pixel search range;

[0040] The second generating unit is used to set a target number threshold of normal pixels within the neighborhood pixel search range, and to perform data screening on the bad pixel coordinate set according to the target number threshold and the non-bad pixel pixel value set to generate a k-th level bad pixel set, where the bad pixels in the k-th level bad pixel set are the k-th layer bad pixels;

[0041] A first calculation unit is used to calculate a spatial distance weight set of each bad pixel in the k-th level bad pixel set according to a non-bad pixel pixel coordinate value set;

[0042] The second calculation unit is used to calculate the gray value difference weight set of each bad pixel in the k-th level bad pixel set according to the image to be corrected;

[0043] A third generating unit, used for generating a combined weight set according to the spatial distance weight set and the gray value difference weight set;

[0044] A first processing unit is used to perform grayscale interpolation processing on bad pixels in the k-th bad pixel set on the image to be corrected according to the combined weight set;

[0045] The second determination unit is used to count the remaining bad pixels that have not been corrected, and determine the k+1th level bad pixel set from the bad pixels that have not been corrected according to the target number threshold and the neighborhood pixel search range, and the bad pixels in the k+1th level bad pixel set are the k+1th layer bad pixels;

[0046] The second processing unit is used to calculate the combination weight set for the second level bad pixel set, and perform grayscale interpolation processing on the image to be corrected according to the combination weight set until all the bad pixels on the image to be corrected are interpolated.

[0047] Optionally, the first computing unit includes:

[0048] The first determination module is used to determine the relative distance between each bad pixel in the k-th level bad pixel set and all normal points in the neighborhood pixel search range according to the non-bad pixel pixel coordinate value set;

[0049] The generation module is used to calculate the spatial distance weights of all normal points and corresponding bad points according to the relative distances, and generate a set of spatial distance weights.

[0050] Optionally, generate modules, including:

[0051] According to the distribution pattern of non-bad points in the neighborhood pixel search range and the size of the neighborhood pixel search range, the distance weight adjustment coefficient of the non-bad points in the neighborhood pixel search range is calculated;

[0052] The spatial distance weights of all normal points and corresponding bad points are calculated according to the distance weight adjustment coefficient and relative distance of non-bad points to generate a set of spatial distance weights.

[0053] Optionally, the second computing unit includes:

[0054] A first determination module is used to determine the pixel values ​​of non-bad pixels in a neighborhood pixel search range on the image to be corrected according to a set of non-bad pixel coordinate values;

[0055] The first calculation module is used to calculate the non-bad pixel pixel mean value in the neighborhood pixel search range according to the pixel values ​​of the non-bad pixels in the neighborhood pixel search range;

[0056] The second calculation module is used to calculate the gray value difference weight set according to the pixel value of the non-bad pixel and the mean value of the non-bad pixel in the neighborhood pixel search range.

[0057] Optionally, the second computing module includes:

[0058] Perform pixel attention calculation for the image to be corrected, and generate an attention value matrix for each pixel on the image to be corrected;

[0059] According to the distribution pattern of non-bad pixels within the neighborhood pixel search range and the attention value matrix, the attention coefficient of non-bad pixels within the neighborhood pixel search range is calculated;

[0060] The gray value difference weight set is calculated based on the attention coefficient of the non-bad pixel, the pixel value of the non-bad pixel in the neighborhood pixel search range, and the mean of the non-bad pixel.

[0061] Optionally, after the second generating unit and before the first calculating unit, the device further includes:

[0062] An updating unit, used for updating the initial search distance and regenerating the neighborhood pixel search range when there is no point information in the k-th level bad pixel set, so as to expand the neighborhood pixel search range;

[0063] The second statistical unit is used to re-count the number of non-bad pixel values ​​and non-bad pixel coordinate value sets of each bad pixel in the bad pixel coordinate set within the neighborhood pixel search range;

[0064] The screening unit is used to re-screen the bad pixel coordinate set according to the target number threshold and the non-bad pixel pixel value set, and record the points that meet the requirements into the k-th level bad pixel set.

[0065] A third aspect of the present application provides a camera sensor bad pixel correction system, comprising:

[0066] Processor, memory, input-output unit, and bus;

[0067] The processor is connected to the memory, the input and output unit, and the bus;

[0068] The memory stores a program, and the processor calls the program to execute the first aspect and any optional method of the first aspect.

[0069] A fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the program executes the first aspect and any optional processing method of the first aspect.

[0070] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0071] In the present application, the camera sensor is firstly calibrated and tested for bad points, and the bad point coordinate set in the camera sensor is determined. The purpose of predetermining the bad points is to correct the bad points. With each bad point in the bad point coordinate set as the center, the initial search distance is set, and the neighborhood pixel search range is generated. An exclusive neighborhood pixel search range is set around each bad point. There may be other bad points and non-bad points (normal points) in each neighborhood pixel search range. The number of non-bad point pixel values ​​and the number of non-bad point pixel coordinate values ​​of each bad point in the neighborhood pixel search range in the bad point coordinate set are counted, that is, the number of normal points in the neighborhood pixel search range for each bad point is determined, and the coordinate position corresponding to the normal point. The target number threshold of normal pixels in the neighborhood pixel search range is set, and the bad point coordinate set is screened according to the target number threshold and the number of non-bad point pixel values, and the k-th level bad point set is generated. The bad points of the k-th level bad point set are the k-th layer bad points. That is, the first level of bad points are first generated and determined. These bad points have N normal points in the neighborhood pixel search range, and N is greater than or equal to the target number threshold. The spatial distance weight set of each bad point in the k-th bad point set is calculated based on the non-bad point pixel coordinate value set, that is, the spatial distance weights of these normal points and the current bad point are determined based on the normal points and the current bad point, and the spatial distance weight of each bad point is determined accordingly. The gray value difference weight set of each bad point in the k-th bad point set is calculated based on the image to be corrected, that is, the pixel values ​​of the surrounding pixels are used to determine which normal points the bad point has the greatest correlation with. A combined weight set is generated based on the spatial distance weight set and the gray value difference weight set, that is, the weight of each bad point to the normal point within the search range of each neighborhood pixel is calculated based on the comprehensive distance and pixel situation. Then the terminal performs grayscale interpolation processing on the bad points in the k-th bad point set on the image to be corrected based on the combined weight set, that is, the bad points are corrected based on the weights. After the grayscale interpolation of the first-level bad pixels is completed, the remaining bad pixels that have not been corrected are counted, and the k+1-level bad pixel set is determined from the bad pixels that have not been corrected according to the target number threshold and the neighborhood pixel search range. The bad pixels in the k+1-level bad pixel set are the k+1-level bad pixels. The combined weight set is calculated for the second-level bad pixel set, and the grayscale interpolation processing is performed on the image to be corrected according to the combined weight set until all the bad pixels on the image to be corrected are interpolated. By automatically and concurrently dividing the bad pixel levels, the bad pixels are corrected layer by layer from the outside to the inside, and then the spatial position and grayscale value difference of the normal pixels in the bad pixel neighborhood pixel search range are combined to perform adaptive weight interpolation correction. In the face of scenes with multiple consecutive bad pixels and complex backgrounds, the bad pixels of the camera sensor itself can also be effectively corrected, greatly improving the bad pixel correction effect of the camera sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0073] Figure 1 A schematic diagram of an embodiment of a method for correcting bad pixels of a camera sensor of the present application;

[0074] Figure 2 This is a schematic diagram of an embodiment of the first stage of the method for calculating a spatial distance weight set according to a set of non-bad pixel coordinate values ​​of the present application;

[0075] Figure 3 This is a schematic diagram of another embodiment of the method for calculating a spatial distance weight set according to a non-bad pixel coordinate value set of the present application;

[0076] Figure 4 This is a schematic diagram of an embodiment of a method for calculating a gray value difference weight set according to an image to be corrected in the present application;

[0077] Figure 5 This is a schematic diagram of another embodiment of the method for calculating a gray value difference weight set according to an image to be corrected in the present application;

[0078] Figure 6 A schematic diagram of an embodiment of a method for re-determining a bad pixel set in the present application;

[0079] Figure 7 A schematic diagram of an embodiment of a device for correcting bad pixels of a camera sensor of the present application;

[0080] Figure 8 A schematic diagram of an embodiment of a system for correcting bad pixels of a camera sensor of the present application. DETAILED DESCRIPTION

[0081] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0082] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0083] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0084] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0085] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0086] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0087] In the prior art, existing methods for pre-correcting bad pixels are mostly based on filtering or interpolation methods, such as first using a camera to shoot, determining the location of the bad pixel, and using the mean or median of the neighboring pixels of these bad pixels to replace the pixel value of the bad pixel location, or using the neighboring pixels to estimate the pixel value of the bad pixel location through interpolation. The bad pixel correction quality of the camera sensor is evaluated based on the effect of pixel value replacement.

[0088] The above correction method can achieve good correction effect for isolated bad pixels and single background scenes, but its correction effect will be greatly affected in some scenes. For example, since filtering will lose image edge detail information and traditional interpolation methods cannot fully utilize local image information, when facing scenes with continuous bad pixels and complex backgrounds, no matter what filtering method or interpolation method is used, the correction effect of the bad pixels of the camera sensor is not ideal, resulting in a decrease in the correction effect of the bad pixels of the camera sensor.

[0089] Based on this, the present application discloses a method, device, system and storage medium for correcting bad pixels of a camera sensor, which are used to improve the bad pixel correction effect of the camera sensor.

[0090] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0091] The method of the present application can be applied to a server, a device, a terminal or other devices with logic processing capabilities, and the present application does not limit this. For the convenience of description, the following description is made by taking the execution subject as an example of a terminal.

[0092] See also Figure 1 The present application provides an embodiment of a method for correcting bad pixels of a camera sensor, comprising:

[0093] 101. Perform bad pixel calibration detection on the camera sensor to determine a bad pixel coordinate set in the camera sensor;

[0094] In this embodiment, the terminal performs bad pixel calibration detection on the camera sensor to determine the bad pixel coordinate set in the camera sensor. In practical applications, the camera sensor can be pre-calibrated with relevant bad pixel detection methods (such as statistical analysis, image threshold segmentation, etc.) to determine the bad pixel coordinates in the sensor and generate a bad pixel correction data set (bad pixel coordinate set) for use in bad pixel correction;

[0095] 102. Taking each bad point in the bad point coordinate set as the center, setting an initial search distance and generating a neighborhood pixel search range;

[0096] The terminal takes the bad pixel as the center and sets the initial search distance s. The neighborhood pixel search range indicates that the search range of the bad pixel neighborhood pixels is (2*s+1)×(2*s+1). If s=1 is set, the search range of the bad pixel neighborhood pixels is 3×3. For images of different resolutions, the search distances used are different.

[0097] 103. Count the number of non-bad pixel values ​​and the non-bad pixel coordinate value set for each bad pixel in the bad pixel coordinate set within the neighborhood pixel search range;

[0098] The terminal counts the number of non-bad pixel pixels within the neighborhood pixel search range (that is, the number of normal pixel points in the bad pixel neighborhood range) and the corresponding non-bad pixel coordinate set according to the neighborhood pixel search range. Then, for each bad pixel, the number of normal pixels within the neighborhood pixel search range and the normal pixel coordinate set within the neighborhood range are obtained. The non-bad pixel value set and the non-bad pixel coordinate value set are recorded as NP and NP_CoSet respectively. For example, if the number of bad pixels is 6000, two sets of values ​​will be obtained. The non-bad pixel value set is recorded as {NPn|n=0,1,2,3,...,5998,5999}, and the non-bad pixel coordinate value set is recorded as {NP_CoSetn|n=0,1,2,3,...,5998,5999}.

[0099] 104. Set a target number threshold of normal pixels within the neighborhood pixel search range, perform data screening on the bad pixel coordinate set according to the target number threshold and the non-bad pixel pixel value set, and generate a k-th level bad pixel set, where the bad pixels in the k-th level bad pixel set are the k-th layer bad pixels;

[0100] The terminal sets the normal pixel number threshold n_th (target number threshold) within the bad pixel neighborhood. For example, if n_th=5 is set, it means that there must be at least 5 normal pixels within the bad pixel neighborhood for subsequent calculation of pixel values ​​at the bad pixel. The specific format needs to be determined according to the resolution of the camera and image.

[0101] The terminal uses the normal pixel number threshold n_th to filter the non-bad pixel value set {NPn|n=0,1,2,3,...,5998,5999}, and filters out the data that is greater than or equal to the normal pixel number threshold n_th. The data index that meets the threshold condition is represented by a set, which is recorded as the k-th level bad pixel set SE_IND={s0,s1,s2,...,sn-1,sn}, where n≤5999.

[0102] If SE_IND!=Null, it means that there is a point in the neighborhood that meets the normal pixel number threshold n_th. The bad pixels that meet the threshold condition can be recorded as bad pixels in the kth layer, and k is initialized to 1.

[0103] 105. Calculate a spatial distance weight set of each bad pixel in the k-th level bad pixel set according to the non-bad pixel pixel coordinate value set;

[0104] 106. Calculate a gray value difference weight set of each bad pixel in the k-th level bad pixel set according to the image to be corrected;

[0105] In this embodiment, if SE_IND!=Null, it means that there are points that meet the normal pixel number threshold n_th in the neighborhood range. The bad pixels that meet the threshold condition can be recorded as bad pixels of the kth layer, and k is initialized to 1. The terminal calculates the spatial distance weight set of each bad pixel in the kth level bad pixel set based on the non-bad pixel pixel coordinate value set. Specifically, the terminal needs to perform adaptive weight interpolation correction on the kth level bad pixels at this time. First, it is necessary to calculate the spatial distance weight set of each bad pixel in the kth level bad pixel set based on the non-bad pixel pixel coordinate value set, and then calculate the gray value difference weight set of each bad pixel in the kth level bad pixel set based on the image to be corrected.

[0106] 107. Generate a combined weight set according to the spatial distance weight set and the gray value difference weight set;

[0107] The terminal generates a combined weight set based on the spatial distance weight set and the gray value difference weight set, both of which are weight values. Specifically, the two can be multiplied.

[0108] 108. Perform grayscale interpolation processing on bad pixels in the k-th bad pixel set on the image to be corrected according to the combined weight set;

[0109] The terminal performs grayscale interpolation processing on the bad pixels in the k-th level bad pixel set on the image to be corrected according to the combined weight set, thereby completing the correction of the bad pixels at this level.

[0110] 109. Count the remaining bad pixels that have not been corrected, and determine the k+1th level bad pixel set from the bad pixels that have not been corrected according to the target number threshold and the neighborhood pixel search range. The bad pixels in the k+1th level bad pixel set are the k+1th layer bad pixels.

[0111] The terminal counts the remaining bad pixels that have not been corrected, and determines the k+1th level bad pixel set from the bad pixels that have not been corrected according to the target number threshold and the neighborhood pixel search range. Because one level of bad pixels has been repaired, the remaining bad pixels are detected again at this time. The corrected bad pixels can be used as normal points to participate in the correction of the remaining bad pixels, or the corrected bad pixels can be not corrected as normal points.

[0112] 110. Calculate a combination weight set for the second level bad pixel set, and perform grayscale interpolation processing on the image to be corrected according to the combination weight set until all bad pixels on the image to be corrected are interpolated.

[0113] The terminal calculates a combination weight set for the second-level bad pixel set, and performs grayscale interpolation processing on the image to be corrected according to the combination weight set until all bad pixels on the image to be corrected are interpolated.

[0114] In this embodiment, the camera sensor is firstly calibrated and tested for bad points to determine the bad point coordinate set in the camera sensor. The purpose of predetermining the bad points is to correct the bad points. With each bad point in the bad point coordinate set as the center, an initial search distance is set to generate a neighborhood pixel search range, and an exclusive neighborhood pixel search range is set around each bad point. Each neighborhood pixel search range may have other bad points and non-bad points (normal points). The number of non-bad point pixel values ​​and the number of non-bad point pixel coordinate values ​​of each bad point in the neighborhood pixel search range are counted, that is, the number of normal points in the neighborhood pixel search range for each bad point in the bad point coordinate set is determined, and the coordinate position corresponding to the normal point is determined. The target number threshold of normal pixels in the neighborhood pixel search range is set, and the bad point coordinate set is screened according to the target number threshold and the number of non-bad point pixel values ​​to generate the k-th level bad point set, and the bad points of the k-th level bad point set are the k-th layer bad points. That is, the first level of bad points are first generated and determined, and these bad points have N normal points in the neighborhood pixel search range, and N is greater than or equal to the target number threshold. The spatial distance weight set of each bad point in the k-th bad point set is calculated based on the non-bad point pixel coordinate value set, that is, the spatial distance weights of these normal points and the current bad point are determined based on the normal points and the current bad point, and the spatial distance weight of each bad point is determined accordingly. The gray value difference weight set of each bad point in the k-th bad point set is calculated based on the image to be corrected, that is, the pixel values ​​of the surrounding pixels are used to determine which normal points the bad point has the greatest correlation with. A combined weight set is generated based on the spatial distance weight set and the gray value difference weight set, that is, the weight of each bad point to the normal point within the search range of each neighborhood pixel is calculated based on the comprehensive distance and pixel situation. Then the terminal performs grayscale interpolation processing on the bad points in the k-th bad point set on the image to be corrected based on the combined weight set, that is, the bad points are corrected based on the weights. After the grayscale interpolation of the first-level bad pixels is completed, the remaining bad pixels that have not been corrected are counted, and the k+1-level bad pixel set is determined from the bad pixels that have not been corrected according to the target number threshold and the neighborhood pixel search range. The bad pixels in the k+1-level bad pixel set are the k+1-level bad pixels. The combined weight set is calculated for the second-level bad pixel set, and the grayscale interpolation processing is performed on the image to be corrected according to the combined weight set until all the bad pixels on the image to be corrected are interpolated. By automatically and concurrently dividing the bad pixel levels, the bad pixels are corrected layer by layer from the outside to the inside, and then the spatial position and grayscale value difference of the normal pixels in the bad pixel neighborhood pixel search range are combined to perform adaptive weight interpolation correction. In the face of scenes with multiple consecutive bad pixels and complex backgrounds, the bad pixels of the camera sensor itself can also be effectively corrected, greatly improving the bad pixel correction effect of the camera sensor.

[0115] Secondly, the terminal automatically divides the bad pixel levels and corrects them layer by layer from the outside to the inside. The core is to suppress the interference of bad pixels in the bad pixel neighborhood on the correction. It has a better correction effect on the situation of multiple consecutive bad pixels, and improves the effectiveness and accuracy of bad pixel correction.

[0116] In addition, adaptive weight interpolation calculates adaptive weights according to the spatial position and grayscale value of normal pixels around the bad pixels, which can correct bad pixels more accurately and better preserve image details while avoiding the blur and distortion that may be caused by traditional interpolation methods.

[0117] Automatic classification of bad pixel level correction and adaptive distance weighting and grayscale weight interpolation have made significant improvements in the correction of bad pixels on display camera sensors. Compared with traditional correction of images taken on display screens, the bad pixel information of the camera sensor itself is used to improve the correction effect.

[0118] See also Figure 2 The present application provides another embodiment of a method for calculating a spatial distance weight set according to a non-bad pixel coordinate value set, including:

[0119] 201. Determine the relative distance between each bad pixel in the k-th level bad pixel set and all normal points in the neighborhood pixel search range according to the non-bad pixel pixel coordinate value set;

[0120] 202. Calculate the spatial distance weights of all normal points and corresponding bad points according to the relative distances, and generate a set of spatial distance weights.

[0121] In this embodiment, the spatial distance weight is first calculated. Specifically, the distance between normal pixels and bad pixels in the neighborhood is considered using the Inverse Distance Weighting (IDW) method. The farther the distance, the smaller the weight. One calculation formula is as follows:

[0122]

[0123] Among them, Dist_weighti represents the spatial distance weight of the i-th normal pixel in the neighborhood. i Indicates the distance between the i-th normal pixel and the bad pixel in the neighborhood. p is a power parameter (usually p≥1), which controls the weight of the spatial distance. A larger p value will make the farther away points have less influence on the interpolation point. In this embodiment, p=2 can be taken here. This calculation method is relatively simple, with a small amount of calculation, and the obtained spatial distance weight is also relatively appropriate.

[0124] See also Figure 3 The present application provides another embodiment of a method for calculating a spatial distance weight set according to a non-bad pixel coordinate value set, including:

[0125] 301. Determine the relative distance between each bad pixel in the k-th level bad pixel set and all normal points in the neighborhood pixel search range according to the non-bad pixel pixel coordinate value set;

[0126] In this embodiment, step 301 is similar to step 201 and will not be described in detail here.

[0127] 302. Calculate the distance weight adjustment coefficient of the non-bad points in the neighborhood pixel search range according to the distribution pattern of the non-bad points in the neighborhood pixel search range and the size of the neighborhood pixel search range;

[0128] The terminal obtains the distribution pattern of non-bad points in the neighborhood pixel search range and the size of the neighborhood pixel search range, that is, obtains the size of the current initial search distance or the size of the neighborhood pixel search range, and then obtains the adjacent connection relationship of the normal points therein, and calculates the distance weight adjustment coefficient of each normal point in the neighborhood pixel search range. The formula is as follows:

[0129]

[0130] in, is the distance weight adjustment coefficient of the i-th point in the neighborhood pixel search range, is the number of pixels of the ith point connected to other normal points in the neighborhood pixel search range. For example, in a 3*3 neighborhood pixel search range, if there is one bad pixel, then every normal point is connected to form a whole, A is 8. If there are two bad pixels, the normal points are split into 5 connected normal points and 2 connected normal points. Then the A of these 5 normal points is 5, and the A of these two normal points is 2. is the number of normal points adjacent to the ith normal point in the neighborhood pixel search range. Each pixel has 4 points in four directions, and the number of normal points of these adjacent points is determined The size of , s is the initial search distance, and s may change with detection.

[0131] 303. Calculate the spatial distance weights of all normal points and corresponding bad points according to the distance weight adjustment coefficient of the non-bad points and the relative distance, and generate a set of spatial distance weights.

[0132] The terminal calculates the spatial distance weights of all normal points and corresponding bad points according to the distance weight adjustment coefficient of the non-bad point and the relative distance, and generates a set of spatial distance weights. In this embodiment, the terminal Figure 2Based on the embodiment, the concept of distance weight adjustment coefficient is introduced. The distance between normal pixels and bad pixels in the neighborhood is considered using the inverse distance weighting (IDW). The farther the distance, the smaller the weight. In addition, the distribution relationship of each normal point in the neighborhood pixel search range is considered. One of the calculation formulas is as follows:

[0133]

[0134] Among them, Dist_weighti represents the spatial distance weight of the i-th normal pixel in the neighborhood. i Indicates the distance between the i-th normal pixel and the bad pixel in the neighborhood. p is a power parameter (usually p≥1), which controls the weight of the spatial distance. A larger p value will make the farther away points have less influence on the interpolation point. In this embodiment, p=2 can be taken here. Although this calculation method increases the amount of calculation, the obtained spatial distance weight is relatively accurate in practical applications.

[0135] See also Figure 4 The present application provides another embodiment of a method for calculating a grayscale value difference weight set according to an image to be corrected, comprising:

[0136] 401. Determine the pixel values ​​of non-bad pixels in a neighborhood pixel search range on the image to be corrected according to a set of non-bad pixel coordinate values;

[0137] 402. Calculate the average value of non-bad pixel pixels in the neighborhood pixel search range according to the pixel values ​​of non-bad pixels in the neighborhood pixel search range;

[0138] 403. Calculate a gray value difference weight set according to the pixel value of the non-bad pixel in the neighborhood pixel search range and the mean value of the non-bad pixel.

[0139] In this embodiment, the terminal determines the pixel values ​​of the non-bad points in the neighborhood pixel search range on the image to be corrected based on the non-bad point pixel coordinate value set. Then, the mean value of the non-bad point pixels in the neighborhood pixel search range is calculated based on the pixel values ​​of the non-bad points in the neighborhood pixel search range. Finally, the gray value difference weight set is calculated based on the pixel values ​​of the non-bad points in the neighborhood pixel search range and the mean value of the non-bad point pixels.

[0140] Specifically, the gray value difference weight is calculated, taking into account the gray value difference of the neighboring pixels. The greater the gray value difference, the smaller the weight. One calculation formula is as follows:

[0141]

[0142] Among them, GrayDiff_weighti represents the gray value difference weight of the i-th normal pixel in the neighborhood.i G represents the pixel value of the i-th normal pixel in the image to be corrected within the neighborhood. m It represents the pixel mean of normal pixels in the neighborhood in the image to be corrected. σ is a very small offset to avoid the denominator being zero;

[0143] When using Figure 2 and Figure 4 When calculating the spatial distance weight and gray value difference weight, it is necessary to combine the spatial distance weight and the gray value difference weight and perform normalization to ensure that the sum of the weights is equal to 1. The calculation formula is as follows:

[0144]

[0145]

[0146] Among them, Combine_weighti represents the combined weight of the i-th normal pixel in the neighborhood. Norm_weighti represents the normalized weight of the i-th normal pixel in the neighborhood. NP is the number of normal pixels in the neighborhood.

[0147] For the subsequent interpolation calculation, each bad pixel in the corrected image or camera sensor is interpolated, and the pixel value of the bad pixel after interpolation is calculated based on the pixel value of the normal pixel in the neighborhood and the normalized weight. The calculation formula is as follows:

[0148]

[0149] Among them, Gray_BP represents the pixel value of the bad pixel after adaptive interpolation correction.

[0150] After completing the bad pixel correction of the kth level, count the number of bad pixels that have not been corrected, and record it as: NO_SE_Nums. If NO_SE_Nums>0, it means that there are bad pixels in the inner layer that have not been corrected. You need to increase k by 1, and then repeat steps 102 to 108 until NO_SE_Nums=0, that is, all bad pixels have been corrected. If NO_SE_Nums=0, it means that all bad pixels have been corrected, and the bad pixel correction process ends.

[0151] Finally, it is worth noting that since the above processing flow for each bad pixel is consistent, the efficiency of bad pixel correction can be greatly improved through parallel computing technology based on the idea of ​​engineering concurrency.

[0152] See also Figure 5 The present application provides another embodiment of a method for calculating a grayscale value difference weight set according to an image to be corrected, comprising:

[0153] 501. Determine the pixel values ​​of non-bad pixels in a neighborhood pixel search range on the image to be corrected according to a set of non-bad pixel coordinate values;

[0154] 502. Calculate the average value of non-bad pixel pixels in the neighborhood pixel search range according to the pixel values ​​of non-bad pixels in the neighborhood pixel search range;

[0155] In this embodiment, step 501 and step 502 are similar to the aforementioned step 401 and step 402, and are not described in detail here.

[0156] 503. Perform pixel attention calculation for the image to be corrected, and generate an attention value matrix for each pixel on the image to be corrected;

[0157] The terminal first performs pixel attention calculation for the image to be corrected, and generates an attention value matrix for each pixel on the image to be corrected. The attention matrix represents the importance or relevance of each pixel in the overall image to be corrected, and can be calculated in a variety of ways.

[0158] 504. Calculate the attention coefficient of the non-bad pixels in the neighborhood pixel search range according to the distribution pattern of the non-bad pixels in the neighborhood pixel search range and the attention value matrix;

[0159] The terminal calculates the attention coefficient of the non-bad pixels in the neighborhood pixel search range according to the distribution pattern of the non-bad pixels in the neighborhood pixel search range and the attention value matrix. The formula is as follows:

[0160]

[0161] in, is the attention coefficient of the i-th point in the neighborhood pixel search range, is the number of pixels of the ith point connected to other normal points within the neighborhood pixel search range, and its meaning is the same as step 302. It can be seen that the larger the whole connected between normal pixels, the greater the attention coefficient of the normal point. is the attention value of the i-th normal point.

[0162] 505. Calculate a gray value difference weight set according to the attention coefficient of the non-bad pixel, the pixel value of the non-bad pixel in the neighborhood pixel search range, and the mean value of the non-bad pixel.

[0163] The terminal calculates the grayscale value difference weight set according to the attention coefficient of the non-bad pixel, the pixel value of the non-bad pixel in the neighborhood pixel search range, and the mean value of the non-bad pixel. Figure 4 Based on the embodiment, the concept of attention coefficient is introduced. Not only the greater the difference in grayscale value, the smaller the weight, but also it needs to be associated with the overall distribution of normal points around it. The grayscale is as close to the normal points with more overall connections as possible. At this time, the formula is as follows:

[0164]

[0165] The addition of the attention coefficient takes into account both the grayscale difference and the distribution of normal points around bad points. The overall normal points have a higher weight, which further improves the accuracy of the weight.

[0166] See also Figure 6 The present application provides another embodiment of a method for re-determining a bad pixel set, comprising:

[0167] 601. When there is no point information in the k-th level bad pixel set, the initial search distance is updated, and the neighborhood pixel search range is regenerated to expand the neighborhood pixel search range;

[0168] 602. Re-count the number of non-bad pixel values ​​and the non-bad pixel coordinate value set for each bad pixel in the bad pixel coordinate set within the neighborhood pixel search range;

[0169] 603. Re-screen the bad pixel coordinate set according to the target quantity threshold and the non-bad pixel pixel value set, and record the points that meet the requirements into the k-th level bad pixel set.

[0170] When there is no point information in the k-th level bad point set, the terminal updates the initial search distance and regenerates the neighborhood pixel search range to expand the neighborhood pixel search range. Re-count the non-bad point pixel value set and non-bad point pixel coordinate value set of each bad point in the bad point coordinate set within the neighborhood pixel search range. Re-screen the bad point coordinate set according to the target number threshold and the non-bad point pixel value set, and record the points that meet the requirements into the k-th level bad point set.

[0171] If the current k-th level bad pixel set SE_IND=Null, it means that there is no bad pixel that meets the normal pixel quantity threshold n_th in the neighborhood range. The initial search distance s needs to be increased by at least 1 pixel, and the target quantity threshold can be increased or unchanged accordingly, and then steps 101, 102, 103, and 104 are repeated.

[0172] See also Figure 7 The present application provides an embodiment of a device for correcting bad pixels of a camera sensor, comprising:

[0173] The first determining unit 701 is used to perform bad pixel calibration detection on the camera sensor and determine a bad pixel coordinate set in the camera sensor;

[0174] The first generating unit 702 is used to set an initial search distance and generate a neighborhood pixel search range with each bad point in the bad point coordinate set as the center;

[0175] The first statistical unit 703 is used to count the number of non-bad pixel values ​​and non-bad pixel coordinate value sets of each bad pixel in the bad pixel coordinate set within the neighborhood pixel search range;

[0176] The second generating unit 704 is used to set a target number threshold of normal pixels within the neighborhood pixel search range, and to perform data screening on the bad pixel coordinate set according to the target number threshold and the non-bad pixel pixel value set to generate a k-th level bad pixel set, where the bad pixels in the k-th level bad pixel set are the k-th layer bad pixels;

[0177] An updating unit 705 is used to update the initial search distance and regenerate the neighborhood pixel search range when there is no point information in the k-th level bad pixel set, so that the neighborhood pixel search range is expanded;

[0178] The second statistical unit 706 is used to re-count the number of non-bad pixel values ​​and non-bad pixel coordinate value sets of each bad pixel in the bad pixel coordinate set within the neighborhood pixel search range;

[0179] The screening unit 707 is used to screen the bad pixel coordinate set again according to the target number threshold and the non-bad pixel pixel value set, and record the points that meet the requirements into the k-th level bad pixel set;

[0180] The first calculation unit 708 is used to calculate the spatial distance weight set of each bad pixel in the k-th level bad pixel set according to the non-bad pixel pixel coordinate value set;

[0181] Optionally, the first computing unit 708 includes:

[0182] The first determination module 7081 is used to determine the relative distance between each bad pixel in the k-th level bad pixel set and all normal points in the neighborhood pixel search range according to the non-bad pixel pixel coordinate value set;

[0183] The generation module 7082 is used to calculate the spatial distance weights of all normal points and corresponding bad points according to the relative distances, and generate a set of spatial distance weights.

[0184] Optionally, the generation module 7082 includes:

[0185] According to the distribution pattern of non-bad points in the neighborhood pixel search range and the size of the neighborhood pixel search range, the distance weight adjustment coefficient of the non-bad points in the neighborhood pixel search range is calculated;

[0186] The spatial distance weights of all normal points and corresponding bad points are calculated according to the distance weight adjustment coefficient and relative distance of non-bad points to generate a set of spatial distance weights.

[0187] The second calculation unit 709 is used to calculate the gray value difference weight set of each bad pixel in the k-th level bad pixel set according to the image to be corrected;

[0188] Optionally, the second calculation unit 709 includes:

[0189] The first determination module 7091 is used to determine the pixel values ​​of non-bad pixels in the neighborhood pixel search range on the image to be corrected according to the non-bad pixel coordinate value set;

[0190] The first calculation module 7092 is used to calculate the non-bad pixel pixel mean value in the neighborhood pixel search range according to the pixel values ​​of the non-bad pixels in the neighborhood pixel search range;

[0191] The second calculation module 7093 is used to calculate the gray value difference weight set according to the pixel value of the non-bad pixel and the mean value of the non-bad pixel in the neighborhood pixel search range.

[0192] Optionally, the second calculation module 7093 includes:

[0193] Perform pixel attention calculation for the image to be corrected, and generate an attention value matrix for each pixel on the image to be corrected;

[0194] According to the distribution pattern of non-bad pixels within the neighborhood pixel search range and the attention value matrix, the attention coefficient of non-bad pixels within the neighborhood pixel search range is calculated;

[0195] The gray value difference weight set is calculated based on the attention coefficient of the non-bad pixel, the pixel value of the non-bad pixel in the neighborhood pixel search range, and the mean of the non-bad pixel.

[0196] The third generating unit 710 is used to generate a combined weight set according to the spatial distance weight set and the gray value difference weight set;

[0197] The first processing unit 711 is used to perform grayscale interpolation processing on bad pixels in the k-th bad pixel set on the image to be corrected according to the combined weight set;

[0198] The second determining unit 712 is used to count the remaining bad pixels that have not been corrected, and determine the k+1th level bad pixel set from the bad pixels that have not been corrected according to the target number threshold and the neighborhood pixel search range, and the bad pixels in the k+1th level bad pixel set are the k+1th layer bad pixels;

[0199] The second processing unit 713 is used to calculate a combination weight set for the second level bad pixel set, and perform grayscale interpolation processing on the image to be corrected according to the combination weight set until all bad pixels on the image to be corrected are interpolated.

[0200] See also Figure 8 The present application provides a camera sensor bad pixel correction system, comprising:

[0201] Processor 801 , memory 802 , input-output unit 803 , and bus 804 .

[0202] The processor 801 is connected to the memory 802 , the input and output unit 803 , and the bus 804 .

[0203] The memory 802 stores a program, and the processor 601 calls the program to execute the following steps: Figure 1 , Figure 2 and Figure 3 , Figure 4 , Figure 5 and Figure 6 The method in .

[0204] The present application provides a computer-readable storage medium, on which a program is stored, and when the program is executed on a computer, the program performs the following steps: Figure 1 , Figure 2 and Figure 3 , Figure 4 , Figure 5 and Figure 6 The method in .

[0205] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0206] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0207] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0208] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0209] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A method for correcting bad pixels of a camera sensor, characterized in that: include: Perform bad pixel calibration detection on the camera sensor to determine a bad pixel coordinate set in the camera sensor; Taking each bad point in the bad point coordinate set as the center, setting an initial search distance and generating a neighborhood pixel search range; Counting the number of non-bad pixel values ​​and the non-bad pixel coordinate value sets of each bad pixel in the bad pixel coordinate set within the neighborhood pixel search range; Setting a target number threshold of normal pixels within the neighborhood pixel search range, performing data screening on the bad pixel coordinate set according to the target number threshold and the non-bad pixel pixel value set, and generating a k-th level bad pixel set, wherein the bad pixels of the k-th level bad pixel set are the k-th layer bad pixels; Calculate a spatial distance weight set of each bad pixel in the k-th level bad pixel set according to the non-bad pixel pixel coordinate value set; Calculate a gray value difference weight set of each bad pixel in the k-th bad pixel set according to the image to be corrected; Generate a combined weight set according to the spatial distance weight set and the gray value difference weight set; Performing grayscale interpolation processing on bad pixels in the k-th level bad pixel set on the image to be corrected according to the combined weight set; Count the remaining bad pixels that have not been corrected, and determine a k+1-th level bad pixel set from the bad pixels that have not been corrected according to the target number threshold and the neighborhood pixel search range, where the bad pixels in the k+1-th level bad pixel set are the k+1-th layer bad pixels; The combination weight set is calculated for the second level bad pixel set, and grayscale interpolation processing is performed on the image to be corrected according to the combination weight set until all bad pixels on the image to be corrected are interpolated.

2. The method according to claim 1, characterized in that Calculating a spatial distance weight set of each bad pixel in the k-th level bad pixel set according to the non-bad pixel pixel coordinate value set, including: Determine the relative distance between each bad pixel in the k-th bad pixel set and all normal points in the neighborhood pixel search range according to the non-bad pixel pixel coordinate value set; The spatial distance weights of all the normal points and the corresponding bad points are calculated according to the relative distances to generate a set of spatial distance weights.

3. The method according to claim 2, characterized in that Calculating the spatial distance weights of all the normal points and the corresponding bad points according to the relative distances, and generating a set of spatial distance weights, including: Calculating a distance weight adjustment coefficient of the non-bad pixels within the neighborhood pixel search range according to a distribution pattern of the non-bad pixels within the neighborhood pixel search range and a size of the neighborhood pixel search range; The spatial distance weights of all the normal points and the corresponding bad points are calculated according to the distance weight adjustment coefficient of the non-bad point and the relative distance to generate a spatial distance weight set.

4. The method according to claim 1, characterized in that Calculating a grayscale value difference weight set of each bad pixel in the k-th level bad pixel set according to the image to be corrected, including: Determine the pixel values ​​of the non-bad pixels in the neighborhood pixel search range on the image to be corrected according to the non-bad pixel coordinate value set; Calculate the non-bad pixel mean value of the neighborhood pixel search range according to the pixel values ​​of the non-bad pixels in the neighborhood pixel search range; A grayscale value difference weight set is calculated according to the pixel value of the non-bad pixel in the neighborhood pixel search range and the mean value of the non-bad pixel.

5. The method according to claim 4, characterized in that Calculating a grayscale value difference weight set according to the pixel value of the non-bad pixel in the neighborhood pixel search range and the mean value of the non-bad pixel, including: Perform pixel attention calculation for the image to be corrected, and generate an attention value matrix for each pixel on the image to be corrected; Calculate the attention coefficient of the non-bad pixels within the neighborhood pixel search range according to the distribution pattern of the non-bad pixels within the neighborhood pixel search range and the attention value matrix; A grayscale value difference weight set is calculated according to the attention coefficient of the non-bad pixel, the pixel value of the non-bad pixel in the neighborhood pixel search range, and the mean value of the non-bad pixel.

6. The method according to any one of claims 1 to 5, characterized in that After setting a target number threshold of normal pixels within the neighborhood pixel search range, performing data screening on the bad pixel coordinate set according to the target number threshold and the non-bad pixel pixel value set to generate a k-th level bad pixel set, and before calculating a spatial distance weight set of each bad pixel in the k-th level bad pixel set according to the non-bad pixel pixel coordinate value set, the method further includes: When the k-th level bad pixel set does not have point information, the initial search distance is updated and the neighborhood pixel search range is regenerated to expand the neighborhood pixel search range; Re-count the number of non-bad pixel values ​​and non-bad pixel coordinate value sets of each bad pixel in the bad pixel coordinate set within the neighborhood pixel search range; The bad pixel coordinate set is re-screened according to the target quantity threshold and the non-bad pixel pixel value set, and the qualified points are recorded into the k-th level bad pixel set.

7. A device for correcting bad pixels of a camera sensor, characterized in that: include: A first determining unit, configured to perform bad pixel calibration detection on the camera sensor and determine a bad pixel coordinate set in the camera sensor; A first generating unit is used to set an initial search distance and generate a neighborhood pixel search range with each bad point in the bad point coordinate set as the center; A first statistical unit is used to count the number of non-bad pixel values ​​and non-bad pixel coordinate value sets of each bad pixel in the bad pixel coordinate set within the neighborhood pixel search range; The second generating unit is used to set a target number threshold of normal pixels within the neighborhood pixel search range, perform data screening on the bad pixel coordinate set according to the target number threshold and the non-bad pixel pixel value set, and generate a k-th level bad pixel set, where the bad pixels of the k-th level bad pixel set are the k-th layer bad pixels; A first calculation unit, configured to calculate a spatial distance weight set of each bad pixel in the k-th level bad pixel set according to the non-bad pixel pixel coordinate value set; A second calculation unit is used to calculate a gray value difference weight set of each bad pixel in the k-th level bad pixel set according to the image to be corrected; A third generating unit, configured to generate a combined weight set according to the spatial distance weight set and the gray value difference weight set; A first processing unit, configured to perform grayscale interpolation processing on bad pixels in the k-th bad pixel set on the image to be corrected according to the combined weight set; A second determining unit is used to count the remaining bad pixels that have not been corrected, and determine a k+1-th level bad pixel set from the bad pixels that have not been corrected according to the target number threshold and the neighborhood pixel search range, where the bad pixels in the k+1-th level bad pixel set are the bad pixels of the k+1-th layer; The second processing unit is used to calculate the combination weight set for the second-level bad pixel set, and perform grayscale interpolation processing on the image to be corrected according to the combination weight set until all the bad pixels on the image to be corrected are interpolated.

8. The device according to claim 7, characterized in that The first computing unit comprises: A first determination module is used to determine the relative distance between each bad pixel in the k-th level bad pixel set and all normal points in the neighborhood pixel search range according to the non-bad pixel pixel coordinate value set; A generation module is used to calculate the spatial distance weights of all the normal points and the corresponding bad points according to the relative distances, and generate a set of spatial distance weights.

9. A camera sensor bad pixel correction system, characterized in that: include: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a program stored thereon, wherein the program, when executed on a computer, performs the method according to any one of claims 1 to 6.

Images