Infrared image dead pixel removing method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of image processing, and discloses an infrared image dead pixel removing method and device, electronic equipment and a storage medium, and the method comprises the steps: calculating a pixel difference value and a pixel sum value between each pixel point in a pre-obtained infrared image and a preset neighborhood pixel point; determining a communication state between the pixel point and the neighborhood pixel point according to the pixel difference value and the pixel sum value; determining a defective pixel category according to the communication state; and carrying out pixel value compensation according to the type of the dead pixels to obtain an infrared image without the dead pixels. Through the implementation of the invention, the to-be-compensated dead pixel needing to be compensated can be determined, the misjudgment that a normal point is taken as a dead pixel in an extreme scene with a relatively large gradient is avoided, the accuracy of dead pixel judgment is improved, meanwhile, pixel value compensation is performed according to the category of the dead pixel, the pixel value point after pixel value compensation can be closer to a true value of a corresponding scene, and the accuracy of dead pixel judgment is improved. And the accuracy of removing the dead pixels of the infrared image is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device, electronic equipment and storage medium for removing bad pixels from infrared images. Background Art

[0002] Bad pixels (hereinafter referred to as bad pixels) refer to some pixels in the infrared pixel array that cannot reflect the infrared radiation intensity information. The value of the bad pixel cannot be used as the information input of the image, and the algorithm should calculate and compensate it according to the surrounding pixel values. The compensated value replaces the original value. There are two types of bad pixels, one is called static bad pixels and the other is called dynamic bad pixels. The characteristic of static bad pixels is that no matter how the infrared radiation intensity changes, its value is at a low level for a long time, at a high level for a long time, or jumps (flickers) for a long time. The static bad pixels of each frame of the image need to be compensated. The characteristic of dynamic bad pixels is that sometimes its value can reflect the radiation intensity information and its value is available, but sometimes its value cannot reflect the radiation intensity information and needs to be compensated.

[0003] In order to avoid the bad pixels in the neighborhood from affecting the judgment of the current point, the existing bad pixel judgment method uses a group judgment strategy. However, when there is a high-frequency texture in the image, there are many points with large value differences around the high-frequency point. Due to the group judgment, some grouped pixels may be misjudged as bad pixels. At the same time, when compensating for bad pixels, the mean value of the normal points around the bad pixel is directly used for compensation, without considering the gradient relationship of the surrounding pixels. The compensated pixel value may not reflect the true value of the bad pixel, resulting in poor accuracy of bad pixel compensation. Summary of the invention

[0004] The present invention provides an infrared image bad pixel removal method, device, electronic equipment and storage medium, the main purpose of which is to solve the problem of poor accuracy of non-uniformity correction of infrared images provided in the related art.

[0005] To achieve the above-mentioned purpose, the present invention provides a method for removing bad pixels from infrared images, comprising: calculating the pixel difference and pixel sum between each pixel point in a pre-acquired infrared image and a preset neighborhood pixel point; determining the connectivity status between the pixel point and the neighborhood pixel point based on the pixel difference and pixel sum; determining the bad pixel category of the pixel point based on the connectivity status; and performing pixel value compensation on the pixel point based on the bad pixel category to obtain a bad pixel removed infrared image.

[0006] The present invention also provides an infrared image bad pixel removal device, comprising: a pixel difference and pixel sum value calculation module, used to calculate the pixel difference and pixel sum value between each pixel point in a pre-acquired infrared image and a preset neighborhood pixel point; a connectivity status determination module, used to determine the connectivity status between the pixel point and the neighborhood pixel point according to the pixel difference and pixel sum value; a bad pixel category determination module, used to determine the bad pixel category of the pixel point according to the connectivity status; a pixel compensation module, used to compensate the pixel value of the pixel point according to the bad pixel category to obtain a bad pixel removed infrared image.

[0007] The present invention also provides an electronic device, comprising: a memory communicatively connected to at least one processor; wherein the processor is used to execute a computer program stored in the memory; the memory stores a computer program executable by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the above-mentioned method for removing bad pixels from infrared images.

[0008] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any one of the above-mentioned methods for removing bad pixels from infrared images.

[0009] The embodiment of the present invention calculates the pixel difference and pixel sum between each pixel and the preset neighboring pixel, and can determine the connectivity between the pixel and the neighboring pixel according to the pixel difference and pixel sum, so as to classify the pixel more carefully; determine the bad pixel category of the pixel according to the connectivity, and determine the bad pixel to be compensated according to the bad pixel category, so as to avoid misjudging normal points as bad pixels in extreme scenes with large gradients; compensate the pixel value according to the bad pixel category, so as to make the pixel value point after pixel value compensation closer to the real value of the corresponding scene, effectively improve the accuracy of bad pixel removal of infrared image, and obtain a bad pixel infrared image with more accurate bad pixel compensation. Therefore, the infrared image bad pixel removal method, device, electronic device and storage medium proposed by the present invention can solve the problem of poor accuracy of bad pixel removal of infrared image. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of a flow chart of a method for removing bad pixels from an infrared image provided by an embodiment of the present invention;

[0011] Figure 2 A schematic diagram of the structure of a pixel point and preset neighborhood pixel points provided by an embodiment of the present invention;

[0012] Figure 3 A schematic diagram of pixels for determining bad pixels in the prior art provided by an embodiment of the present invention;

[0013] Figure 4A schematic diagram of pixel points within a preset gradient neighborhood provided by an embodiment of the present invention;

[0014] Figure 5 A schematic diagram of the structure of horizontal and vertical bright and dark stripes in an infrared image after pixel value compensation provided by an embodiment of the present invention;

[0015] Figure 6 A schematic diagram of the structure of the stripes at the turning point in the infrared image provided by an embodiment of the present invention;

[0016] Figure 7 A schematic diagram of bad pixels of an infrared image in an extreme scene with a large gradient provided by an embodiment of the present invention;

[0017] Figure 8 A schematic diagram of pixel value compensation for bad pixels in an extreme scene with a large gradient provided by an embodiment of the present invention;

[0018] Fig. 9 A schematic diagram of performing pixel value compensation on two adjacent bad pixels in an infrared image provided by an embodiment of the present invention;

[0019] Fig.10 A schematic diagram of the structure of an infrared image bad pixel removal device provided by an embodiment of the present invention;

[0020] Fig.11 A schematic diagram of the structure of an electronic device for implementing a method for removing bad pixels from infrared images provided by an embodiment of the present invention.

[0021] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0022] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0023] The embodiment of the present application provides a method for removing bad pixels from infrared images, and the execution subject of the method for removing bad pixels from infrared images includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, a method for removing bad pixels from infrared images can be executed by software or hardware installed on a terminal device or a server device. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0024] Reference Figure 1 The figure is a schematic flow chart of a method for removing bad pixels from an infrared image provided by an embodiment of the present invention, which specifically includes the following process:

[0025] S1. Calculate the pixel difference and pixel sum between each pixel point in the pre-acquired infrared image and the preset neighborhood pixel points.

[0026] In one embodiment, the preset neighborhood pixel points are eight pixel points adjacent to the current pixel point, namely, the neighborhood pixel points formed by the eight adjacent pixel points of top, bottom, left, right, upper left, upper right, lower left, and lower right. The specific neighborhood pixel points are as follows: Figure 2 shown.

[0027] Specifically, the pixel difference value represents the difference between the pixel value of the pixel point and the pixel values ​​of each neighboring pixel point, and the pixel sum value is the sum of the pixel values ​​of the pixel point and all the neighboring pixel points.

[0028] Among them, Figure 2 As shown, for the pixel value P of the pixel point coordinate (m,n) m,n , calculate P m,n The difference and sum of the pixel values ​​of the eight adjacent neighborhood pixels respectively.

[0029] Specifically, the pixel difference and pixel sum value may be calculated using preset pixel difference and pixel sum value formulas, which are expressed as follows:

[0030]

[0031] Among them, lr diff (m,n), lr diff (m,n-1), ud diff (m,n), ud diff (m-1,n), lurd diff (m,n), lurd diff (m-1,n-1),ruld diff (m,n),ruld diff (m-1,n+1) represents the pixel difference between the pixel at coordinate (m,n) and the neighboring pixel at coordinates (m,n+1), (m,n-1), (m+1,n), (m-1,n), (m+1,n+1), (m-1,n-1), (m+1,n-1), (m-1,n+1), (m-1,n+1), P m,n , P m,n+1 , P m,n-1 , P m+1,n , P m-1,n , P m+1,n+1 , P m-1,n-1 , Pm+1,n-1 , P m-1,n+1 They represent the pixel values ​​at (m,n), (m,n+1), (m,n-1), (m+1,n), (m-1,n), (m+1,n+1), (m-1,n-1), (m+1,n-1), (m-1,n+1), (m-1,n+1), abs represents the absolute value, lr represents the absolute value, sum (m,n), lr sum (m,n-1), ud sum (m,n), ud sum (m-1,n), lurd sum (m,n), lurd sum (m-1,n-1),ruld sum (m,n),ruld sum (m-1,n+1) represents the pixel sum value at the coordinates (m,n) and its neighboring pixels at the coordinates (m,n+1), (m,n-1), (m+1,n), (m-1,n), (m+1,n+1), (m-1,n-1), (m+1,n-1), and (m-1,n+1).

[0032] Furthermore, for the pixel points at the edge of the infrared image, the infrared image can be expanded using the row and column where the pixel points are located. For example, for the pixel points in the first column and the last column of the infrared image, the length of the infrared image is expanded using the first column and the last column. For the pixel points in the first row and the last row of the infrared image, the width of the infrared image is expanded using the first row and the last row. For the four edge pixel points located at coordinates (1,1), (M,1), (1,N), and (M,N), where M and N represent the length and width of the infrared image respectively, (1,1), (M,1), (1,N), and (M,N) are used to represent the pixel points of the expanded infrared image at coordinates (1-1,1-1), (M+1,1-1), (1-1,N+1), and (M+1,N+1), respectively, so that the pixel value difference between each pixel point and its neighboring pixel points can be calculated.

[0033] S2. Determine the connectivity between the pixel point and neighboring pixel points based on the pixel difference and pixel sum value.

[0034] In one embodiment, the bright spots and dark spots in the infrared image can be determined more accurately through the connectivity state, and the original high-frequency texture in the infrared image is maintained without causing misjudgment.

[0035] In one embodiment, the method of determining the connectivity status between a pixel point and neighboring pixel points based on a pixel difference and a pixel sum value includes: calculating a pixel ratio between a pixel difference and a pixel sum value; comparing the pixel difference with a preset difference threshold, and comparing the pixel ratio with a preset ratio threshold; when the pixel difference is greater than the difference threshold and the pixel ratio is greater than the ratio threshold, the connectivity status between the pixel point and the neighboring pixel points is disconnected; when the pixel difference is less than or equal to the difference threshold or the pixel ratio is less than or equal to the ratio threshold, the connectivity status between the pixel point and the neighboring pixel points is connected.

[0036] In one embodiment, the difference threshold and ratio threshold corresponding to the neighborhood pixel difference and pixel ratio are set to be t diff and t ratio , if the pixel difference is greater than t diff , and the pixel ratio is greater than t ratio , indicating that the pixel point is not connected to the neighboring pixels. If one of them is less than the threshold, it means that the pixel point is connected to the neighboring pixels.

[0037] Specifically, you can use MASK lr2 (m,n) and MASK lr1 (m,n) marks whether the central pixel is connected to the left and right pixels, using MASK ud2 (m,n) and MASK ud1 (m,n) marks whether the central pixel is connected to the upper and lower pixels, using MASK lurd2 (m,n) and MASK lurd1 (m,n) marks whether the central pixel is connected to the upper left and lower right pixels, using MASK ruld2 (m,n) and MASK ruld1 (m,n) marks whether the central pixel is connected to the upper right and lower left pixels. 0 means no connection, 1 means connected to one of the pixels, and 2 means connected to both pixels. The specific judgment process is as follows:

[0038] If lr diff (m,n) <t diff or Then (m,n) is connected to (m,n+1), MASK lr1 (m,n)=1; otherwise, MASK lr1 (m,n)=0.

[0039] If lr diff (m,n-1) <t diff or Then (m,n) is connected to (m,n-1), MASK lr2 (m,n)=MASK lr1(m,n)+1; otherwise, MASK lr2 (m,n)=MASK lr1 (m,n).

[0040] If diff (m,n) <t diff or Then (m,n) is connected to (m+1,n), MASK ud1 (m,n)=1; otherwise, MASK ud1 (m,n)=0.

[0041] If diff (m-1,n) <t diff or Then (m,n) is connected to (m-1,n), MASK ud2 (m,n)=MASK ud1 (m,n)+1; otherwise, MASK ud2 (m,n)=MASK ud1 (m,n).

[0042] If lurd diff (m,n) <t diff or Then (m,n) is connected to (m+1,n+1), MASK lurd1 (m,n)=1; otherwise, MASK lurd1 (m,n)=0.

[0043] If lurd diff (m-1,n-1) <t diff or Then (m,n) is connected to (m-1,n-1), MASK lurd2 (m,n)=MASK lurd1 (m,n)+1; otherwise, MASK lurd2 (m,n)=MASK lurd1 (m,n).

[0044] If ruld diff (m,n) <t diff or Then (m,n) is connected to (m+1,n-1), MASK ruld1 (m,n)=1; otherwise, MASK rukd1 (m,n)=0.

[0045] If ruld diff (m-1,n+1) <t diff or Then (m,n) is connected to (m-1,n+1), MASKruld2 (m,n)=MASK ruld1 (m,n)+1; otherwise, MASK ruld2 (m,n)=MASK ruld1 (m,n).

[0046] Exemplarily, other relationships such as the "gradient" between the current pixel and the neighboring points may be calculated and a threshold may be set or a conditional segmentation threshold may be used to determine the connectivity state.

[0047] S3. Determine the bad pixel category of the pixel according to the connectivity status.

[0048] In one embodiment, the bad pixel categories include two categories, one is the first category of bad pixels in which the pixel point is not connected to any of the eight neighboring pixel points, and the other is the second category of bad pixels in which the pixel point is only connected to one neighboring pixel point. Different bad pixel categories are used to perform more accurate bad pixel compensation.

[0049] Specifically, the method of determining the bad pixel category of a pixel point based on the connectivity status includes: when the connectivity status of the pixel point and its neighboring pixel points is that both are not connected, determining the pixel point as a first type of bad pixel; when the connectivity status of the pixel point and its neighboring pixel points is that only one is connected, determining the pixel point as a second type of bad pixel; and combining the first type of bad pixels and the second type of bad pixels to obtain the bad pixel category of the pixel point, wherein when the connectivity status of the pixel point and its neighboring pixel points is that two or more are connected, the pixel point is a normal point.

[0050] For example, use the following formula to calculate the overall connectivity of pixels:

[0051] MASK(m,N)

[0052] =MASK lr2 (m,n)+MASK ud2 (m,n)+MASK lurd2 (m,n)+MASK ruld2 (m,n)

[0053] Among them, MASK(m,n)=0 means that the pixel is not connected to the eight neighboring pixels, which is defined as the first type of bad pixel; MASK(m,n)=1 means that the pixel is connected to one of the pixels, which is defined as the second type of bad pixel. If MASK(m,n) is other values, it is considered a normal point and no pixel value compensation is performed.

[0054] In one embodiment, there are two types of bad pixels in infrared images, one is called static bad pixels and the other is called dynamic bad pixels. The characteristic of static bad pixels is that no matter how the infrared radiation intensity changes, its value is at a low level for a long time, at a high level for a long time, or jumps (flickers) for a long time, and the static bad pixels of each frame image need to be compensated. The characteristic of dynamic bad pixels is that sometimes its value can reflect the radiation intensity information and its value is usable, but sometimes its value cannot reflect the radiation intensity information and needs to be compensated.

[0055] Among them, in the prior art, the position of the static bad pixel can be determined according to the characteristics of the static bad pixel during the calibration process of the infrared sensor, and then the value of the position of the static bad pixel can be compensated by using an algorithm during the use of the infrared sensor. However, it is impossible to measure in advance when and where the dynamic bad pixel will appear. Therefore, it is only possible to determine which pixels are the bad pixels of the current frame according to the image characteristics frame by frame during the use of the infrared sensor, and then compensate immediately in the current frame.

[0056] Specifically, in the prior art, the current bad pixel correction algorithm is to take multiple sets of uniform irradiation images at different irradiation intensities, find the static bad pixels and record their position information. When using an infrared sensor, the dynamic bad pixel position of the current frame must be found for each frame. After the normal pixel points are corrected, the bad pixels are compensated. At this time, the compensated bad pixels include both the dynamic bad pixels of the current frame and the static bad pixels found in the correction stage.

[0057] Furthermore, in the process of bad pixel correction for infrared images, we first need to take several uniformly irradiated infrared images under two irradiation intensities, and calculate the response value of each pixel in the infrared image under different irradiation intensities, that is, the infrared radiation intensity, which is usually expressed in grayscale values. For example, under the uniform irradiation intensity of T1, n infrared images are taken, and the response value of each infrared image at the pixel coordinate (i, j) is V T1,n (i, j), then the uniform irradiance of T1 is the uniform response value of the pixel at the pixel coordinate (i, j) as shown below:

[0058]

[0059] in, represents the uniform response value of the pixel coordinate (i, j) under the uniform irradiation intensity of T1, n represents the total number of infrared images under the uniform irradiation intensity of T1, V T1,k (i, j) represents the response value of the kth infrared image at the pixel coordinate (i, j) under the uniform irradiation intensity of T1.

[0060] Similarly, under the uniform irradiation intensity of T2, n irradiation images are taken, and the response value of the pixel at the coordinate (i, j) in each image is V T2,n(i, j), the uniform response value of the pixel at the pixel coordinate (i, j) is:

[0061]

[0062] in, represents the uniform response value of the pixel coordinate (i, j) under the uniform irradiation intensity of T2, n represents the total number of infrared images under the uniform irradiation intensity of T2, V T2,k (i, j) represents the response value of the kth infrared image at the pixel coordinate (i, j) under the uniform irradiation intensity of T1.

[0063] Furthermore, the temperature response rate at the pixel coordinate (i, j) and the average temperature response rate of all pixels can be calculated according to the response value of the infrared image at the pixel coordinate (i, j) under different irradiation intensities, as shown in the following formula:

[0064]

[0065] Among them, r i,j represents the temperature response rate, represents the uniform response value of the pixel coordinate (i, j) under the uniform irradiation intensity of T2, represents the uniform response value of the pixel coordinate (i, j) under the uniform irradiation intensity of T1, represents the average temperature response rate, T2 and T1 represent the uniform irradiation intensity, and M and N represent the length and width of the infrared image, respectively.

[0066] Specifically, the static bad pixel in the infrared image is judged as a static bad pixel if the pixel with the coordinates (i, j) meets any one of the following two conditions:

[0067]

[0068] Among them, r i,j represents the temperature response rate, represents the average temperature response rate, T s_low , T s_high Respectively represent the preset first threshold and second threshold.

[0069] The typical values ​​of the first threshold and the second threshold are 0.8 and 1.2, and a marker parameter MASK is introduced. s , if the pixel point with coordinates (i, j) is a static bad pixel, record MASK s (i,j)=0, if the pixel with coordinates (i,j) is not a static bad pixel, record MASK s (i,j)=1.

[0070] In one embodiment, dynamic bad pixels are determined in each frame according to the image content, and for each pixel in the current frame, it is determined whether it is a dynamic bad pixel. The determination of dynamic bad pixels includes the determination of bright spots and the determination of dark spots. Bright spots refer to the current pixel value being significantly higher than the pixel values ​​of surrounding pixels, and dark spots refer to the current pixel value being significantly lower than the pixel values ​​of surrounding pixels.

[0071] Specifically, to determine whether a pixel with coordinates (i, j) is a bright spot, eight surrounding pixels are needed for determination, where V 11 、V 13 、V 31 and V 33 For Group A, V 12 、V 21 、V 23 and V 32 For Group B, V 22 is the pixel point with coordinates (i, j), such as Figure 3 shown.

[0072] Among them, record the four values ​​V of group A 11 、V 13 、V 31 and V 33 They are g1, g2, g3 and g4 respectively, and a marking parameter MASK is introduced d_l_1 :

[0073]

[0074] Among them, T d_l Represents the preset parameters. According to the above formula, if MASK d_l_1 (i,j)=0, then the pixel with coordinates (i,j) is a bright spot among dynamic bad pixels. d_l_1 (i,j)=1, which is temporarily considered as a non-bright spot.

[0075] Remember the four values ​​V of group B 12 、V 21 、V 23 and V 32 They are g1, g2, g3 and g4 respectively, and a marking parameter MASK is introduced d_l_2 , repeat the above steps in group A. If MASK d_l_2 (i,j)=0, then the pixel with coordinates (i,j) is a bright spot among dynamic bad pixels. If MASK d_l_2 (i,j)=1, which is temporarily considered as a non-bright spot.

[0076] MASK d_l (i,j)=MASK d_l_1 (i,j)&MASK d_l_2(i,j)

[0077] If MASK d_l (i, j) = 0, the pixel with coordinates (i, j) is a bright spot among dynamic bad pixels. If MASK d_l (i,j)=1, the pixel with coordinates (i,j) is not a bright spot.

[0078] Specifically, to determine whether the pixel with coordinates (i, j) is a dark spot, the eight surrounding pixels are used for determination. 11 and V 33 For group C, V 12 and V 32 For group D, V 13 and V 31 For Group E, V 23 and V 21 For Group F, V 22 is the pixel point with coordinates (i, j), such as Figure 3 shown.

[0079] Among them, record the two values ​​V of group C 11 and V 33 For g5 and g6, introduce a marker parameter MASK d_d_1 :

[0080]

[0081] Among them, T d_d Indicates the preset highlight parameters. According to the above formula, if MASK d_d_1 (i, j) = 0, then the pixel with coordinates (i, j) is a dark pixel in the dynamic bad pixel. d_d_1 (i,j)=1, temporarily considered as a non-dark spot.

[0082] Let the two values ​​V of group D be 12 and V 32 For g5 and g6, introduce a marker parameter MASK d_d_2 Repeat the above steps in group C. If MASK d_d_2 (u,j)=0, then the pixel with coordinates (u,j) is a dark pixel in the dynamic bad pixel. d_d_2 (i,j)=1, temporarily considered as a non-dark spot.

[0083] Let the two values ​​V of group E be 13 and V 31 For g5 and g6, introduce a marker parameter MASK d_d_3 Repeat the above steps in group C. If MASK d_d_3 (u,j)=0, then the pixel with coordinates (u,j) is a dark pixel in the dynamic bad pixel.d_d_3 (i,j)=1, temporarily considered as a non-dark spot.

[0084] Let V be the two values ​​of group F. 23 and V 21 For g5 and g6, introduce a marker parameter MASK d_d_4 Repeat the above steps in group C. If MASK d_d_4 (i, j) = 0, then the pixel with coordinates (i, j) is a dark pixel in the dynamic bad pixel. d_d_4 (i,j)=1, temporarily considered as a non-dark spot.

[0085] MASK d_d (i,j)

[0086] =MASK d_d_1 (i,j)&MASK d_d_2 (i,j)&MASK d_d_3 (i,j)&MASK d_d_4 (i,j)

[0087] If MASK d_d (i, j) = 0, the pixel with coordinates (i, j) is a dark pixel in the dynamic bad pixel. If MASK d_d (i,j)=1, the pixel with coordinates (i,j) is not a dark point.

[0088] Dynamic bad pixel summary

[0089] MASK d (i,j)=MASK d_l (i,j)&MASK d_d (i,j)

[0090] If MASK d (i,j)=0, the pixel with coordinates (i,j) is a dynamic bad pixel. If MASK d (i,j)=1, the pixel with coordinates (i,j) is not a dynamic bad pixel.

[0091] Furthermore, for the marker parameter MASK:

[0092] MASK(i,j)=MASK s (i,j)&MASK d (i,j)

[0093] If MASK(i,j)=0, the pixel with coordinates (u,j) is a bad pixel. If MASK(i,j)=1, the pixel with coordinates (i,j) is a normal pixel.

[0094] For a bad pixel with coordinates (u, j), eight pixels around it are used for repair, as shown in Table 4.

[0095] Among them, Figure 3 As shown, V 11 and V 33 For Group G, V 12 and V 32 For group H, V 13 and V 31 For group I, V 23 and V 21 For group J. If two pixels in group G have bad pixels, all pixels in group G are unavailable; if two pixels in group H have bad pixels, all pixels in group H are unavailable; if two pixels in group I have bad pixels, all pixels in group I are unavailable; if two pixels in group J have bad pixels, all pixels in group J are unavailable.

[0096] Use the available eight pixels around the (i, j) bad pixel to find their average value as the compensation value of the (i, j) bad pixel, as shown in the following formula:

[0097] V(i,j) ′ =[(V 11 +V 33 )×(MASK 11 ×MASK 33 )+(V 12 +V 32 )×(MASK 12 ×MASK 32 )+(V 13 +V 31 )×(MASK 13 ×MASK 31 )+(V 23 +V 21 )×(MASK 23 ×MASK 21 )] / (MASK 11 ×MASK 33 +MASK 12 ×MASK 32 +MASK 13 ×MASK 31 +MASK 23 ×MASK 21 )

[0098] In one embodiment, the method of finding dynamic bad pixels in the prior art has a relatively simple determination process. For infrared image areas with smaller gradients, dynamic bad pixels can be determined more accurately. However, if the gradient of a certain area of ​​the infrared image is larger and the image frequency is higher, the existing dynamic bad pixel determination method is prone to misjudgment, and some normal points at high frequencies are determined as dynamic bad pixels. At the same time, after determining the position of the bad pixel, when compensating for the bad pixel, the average value of the normal points around the bad pixel is directly used for compensation, without considering the gradient relationship of the surrounding pixels. The compensated pixel value may not necessarily reflect the true value of the compensation point, resulting in poor bad pixel removal effect in the infrared image.

[0099] S4. Compensate the pixel values ​​of the pixels according to the bad pixel categories to obtain a bad pixel-removed infrared image.

[0100] In one embodiment, the bad pixels include bad pixels that need to be compensated for pixel values ​​and pixel points that do not need to be compensated, which can be further determined by the connectivity state to more accurately remove the bad pixels in the infrared image.

[0101] Specifically, pixel value compensation is performed on pixel points according to bad pixel categories to obtain an infrared image without bad pixels, including: identifying bad pixels to be compensated in pixels according to bad pixel categories; calculating directional gradients of the bad pixels to be compensated in a preset direction; and pixel value compensation is performed on the bad pixels to be compensated according to the directional gradient to obtain an infrared image without bad pixels.

[0102] Furthermore, bad pixels to be compensated in pixels are identified according to bad pixel categories, including: calculating neighborhood pixel differences between a first category of bad pixels in the bad pixel category and pixels in a preset neighborhood; when the neighborhood pixel differences are all less than zero or all greater than zero, determining that the pixel corresponding to the first category of bad pixels is a first category of bad pixel to be compensated; judging whether there are second category bad pixels in the neighborhood of a second category of bad pixels in the bad pixel category; when there are second category bad pixels, judging the connectivity between the pixel and the second category of pixels; if the connectivity between the pixel and the second category of pixels is connectivity, the pixel is a second category of bad pixel to be compensated; and collecting the first category of bad pixels to be compensated and the second category of bad pixels to be compensated to obtain bad pixels to be compensated in pixels.

[0103] Further, for the first type of bad pixels, if the pixel point does not satisfy that the point is greater than all eight points in the neighborhood or the point is less than all eight points in the neighborhood, then the first type of bad pixels does not need to be compensated; for the second type of bad pixels, if the point is not connected to any second type of bad points in the neighborhood, then the point does not need to be compensated; if there are no second type of bad points in the neighborhood of the point, then the point does not need to be compensated, wherein the pixel points in the preset neighborhood can be the neighborhood corresponding to each pixel point in S1 above, that is, the neighborhood composed of eight adjacent points of top, bottom, left, right, upper left, upper right, lower left, and lower right.

[0104] Furthermore, the directional gradient may include gradients in the horizontal direction, the vertical direction, the upper right-lower left direction, and the upper left-lower right direction, which may be calculated based on each bad pixel to be compensated and the pixel points in the preset gradient neighborhood, wherein the pixel points in the preset gradient neighborhood may be as follows: Figure 4 shown.

[0105] in, Figure 4 Medium V x Indicates the bad pixel to be compensated, and the remaining points are the pixel values ​​corresponding to the pixel points in the gradient neighborhood.

[0106] In one embodiment, a preset gradient calculation formula is used to calculate the directional gradient of the bad pixel to be compensated in a preset direction. The gradient calculation formula is expressed as:

[0107]

[0108] Among them, G1 represents the horizontal gradient, G2 represents the vertical gradient, G3 represents the upper right-lower left gradient, and G4 represents the upper left-lower right gradient.

[0109] Furthermore, pixel value compensation is performed on the bad pixels to be compensated according to the directional gradient to obtain an infrared image without bad pixels, including: determining the compensation direction according to the size of the directional gradient; performing pixel value compensation on the bad pixels to be compensated according to the compensation direction using a preset compensation formula to obtain compensated pixel values; and replacing the pixel values ​​corresponding to the bad pixels to be compensated in the infrared image using the compensated pixel values ​​to obtain an infrared image without bad pixels.

[0110] In one embodiment, the direction corresponding to the minimum value of the directional gradient is selected as the compensation direction.

[0111] If the horizontal gradient is the minimum value, the compensation formula is expressed as:

[0112]

[0113] Among them, U x Indicates the compensation pixel value corresponding to the xth bad pixel to be compensated, V 20 、V 21 、V 23 、V 24 、V 12 、V 32 They respectively represent the pixel values ​​corresponding to the pixel points preset in the horizontal direction of the gradient in the gradient neighborhood.

[0114] If the vertical gradient is the minimum value, the compensation formula is as follows:

[0115]

[0116] Among them, U xIndicates the compensation pixel value corresponding to the xth bad pixel to be compensated, V 02 、V 12 、V 32 、V 42 、V 21 、V 23 They respectively represent the pixel values ​​corresponding to the pixel points preset in the vertical direction of the gradient in the gradient neighborhood.

[0117] If the value of the gradient in the upper right-lower left direction is the minimum, the compensation formula is as follows:

[0118]

[0119] Among them, U x Indicates the compensation pixel value corresponding to the xth bad pixel to be compensated, V 12 、V 13 、V 21 、V 23 、V 31 、V 32 They respectively represent the pixel values ​​corresponding to the pixel points preset in the upper right-lower left direction of the gradient in the gradient neighborhood.

[0120] If the value of the gradient in the upper left-lower right direction is the minimum, the compensation formula is as follows:

[0121]

[0122] Among them, U x Indicates the compensation pixel value corresponding to the xth bad pixel to be compensated, V 11 、V 12 、V 21 、V 23 、V 32 、V 33 They respectively represent the pixel values ​​corresponding to the pixel points preset in the upper left-lower right direction of the gradient in the gradient neighborhood.

[0123] Furthermore, for the pixel points at the edge of the infrared image, the above-mentioned method for calculating the pixel value difference can be used to expand the infrared image to obtain the pixel points in the gradient neighborhood corresponding to each bad point.

[0124] Preferably, after pixel value compensation, the horizontal and vertical bright and dark stripes in the infrared image are Figure 5 As shown, the original high-frequency texture is maintained and no misjudgment occurs.

[0125] At the same time, it can correctly judge the stripes at the corners with a width of only one pixel, such as Figure 6 As shown, the original high-frequency texture is maintained and no misjudgment occurs.

[0126] In one embodiment, by using non-bad pixel points in the neighborhood to perform gradient judgment, the pixel value point after pixel value compensation is closer to the real value of the corresponding scene, which can effectively improve the accuracy of bad pixel removal in infrared images.

[0127] In one embodiment, by calculating the number of connected pixel points, the points with each connected number can be classified more carefully, and the points with a connected number of 0 pixels can be further judged as "bad pixels". If the difference of pixel values ​​is greater than or less than the neighboring points, it will be judged as a bad pixel, otherwise it will not be judged as a bad pixel. This can avoid misjudging normal points as bad points in extreme scenes with large gradients. Figure 7 As shown, the original image is an infrared image, and the local original image is an extreme scene with a large gradient. Figure 8 For Figure 7 The infrared image after pixel value compensation of the local original image shown in FIG. can effectively find two adjacent bad pixels in a scene with many bad pixels. Fig. 9 As shown, Fig. 9 The original image is an infrared image, the local original image is two adjacent bad pixels, and the effect of the scheme after compensating the pixel values ​​of the two adjacent bad pixels can ensure the accuracy of bad pixel removal in the infrared image.

[0128] like Fig.10 , which is a schematic diagram of the structure of an infrared image bad pixel removal device 100 provided by an embodiment of the present invention. The device includes a pixel difference and pixel sum calculation module 101, which is used to calculate the pixel difference and pixel sum between each pixel in the pre-acquired infrared image and the preset neighboring pixel; a connectivity determination module 102, which is used to determine the connectivity between the pixel and the neighboring pixel according to the pixel difference and pixel sum; a bad pixel category determination module 103, which is used to determine the bad pixel category of the pixel according to the connectivity; and a pixel compensation module 104, which is used to compensate the pixel value of the pixel according to the bad pixel category to obtain a bad pixel removed infrared image.

[0129] In detail, in one embodiment, the infrared image bad pixel removal device 100 adopts the same technical means as the infrared image bad pixel removal method in the accompanying drawings when used, and can produce the same technical effect, which will not be repeated here.

[0130] like Fig.11 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing a method for removing bad pixels from infrared images provided by an embodiment of the present invention.

[0131] The electronic device 110 may include a processor 111 , a memory 112 , a communication bus 113 , and a communication interface 114 , and may also include a computer program stored in the memory 112 and executable on the processor 111 , such as an infrared image bad pixel removal program.

[0132] Among them, the processor 111 can be composed of an integrated circuit in some embodiments, for example, it can be composed of a single packaged integrated circuit, or it can be composed of multiple integrated circuits packaged with the same function or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors and various control chips.

[0133] The memory 112 includes at least one type of readable storage medium, including flash memory, mobile hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 112 may be an internal storage unit of the electronic device, such as a mobile hard disk of the electronic device.

[0134] The communication bus 113 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 112 and at least one processor 111, etc.

[0135] The communication interface 114 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface or a wireless interface.

[0136] Fig.11 Only an electronic device with components is shown, and those skilled in the art will understand that Fig.11 The structure shown does not constitute a limitation on the electronic device 110 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0137] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited by this structure.

[0138] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement a method for removing bad pixels from an infrared image in any of the above-mentioned embodiments. It should be noted that the computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device that can carry computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM). In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules is only a logical function division, and there may be other division methods in actual implementation.

[0139] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0140] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0141] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for removing bad pixels from infrared images, characterized in that: The method comprises: Calculating the pixel difference and pixel sum between each pixel point in the pre-acquired infrared image and the preset neighborhood pixel points; Determine the connectivity state between the pixel point and the neighboring pixel points according to the pixel difference value and the pixel sum value; Determine the bad pixel category of the pixel according to the connectivity state; The pixel values ​​of the pixels are compensated according to the bad pixel categories to obtain a bad pixel removed infrared image.

2. The method for removing bad pixels from infrared images as claimed in claim 1, characterized in that: The step of calculating the pixel difference between the pixel values ​​of each pixel point in the infrared image and each neighboring pixel point, and calculating the pixel sum value of the pixel point and each neighboring pixel point in the infrared image, comprises: The pixel difference and pixel sum are calculated using the preset pixel difference and pixel sum formulas, which are expressed as: Among them, lr diff (m,n), lr diff (m,n-1), ud diff (m,n), ud diff (m-1,n), lurd diff (m,n), lurd diff (m-1,n-1),ruld diff (m,n),ruld diff (m-1,n+1) represents the pixel difference between the pixel at coordinate (m,n) and the neighboring pixel at coordinates (m,n+1), (m,n-1), (m+1,n), (m-1,n), (m+1,n+1), (m-1,n-1), (m+1,n-1), (m-1,n+1), (m-1,n+1), P m,n , P m,n+1 , P m,n-1 , P m+1,n , P m-1,n , P m+1,n+1 , P m-1,n-1 , P m+1,n-1 , P m-1,n+1 They represent the pixel values ​​at (m,n), (m,n+1), (m,n-1), (m+1,n), (m-1,n), (m+1,n+1), (m-1,n-1), (m+1,n-1), (m-1,n+1), (m-1,n+1), abs represents the absolute value, lr represents the absolute value, sum (m,n), lr sum (m,n-1), ud sum (m,n), ud sum (m-1,n), lurd sum (m,n), lurd sum (m-1,n-1),ruld sum (m,n),ruld sum (m-1,n+1) represents the pixel sum value at the coordinates (m,n) and its neighboring pixels at the coordinates (m,n+1), (m,n-1), (m+1,n), (m-1,n), (m+1,n+1), (m-1,n-1), (m+1,n-1), and (m-1,n+1).

3. The method for removing bad pixels from infrared images as claimed in claim 1, characterized in that: The determining the connectivity state between the pixel point and the neighboring pixel points according to the pixel difference value and the pixel sum value includes: Calculating a pixel ratio between the pixel difference value and the pixel sum value; Comparing the pixel difference with a preset difference threshold, and comparing the pixel ratio with a preset ratio threshold; When the pixel difference is greater than the difference threshold and the pixel ratio is greater than the ratio threshold, the connectivity state between the pixel point and the neighboring pixel point is disconnected; When the pixel difference is less than or equal to the difference threshold or the pixel ratio is less than or equal to the ratio threshold, the connectivity state between the pixel point and the neighborhood pixel point is connected.

4. The method for removing bad pixels from infrared images as claimed in claim 1, characterized in that: The determining the bad pixel category of the pixel according to the connectivity state includes: When the connectivity between the pixel point and the neighboring pixel points is disconnected, determining that the pixel point is a first-type bad pixel; When the connection state between the pixel point and the neighboring pixel points is that only one of them is connected, determining that the pixel point is a second-type bad pixel; The first type of bad pixels and the second type of bad pixels are collected to obtain the bad pixel category of the pixel.

5. The method for removing bad pixels from infrared images as claimed in claim 1, characterized in that: The step of compensating the pixel values ​​of the pixels according to the bad pixel categories to obtain a bad pixel removed infrared image includes: Identify bad pixels to be compensated among the pixels according to the bad pixel category; Calculating the directional gradient of the bad pixel to be compensated in a preset direction; The pixel value of the bad pixel to be compensated is compensated according to the directional gradient to obtain an infrared image with bad pixels removed.

6. The method for removing bad pixels from infrared images as claimed in claim 5, characterized in that: The step of identifying bad pixels to be compensated in the pixels according to the bad pixel categories includes: Calculating neighborhood pixel differences between a first type of bad pixel in the bad pixel category and pixels in a preset neighborhood; When the neighborhood pixel differences are all less than zero or all greater than zero, determining the pixel points corresponding to the first type of bad pixels as the first type of bad pixels to be compensated; Determine whether there is a second type of bad pixel in the neighborhood of the second type of bad pixel in the bad pixel category; When there are second-type bad pixels, determining the connectivity between the pixel and the second-type pixel; If the connection state between the pixel point and the second type of pixel points is connected, the pixel point is a second type of bad pixel to be compensated; The first category of bad pixels to be compensated and the second category of bad pixels to be compensated are collected to obtain bad pixels to be compensated in the pixels.

7. The method for removing bad pixels from infrared images as claimed in claim 5, characterized in that: The step of compensating the pixel values ​​of the bad pixels to be compensated according to the directional gradient to obtain the bad pixel removed infrared image includes: Determining a compensation direction according to the magnitude of the directional gradient; Performing pixel value compensation on the bad pixel to be compensated by using a preset compensation formula according to the compensation direction to obtain a compensated pixel value; The pixel values ​​corresponding to the bad pixels to be compensated in the infrared image are replaced by the compensation pixel values ​​to obtain an infrared image with bad pixels removed.

8. An infrared image bad pixel removal device, characterized in that: The device comprises: A pixel difference and difference-sum ratio calculation module is used to calculate the pixel difference and pixel sum between each pixel point in the pre-acquired infrared image and the preset neighborhood pixel points; A connectivity state determination module, used to determine the connectivity state between the pixel point and the neighboring pixel points according to the pixel difference value and the pixel sum value; A bad pixel category determination module, used to determine the bad pixel category of the pixel according to the connectivity state; The pixel compensation module is used to compensate the pixel values ​​of the pixels according to the bad pixel categories to obtain a bad pixel-free infrared image.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The processor is used to execute the computer program stored in the memory; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the infrared image bad pixel removal method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the infrared image bad pixel removal method described in any one of claims 1 to 7 is implemented.

Images