Infrared image blind pixel replacement method based on neighborhood high-frequency information

By decomposing high-frequency and low-frequency signals in infrared image processing and combining direction derivatives and neighborhood averaging technologies, the problem of insufficient utilization of high-frequency signals in the existing technology is solved, the effect of blind element replacement is improved, and the imaging quality of the detector is improved .

CN118967458APending Publication Date: 2024-11-15KUNMING INST OF PHYSICS
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410890570.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-04
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing infrared image blind element replacement method lacks the extraction and utilization of high-frequency signals, resulting in a decrease in the replacement effect in scenes with a lot of high-frequency information, affecting the imaging quality of the detector.

Method used

By decomposing the infrared image signal into high-frequency and low-frequency signal components, a mathematical model of blind element replacement is established, the direction derivative is used to estimate high-frequency information and neighborhood average estimate low-frequency information, and the blind element replacement in the image is combined with high-frequency and low-frequency signals. .

Benefits of technology

It improves the compensation and replacement effect of the blind element and improves the imaging quality of the detector.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967458A_ABST
    Figure CN118967458A_ABST
Patent Text Reader

Abstract

The invention relates to an infrared image blind pixel replacement method based on neighborhood high-frequency information, which is characterized by comprising the following steps of: S1, carrying out the following blind pixel detection on an input continuous m-frame x-row y-column image IRk, k in the image IRk being equal to 1, 2, 3... m: S2, expressing a pixel IR (x, y) located at a coordinate (x, y) and a pixel IR (x + p, y + q) with a distance of (p, q) by a formula; and S3, obtaining an estimated pixel value IR * (x, y) in combination with the high-frequency information and the low-frequency information, and replacing the pixel value IR (x, y) at the blind pixel with the estimated pixel value IR * (x, y). According to the invention, the directional derivative of the neighborhood pixel of the blind pixel is used for approximately estimating the high-frequency information at the blind pixel, the neighborhood pixel average is used for estimating the low-frequency information at the blind pixel, and the blind pixel can be more effectively compensated and replaced by combining the compensation method of the high-frequency information and the low-frequency information, so that the imaging quality of the detector is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of infrared image processing, and in particular relates to an infrared image blind pixel replacement method based on neighborhood high-frequency information. Background Art

[0002] Due to the influence of materials, manufacturing process and other aspects, the pixels of infrared focal plane array will have uneven response and defects. The national standard divides blind pixels into overheated pixels and dead pixels. Overheated pixels are pixels with a response rate 10 times higher than the average response rate, and dead pixels are pixels with a response rate lower than 1 / 10 of the average response rate. They are usually manifested in the form of dark spots or bright spots on the image. Blind pixels will affect the imaging quality of the detector, so it is necessary to detect and replace blind pixels on the focal plane array.

[0003] Blind pixel replacement is generally performed after non-uniformity correction. Compared with other infrared image processing, there is less research on blind pixel replacement algorithms. Currently, the most widely studied and applied blind pixel replacement algorithms are mainly nearest neighbor algorithms. Typical algorithms include neighboring pixel replacement, median filtering, neighborhood pixel average replacement, neighborhood pixel Gaussian weighted average, etc. Due to its simple calculation, neighborhood pixel average is the most commonly used algorithm and is widely used in blind pixel replacement processing of various types of infrared focal plane detection.

[0004] The currently commonly used neighborhood blind pixel replacement method lacks the extraction and utilization of neighborhood high-frequency signals. Although the blind pixel effect is better at low-frequency information, the replacement effect will decrease in scenarios with more high-frequency information. Therefore, it is necessary to improve the existing technology. Summary of the invention

[0005] In order to improve the imaging quality of the detector, the present invention provides an infrared image blind pixel replacement method based on neighborhood high-frequency information.

[0006] The present invention decomposes infrared image signals into high-frequency signal components and low-frequency signal components, characterizes the relationship between blind pixels and neighboring pixels through mathematical methods, establishes a mathematical model for blind pixel replacement, estimates the low-frequency signal components at the blind pixels through neighborhood averaging, estimates the high-frequency information components at the blind pixels through directional derivatives, and utilizes the superposition of the estimated high-frequency signal components and low-frequency signal components to better replace the blind pixels in the image.

[0007] The present invention is implemented by the following technical scheme: a method for replacing blind pixels of infrared images based on neighborhood high-frequency information, comprising the following steps:

[0008] Step S1: input m consecutive frames of image IR with x rows and y columns k Perform the following blind pixel detection, the image IR k where k = 1, 2, 3…m:

[0009] S1.1) Use wavelet transform to transform image IR k Decompose into high-frequency information image H k and low-frequency information image L k , and perform binarization processing;

[0010] S1.2) Set the initial blind cell table bp of row x and column y, and set all elements in the initial blind cell table bp to 1;

[0011] S1.3) Based on the high frequency information image H k and low-frequency information image L k , for image IR k The following position classification is performed for the middle pixels: when the pixel (x, y) is greater than 0 in the high-frequency information image Hk(x, y), the pixel is judged to be in the high-frequency information; when the pixel (x, y) is greater than 0 in the low-frequency information image Lk(x, y), the pixel is judged to be in the low-frequency information;

[0012] S1.4) Set a threshold h, and adjust the image IR according to the position of the pixel. k The following blind pixel discrimination is performed for the middle pixel:

[0013] S1.4.1) When a pixel is located at a high frequency information location, and the absolute value of the average difference between the pixel and the pixel in the neighborhood that is located at a high frequency information location is greater than the threshold h, it is determined to be a blind pixel;

[0014] S1.4.2) When a pixel is located at a low-frequency information location, and the absolute value of the average difference between the pixel and the pixel in the neighborhood that is located at a low-frequency information location is greater than the threshold h, it is determined to be a blind pixel;

[0015] S1.4.3) If a certain pixel IR(x, y) is judged as a blind pixel for n consecutive frames (n≥m / 2), the corresponding blind pixel table bp(x, y) is set to 0;

[0016] Step S2, for the pixel IR(x, y) at the coordinate (x, y), the pixel IR(x+p, y+q) at the distance (p, q) is expressed by the following formula:

[0017] IR(x,y)=IR(x,y)-IR(x+p,y+q)+IR(x+p,y+q)

[0018] Transforming the formula yields the following equations for any pixel and its neighboring pixels:

[0019]

[0020] The first summation term on the right side of the equation is the average value of r valid pixels in the neighborhood, representing the low-frequency information part of the pixel IR(x, y). The second summation term is the average of the difference between the pixel IR(x, y) and the adjacent pixels, that is, the average of the directional derivatives in s directions, representing the high-frequency information part of the pixel IR(x, y).

[0021] Through the neighboring pixels, the high-frequency information and low-frequency information of the pixel IR (x, y) are estimated as follows:

[0022] S2.1, using the neighborhood directional derivative approximation, the high-frequency information at the pixel IR(x, y) is estimated as follows:

[0023] S2.1.1), determine the neighborhood information of the blind pixel, and classify the distribution of the blind pixels in the neighborhood according to whether there are blind pixels in the neighborhood;

[0024] S2.1.2), according to the classification of S2.1.1), use different directional derivative operator templates to obtain the directional derivatives of the neighborhood, and use the directional derivatives of the neighborhood to approximate the directional derivatives at the pixel IR(x, y), that is, the high-frequency information H(x, y) of the pixel IR(x, y) is obtained, as shown in the following formula:

[0025]

[0026] S2.2, using the pixels in the non-blind pixel neighborhood to average, the low-frequency information L(x, y) at the pixel IR(x, y) is estimated as follows:

[0027]

[0028] Step S3, combining the high-frequency information and the low-frequency information to obtain the estimated pixel value IR*(x, y), and using the estimated pixel value IR*(x, y) to replace the pixel value IR(x, y) at the blind pixel, the replacement formula is as follows:

[0029]

[0030] In step S2.1.2, n is adjusted according to different distributions of blind pixels in the neighborhood. When a fixed compensation order and a fixed size neighborhood are used, the directional derivative operator is determined.

[0031] When estimating the low-frequency information L(x, y) at the pixel IR(x, y) in step S2.2, the required neighborhood size is adjusted by changing the coefficients p and q.

[0032] In step S3, when the blind pixel is replaced, the coefficients a and b are adjusted according to the actual situation.

[0033] The beneficial effect of the present invention is: by adopting the above-mentioned technical scheme, that is, using the directional derivatives of the blind pixel neighborhood pixels to approximately estimate the high-frequency information at the blind pixel, and using the average of the neighborhood pixels to estimate the low-frequency information at the blind pixel, combined with the replacement method of the high-frequency information and the low-frequency information, the blind pixel can be more effectively compensated and replaced, thereby improving the imaging quality of the detector. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are used to further understand the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention.

[0035] Figure 1 is a flow chart of the present invention;

[0036] Figure 2 This is a comparison diagram of the effects of the present invention and the traditional neighborhood replacement algorithm, where:

[0037] (a) is the image without blind pixel replacement, and the blind pixel is set on the four-bar target of the characteristic frequency;

[0038] (b) is the effect image of the neighborhood average algorithm;

[0039] (c) is an effect image of the present invention;

[0040] Figure 3 This is an effect image of the present invention, wherein:

[0041] (a) is the image without blind pixel replacement, and the blind pixel is set on the four-bar target of the characteristic frequency;

[0042] (b) is the effect image of this algorithm;

[0043] Figure 4 This is a schematic diagram of estimating the directional derivative of a blind pixel at a location marked in Example 1, where:

[0044] The left side is a schematic diagram of horizontal derivative estimation;

[0045] The right side is a schematic diagram of the vertical derivative estimation;

[0046] Figure 5 This is a schematic diagram of blind pixel directional derivative estimation in Example 2;

[0047] The left side is a schematic diagram of horizontal derivative estimation;

[0048] On the right is a schematic diagram of vertical derivative estimation. DETAILED DESCRIPTION

[0049] The principles and features of the present invention are further described below in conjunction with the accompanying drawings and embodiments. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0050] Example 1

[0051] Combined with Figure 1 , with Figure 2 For example, the infrared image blind pixel replacement method based on neighborhood high frequency information provided by the present invention includes the following steps:

[0052] Step S1: input 10 consecutive frames of 640 rows and 512 columns of image IR k The following blind pixel detection is performed, the image IR k where k = 1, 2, 3…10:

[0053] S1.1) Use the conventional wavelet transform method to transform the image IR k Decompose into high-frequency information image H k and low-frequency information image L k , and perform conventional binarization processing;

[0054] S1.2) Set an initial blind cell table bp with 640 rows and 512 columns, and set all elements in the initial blind cell table bp to 1;

[0055] S1.3) Based on the high frequency information image H k and low-frequency information image L k , for image IR k The following position classification is performed for the pixels in the image: When the pixel (x, y) is in the high-frequency information image H k If (x, y) is greater than 0, the pixel is judged to be in the high-frequency information; when the pixel (x, y) is in the low-frequency information image L k If (x, y) is greater than 0, the pixel is judged to be in the low-frequency information area;

[0056] S1.4) Set a threshold h = 25, and perform image IR k The following blind pixel discrimination is performed for the middle pixel:

[0057] S1.4.1) When a pixel is located at a high frequency information location, and the absolute value of the average difference between the pixel and the pixel in the neighborhood that is located at a high frequency information location is greater than a threshold of 25, it is determined to be a blind pixel;

[0058] S1.4.2) When a pixel is located at a low-frequency information location, and the absolute value of the average difference between the pixel and the pixel in the neighborhood that is located at a low-frequency information location is greater than a threshold of 25, it is determined to be a blind pixel;

[0059] S1.4.3) According to the attached Figure 2 (a) There are four blind pixels in the image, and the corresponding positions in the blind pixel table bp are set to 0;

[0060] Step S2, for the pixel IR(x, y) at the coordinate (x, y), the pixel IR(x+p, y+q) at the distance (p, q) is expressed by the following formula:

[0061] IR(x,y)=IR(x,y)-IR(x+p,y+q)+IR(x+p,y+q)

[0062] Transforming the formula yields the following equations for any pixel and its neighboring pixels:

[0063]

[0064] The first summation term on the right side of the equation is the average value of r valid pixels in the neighborhood, representing the low-frequency information part of the pixel IR(x, y). The second summation term is the average of the difference between the pixel IR(x, y) and the adjacent pixels, that is, the average of the directional derivatives in s directions, representing the high-frequency information part of the pixel IR(x, y).

[0065] Through the neighboring pixels, the high-frequency information and low-frequency information of the pixel IR (x, y) are estimated as follows:

[0066] S2.1, using the neighborhood directional derivative approximation, the high-frequency information at the pixel IR(x, y) is estimated as follows:

[0067] S2.1.1), determine the neighborhood information of the blind pixel, and classify the distribution of the blind pixels in the neighborhood according to whether there are blind pixels in the neighborhood;

[0068] When using directional derivatives to extract high-frequency information, the distribution of blind pixels in the image neighborhood is judged and classified, and conventional different directional derivative operator templates are used to extract high-frequency information from the blind pixel neighborhood in the input image IR (x, y):

[0069] Attached Figure 2 In (a), the number of blind pixels is four, as shown in the following figure. Figure 4 As shown in the figure: the black part represents the blind pixel, and the white part represents the normal pixel. Taking the blind pixel at the mark 1 as an example, its directional derivative is approximately distributed, as shown by the red solid arrow in the figure, and the directional derivative represented by its neighboring pixel is shown by the blue dotted arrow. The derivative represented by the blue dotted arrow is used to estimate the derivative represented by the red solid arrow;

[0070] S2.1.2), according to the classification of S2.1.1), use conventional different directional derivative operator templates to obtain the directional derivatives of the neighborhood, and use the directional derivatives of the neighborhood to approximate the directional derivatives at the pixel IR(x, y), that is, the high-frequency information H(x, y) of the pixel IR(x, y) is obtained as follows:

[0071]

[0072] S2.2, use the pixels in the non-blind pixel neighborhood to average and estimate the low-frequency information L(x, y) at the pixel IR(x, y), that is, among the neighborhood pixels of the blind pixel at mark 1, there are five non-blind pixel pixels. Use the neighborhood averaging algorithm to extract the low-frequency information of the blind pixel neighborhood in the input image IR(x, y):

[0073]

[0074] Step S3, combining the high-frequency information and the low-frequency information to obtain the estimated pixel value IR*(x, y), and using the estimated pixel value IR*(x, y) to replace the pixel value IR(x, y) at the blind pixel, the replacement formula is as follows:

[0075]

[0076] Take a=1,b=1, we get:

[0077]

[0078] The replacement effect is as follows Figure 2 (c) as shown.

[0079] Example 2

[0080] Combined with Figure 1 , with Figure 3 For example, the infrared image blind pixel replacement method based on neighborhood high frequency information provided by the present invention includes the following steps:

[0081] Step S1: input 10 consecutive frames of 640 rows and 512 columns of image IR k Perform the following blind pixel detection, the image IR k where k = 1, 2, 3…10:

[0082] S1.1) Use the conventional wavelet transform method to transform the image IR k Decompose into high-frequency information image H k and low-frequency information image L k , and perform conventional binarization processing;

[0083] S1.2) Set an initial blind cell table bp with 640 rows and 512 columns, and set all elements in the initial blind cell table bp to 1;

[0084] S1.3) Based on the high frequency information image H k and low-frequency information image L k , for image IR k The following position classification is performed for the pixels in the image: When the pixel (x, y) is in the high-frequency information image H kWhen the pixel (x, y) is greater than 0 in the low-frequency information image Lk(x, y), the pixel is determined to be in the high-frequency information. When the pixel (x, y) is greater than 0 in the low-frequency information image Lk(x, y), the pixel is determined to be in the low-frequency information.

[0085] S1.4) Set a threshold h = 25, and perform image IR k The following blind pixel discrimination is performed for the middle pixel:

[0086] S1.4.1) When a pixel is located at a high frequency information location, and the absolute value of the average difference between the pixel and the pixel in the neighborhood that is located at a high frequency information location is greater than a threshold of 25, it is determined to be a blind pixel;

[0087] S1.4.2) When a pixel is located at a low-frequency information location, and the absolute value of the average difference between the pixel and the pixel in the neighborhood that is located at a low-frequency information location is greater than a threshold of 25, it is determined to be a blind pixel;

[0088] S1.4.3) If the pixel IR(x, y) is judged as a blind pixel for 5 consecutive frames, the corresponding blind pixel table bp(x, y) is set to 0;

[0089] Step S2, for the pixel IR(x, y) at the coordinate (x, y), the pixel IR(x+p, y+q) at the distance (p, q) is expressed by the following formula:

[0090] IR(x,y))=IR(x,y)-IR(x+p,y+q)+IR(x+p,y+q)

[0091] Transforming the formula yields the following equations for any pixel and its neighboring pixels:

[0092]

[0093] The first summation term on the right side of the equation is the average value of r valid pixels in the neighborhood, representing the low-frequency information part of the pixel IR(x, y). The second summation term is the average of the difference between the pixel IR(x, y) and the adjacent pixels, that is, the average of the directional derivatives in s directions, representing the high-frequency information part of the pixel IR(x, y).

[0094] Through the neighboring pixels, the high-frequency information and low-frequency information of the pixel IR (x, y) are estimated as follows:

[0095] S2.1, using the neighborhood directional derivative approximation, the high-frequency information at the pixel IR(x, y) is estimated as follows:

[0096] S2.1.1), determine the neighborhood information of the blind pixel, and classify the distribution of the blind pixels in the neighborhood according to whether there are blind pixels in the neighborhood;

[0097] When using directional derivatives to extract high-frequency information, the distribution of blind pixels in the image neighborhood is judged and classified, and different directional derivative operators are used to extract high-frequency information from the blind pixel neighborhood in the input image IR (x, y);

[0098] Attached Figure 3 In (a), the number of blind pixels is 1, as shown in the following figure. Figure 4 , 5 As shown, the black part represents the blind pixel, the gray part represents the normal pixel, the approximate distribution of the blind pixel prescription derivative is shown by the red solid arrow in the figure, and the directional derivative represented by its neighboring pixel is represented by the blue dotted arrow. The derivative represented by the blue dotted arrow is used to estimate the derivative represented by the red solid arrow.

[0099] S2.1.2), according to the classification of S2.1.1), use conventional different directional derivative operator templates to obtain the directional derivatives of the neighborhood, and use the directional derivatives of the neighborhood to approximate the directional derivatives at the pixel IR(x, y), that is, to obtain the high-frequency information H(x, y) of the pixel IR(x, y), which is as follows:

[0100]

[0101] S2.2, use the average of the pixels at the non-blind pixel in the neighborhood to estimate the low-frequency information L(x, y) at the pixel IR(x, y), that is, among the neighborhood pixels of the blind pixel, the number of non-blind pixel pixels is 8, and the neighborhood averaging algorithm is used to extract the low-frequency information of the blind pixel neighborhood in the input image IR(x, y):

[0102]

[0103] Step S3, combining the extracted high-frequency information and low-frequency information to replace the blind pixels, the replacement formula is as follows:

[0104]

[0105] Take a=1,b=1, we get:

[0106]

[0107] The replacement effect is as follows Figure 3 shown.

Claims

1. A blind pixel replacement method for infrared images based on neighborhood high-frequency information, characterized in that The following steps are involved: Step S1: input m consecutive frames of image IR with x rows and y columns k The following blind pixel detection is performed, the image IR k where k = 1, 2, 3…m: S1.1) Use wavelet transform to transform image IR k Decompose into high-frequency information image H k and low-frequency information image L k , and perform binarization processing; S1.2) Set the initial blind cell table bp of row x and column y, and set all elements in the initial blind cell table bp to 1; S1.3) Based on the high frequency information image H k and low-frequency information image L k , for image IR k The following position classification is performed for the pixels in the image: When the pixel (x, y) is in the high-frequency information image H k If (x, y) is greater than 0, the pixel is judged to be in the high-frequency information; when the pixel (x, y) is in the low-frequency information image L k If (x, y) is greater than 0, the pixel is judged to be in the low-frequency information area; S1.4) Set a threshold h, and adjust the image IR according to the position of the pixel. k The following blind pixel discrimination is performed for the middle pixel: S1.4.1) When a pixel is located at a high frequency information location, and the absolute value of the average difference between the pixel and the pixel in the neighborhood that is located at a high frequency information location is greater than the threshold h, it is determined to be a blind pixel; S1.4.2) When a pixel is located at a low-frequency information location, and the absolute value of the average difference between the pixel and the pixel in the neighborhood that is located at a low-frequency information location is greater than the threshold h, it is determined to be a blind pixel; S1.4.3) If a certain pixel IR(x, y) is judged as a blind pixel for n consecutive frames (n≥m / 2), the corresponding blind pixel table bp(x, y) is set to 0; Step S2, for the pixel IR(x, y) at the coordinate (x, y), the pixel IR(x+p, y+q) at the distance (p, q) is expressed by the following formula: IR(x,y)=IR(x,y)-IR(x+p,y+q)+IR(x+p,y+q) Transforming the formula yields the following equations for any pixel and its neighboring pixels: The first summation term on the right side of the equation is the average value of r valid pixels in the neighborhood, representing the low-frequency information part of the pixel IR(x, y). The second summation term is the average of the difference between the pixel IR(x, y) and the adjacent pixels, that is, the average of the directional derivatives in s directions, representing the high-frequency information part of the pixel IR(x, y). Through the neighboring pixels, the high-frequency information and low-frequency information of the pixel IR (x, y) are estimated as follows: S2.1, use the neighborhood directional derivative approximation to estimate the high-frequency information at the pixel IR(x, y), as follows: S2.1.1), determine the neighborhood information of the blind pixel, and classify the distribution of the blind pixels in the neighborhood according to whether there are blind pixels in the neighborhood; S2.1.2), according to the classification of S2.1.1), use different directional derivative operator templates to obtain the directional derivatives of the neighborhood, and use the directional derivatives of the neighborhood to approximate the directional derivatives at the pixel IR(x, y), that is, the high-frequency information H(x, y) of the pixel IR(x, y) is obtained as follows: S2.2, use the pixels at the non-blind pixels in the neighborhood to average and estimate the low-frequency information L(x, y) at the pixel IR(x, y), as follows; Step S3, combining the high-frequency information and the low-frequency information to obtain the estimated pixel value IR*(x, y), and using the estimated pixel value IR*(x, y) to replace the pixel value IR(x, y) at the blind pixel, the replacement formula is as follows:

2. The infrared image blind pixel replacement method based on neighborhood high frequency information according to claim 1 is characterized in that In step S2.1.2, n is adjusted according to different distributions of blind pixels in the neighborhood. When a fixed compensation order and a fixed size neighborhood are used, the directional derivative operator is determined.

3. The infrared image blind pixel replacement method based on neighborhood high frequency information according to claim 1 is characterized in that When estimating the low-frequency information L(x, y) at the pixel IR(x, y) in step S2.2, the required neighborhood size is adjusted by changing the coefficients p and q.

4. The infrared image blind pixel replacement method based on neighborhood high frequency information according to claim 1 is characterized in that In step S3, when the blind pixel is replaced, the coefficients a and b are adjusted according to the actual situation.

Citation Information

Patent Citations

  • Infrared image correction method

    CN115100068A

  • Infrared sensing devices and methods

    US20170016762A1