Multidirectional infrared image blind pixel detection system and method of convolution sliding window

Abstract

The invention discloses a multidirectional infrared image blind pixel detection system and method of a convolution sliding window, and belongs to the technical field of infrared thermal imaging. The system comprises an acquisition module, a display module and a detection module. The detection method comprises the following steps: acquiring infrared image data needing to be processed to obtain a complete infrared image, and then performing sliding convolution operation on the image to obtain a threshold matrix; meanwhile, respectively calculating the maximum value and the minimum value of one-dimensional operators in the cross direction and the diagonal direction of the pixel point to be measured, and respectively calculating the maximum value and the minimum value in the two-dimensional direction according to the obtained values; obtaining a threshold matrix according to the solved extreme value and the threshold matrix, judging whether the gray value of the pixel point is within a threshold range, and if the gray value is greater than the threshold, judging that the point is a suspected overheating failure element; and if the gray value is smaller than the threshold value, the point is judged to be a suspected supercooling failure element. And carrying out extrapolation on the pixel points until the whole image is completely traversed, and at the moment, obtaining a plurality of points suspected to be failure elements.

Description

Technical Field

[0001] The invention belongs to the field of infrared thermal imaging technology, and in particular relates to an infrared image blind pixel detection system and method. Background Art

[0002] As a new generation of infrared detection devices, infrared focal plane arrays have improved the spatial resolution and system sensitivity of infrared systems, and their applications are becoming more and more extensive. However, during its manufacturing and use, factors such as semiconductor material properties, manufacturing processes or working environments may cause inconsistencies in the photoelectric conversion characteristic curves of each detection unit of the infrared focal plane array, and blind pixels are caused by the excessive inconsistency of this characteristic curve. Specifically, some pixels in the infrared focal plane array are abnormal, appearing to be always bright, always dark or periodically flickering.

[0003] According to the national definition of blind pixels, blind pixels can be divided into dead pixels and overheated pixels. If the pixel response rate is less than 0.1 times the average response rate of all effective pixels, it is defined as a dead pixel. If the pixel noise voltage is greater than 10 times the average noise voltage of all effective pixels, it is defined as an overheated pixel.

[0004] According to statistical characteristics, blind pixels can be divided into two categories: fixed blind pixels and random blind pixels. Fixed blind pixels are mainly due to the limitations of the manufacturing process of the infrared focal plane array detection device itself, which makes the fixed position always have over-response or under-response blind pixels. Random blind pixels are caused by channel obstacles or external factors that cause the signal of the relevant unit to attenuate, making the generation of blind pixels uncertain in time and space.

[0005] In the industrial production of infrared thermal imagers, whether it is the field of infrared image processing that needs to compensate for blind pixels or the field of infrared inspection that judges blind pixels, the problem of blind pixel detection cannot be avoided. However, the current infrared image blind pixel detection technology is limited by the performance of the algorithm, and there are problems such as long detection time, high misjudgment rate, and poor generalization; on the other hand, it is limited by the software development language and lacks a simple, efficient and compatible universal blind pixel detection system. The blind pixels of infrared thermal imagers and their related indicators are important test indicators for evaluating the performance of infrared thermal imagers, and have important guiding significance for improving production processes and enhancing user experience. In view of the above situation, it is of great significance to improve the blind pixel detection algorithm and establish an infrared image blind pixel detection system with universality, compatibility and generalization through software calculation processing. Summary of the invention

[0006] The present invention proposes a multi-directional infrared image blind pixel detection system and method with a highly generalized convolution sliding window, which has strong versatility and compatibility and can accurately identify blind pixels with different characteristics.

[0007] A convolution sliding window multi-directional infrared image blind pixel detection system, characterized by comprising an acquisition module, a display module and a detection module, wherein the acquisition module obtains the pixel value of the infrared image data to be tested, is connected to the single channel point of the display module, and provides the display module with an image pixel matrix in the form of grayscale image data output; the display module displays the infrared image and intercepts and records the image for storage; the display module is bidirectionally electrically connected to the detection module, the detection module performs sliding convolution on the infrared image displayed by the display module, generates a threshold matrix and determines the blind pixel point after calculation, the module has a built-in blind pixel table, stores the number and position of the blind pixels, and transmits them to the display module;

[0008] A convolution sliding window multi-directional infrared image blind pixel detection system method, which is characterized by comprising the following steps:

[0009] S1, read the infrared image pixel value through the acquisition module, and output it in the form of a matrix to be displayed on the display module;

[0010] S2-1, expanding the edge of the image through the detection module to expand the size of the output image;

[0011] S2-2, respectively calculate the maximum and minimum values ​​of the one-dimensional operator in the cross direction and diagonal direction of the infrared image pixel point in S1, and the maximum and minimum values ​​of the one-dimensional operator in the x direction are respectively calculated by I xmax and I xmin Indicates that the maximum and minimum values ​​of the one-dimensional operator in the y direction are represented by I ymax and I ymin The maximum and minimum values ​​of the diagonal one-dimensional operator are represented by I xymax ,I xymin ,I yxmax and I yxmin The calculation formula is as follows: xmax =max{(2*P i,j-1 -P ij-2 ),(2*P i,j+1 -P i,j+2 ),(P i,j-1 ),(P i,j+1 )},

[0012] I xmin =min{(2*P i,j-1 -P i,j-2 ),(2*P i,j+1 -P i,j+2 ),(P i,j-1 ),(P i,j+1 )},

[0013] I ymax =max{(2*P i-1,j -P i-2,j),(2*P i+1,j -P i+2,j ),(P i-1,j ),(P i+1,j )},

[0014] I ymin =min{(2*P i-1,j -P i-2,j ),(2*P i+1,j -P i+2,j ),(P i-1,j ),(P i+1,j )},

[0015] I xymax =max{(2*P i-1,j-1 -P i-2,j-2 ),(2*P i+1,j+1 -P i+2,j+2 ),(P i-1,j-1 ),(P i+1,j+1 )},

[0016] I xymin =min{(2*P i-1,j-1 -P i-2,j-2 ),(2*P i+1,j+1 -P i+2,j+2 ),(P i-1,j-1 ),(P i+1,j+1 )},I yxmax =max{(2*P i-1,j+1 -P i-2,j+2 ),(2*P i+1,j-1 -P i+2,j-2 ),(P i-1,j+1 ),(P i+1,j-1 )},I yxmin =min{(2*P i-1,j+1 -P i-2,j+2 ),(2*P i+1,j-1 -P i+2,j-2 ),(P i-1,j+1 ),(P i+1,j-1 )};

[0017] Wherein, P represents the pixel point of the image, i and j represent the x- and y-coordinates of the pixel point respectively.

[0018] S3-1, performing sliding convolution on the image output in S2-1 to obtain a threshold matrix;

[0019] S3-2, according to the maximum value and minimum value of the one-dimensional operator calculated in S2-2, respectively calculate the maximum value and minimum value in the two-dimensional direction, the maximum value and minimum value in the two-dimensional direction are respectively I 1max ,I 1min ,I2max and I 2min The calculation formula is as follows:

[0020] I 1max =max{(I xmax ),(I ymax )},

[0021] I 1mi =min{(I xmin ),(I ymin )},

[0022] I 2max =max{(I xymax ),(I yxmax )},

[0023] I 2mi =min{(I xymin ),(I yxmin )};

[0024] Among them, I 1max and I 1min Respectively represent the maximum and minimum values ​​of the two-dimensional operator in the x-direction and y-direction; I 2max and I 2min Respectively represent the maximum and minimum values ​​of the two-dimensional operator in the oblique downward and upward directions;

[0025] S4, the threshold value T of the corresponding coordinate in the threshold matrix obtained in S3-1 i,j The threshold is calculated by combining the maximum and minimum values ​​calculated in S3-2. The calculation formula is as follows:

[0026] |I 1max +0.15*T i,j |or|I 2max +0.15*T i,j |,

[0027] |I 1min -0.15*T i,j |or|I 2min -0.15*T i,j |;

[0028] Gray value P of the target pixel i,j is greater than the threshold, that is, P i,j ≥|I 1max +0.15*T i,j | or P i,j ≥|I 2max +0.15*T i,j |, it is judged as a suspected overheating failure element,

[0029] Gray value P of the target pixeli,j is less than the threshold, that is, P i,j ≤|I 1min -0.15*T i,j | or P i,j ≤|I 2min -0.15*T i,j |;, judged as a suspected overcooling failure element;

[0030] S5-Extrapolate the pixel points according to the above method until the entire image is completely traversed to obtain multiple points suspected to be failure elements. When the two frames of infrared images are consistent overheating failure elements or consistent undercooling failure elements, the point is judged as a blind pixel point.

[0031] Furthermore, in S2, the edge expansion of the image is performed by mirror expansion. The size of the image is M×N, and the 2 rows and 2 columns of its edge are mirror expanded respectively. The mirror expansion fills the two edge columns of the original image into the required expansion area after mirroring them symmetrically, and fills the places without corresponding values ​​at the four corners with 0, and finally obtains an image of size (M+4)×(N+4).

[0032] Furthermore, when the detection object is a video, the next frame of infrared image data is collected, and the points in the data are detected for suspected failure elements. If the failure element positions of the two frames of infrared image data are consistent, the consistent points are determined as blind pixels.

[0033] The beneficial effect of the present invention is that by using the threshold matrix to participate in the subsequent threshold judgment, not only the error caused by the neighboring abnormal points in the blind pixel judgment is eliminated, but also the difference between the local features of the image is retained, which is more accurate and reliable than the traditional method of using the image grayscale average value as the blind pixel judgment threshold. This method has strong versatility and compatibility, and can be used for the detection and processing of blind pixels in infrared images of different formats and specifications. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of a multi-directional infrared image blind pixel detection system based on a convolution sliding window.

[0035] Figure 2 This is a flow chart of a convolution sliding window blind pixel detection method for multi-directional infrared images.

[0036] Figure 3 This is a sample image of the mirror edge expansion in Example 1.

[0037] Figure 4 This is a window diagram for performing sliding convolution in Example 1.

[0038] Figure 5 This is the two-dimensional operator distribution diagram in Example 1. DETAILED DESCRIPTION

[0039] Embodiment 1: A convolution sliding window multi-directional infrared image blind pixel detection system includes an acquisition module, a display module and a detection module.

[0040] The acquisition module obtains the pixel value of the infrared image data to be tested, is electrically connected to the display module through a single channel, and provides the display module with an image pixel matrix in the form of grayscale image data output; the input of this module is compatible with the general picture format, general video format and CameraLink interface format, realizing the acquisition of different image sources. At the same time, in order to make the system more widely used, multiple API interfaces are reserved for the subsequent secondary development of interface formats, so that the system can be compatible with cameras with more interface formats.

[0041] The imaging module and the detection module are electrically connected to each other.

[0042] The display module displays infrared images, including pictures, videos and real-time camera videos, and can capture, record and save images. At the same time, considering the different parameters such as resolution, bit width and depth of the image data source, the system sets the above parameters to be compatible with more types of image data. In addition, the module has the function of displaying the image histogram and viewing the grayscale value of a single pixel to obtain the required information.

[0043] The detection module performs sliding convolution on the infrared image displayed by the display module to generate a threshold matrix; uses an operator to calculate the maximum and minimum values ​​of the one-dimensional pixel points; then calculates the maximum and minimum values ​​in the two-dimensional direction; calculates the blind pixel detection threshold corresponding to the pixel point; determines whether the pixel point is a suspected failure point based on the threshold range; and finally compares the suspected failure points of multiple frames of images, and the overlapping points can be determined as blind pixel points. In addition, the module has a built-in blind pixel table that can store the number and position of blind pixels and pass them to the display module.

[0044] The method for detecting blind pixels of multi-directional infrared images using a convolution sliding window using the system is used to detect video data, including the following steps:

[0045] S1, read the infrared image pixel value through the acquisition module, and output it in the form of a matrix to be displayed on the display module.

[0046] S2-1, expand the edge of the image through the detection module to expand the size of the output image. Edge expansion refers to filling the edges of the input image before the convolution operation to expand the size of the output feature map. Its purpose is to cover the edge information of the original image. It is a convolution operation technique that can effectively prevent the loss of edge information and improve the generalization ability of the model. Among them, the expansion methods include filling constants, filling image edges, and mirroring at the filling edges. This method uses mirror expansion processing. Assuming that the size of the image is M×N, we mirror the 2 rows and 2 columns of its edges respectively, and fill in 0 operations at the top corners. Finally, we get an image of size (M+4)×(N+4), as shown in Figure 3 shown.

[0047] S2-2, while expanding the edge, respectively calculate the maximum and minimum values ​​of the one-dimensional operator in the cross direction and diagonal direction of the pixel to be tested, and the maximum and minimum values ​​of the one-dimensional operator in the x direction are respectively calculated by I xmax and I xmin Indicates that the maximum and minimum values ​​of the one-dimensional operator in the y direction are represented by I ymax and I ymin The maximum and minimum values ​​of the diagonal one-dimensional operator are represented by I xymax ,I xymin ,I yxmax and I yxmin The calculation formula is as follows:

[0048] I xmax =max{(2*P i,j-1 -P ij-2 ),(2*P i,j+1 -P i,j+2 ),(P i,j-1 ),(P i,j+1 )},

[0049] I xmin =min{(2*P i,j-1 -P i,j-2 ),(2*P i,j+1 -P i,j+2 ),(P i,j-1 ),(P i,j+1 )},

[0050] I ymax =max{(2*P i-1,j -P i-2,j ),(2*P i+1,j -P i+2,j ),(P i-1,j ),(P i+1,j )},

[0051] I ymin=min{(2*P i-1,j -P i-2,j ),(2*P i+1,j -P i+2,j ),(P i-1,j ),(P i+1,j )},

[0052] I xymax =max{(2*P i-1,j-1 -P i-2,j-2 ),(2*P i+1,j+1 -P i+2,j+2 ),(P i-1,j-1 ),(P i+1,j+1 )},

[0053] I xymin =min{92*P i-1,j-1 -P i-2,j-2 ),(2*P i+1,j+1 -P i+2,j+2 ),(P i-1,j-1 ),(P i+1,j+1 )},I yxmax =max{(2*P i-1,j+1 -P i-2,j+2 ),(2*P i+1,j-1 -P i+2,j-2 ),(P i-1,j+1 ),(P i+1,j-1 )},I yxmin =min{92*P i-1,j+1 -P i-2,j+2 ),(2*P i+1,j-1 -P i+2,j-2 ),(P i-1,j+1 ),(P i+1,j-1 )}.

[0054] Wherein, P represents the pixel point of the image, i and j represent the x- and y-coordinates of the pixel point respectively.

[0055] S3-1, use a 5*5 window to perform sliding convolution on the image output in S2-1 to obtain a threshold matrix. The window follows the distance weighted principle, because the neighboring pixels close to the target pixel can better reflect its characteristics. This operation can well adapt to the local characteristics of the image. The selected window is as follows Figure 4 shown.

[0056] S3-2, according to the maximum value and minimum value of the one-dimensional operator calculated in S2-2, respectively calculate the maximum value and minimum value in the two-dimensional direction, the maximum value and minimum value in the two-dimensional direction are respectively I 1max ,I 1min ,I 2max and I 2minThe calculation formula is as follows:

[0057] I 1max =max{(I xmax ),(I ymax )},

[0058] I 1min =min{(I xmin ),(I ymin )},

[0059] I 2max =max{(I xymax ),(I yxmax )},

[0060] I 2min =min{(I xymin ),(I yxmin )},

[0061] Among them, I 1max and I 1min Respectively represent the maximum and minimum values ​​of the two-dimensional operator in the horizontal direction (x direction and y direction); I 2max and I 2min They represent the maximum and minimum values ​​of the two-dimensional operator in the diagonal direction (oblique downward and oblique upward) respectively;

[0062] S4, the threshold value T of the corresponding coordinate in the threshold matrix obtained in S3-1 i,j The threshold is calculated by combining the maximum and minimum values ​​calculated in S3-2. The calculation formula is as follows:

[0063] |I 1max +0.15*T i,j |or|I 2max +0.15*T i,j |,

[0064] |I 1min -0.15*T i,j |or|I 2min -0.15*T i,j |;

[0065] Gray value P of the target pixel i,j is greater than the threshold, that is, P i,j ≥|I 1max +0.15*T i,j | or P i,j ≥|I 2max +0.15*T i,j |, it is judged as a suspected overheating failure element,

[0066] Gray value P of the target pixel i,jis less than the threshold, that is, P i,j ≤|I 1min -0.15*T i,j | or P i,j ≤|I 2min -0.15*T i,j |;, judged as a suspected overcooling failure element;

[0067] S5, randomly collect another frame of image, and perform suspected failure element detection according to the above steps. When the two frames of infrared images are consistent with the overheating failure element or the undercooling failure element, the point is judged as a blind point.

Claims

1. A convolution sliding window multi-directional infrared image blind pixel detection system, characterized in that It includes an acquisition module, a display module and a detection module, wherein the acquisition module obtains the pixel value of the infrared image data to be tested, is connected to the single channel point of the reality module, and provides the image pixel matrix for the display module in the output mode of grayscale image data; The display module displays the infrared image and captures, records and saves the image; the display module is bidirectionally electrically connected to the detection module, the detection module performs sliding convolution on the infrared image displayed by the display module, generates a threshold matrix and determines the blind pixel point after calculation, the module has a built-in blind pixel table, stores the number and position of the blind pixels, and transmits them to the display module; A convolution sliding window multi-directional infrared image blind pixel detection system method, Its characteristics include the following steps: S1, read the infrared image pixel value through the acquisition module, and output it in the form of a matrix to be displayed on the display module; S2-1, expanding the edge of the image through the detection module to expand the size of the output image; S2-2, respectively calculate the maximum and minimum values ​​of the one-dimensional operator in the cross direction and diagonal direction of the infrared image pixel point in S1, and the maximum and minimum values ​​of the one-dimensional operator in the x direction are respectively calculated by I xmax and I xmin Indicates that the maximum and minimum values ​​of the one-dimensional operator in the y direction are represented by I ymax and I ymin The maximum and minimum values ​​of the diagonal one-dimensional operator are represented by I xymax ,I xymin ,I yxmax and I yxmin The calculation formula is as follows: I xmax =max{(2*P i,j-1 -P ij-i ),(2*P i,j+1 -P i,j+2 ),(P i,j-1 ),(P i,j+1 )}, I xmin =min{(2*P i,j-1 -P i,j-2 ),(2*P i,j+1 -P i,j+2 ),(P i,j-1 ),(P i,j+1 )}, I ymax =max{(2*P i-1,j -P i-2,j ),(2*P i+1,j -P i+2,j ),(P i-1,j ),(P i+1,j )}, I ymin =min{(2*P i-1,j -P i-2,j ),(2*P i+1,j -P i+2,j ),(P i-1,j ),(P i+1,j )}, I xymax =max{(2*P i-1,j-1 -P i-2,j-2 ),(2*P i+1,j+1 -P i+2,j+2 ),(P i-1,j-1 ),(P i+1,j+1 )}, I xymin =min{(2*P i-1,j-1 -P i-2,j-2 ),(2*P i+1,j+1 -P i+2,j+2 ),(P i-1,j-1 ),(P i+1,j+1 )},I yxmax =max{(2*P i-1,j+1 -P i-2,j+2 ),(2*P i+1,j-1 -P i+2,j-2 ),(P i-1,j+1 ),(P i+1,j-1 )},I yxmin =min{(2*P i-1,j+1 -P i-2,j+2 ),(2*P i+1,j-1 -P i+2,j-2 ),(P i-1,j+1 ),(P i+1,j-1 )}; Wherein, P represents the pixel point of the image, i and j represent the x-direction and y-direction coordinates of the pixel point respectively. S3-1, performing sliding convolution on the image output in S2-1 to obtain a threshold matrix; S3-2, according to the maximum value and minimum value of the one-dimensional operator calculated in S2-2, respectively calculate the maximum value and minimum value in the two-dimensional direction, the maximum value and minimum value in the two-dimensional direction are respectively I 1max ,I 1min ,I 2max and I 2min The calculation formula is as follows: I 1max =max{(I xmax ),(I ymax )}, IN 1min =min{(I xmin ),(IN ymin )}, I 2max =max{(I xymax ),(I yxmax )}, IN 2min =min{(I xymin ),(IN yxmin )}; Among them, I 1max and I 1min Respectively represent the maximum and minimum values ​​of the two-dimensional operator in the x-direction and y-direction; I 2max and I 2min Respectively represent the maximum and minimum values ​​of the two-dimensional operator in the oblique downward and upward directions; S4, the threshold value T of the corresponding coordinate in the threshold matrix obtained in S3-1 i,j The threshold is calculated by combining the maximum and minimum values ​​calculated in S3-2. The calculation formula is as follows: |I 1max +0.15*T i,j | or |I 2max +0.15*T i,j |, |I 1min -0.15*T i,j | or | I 2min -0.15*T i,j |; Gray value P of the target pixel i,j Greater than the threshold, P i,j ≥|I 1max +0.15*T i,j | or P i,j ≥|I 2max +0.15*T i,j |, it is judged as a suspected overheating failure element, Gray value P of the target pixel i,j is less than the threshold, that is, P i,j ≤|I 1min -0.15*T i,j | or P i,j ≤|I 2min -0.15*T i,j |;, judged as a suspected overcooling failure element; S5-Extrapolate the pixel points according to the above method until the entire image is completely traversed to obtain multiple points suspected to be failure elements. When the two frames of infrared images are consistent overheating failure elements or consistent undercooling failure elements, the point is judged as a blind pixel point.

2. A convolution sliding window multi-directional infrared image blind pixel detection system method as claimed in claim 1, characterized in that In S2, the edge expansion of the image is performed by mirror expansion. The size of the image is M×N, and the 2 rows and 2 columns of its edge are mirror expanded respectively. The mirror expansion fills the two columns of the edge of the original image into the required expansion area after mirroring symmetry, and fills 0 in the places where there are no corresponding values ​​at the four corners, and finally obtains an image of size (M+4)×(N+4).

3. The multi-directional infrared image blind pixel detection system method of the convolution sliding window as claimed in claim 1 is characterized in that When the detection object is a video, the next frame of infrared image data is collected, and the points in the data are detected for suspected failure elements. If the failure element positions of the two frames of infrared image data are consistent, the consistent points are determined as blind pixels.

Images