An infrared focal plane array attitude determination image defective pixel repair method and device

Abstract

The application discloses an infrared focal plane array attitude measurement image defect pixel repairing method and device, and belongs to the technical field of infrared imaging and image processing, which comprises the following steps: performing linear fitting on each row and each column of an infrared focal plane array attitude measurement image containing a fault defect, and calculating row residual sum of squares and column residual sum of squares; arranging the row residual sum of squares and the column residual sum of squares in descending order, taking the intersection of the row and the column corresponding to the K largest residuals, and generating an initial positioning mask of the defect pixel; for each defect pixel, adaptively disabling the pollution direction in four one-dimensional windows of a horizontal direction, a vertical direction and double diagonal lines, and establishing a multi-direction robust regression model by using the pixel samples which are not polluted, and then extrapolating to obtain a multi-direction prediction value of the defect pixel; and using the average value of the multi-direction prediction value of each defect pixel as the repair gray value of the defect pixel. The application can realize blind diagnosis and high-precision repair of defects under the conditions of a single frame, no training and no external reference.

Description

Technical Field

[0001] This invention relates to a method and apparatus for repairing defective pixels in infrared focal plane array attitude measurement images, belonging to the field of infrared imaging and image processing technology. Background Technology

[0002] Infrared focal plane arrays are the core photosensitive devices in missile-borne and airborne high-precision attitude measurement systems. The grayscale fidelity of their output images directly determines the accuracy of star point extraction and attitude calculation. Due to manufacturing defects, temperature cycling, and space radiation, short linear bad pixels often appear on the focal plane, leading to abrupt changes in local grayscale and thus introducing attitude errors. Therefore, fault diagnosis and restoration of true grayscale in the original image before star point extraction are essential steps to improve the reliability of the attitude measurement system.

[0003] Currently, fault diagnosis of missile-borne infrared focal plane array sensors mainly relies on a "texture comparison + removal" scheme, and its primary purpose is to diagnose fault surface sources, without addressing grayscale restoration after diagnosis. For example, Chinese patent CN117968734A discloses a fault diagnosis method and device for missile-borne infrared focal plane array sensors. Its principle is as follows: the missile-borne infrared focal plane array first corrects the output using the attenuation coefficient of each element, and then maps the corrected matrix into a real-time infrared image; using the previous frame's fault-free elevation angle as a reference, a fault-free LBP texture matrix and its complementary matrix are generated; the LBP texture of the current frame is calculated by improving the LBP algorithm and compared with the reference to complete the fault element diagnosis. Although this patent can achieve fault element diagnosis, it does not involve grayscale restoration. In addition, traditional fault diagnosis methods are sensitive to short linear continuous defects and are prone to misjudging entire rows as bad elements, lacking sufficient detection capability for short linear defects.

[0004] Therefore, existing technologies lack an integrated method that can simultaneously perform blind diagnosis of short linear defects and high-precision grayscale restoration within a single frame image without requiring external benchmarks or training data. Summary of the Invention

[0005] The purpose of this invention is to provide a method and device for repairing defective pixels in infrared focal plane array pose measurement images. It can quickly locate defects from the original image through planar residual coupling technology and perform grayscale repair through robust regression technology, resulting in small repair error. This invention can achieve blind defect diagnosis and high-precision repair under single-frame, no training, and no external reference conditions.

[0006] To achieve the above objectives / to solve the above technical problems, the present invention is implemented using the following technical solution:

[0007] In a first aspect, the present invention provides a method for repairing defective pixels in an infrared focal plane array attitude measurement image, comprising:

[0008] Linear fitting is performed row by row and column by column on the attitude measurement images of the infrared focal plane array containing faults, and the sum of squares of row residuals and column residuals are calculated.

[0009] Arrange the sum of squared row residuals and the sum of squared column residuals in descending order, and take the intersection of the rows and columns corresponding to the K largest residuals to generate the initial localization mask for the defective pixels;

[0010] For each defective pixel within the initial localization mask, the contamination direction is adaptively disabled within four one-dimensional windows: horizontal, vertical, and double diagonal. A robust regression model with multiple directions is established using uncontaminated pixel samples, and the predicted value of the defective pixel with multiple directions is extrapolated.

[0011] Grayscale restoration is performed on defective pixels based on the multi-directional predicted values ​​of each defective pixel.

[0012] In conjunction with the first aspect, further, the step of performing linear fitting row-by-row and column-by-column on the attitude measurement image of the infrared focal plane array containing fault defects, and calculating the sum of squared residuals for rows and columns, includes:

[0013] For each row of pixels in the infrared focal plane array attitude measurement image, with column number c as the independent variable, the pixel grayscale value... Establish a least squares linear model with the variable as the dependent variable:

[0014] ;

[0015] in, Let r be the intercept of the line. Let r be the slope of line r. Let r be the residual of r rows and c columns, r = 1, 2, ..., H, c = 1, 2, ..., W, where H is the total number of rows in the infrared focal plane array attitude measurement image and W is the total number of columns in the infrared focal plane array attitude measurement image.

[0016] After fitting the data to all pixels in r rows, calculate the sum of squared row residuals for r rows. :

[0017] ;

[0018] For each column of pixels in the infrared focal plane array attitude measurement image, with row number r as the independent variable, the pixel grayscale value... Establish a least squares linear model with the variable as the dependent variable:

[0019] ;

[0020] in, The intercept of column c, The slope of column c;

[0021] After fitting the data to all pixels in column c, calculate the sum of squared column residuals for column c. :

[0022] .

[0023] In conjunction with the first aspect, further, the step of arranging the row residual sum of squares and column residual sum of squares in descending order, taking the row and column intersections corresponding to the top K largest residuals, and generating the initial localization mask for the defective pixels includes:

[0024] Arrange the sum of squared row residuals of all rows in the infrared focal plane array attitude measurement image in descending order, and take the row numbers corresponding to the first K largest residuals to form a row number set R;

[0025] Sort the sum of squared column residuals of all columns in the infrared focal plane array attitude measurement image in descending order, and take the column numbers corresponding to the first K largest residuals to form a column number set C;

[0026] Based on the set of line numbers and column set An initial positioning mask is generated through set operations. , Each element in the dataset is generated according to the following rules:

[0027] ;

[0028] ;

[0029] in, Indicates the initial positioning mask The element in row r and column c, This indicates that the pixels in row r and column c of the infrared focal plane array attitude measurement image are the initially located defect pixels. This indicates that the pixel in row r and column c of the infrared focal plane array attitude measurement image is a defect pixel that is not initially located. r = 1, 2, ..., H, c = 1, 2, ..., W, where H is the total number of rows in the infrared focal plane array attitude measurement image and W is the total number of columns in the infrared focal plane array attitude measurement image.

[0030] In conjunction with the first aspect, the formula for calculating K is as follows:

[0031] ;

[0032] in, This is the preset maximum pixel length for defects.

[0033] In conjunction with the first aspect, further, the adaptive disabling of contamination directions within four one-dimensional windows (horizontal, vertical, and double diagonal) for each defective pixel within the initial positioning mask includes:

[0034] For the defective pixel in row r and column c within the initial positioning mask, calculate the number of defects in its horizontal neighborhood. Number of defects in vertical neighborhood :

[0035] ;

[0036] ;

[0037] in, This represents the element in row r and column j of the initial positioning mask. This represents the element in row i and column c of the initial positioning mask. , R is the set of row numbers corresponding to the first K largest residuals, and C is the set of column numbers corresponding to the first K largest residuals;

[0038] like If the defective pixel is not visible, the horizontal window is disabled; otherwise, the vertical window is disabled.

[0039] The double diagonal window for all defective pixels within the initial positioning mask is always enabled.

[0040] In conjunction with the first aspect, further, the step of establishing a multi-directional robust regression model using uncontaminated pixel samples to extrapolate and obtain multi-directional predicted values ​​for the defective pixel includes:

[0041] Let the one-dimensional coordinates of the robust regression of the horizontal window be... Then the set of pure samples in the horizontal window for:

[0042] ;

[0043] Where L is the half-length of the one-dimensional window, This refers to the column number corresponding to the defective pixel. This represents the element in row r and column t of the initial localization mask, where r is the row number corresponding to the defective pixel;

[0044] Based on the set of pure samples in the horizontal window Establish a robust regression model with a horizontal window:

[0045] ;

[0046] in, This is the intercept of the horizontal window. The slope of the horizontal window. This represents the pixel grayscale value in row r and column t of the infrared focal plane array attitude measurement image. For piecewise weight functions;

[0047] The expression is as follows:

[0048] ;

[0049] ;

[0050] in, As variables, , The standard deviation of the pure sample;

[0051] Based on the robust regression model of the horizontal window, at the defect point Extrapolation to obtain lateral prediction values :

[0052] ;

[0053] Let the one-dimensional coordinates of the vertical window robust regression be... The set of pure samples in the vertical window. for:

[0054] ;

[0055] in, This represents the element in row g and column c of the initial positioning mask;

[0056] Based on the set of pure samples in the vertical window Establish a robust regression model with a vertical window:

[0057] ;

[0058] in, This is the intercept of the vertical window. The slope of the vertical window. This represents the pixel grayscale value in row g and column c of the infrared focal plane array attitude measurement image;

[0059] Based on the robust regression model of the vertical window, at the defect point extrapolation of longitudinal predicted values :

[0060] ;

[0061] Let the robust regression one-dimensional coordinates in the diagonal↗ direction be... The pixel coordinates in the infrared focal plane array attitude measurement image are based on the current defect pixel coordinates. For standard conversion The clean sample set of the window in the diagonal ↗ direction for:

[0062] ;

[0063] in, Indicates the initial positioning mask OK, Column elements;

[0064] Based on the set of pure samples in the diagonal ↗ direction window Establish a robust regression model for the diagonal↗ direction window:

[0065] ;

[0066] in, The intercept of the window in the diagonal ↗ direction. The slope of the window in the diagonal ↗ direction. Indicating the infrared focal plane array attitude measurement image OK, The pixel grayscale value of the column;

[0067] Based on the robust regression model of the diagonal ↗ direction window, s is centered to 0, and the predicted diagonal value in the ↗ direction is extrapolated. :

[0068] ;

[0069] Let the robust regression one-dimensional coordinates in the diagonal↖ direction be... The pixel coordinates in the infrared focal plane array attitude measurement image are based on the current defect pixel coordinates. For standard conversion The clean sample set of the window in the diagonal direction ↖ for:

[0070] ;

[0071] in, Indicates the initial positioning mask OK, Column elements;

[0072] Based on the set of pure samples in the diagonal↖ direction window Establish a robust regression model for the diagonal↖ direction window:

[0073] ;

[0074] in, The intercept of the window in the diagonal ↖ direction. The slope of the window in the diagonal ↖ direction. Indicating the infrared focal plane array attitude measurement image OK, The pixel grayscale value of the column;

[0075] Based on the robust regression model of the diagonal ↖ direction window, with f centered at 0, the predicted diagonal value in the ↖ direction is extrapolated. :

[0076] .

[0077] In conjunction with the first aspect, furthermore, the average value of the multi-directional predicted values ​​of each defective pixel is used as the repair grayscale value of the defective pixel for grayscale repair.

[0078] The average value of the multi-directional predicted values ​​of the defective pixels is as follows:

[0079] ;

[0080] in, This represents the average of the multi-directional predicted values ​​of the defect pixels in row r and column c of the infrared focal plane array attitude measurement image. This represents the lateral predicted value of the defective pixel. This represents the vertical predicted value of the defective pixel. This is the predicted diagonal value in the ↗ direction for the defective pixel. This is the predicted diagonal value in the ↖ direction for the defective pixel.

[0081] In a second aspect, the present invention provides an infrared focal plane array attitude measurement image defect pixel repair device, comprising:

[0082] The planar residual coupling module is used to perform linear fitting row by row and column by column on the attitude measurement image of the infrared focal plane array containing faults and defects, and to calculate the sum of squares of the row residuals and the sum of squares of the column residuals.

[0083] The fault diagnosis module is used to sort the row residual sum of squares and column residual sum of squares in descending order, take the row and column intersection corresponding to the K largest residuals, and generate the initial localization mask for defect pixels.

[0084] The robust regression module is used to adaptively disable contamination directions in four one-dimensional windows (horizontal, vertical, and double diagonal) for each defective pixel within the initial localization mask, and to build a multi-directional robust regression model using uncontaminated pixel samples to extrapolate the multi-directional predicted value of the defective pixel.

[0085] The grayscale restoration module is used to restore the grayscale of defective pixels based on the multi-directional prediction values ​​of each defective pixel.

[0086] In conjunction with the second aspect, further, the planar residual coupling module is specifically used for

[0087] For each row of pixels in the infrared focal plane array attitude measurement image, with column number c as the independent variable, the pixel grayscale value... Establish a least squares linear model with the variable as the dependent variable:

[0088] ;

[0089] in, Let r be the intercept of the line. Let r be the slope of line r. Let r be the residual of r rows and c columns, r = 1, 2, ..., H, c = 1, 2, ..., W, where H is the total number of rows in the infrared focal plane array attitude measurement image and W is the total number of columns in the infrared focal plane array attitude measurement image.

[0090] After fitting the data to all pixels in r rows, calculate the sum of squared row residuals for r rows. :

[0091] ;

[0092] For each column of pixels in the infrared focal plane array attitude measurement image, with row number r as the independent variable, the pixel grayscale value... Establish a least squares linear model with the variable as the dependent variable:

[0093] ;

[0094] in, The intercept of column c, The slope of column c;

[0095] After fitting the data to all pixels in column c, calculate the sum of squared column residuals for column c. :

[0096] .

[0097] In conjunction with the second aspect, the robust regression module is further used specifically for:

[0098] For the defective pixel in row r and column c within the initial positioning mask, calculate the number of defects in its horizontal neighborhood. Number of defects in vertical neighborhood :

[0099] ;

[0100] ;

[0101] in, This represents the element in row r and column j of the initial positioning mask. This represents the element in row i and column c of the initial positioning mask. , R is the set of row numbers corresponding to the first K largest residuals, and C is the set of column numbers corresponding to the first K largest residuals;

[0102] like If the defective pixel is not visible, the horizontal window is disabled; otherwise, the vertical window is disabled.

[0103] The double diagonal window for all defective pixels within the initial positioning mask is always enabled;

[0104] Let the one-dimensional coordinates of the robust regression of the horizontal window be... Then the set of pure samples in the horizontal window for:

[0105] ;

[0106] Where L is the half-length of the one-dimensional window, This refers to the column number corresponding to the defective pixel. This represents the element in row r and column t of the initial localization mask, where r is the row number corresponding to the defective pixel;

[0107] Based on the set of pure samples in the horizontal window Establish a robust regression model with a horizontal window:

[0108] ;

[0109] in, This is the intercept of the horizontal window. The slope of the horizontal window. This represents the pixel grayscale value in row r and column t of the infrared focal plane array attitude measurement image. For piecewise weight functions;

[0110] The expression is as follows:

[0111] ;

[0112] ;

[0113] in, As variables, , The standard deviation of the pure sample;

[0114] Based on the robust regression model of the horizontal window, at the defect point Extrapolation to obtain lateral prediction values :

[0115] ;

[0116] Let the one-dimensional coordinates of the vertical window robust regression be... The set of pure samples in the vertical window. for:

[0117] ;

[0118] in, This represents the element in row g and column c of the initial positioning mask;

[0119] Based on the set of pure samples in the vertical window Establish a robust regression model with a vertical window:

[0120] ;

[0121] in, This is the intercept of the vertical window. The slope of the vertical window. This represents the pixel grayscale value in row g and column c of the infrared focal plane array attitude measurement image;

[0122] Based on the robust regression model of the vertical window, at the defect point extrapolation of longitudinal predicted values :

[0123] ;

[0124] Let the robust regression one-dimensional coordinates in the diagonal↗ direction be... The pixel coordinates in the infrared focal plane array attitude measurement image are based on the current defect pixel coordinates. For standard conversion The clean sample set of the window in the diagonal ↗ direction for:

[0125] ;

[0126] in, Indicates the initial positioning mask OK, Column elements;

[0127] Based on the set of pure samples in the diagonal ↗ direction window Establish a robust regression model for the diagonal↗ direction window:

[0128] ;

[0129] in, The intercept of the window in the diagonal ↗ direction. The slope of the window in the diagonal ↗ direction. Indicating the infrared focal plane array attitude measurement image OK, The pixel grayscale value of the column;

[0130] Based on the robust regression model of the diagonal ↗ direction window, s is centered to 0, and the predicted diagonal value in the ↗ direction is extrapolated. :

[0131] ;

[0132] Let the robust regression one-dimensional coordinates in the diagonal↖ direction be... The pixel coordinates in the infrared focal plane array attitude measurement image are based on the current defect pixel coordinates. For standard conversion The clean sample set of the window in the diagonal direction ↖ for:

[0133] ;

[0134] in, Indicates the initial positioning mask OK, Column elements;

[0135] Based on the set of pure samples in the diagonal↖ direction window Establish a robust regression model for the diagonal↖ direction window:

[0136] ;

[0137] in, The intercept of the window in the diagonal ↖ direction. The slope of the window in the diagonal ↖ direction. Indicating the infrared focal plane array attitude measurement image OK, The pixel grayscale value of the column;

[0138] Based on the robust regression model of the diagonal ↖ direction window, with f centered at 0, the predicted diagonal value in the ↖ direction is extrapolated. :

[0139] .

[0140] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0141] This invention proposes a method and apparatus for repairing defective pixels in infrared focal plane array pose measurement images. By performing intersection calculations on the row and column residuals of the current frame, defects are located. Furthermore, by adaptively disabling contamination directions, the overlap rate between the fault location mask and the actual defective pixels is improved, achieving more accurate fault diagnosis. Building upon fault diagnosis, this invention also introduces grayscale restoration technology, employing multi-directional robust regression for grayscale value restoration. This results in small grayscale value restoration errors, requiring no external reference image or training data, and enabling blind detection and restoration within the infrared focal plane. This significantly reduces pose measurement errors introduced by bad pixels, improving system reliability and imaging quality.

[0142] This invention enables blind defect diagnosis and high-precision repair under single-frame, no-training, and no-external-reference conditions, and can be widely used in attitude measurement systems in no-reference scenarios such as missile-borne and space-borne systems. Attached Figure Description

[0143] Figure 1 The diagram shown is a flowchart illustrating the steps of a method for repairing defective pixels in an infrared focal plane array attitude measurement image provided by the present invention.

[0144] Figure 2 The image shown is a schematic diagram of an infrared focal plane array attitude measurement image without any faults or defects in an embodiment of the present invention.

[0145] Figure 3 The image shown is a schematic diagram of an infrared focal plane array attitude measurement image with fault defects incorporated in an embodiment of the present invention.

[0146] Figure 4 The image shown is a schematic diagram of the infrared focal plane array attitude measurement image repaired by the method of the present invention in an embodiment of the present invention;

[0147] Figure 5 The diagram shown is a structural schematic of an infrared focal plane array attitude measurement image defect pixel repair device provided by the present invention. Detailed Implementation

[0148] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0149] Example 1

[0150] This embodiment describes a method for repairing defective pixels in infrared focal plane array attitude measurement images, such as... Figure 1 As shown, the specific steps include the following:

[0151] Step A: Obtain the original image containing faults and defects from the infrared focal plane output.

[0152] Step B: Perform linear fitting on the original image row by row and column by column, calculate the sum of squared residuals of the rows and columns, and complete the planar residual coupling.

[0153] Suppose the original image has H rows and W columns of pixels. For each row r (r=1,2,…,H) in the original image, with column number c as the independent variable, the pixel grayscale value… Establish a least squares linear model with the variable as the dependent variable:

[0154] (1)

[0155] in, Let r be the intercept of the line. Let r be the slope of line r. The residual is the difference between rows r and columns c.

[0156] After fitting the data to all pixels in r rows, calculate the sum of squared row residuals for r rows. :

[0157] (2)

[0158] In this invention, The larger the value, the worse the linearity of the row, and the higher the possibility that the row has defects.

[0159] Similarly, for each column c (c=1,2,…,W) in the original image, with row number r as the independent variable, the pixel grayscale value... Establish a least squares linear model with the variable as the dependent variable:

[0160] (3)

[0161] in, The intercept of column c, The slope of column c.

[0162] After fitting the data to all pixels in column c, calculate the sum of squared column residuals for column c. :

[0163] (4)

[0164] In this invention, The larger the value, the worse the linearity of the column, and the higher the possibility that the column has defects.

[0165] This invention achieves “planar residual coupling” through formulas (2) and (4).

[0166] Step C: Sort the row residual sum of squares and column residual sum of squares in descending order, and take the row and column intersections corresponding to the K largest residuals to generate the initial localization mask for defect pixels.

[0167] right and Sort them in descending order and take the set of row numbers corresponding to the K largest residuals. and column set ,in:

[0168] (5)

[0169] in, The maximum pixel length of the defect is preset to ensure that the number of locations matches the defect size. This invention can set the maximum pixel length of the defect based on experience. Since this invention focuses on diagnosing short linear defects, The values ​​are generally not large.

[0170] Based on the set of line numbers and column set An initial positioning mask is generated through set operations. , Each element in the dataset is generated according to the following rules:

[0171] (6)

[0172] (7)

[0173] in, Indicates the initial positioning mask The element in row r and column c, This indicates that the pixels in row r and column c of the infrared focal plane array attitude measurement image are the initially located defect pixels. This indicates that the pixels in row r and column c of the infrared focal plane array attitude measurement image are defective pixels that are not initially located.

[0174] Step D: For each defective pixel within the initial positioning mask, adaptively disable the contamination direction within four one-dimensional windows (horizontal, vertical, and double diagonal), and use uncontaminated pixel samples to establish a multi-directional robust regression model to extrapolate the multi-directional predicted value of the defective pixel.

[0175] Step D01, Regarding the disabling of adaptive orientation Each defective pixel Calculate the number of defects in its horizontal neighborhood. Number of defects in vertical neighborhood The calculation formula is as follows:

[0176] (8)

[0177] (9)

[0178] in, This represents the element in row r and column j of the initial positioning mask. This represents the element in row i and column c of the initial positioning mask.

[0179] like If the defective pixel is not detected, horizontal extrapolation is disabled; otherwise, vertical extrapolation is disabled. In this invention, the double diagonal window is always enabled to ensure the purity of the regression sample.

[0180] Step D02: Based on the determination of the disabled contamination direction, a multi-directional robust regression model is established using uncontaminated pixel samples (also known as clean samples).

[0181] The robust regression model for the four one-dimensional windows (horizontal, vertical, and double diagonal) is as follows:

[0182] (1) Let the one-dimensional coordinates of the horizontal window robust regression (when not disabled) be as follows: Then the set of pure samples in the horizontal window It can be represented as:

[0183] (10)

[0184] Where L is the half-length of the one-dimensional window, This represents the element in row r and column t of the initial positioning mask.

[0185] Based on the set of pure samples in the horizontal window A robust regression model with a horizontal window is established, as follows:

[0186] (11)

[0187] in, This is the intercept of the horizontal window. The slope of the horizontal window. This represents the pixel grayscale value in row r and column t of the infrared focal plane array attitude measurement image. The piecewise weight function is expressed as follows:

[0188] (12)

[0189] (13)

[0190] in, As variables, , This represents the standard deviation of the pure sample.

[0191] (2) Let g be the one-dimensional coordinate of the vertical window robust regression (when not disabled), then the pure sample set of the vertical window is... It can be represented as:

[0192] (14)

[0193] in, This represents the element in row g and column c of the initial positioning mask.

[0194] Based on the set of pure samples in the vertical window A robust regression model for the vertical window is established, as follows:

[0195] (15)

[0196] in, This is the intercept of the vertical window. The slope of the vertical window. This represents the pixel grayscale value in row g and column c of the infrared focal plane array attitude measurement image.

[0197] (3) Robust regression with double diagonal window is always enabled. For the diagonal ↗ direction, let the one-dimensional coordinate be... The pixel coordinates in the infrared focal plane array attitude measurement image are based on the current defect pixel coordinates. For standard conversion Then the pure sample set of the window in the diagonal ↗ direction is uncontaminated. It can be represented as:

[0198] (16)

[0199] in, Indicates the initial positioning mask OK, The elements of the column.

[0200] Based on the set of pure samples in the diagonal ↗ direction window A robust regression model for the diagonal↗ direction window is established, as follows:

[0201] (17)

[0202] in, The intercept of the window in the diagonal ↗ direction. The slope of the window in the diagonal ↗ direction. Indicating the infrared focal plane array attitude measurement image OK, The pixel grayscale value of the column.

[0203] Similarly, for the diagonal ↖ direction, let the one-dimensional coordinate be... The pixel coordinates in the infrared focal plane array attitude measurement image are based on the current defect pixel coordinates. For standard conversion Then the pure sample set of the window in the diagonal ↖ direction is uncontaminated. It can be represented as:

[0204] (18)

[0205] in, Indicates the initial positioning mask OK, The elements of the column.

[0206] Based on the set of pure samples in the diagonal↖ direction window A robust regression model for the diagonal↖ direction window is established, as follows:

[0207] (19)

[0208] in, The intercept of the window in the diagonal ↖ direction. The slope of the window in the diagonal ↖ direction. Indicating the infrared focal plane array attitude measurement image OK, The pixel grayscale value of the column.

[0209] Step D03: Based on the multi-directional robust regression model, extrapolate the multi-directional predicted value of the defective pixel.

[0210] Based on the robust regression model of the horizontal window, at the defect point Extrapolation to obtain lateral prediction values :

[0211] (20)

[0212] Based on the robust regression model of the vertical window, at the defect point extrapolation of longitudinal predicted values :

[0213] (twenty one)

[0214] Based on the robust regression model of the diagonal ↗ direction window, s is centered to 0, and the predicted diagonal value in the ↗ direction is extrapolated. :

[0215] (twenty two)

[0216] Based on the robust regression model of the diagonal ↖ direction window, with f centered at 0, the predicted diagonal value in the ↖ direction is extrapolated. :

[0217] (twenty three)

[0218] In this embodiment of the invention, robust regression uses only pixels that are not marked by a localization mask as training samples.

[0219] Step E: Perform grayscale repair on the defective pixels based on the multi-directional prediction values ​​of each defective pixel.

[0220] Specifically, in this embodiment of the invention, the average value of the multi-directional predicted values ​​of each defective pixel is used as the repair grayscale value of the defective pixel and then backfilled to achieve image restoration.

[0221] The average of the multi-directional predictions is:

[0222] (twenty four)

[0223] In this invention, only Backfill to the initial positioning mask At this point, the remaining pixels retain their original values, thus completing the information recovery.

[0224] The present invention can also use weighted average or median fusion to repair grayscale values.

[0225] This invention enables blind defect diagnosis and high-precision repair under single-frame, no-training, and no-external-reference conditions, and can be widely used in attitude measurement systems in no-reference scenarios such as missile-borne and space-borne systems.

[0226] To verify the fault diagnosis and repair effects of the present invention, the following experiments are provided in the embodiments of the present invention:

[0227] In such Figure 2 The original attitude measurement image output by the infrared focal plane array is shown with a fault defect inserted. To verify the diagnostic and repair capabilities of this invention for short linear continuous defects, this embodiment of the invention randomly generates two 1-3 pixel short line defects (horizontal or vertical) in the image. The defect pixel value is set to 0, while the remaining pixels retain their original values. The image after inserting the fault defect is shown below. Figure 3 As shown.

[0228] Using the method of the present invention to Figure 3 The faults and defects in the image are detected and repaired using grayscale. The repaired image is as follows: Figure 4 As shown, comparisons were made using professional tools. Figure 2 and Figure 4 It can be seen that the grayscale restoration error of the method of the present invention is only 3.62%, and the present invention achieves high-precision grayscale restoration.

[0229] Example 2

[0230] Based on the same inventive concept as Embodiment 1, this embodiment introduces a device for repairing defective pixels in infrared focal plane array attitude measurement images, such as... Figure 5 As shown, it includes:

[0231] The planar residual coupling module is used to perform linear fitting on row-by-row and column-by-column of the attitude measurement image of the infrared focal plane array containing faults and defects, and to calculate the sum of squares of the row residuals and the sum of squares of the column residuals.

[0232] The fault diagnosis module is used to sort the row residual sum of squares and column residual sum of squares in descending order, take the row and column intersection corresponding to the top K largest residuals, and generate the initial localization mask for defective pixels.

[0233] The robust regression module is used to adaptively disable contamination directions in four one-dimensional windows (horizontal, vertical, and double diagonal) for each defective pixel within the initial localization mask, and to build a multi-directional robust regression model using uncontaminated pixel samples to extrapolate the multi-directional predicted value of the defective pixel.

[0234] The grayscale restoration module is used to restore grayscale by using the average of the multi-directional predicted values ​​of each defective pixel as the restoration grayscale value of that defective pixel.

[0235] The specific functions of each module described above are explained in the relevant content of the method in Embodiment 1, and will not be repeated here.

[0236] In summary, compared with the prior art, the present invention has the following significant advantages:

[0237] 1. This invention can locate defects by performing intersection operations on the row and column residuals of the current frame, and is applicable to high-dynamic missile-borne scenarios with no reference in a single frame.

[0238] 2. Traditional statistical methods are sensitive to short, continuous linear defects and are prone to misjudging entire rows as bad pixels. However, this invention disables the contamination direction in real time, resulting in a very high overlap rate between the positioning mask and the actual defect pixels, making fault diagnosis more accurate.

[0239] 3. This invention uses a robust weight function to resist outliers and multi-directional average backfilling for repair, resulting in small grayscale value repair error.

[0240] 4. The entire process of this invention can be run within an embedded processing unit inside the infrared focal plane, without the need for additional storage or training data, saving valuable missile-borne and spaceborne resources and significantly improving system reliability.

[0241] This invention integrates "diagnosis" and "repair" into the same framework. It can complete blind detection and blind repair within the infrared focal plane without the need for external reference images or training data. This significantly reduces the attitude measurement error introduced by bad elements, improves system reliability and imaging quality, and effectively recovers image information and improves measurement accuracy compared to traditional schemes that only remove without repair.

[0242] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0243] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0244] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0245] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0246] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for repairing defective pixels in an infrared focal plane array attitude measurement image, characterized in that, include: Linear fitting is performed row by row and column by column on the attitude measurement images of the infrared focal plane array containing faults, and the sum of squares of row residuals and column residuals are calculated. Arrange the sum of squared row residuals and the sum of squared column residuals in descending order, and take the intersection of the rows and columns corresponding to the K largest residuals to generate the initial localization mask for the defective pixels; For each defective pixel within the initial localization mask, the contamination direction is adaptively disabled within four one-dimensional windows: horizontal, vertical, and double diagonal. A robust regression model with multiple directions is established using uncontaminated pixel samples, and the predicted value of the defective pixel with multiple directions is extrapolated. Grayscale restoration is performed on defective pixels based on the multi-directional prediction values ​​of each defective pixel; The process of establishing a multi-directional robust regression model using uncontaminated pixel samples and extrapolating to obtain multi-directional predicted values ​​for the defective pixel includes: Let the one-dimensional coordinates of the robust regression of the horizontal window be... Then the set of pure samples in the horizontal window for: ； Where L is the half-length of the one-dimensional window, This refers to the column number corresponding to the defective pixel. This represents the element in row r and column t of the initial localization mask, where r is the row number corresponding to the defective pixel; Based on the set of pure samples in the horizontal window Establish a robust regression model with a horizontal window: ； in, This is the intercept of the horizontal window. The slope of the horizontal window. This represents the pixel grayscale value in row r and column t of the infrared focal plane array attitude measurement image. For piecewise weight functions; The expression is as follows: ； ； in, As variables, , The standard deviation of the pure sample; Based on the robust regression model of the horizontal window, at the defect point Extrapolation to obtain lateral prediction values : ； Let the one-dimensional coordinates of the vertical window robust regression be... The set of pure samples in the vertical window. for: ； in, This represents the element in row g and column c of the initial positioning mask; Based on the set of pure samples in the vertical window Establish a robust regression model with a vertical window: ； in, This is the intercept of the vertical window. The slope of the vertical window. This represents the pixel grayscale value in row g and column c of the infrared focal plane array attitude measurement image; Based on the robust regression model of the vertical window, at the defect point extrapolation of longitudinal predicted values : ； Let the robust regression one-dimensional coordinates in the diagonal↗ direction be... The pixel coordinates in the infrared focal plane array attitude measurement image are based on the current defect pixel coordinates. For standard conversion The clean sample set of the window in the diagonal ↗ direction for: ； in, Indicates the initial positioning mask OK, Column elements; Based on the set of pure samples in the diagonal ↗ direction window Establish a robust regression model for the diagonal↗ direction window: ； in, The intercept of the window in the diagonal ↗ direction. The slope of the window in the diagonal ↗ direction. Indicating the infrared focal plane array attitude measurement image OK, The pixel grayscale value of the column; Based on the robust regression model of the diagonal ↗ direction window, s is centered to 0, and the predicted diagonal value in the ↗ direction is extrapolated. : ； Let the robust regression one-dimensional coordinates in the diagonal↖ direction be... The pixel coordinates in the infrared focal plane array attitude measurement image are based on the current defect pixel coordinates. For standard conversion The clean sample set of the window in the diagonal direction ↖ for: ； in, Indicates the initial positioning mask OK, Column elements; Based on the set of pure samples in the diagonal↖ direction window Establish a robust regression model for the diagonal↖ direction window: ； in, The intercept of the window in the diagonal ↖ direction. The slope of the window in the diagonal ↖ direction. Indicating the infrared focal plane array attitude measurement image OK, The pixel grayscale value of the column; Based on the robust regression model of the diagonal ↖ direction window, with f centered at 0, the predicted diagonal value in the ↖ direction is extrapolated. : 。 2. The method for repairing defective pixels in infrared focal plane array attitude measurement images according to claim 1, characterized in that, The process of performing linear fitting row-by-row and column-by-column on the attitude measurement image of the infrared focal plane array containing faults, and calculating the sum of squared residuals for rows and columns, includes: For each row of pixels in the infrared focal plane array attitude measurement image, with column number c as the independent variable, the pixel grayscale value... Establish a least squares linear model with the dependent variable as the variable: ； in, Let r be the intercept of the line. Let r be the slope of line r. Let r be the residual of r rows and c columns, r = 1, 2, ..., H, c = 1, 2, ..., W, where H is the total number of rows in the infrared focal plane array attitude measurement image and W is the total number of columns in the infrared focal plane array attitude measurement image. After fitting the data to all pixels in r rows, calculate the sum of squared row residuals for r rows. : ； For each column of pixels in the infrared focal plane array attitude measurement image, with row number r as the independent variable, the pixel grayscale value... Establish a least squares linear model with the dependent variable as the variable: ； in, The intercept of column c, The slope of column c; After fitting the data to all pixels in column c, calculate the sum of squared column residuals for column c. : 。 3. The method for repairing defective pixels in infrared focal plane array attitude measurement images according to claim 1, characterized in that, The step of arranging the row residual sum of squares and column residual sum of squares in descending order, taking the row and column intersections corresponding to the top K largest residuals, and generating the initial localization mask for defective pixels includes: Arrange the sum of squared row residuals of all rows in the infrared focal plane array attitude measurement image in descending order, and take the row numbers corresponding to the first K largest residuals to form a row number set R; Sort the sum of squared column residuals of all columns in the infrared focal plane array attitude measurement image in descending order, and take the column numbers corresponding to the first K largest residuals to form a column number set C; Based on the set of line numbers and column set An initial positioning mask is generated through set operations. , Each element in the dataset is generated according to the following rules: ； ； in, Indicates the initial positioning mask The element in row r and column c, This indicates that the pixels in row r and column c of the infrared focal plane array attitude measurement image are the initially located defect pixels. This indicates that the pixel in row r and column c of the infrared focal plane array attitude measurement image is a defect pixel that is not initially located. r = 1, 2, ..., H, c = 1, 2, ..., W, where H is the total number of rows in the infrared focal plane array attitude measurement image and W is the total number of columns in the infrared focal plane array attitude measurement image.

4. The method for repairing defective pixels in infrared focal plane array attitude measurement images according to claim 3, characterized in that, The formula for calculating K is as follows: ； in, This is the preset maximum pixel length for defects.

5. The method for repairing defective pixels in infrared focal plane array attitude measurement images according to claim 1, characterized in that, The adaptive disabling of contamination directions within four one-dimensional windows (horizontal, vertical, and double diagonals) for each defective pixel within the initial positioning mask includes: For the defective pixel in row r and column c within the initial positioning mask, calculate the number of defects in its horizontal neighborhood. Number of defects in vertical neighborhood : ； ； in, This represents the element in row r and column j of the initial positioning mask. This represents the element in row i and column c of the initial positioning mask. , R is the set of row numbers corresponding to the first K largest residuals, and C is the set of column numbers corresponding to the first K largest residuals; like If the defective pixel is not visible, the horizontal window is disabled; otherwise, the vertical window is disabled. The double diagonal window for all defective pixels within the initial positioning mask is always enabled.

6. The method for repairing defective pixels in infrared focal plane array attitude measurement images according to claim 1, characterized in that, The average value of the multi-directional predicted values ​​of each defective pixel is used as the repair grayscale value for that defective pixel for grayscale repair. The average value of the multi-directional predicted values ​​of the defective pixels is as follows: ； in, This represents the average of the multi-directional predicted values ​​of the defect pixels in row r and column c of the infrared focal plane array attitude measurement image. This represents the lateral predicted value of the defective pixel. This represents the vertical predicted value of the defective pixel. This is the predicted diagonal value in the ↗ direction for the defective pixel. This is the predicted diagonal value in the ↖ direction for the defective pixel.

7. A device for repairing defective pixels in an infrared focal plane array attitude measurement image, characterized in that, include: The planar residual coupling module is used to perform linear fitting row by row and column by column on the attitude measurement image of the infrared focal plane array containing faults and defects, and to calculate the sum of squares of the row residuals and the sum of squares of the column residuals. The fault diagnosis module is used to sort the row residual sum of squares and column residual sum of squares in descending order, take the row and column intersection corresponding to the K largest residuals, and generate the initial localization mask for defective pixels. The robust regression module is used to adaptively disable contamination directions in four one-dimensional windows (horizontal, vertical, and double diagonal) for each defective pixel within the initial localization mask, and to build a multi-directional robust regression model using uncontaminated pixel samples to extrapolate the multi-directional predicted value of the defective pixel. The grayscale restoration module is used to perform grayscale restoration on defective pixels based on the multi-directional prediction values ​​of each defective pixel. The process of establishing a multi-directional robust regression model using uncontaminated pixel samples and extrapolating to obtain multi-directional predicted values ​​for the defective pixel includes: Let the one-dimensional coordinates of the robust regression of the horizontal window be... Then the set of pure samples in the horizontal window for: ； Where L is the half-length of the one-dimensional window, This refers to the column number corresponding to the defective pixel. This represents the element in row r and column t of the initial localization mask, where r is the row number corresponding to the defective pixel; Based on the set of pure samples in the horizontal window Establish a robust regression model with a horizontal window: ； in, This is the intercept of the horizontal window. The slope of the horizontal window. This represents the pixel grayscale value in row r and column t of the infrared focal plane array attitude measurement image. For piecewise weight functions; The expression is as follows: ； ； in, As variables, , The standard deviation of the pure sample; Based on the robust regression model of the horizontal window, at the defect point Extrapolation to obtain lateral prediction values : ； Let the one-dimensional coordinates of the vertical window robust regression be... The set of pure samples in the vertical window. for: ； in, This represents the element in row g and column c of the initial positioning mask; Based on the set of pure samples in the vertical window Establish a robust regression model with a vertical window: ； in, This is the intercept of the vertical window. The slope of the vertical window. This represents the pixel grayscale value in row g and column c of the infrared focal plane array attitude measurement image; Based on the robust regression model of the vertical window, at the defect point extrapolation of longitudinal predicted values : ； Let the robust regression one-dimensional coordinates in the diagonal↗ direction be... The pixel coordinates in the infrared focal plane array attitude measurement image are based on the current defect pixel coordinates. For standard conversion The clean sample set of the window in the diagonal ↗ direction for: ； in, Indicates the initial positioning mask OK, Column elements; Based on the set of pure samples in the diagonal ↗ direction window Establish a robust regression model for the diagonal↗ direction window: ； in, The intercept of the window in the diagonal ↗ direction. The slope of the window in the diagonal ↗ direction. Indicating the infrared focal plane array attitude measurement image OK, The pixel grayscale value of the column; Based on the robust regression model of the diagonal ↗ direction window, s is centered to 0, and the predicted diagonal value in the ↗ direction is extrapolated. : ； Let the robust regression one-dimensional coordinates in the diagonal↖ direction be... The pixel coordinates in the infrared focal plane array attitude measurement image are based on the current defect pixel coordinates. For standard conversion The clean sample set of the window in the diagonal direction ↖ for: ； in, Indicates the initial positioning mask OK, Column elements; Based on the set of pure samples in the diagonal↖ direction window Establish a robust regression model for the diagonal↖ direction window: ； in, The intercept of the window in the diagonal ↖ direction. The slope of the window in the diagonal ↖ direction. Indicating the infrared focal plane array attitude measurement image OK, The pixel grayscale value of the column; Based on the robust regression model of the diagonal ↖ direction window, with f centered at 0, the predicted diagonal value in the ↖ direction is extrapolated. : 。 8. The infrared focal plane array attitude measurement image defect pixel repair device according to claim 7, characterized in that, The planar residual coupling module is specifically used for: For each row of pixels in the infrared focal plane array attitude measurement image, with column number c as the independent variable, the pixel grayscale value... Establish a least squares linear model with the dependent variable as the variable: ； in, Let r be the intercept of the line. Let r be the slope of line r. Let r be the residual of r rows and c columns, r = 1, 2, ..., H, c = 1, 2, ..., W, where H is the total number of rows in the infrared focal plane array attitude measurement image and W is the total number of columns in the infrared focal plane array attitude measurement image. After fitting the data to all pixels in r rows, calculate the sum of squared row residuals for r rows. : ； For each column of pixels in the infrared focal plane array attitude measurement image, with row number r as the independent variable, the pixel grayscale value... Establish a least squares linear model with the dependent variable as the variable: ； in, The intercept of column c, The slope of column c; After fitting the data to all pixels in column c, calculate the sum of squared column residuals for column c. : 。 9. The infrared focal plane array attitude measurement image defect pixel repair device according to claim 7, characterized in that, The robust regression module is specifically used for: For the defective pixel in row r and column c within the initial positioning mask, calculate the number of defects in its horizontal neighborhood. Number of defects in vertical neighborhood : ； ； in, This represents the element in row r and column j of the initial positioning mask. This represents the element in row i and column c of the initial positioning mask. , R is the set of row numbers corresponding to the first K largest residuals, and C is the set of column numbers corresponding to the first K largest residuals; like If the defective pixel is not visible, the horizontal window is disabled; otherwise, the vertical window is disabled. The double diagonal window for all defective pixels within the initial positioning mask is always enabled.

Images