Image correction method for linear array camera

Abstract

The embodiment of the invention provides an image correction method and device for a line-scan digital camera. The method comprises the following steps: acquiring a plurality of calibration images of a line-scan digital camera for a geometric calibration field; applying image space affine transformation to a pre-obtained rational polynomial coefficient model for the geometric calibration field to obtain a first correction model; correcting external orientation parameters corresponding to each scanning line in the first correction model to obtain a second correction model; for each scanning row in each calibration image, determining a plurality of residual errors between the plurality of observation image coordinates and the plurality of prediction image coordinates, and respectively determining distortion samples of a plurality of corresponding columns in the vertical rail direction and the along rail direction in the line-scan digital camera according to the plurality of residual errors; and determining first distortion values corresponding to the plurality of columns according to the distortion samples of the plurality of columns, performing data processing on the first distortion values based on a preset guide filtering algorithm to obtain second distortion values, and correcting the calibration image according to the second distortion values to obtain a corrected image.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing satellite image processing, and in particular to an image correction method for a linear array camera. Background Art

[0002] Compared with narrow-angle remote sensing satellite payloads, wide field of view (WFV) payloads can obtain a wider range of earth observation data in a single imaging, and can achieve higher temporal resolution with the same number of satellites. The Gaofen-1 (GF-1) WFV has similar spatial resolution and spectral bands as the Gaofen-6 (GF-6) WFV and the HJ-2A / B (HJ-2A / B) CCD payloads. Through network observation, these satellite payloads can further improve the temporal resolution of their joint earth observation data sets to less than one day. However, the first-level products publicly released by these payloads usually only provide rational polynomial coefficients (RPCs) and lack a strict sensor model. Usually, a large number of well-distributed ground control points (GCPs) are required, and complex methods with poor stability are used for geometric correction, such as terrain-dependent RPC models or thin plate spline (TPS) interpolation methods. For image products with only RPC models, due to the lack of a strict imaging geometry model, the traditional on-orbit geometric calibration method that relies on the separation of internal and external orientation parameters is no longer applicable.

[0003] Therefore, an image correction method for a linear array camera is needed. Summary of the invention

[0004] The embodiment of the present invention proposes an image correction method and device for a linear array camera. The method can focus only on the scan lines of the calibration image and minimize the influence of posture changes, thereby better separating the errors caused by external orientation parameters and CCD distortion.

[0005] The technical solution adopted by the present invention to solve the above technical problems is, on the one hand, to provide an image correction method for a line array camera, the method comprising:

[0006] Acquire multiple calibration images of the line array camera for the geometric calibration field;

[0007] Applying image-space affine transformation to the pre-acquired rational polynomial coefficient model for the geometric calibration field to obtain a first correction model;

[0008] For each calibration image, a plurality of ground control points are determined; for each scan line of each calibration image, based on the ground control points, the exterior orientation parameters corresponding to each scan line in the first correction model are corrected to obtain a second correction model;

[0009] For each calibration image, a plurality of ground check points are determined; for each scan line in each calibration image, a plurality of residuals between a plurality of observed image coordinates obtained through the plurality of ground check points and a plurality of predicted image coordinates obtained through the second correction model are determined, and distortion samples of a plurality of columns in the vertical track and along the track corresponding to the linear array camera are respectively confirmed according to the plurality of residuals;

[0010] According to the distortion samples of the multiple columns, first distortion values ​​corresponding to the multiple columns are determined, data processing is performed on the first distortion values ​​based on a preset guided filtering algorithm to obtain second distortion values, and the calibration image is corrected according to the second distortion values ​​to obtain a corrected image.

[0011] Specifically, determining a plurality of ground control points includes: determining a plurality of ground control points and / or

[0012] Determining a plurality of ground checkpoints includes: determining a plurality of ground checkpoints based on a pre-acquired high-precision geometric calibration field.

[0013] Specifically, according to the distortion samples of the multiple columns, first distortion values ​​corresponding to the multiple columns are determined, data processing is performed on the first distortion values ​​based on a preset guided filtering algorithm to obtain second distortion values, and the calibration image is corrected according to the second distortion value to obtain a corrected image, including:

[0014] Determine the number of distorted samples respectively possessed by each of the multiple columns, and determine known columns and unknown columns among the multiple columns, wherein the number of distorted samples possessed by the known columns is greater than or equal to a predetermined threshold, and the number of distorted samples possessed by the unknown columns is less than the predetermined threshold;

[0015] Determine, according to the distortion samples of the known column, the distortion value corresponding to the known column;

[0016] According to the distortion values ​​corresponding to the known columns, a thin plate spline function model for the distortion values ​​of the unknown columns is constructed, and the distortion values ​​of the unknown columns are obtained based on the thin plate spline function model;

[0017] Inputting the first distortion value of each column of the sensor as a guidance signal into the guidance filter to obtain the second distortion value of each column of the sensor;

[0018] The calibration image is corrected according to the distortion value to obtain a corrected image.

[0019] Specifically, the image-space affine transformation is applied to the pre-acquired rational polynomial coefficient model for the geometric calibration field, which can be expressed as:

[0020]

[0021] Where c and r are the normalized column and row coordinates of the image point on the image plane, respectively; (X, Y, Z) is the normalized object space coordinate, N c ,D c ,N r and D r is the polynomial term defined by the RPC model coefficients, and a0, a1, a2, b0, b1, b2 are the affine transformation parameters.

[0022] Specifically, a0, a1, b0, b1 are estimated by the objective function Q, which can be expressed as:

[0023]

[0024] Among them, Q is the objective function, (c i ,r i ) and (X i ,Y i ,Z i ) represent the normalized coordinates of the ith control point in the image plane and object space, n is the total number of control points in the row, N c ,D c ,N r and D r is the polynomial term defined by the RPC model coefficients, and a0, a1, b0, b1 are the affine transformation parameters.

[0025] Specifically, the distortion samples of the corresponding multiple columns in the vertical track and along the track in the linear array camera are respectively confirmed according to the multiple residuals, which can be expressed as:

[0026]

[0027] in, (c i ,r i ) and (X i ,Y i ,Z i ) represent the normalized coordinates of the ith control point in the image plane and object space, respectively, N c ,D c ,N r and D r is the polynomial term defined by the RPC model coefficients, a0, a1, b0, b1 are the affine transformation parameters, Δc i and Δr i Respectively represent the i-th ground checkpoint (c-th i The distortion samples in the vertical and along-track directions are shown in Figure 1.

[0028] Specifically, the thin plate spline function model can be expressed as:

[0029]

[0030] Among them, x represents the coordinates of the known column to be interpolated, x i represents the coordinates of the i-th ground control point, ψ(r) = r 2 logr is the radial basis kernel function, α i , β0 and β1 represent calculation coefficients, K n is the number of adjacent ground control points used for interpolation.

[0031] Preferably, the predetermined threshold is 20; determining the distortion value corresponding to the known column according to the distortion samples of the known column includes: determining the distortion value corresponding to the known column according to the median of the distortion samples of the known column.

[0032] Preferably, the guided filtering algorithm can be expressed as:

[0033]

[0034] Among them, f′(x) is the output signal, f(x) is the guidance signal, |ω x | represents the number of all windows containing x, b k =μ k -a k μ k ,μ k and They are respectively the guiding signal f(x) in the window ω k The mean and variance in , ∈ is the regularization parameter used to control the degree of smoothing, and ω is the window size.

[0035] In a second aspect, a linear array camera geometric distortion on-orbit calibration device is provided, the device comprising:

[0036] An acquisition unit is configured to acquire a plurality of calibration images of the linear array camera for a geometric calibration field;

[0037] A first model correction unit is configured to apply an image-space affine transformation to a pre-acquired rational polynomial coefficient model for the geometric calibration field to obtain a first correction model;

[0038] The second model correction unit is configured to determine a plurality of ground control points for each calibration image; for each scan line of each calibration image, based on the ground control points, correct the exterior orientation parameters corresponding to each scan line in the first correction model to obtain a second correction model;

[0039] The processing unit is configured to determine, for each calibration image, a plurality of ground check points; for each scan line in each calibration image, determine a plurality of residuals between a plurality of observed image coordinates obtained through the plurality of ground check points and a plurality of predicted image coordinates obtained through the second correction model, and confirm, according to the plurality of residuals, the corresponding plurality of distortion samples in the vertical and along-track directions of the line array camera;

[0040] The image correction unit is configured to determine first distortion values ​​corresponding to the multiple columns based on the distortion samples of the multiple columns, apply data processing to the first distortion value based on a preset guided filtering algorithm to obtain a second distortion value, and correct the calibrated image according to the second distortion value to obtain a corrected image. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0042] Figure 1 A flowchart of an image correction method for a line array camera provided by an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of an image correction method for a line array camera provided by an embodiment of the present invention;

[0044] Figure 3 A structural diagram of an image correction device for a line array camera provided by an embodiment of the present invention;

[0045] Figure 4 A data graph of a target camera provided by an embodiment of the present invention;

[0046] Figure 5 A residual map of a target camera distortion sample provided by an embodiment of the present invention;

[0047] Figure 6 A geometric distortion diagram of the target camera provided by an embodiment of the present invention in the vertical track direction and the along track direction;

[0048] Figure 7 The error distribution diagram before and after the image calibration provided by the embodiment of the present invention;

[0049] Figure 8 Image geometric error distribution and error map provided by the embodiments of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0052] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "plurality" means two or more.

[0053] In the description of the embodiments of the present application, the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or are the positions or positional relationships in which the product of the application is usually placed when in use. They are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0054] In the description of the embodiments of the present application, the terms "horizontal", "vertical", "overhanging", "vertical" and the like do not mean that the components are required to be absolutely horizontal or overhanging, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0055] Conventional on-orbit geometric calibration methods usually require the construction of a rigorous imaging geometry model with calibration parameters, and then use ground control points to correct the external orientation parameters of each calibration image, and then combine the ground control points of each calibration image to solve the internal calibration parameters. In an example, a rigorous imaging geometry model can be first constructed based on the imaging parameters of the sensor, which can be expressed as:

[0056]

[0057] Where, m is the imaging ratio; is the position vector of the satellite in the WGS84 coordinate system during imaging; Image point The corresponding object point coordinates in the WGS84 coordinate system; is the transformation matrix determined by the installation angle of the camera in the body coordinate system; R U Bias matrix to compensate for attitude and track measurement errors, measurement equipment and camera installation errors; It is the transformation matrix between the body coordinate system and the J2000 coordinate system obtained by the attitude determination equipment measurement and processing; is the transformation matrix of the J2000 coordinate system relative to the WGS84 coordinate system; (ψ x ,ψ y ) is the pointing angle of the pixel (x, y) in the camera coordinate system, and s is the column coordinate of the pixel.

[0058] Furthermore, the pixel pointing angle model can be given by formula (2):

[0059]

[0060] Where (x, y) is the image plane coordinate, (x0, y0) is the image principal point, f is the principal distance, and the polynomial coefficients (a0, a1, a2, a3, b0, b1, b2, b3) are internal calibration parameters. The initial values ​​of (a0, a1, a2, a3, b0, b1, b2, b3) can be calculated based on the image plane coordinates and the initial values ​​of the image principal point and principal distance.

[0061] Secondly, the bias matrix is ​​solved: for multiple calibration images, multiple ground control points are obtained by referring to the geometric calibration field. For multiple ground control points, in formula (1) are all known quantities, tanψ x (s) and tanψ y (s) can be calculated based on the initial values ​​of (a0, a1, a2, a3, b0, b1, b2, b3); using the ground control points of each image, the bias matrix R of each image can be estimated separately U .

[0062] Then solve the internal orientation elements: the bias matrix R estimated by each image U Substitute into equation (1), and simultaneously combine multiple ground control points of multiple calibration images to jointly solve the internal calibration parameters (a0, a1, a2, a3, b0, b1, b2, b3). Construct a rational function model and convert the solved bias matrix R U The internal calibration parameters (a0, a1, a2, a3, b0, b1, b2, b3) are substituted into formula (1), and a virtual control point grid is generated on multiple elevation surfaces according to the rigorous imaging geometry model. The virtual control points are then used to solve the rational function model that is independent of the terrain.

[0063] However, conventional geometric calibration methods require the construction of a rigorous imaging geometric model based on the imaging parameters of the sensor in order to separate and solve the external calibration parameters and the internal calibration parameters. However, conventional geometric calibration methods are not applicable when only rational function model data is provided and accurate imaging parameters cannot be obtained.

[0064] In order to solve the above technical problems, this specification proposes an image correction method for a linear array camera, through which the influence of camera posture changes can be minimized by focusing only on a single image row, so that the errors caused by external orientation parameters and CCD distortion can be better separated, thereby improving the accuracy of linear array push-broom remote sensing images that lack a strict imaging geometry model.

[0065] Figure 1 Flow chart of an image correction method for a linear array camera provided by an embodiment of the present invention. Figure 1 As shown, the method comprises at least the following steps:

[0066] Step 101: Acquire multiple calibration images of the line array camera for the geometric calibration field. Calibration images refer to images that have been calibrated (calibrated) in the fields of remote sensing, photogrammetry, etc.; geometric calibration field refers to a specific field specifically used for geometric calibration of sensors (especially remote sensing and imaging equipment). In different embodiments, the specific method of acquiring multiple calibration images for the geometric calibration field may be different. Figure 2 Schematic diagram of an image correction method for a line array camera provided by an embodiment of the present invention. Figure 2 In the illustrated embodiment, the reference image can first be reprojected to the image plane space of the calibration image using the initial rational polynomial coefficient (RPC) model and digital elevation model (DEM) data of the calibration image for matching the ground control points or ground checkpoints of a specific scan line. At this time, the geometric deviation between the reprojected reference image and the calibration image can be locally approximated as a translation.

[0067] Step 102: Apply image-space affine transformation to the pre-acquired rational polynomial coefficient model for the geometric calibration field to obtain a first correction model.

[0068] In one embodiment, the image-space affine transformation is applied to the pre-acquired rational polynomial coefficient model for the geometric calibration field, which can be expressed as:

[0069]

[0070] Where c and r are the normalized column and row coordinates of the image point on the image plane, respectively; (X, Y, Z) is the normalized object space coordinate, N c ,D c ,N r and D r is the polynomial term defined by the RPC model coefficients; a0, a1, a2, b0, b1, b2 are the affine transformation parameters.

[0071] For example, due to the inheritance of the errors of the internal and external orientation parameters in the strict imaging model, the image-space affine transformation is usually used to correct the deviation of the RPC model. The image-space affine transformation can be expressed as:

[0072]

[0073] Where c and r are the normalized column and row coordinates of the image point on the image plane, respectively; (X, Y, Z) is the normalized object space coordinate, N c ,D c ,N r and D r is the polynomial term defined by the RPC model coefficients; a0, a1, a2, b0, b1, b2 are the affine transformation parameters.

[0074] Step 103: for each calibration image, a plurality of ground control points are determined; for each scan line of each calibration image, based on the ground control points, the exterior orientation parameters corresponding to each scan line in the first correction model are corrected to obtain a second correction model;

[0075] In one embodiment, determining a plurality of ground control points includes: determining a plurality of ground control points based on a pre-acquired global reference image GRI and / or determining a plurality of ground check points includes: determining a plurality of ground check points based on a pre-acquired high-precision geometric calibration field.

[0076] For example, in order to achieve sub-pixel accuracy, a phase correlation (PC) image matching method is used to perform phase correlation matching in the frequency domain on a pair of image blocks on the reprojected reference image and the calibration image to obtain a matching point with ultra-high accuracy. Through the above method, multiple ground control point sets (S1) and multiple ground check point sets (S2) can be matched. S1 is sparsely and evenly distributed in the row, and the number can be about 100; S2 is densely sampled in the overlapping area between a specific scan line of the calibration image and the geometric calibration field, and the number can be equivalent to the number of image columns in the overlapping area. The first correction model is the RPC row refinement model. Directly applying formula (3) to the ground control points of the scan line will lead to ill-posedness. To solve this problem, the refinement process only focuses on the offset and drift terms of each scan line along the column direction, that is, only a0, a1, b0, b1 in formula (3) are retained, and row-by-row RPC refinement can be effectively performed.

[0077] In one embodiment, a0, a1, b0, b1 are estimated by an objective function Q, which can be expressed as:

[0078]

[0079] Among them, Q is the objective function, (c i ,r i ) and (X i ,Y i ,Z i ) represent the normalized coordinates of the ith control point in the image plane and object space, n is the total number of control points in the row, N c ,D c ,N r and D r is the polynomial term defined by the RPC model coefficients, and a0, a1, b0, b1 are the affine transformation parameters.

[0080] For example, the parameters a0, a1, b0, b1 are estimated by minimizing the objective function Q in Equation 4:

[0081]

[0082] Among them, Q is the objective function, (c i ,r i ) and (X i ,Y i ,Z i ) represent the normalized coordinates of the i-th ground control point in the image plane and object space, n is the total number of ground control points in the scan line, N c ,D c ,N r and Dr is a polynomial term defined by the RPC model coefficients, and a0, a1, b0, and b1 are affine transformation parameters. a0 and a1 can compensate for the external orientation error and a portion of the CCD distortion in the vertical track direction (zero-order and first-order terms). The remaining minimized residual error in the column direction can then be attributed to the high-order distortion in the vertical track direction, while the residual error in the row direction reflects the distortion in the along-track direction. It should be noted that although the residual distortion after row refinement may not perfectly represent the true CCD distortion, the deviation introduced by the zero-order and first-order terms can be effectively eliminated by conventional affine transformations, and therefore will not affect the accuracy of subsequent controlled geometry processing.

[0083] Step 104: for each calibration image, determine a plurality of ground check points; for each scan line in each calibration image, determine a plurality of residuals between a plurality of observed image coordinates obtained through the plurality of ground check points and a plurality of predicted image coordinates obtained through the second correction model; and confirm a plurality of corresponding distortion samples in the vertical and along-track directions of the line array camera according to the plurality of residuals;

[0084] In one embodiment, the distortion samples of the corresponding multiple columns in the vertical track and along the track in the linear array camera are respectively confirmed according to the multiple residuals, which can be expressed as:

[0085]

[0086] in, (c i ,r i ) and (X i ,Y i ,Z i ) represent the normalized coordinates of the ith control point in the image plane and object space, respectively, N c ,D c ,N r and D r is the polynomial term defined by the RPC model coefficients, a0, a1, b0, b1 are the affine transformation parameters, Δc i and Δr i Respectively represent the i-th ground checkpoint (c-th i The distortion samples in the vertical and along-track directions are shown in Figure 1.

[0087] Exemplarily, since some publicly distributed line scan camera products do not provide strict imaging parameters, the ground control point set S1 and the ground check point set S2 obtained in the embodiment are used to calculate the distortion information of a specific row of the calibration image, specifically including: RPC model deviation estimation: using the uniformly distributed but relatively sparse ground control point set S1, the row-by-row RPC refinement model (4) is used to correct the external orientation parameter deviation in the scan line to obtain a second correction model, which is the row-corrected RPC model. For the ground check point set S2 in the overlapping area between the specific scan line of the calibration image and the geometric calibration field, the corresponding multiple columns of the line scan camera in the vertical and along-track directions are respectively confirmed according to the multiple residuals, which can be expressed as:

[0088]

[0089] in, (c i ,r i ) and (X i ,Y i ,Z i ) represent the normalized coordinates of the ith control point in the image plane and object space, respectively, N c ,D c ,N r and D r is the polynomial term defined by the RPC model coefficients, a0, a1, b0, b1 are the affine transformation parameters, Δc i and Δr i Respectively represent the i-th ground checkpoint (c-th i The invention provides distortion samples in the vertical and along-track directions of the image (column). A key difference compared to the traditional RPC model deviation compensation method is that the RPC refined parameters a0, a1, b0, b1 of the present invention may vary between different rows. This takes into account the potential variation of external orientation parameters between different rows, allowing the combination of distortion samples from different images and rows for a more comprehensive analysis.

[0090] Step 105: Determine first distortion values ​​corresponding to the multiple columns based on the distortion samples of the multiple columns, apply data processing to the first distortion value based on a preset guided filtering algorithm to obtain a second distortion value, and correct the calibrated image based on the second distortion value to obtain a corrected image.

[0091] In one embodiment, determining first distortion values ​​corresponding to the plurality of columns according to the distortion samples of the plurality of columns, applying data processing to the first distortion values ​​based on a preset guided filtering algorithm to obtain second distortion values, and correcting the calibration image according to the second distortion values ​​to obtain a corrected image includes:

[0092] Determine the number of distorted samples respectively possessed by each of the multiple columns, and determine known columns and unknown columns among the multiple columns, wherein the number of distorted samples possessed by the known columns is greater than or equal to a predetermined threshold, and the number of distorted samples possessed by the unknown columns is less than the predetermined threshold;

[0093] Determine, according to the distortion samples of the known column, the distortion value corresponding to the known column;

[0094] According to the distortion values ​​corresponding to the known columns, a thin plate spline function model for the distortion values ​​of the unknown columns is constructed, and the distortion values ​​of the unknown columns are obtained based on the thin plate spline function model;

[0095] Inputting the first distortion value of each column of the sensor as a guidance signal into the guidance filter to obtain the second distortion value of each column of the sensor;

[0096] The calibration image is corrected according to the distortion value to obtain a corrected image.

[0097] In one embodiment, the thin plate spline function model can be expressed as:

[0098]

[0099] Among them, x represents the coordinates of the known column to be interpolated, x i represents the coordinates of the i-th ground control point, ψ(r) = r 2 logr is the radial basis kernel function, α i , β0 and β1 represent calculation coefficients, K n is the number of adjacent ground control points used for interpolation.

[0100] In one embodiment, the predetermined threshold is 20; determining the distortion value corresponding to the known column according to the distortion samples of the known column includes: determining the distortion value corresponding to the known column according to the median of the distortion samples of the known column.

[0101] For example, millions of distortion samples can be collected from different scan lines of different calibration images. The next step is to estimate the column-by-column distortion of the linear array camera J2000. Determine the number of distortion samples in each of the multiple columns, and determine the known columns and unknown columns in the multiple columns, wherein the number of distortion samples in the known columns is greater than or equal to a predetermined threshold, and the number of distortion samples in the unknown columns is less than the predetermined threshold; wherein the predetermined threshold can be 20. Based on the distortion samples in the known columns, the distortion values ​​corresponding to the known columns can be determined; based on the distortion values ​​corresponding to the known columns, a thin plate spline function (TPS) model (Formula 6) for the distortion values ​​of the unknown columns can be constructed to interpolate the distortion values ​​of the unknown columns. The thin plate spline function model can be expressed as:

[0102]

[0103] Among them, x represents the coordinates of the known column to be interpolated, x i represents the coordinates of the i-th ground control point, ψ(r) = r 2 logr is the radial basis kernel function, α i , β0 and β1 represent calculation coefficients, K n is the number of adjacent ground control points used for interpolation. And it can be obtained by minimizing the objective function E(f tps ) to determine the coefficients of the thin plate spline model:

[0104]

[0105] Among them, y i is the known distortion value (along-track or vertical-track direction) at the ith checkpoint, and λ is a non-negative regularization parameter used to control f tps The smoothness of the image is determined by the empirical value λ=5. The embodiment of the present invention selects the TPS model instead of the high-order polynomial model to improve the versatility of the method. Specifically, the TPS model can effectively capture complex distortion patterns without assuming a specific parameter form, which makes them very suitable for non-uniform distortion that may exist in the first-level product of the linear array camera, because the initial RPC may already contain part of the CCD distortion.

[0106] In one embodiment, the guided filtering algorithm can be expressed as:

[0107]

[0108] Among them, f′(x) is the output signal, f(x) is the guidance signal, |ω x | represents the number of all windows containing x, b k =μ k -a k μ k ,μ k and They are respectively the guiding signal f(x) in the window ω k The mean and variance in , ∈ is the regularization parameter used to control the degree of smoothing, and ω is the window size.

[0109] For example, although the median of the sample value is used to suppress random noise when calculating the distortion value of each column, due to the existence of matching errors, the calculated distortion values ​​of each column will still show local fluctuations. In order to obtain a smoother distortion curve, a direct idea is to regard the distortion values ​​of each column of the sensor as a one-dimensional continuous signal and use a traditional time series filter (such as SavitzkyGolay filter) for smoothing. However, while suppressing noise, this type of filter often smoothes out the mutation information in the signal, which is not conducive to effectively retaining high-frequency irregular distortion and CCD splicing misalignment errors. In order to overcome the shortcomings of traditional filters, this paper introduces a guided filtering algorithm to smooth the distortion values ​​of each column to achieve edge-preserving filtering. The core idea of ​​guided filtering is to use the structural information of the guided image (or signal) to guide the filtering process, so as to effectively retain the edge and detail information while smoothing the noise. Specifically, in this application scenario, we can use the original distortion value signals of each column as guided signals, and obtain a smoothed output signal f′(x) after guided filtering. The output signal f′(x) can be expressed as:

[0110]

[0111] Among them, f′(x) is the output signal, f(x) is the guidance signal, |ω x | represents the number of all windows containing x, b k =μ k -a k μ k ,μ k and They are respectively the guiding signal f(x) in the window ω k The mean and variance in , ∈ is a regularization parameter for controlling the degree of smoothness, and ω is the window size. For the HJ-2A CCD1 camera, the present invention uses empirical values ​​of ω=50 and ∈=0.1.

[0112] After the column-by-column distortion estimation is completed, the distortion correction can be performed pixel by pixel for any scene image acquired by the HJ-2A CCD1 camera. Compared with the traditional correction method based on the rational function model (RPC), this direct correction strategy can effectively avoid the additional error introduced by regenerating the RPC model. Previous studies have shown that regenerating the RPC model may introduce an accuracy loss of about 0.3 pixels.

[0113] According to yet another embodiment, an on-orbit calibration device for geometric distortion of a linear array camera is provided. Figure 3 A structural diagram of an image correction device for a line array camera provided by an embodiment of the present invention, such as Figure 3 As shown, the device 300 includes:

[0114] Provided is an on-orbit calibration device for geometric distortion of a linear array camera, the device comprising:

[0115] An acquisition unit 301 is configured to acquire a plurality of calibration images of a linear array camera for a geometric calibration field;

[0116] A first model correction unit 302 is configured to apply an image-space affine transformation to a pre-acquired rational polynomial coefficient model for the geometric calibration field to obtain a first correction model;

[0117] The second model correction unit 303 is configured to determine a plurality of ground control points for each calibration image; for each scan line of each calibration image, based on the ground control points, correct the exterior orientation parameters corresponding to each scan line in the first correction model to obtain a second correction model;

[0118] The processing unit 304 is configured to determine, for each calibration image, a plurality of ground check points; for each scan line in each calibration image, determine a plurality of residuals between a plurality of observed image coordinates obtained through the plurality of ground check points and a plurality of predicted image coordinates obtained through the second correction model, and confirm, according to the plurality of residuals, a plurality of corresponding distortion samples in the vertical and along-track directions of the line array camera;

[0119] The image correction unit 305 is configured to determine the first distortion values ​​corresponding to the multiple columns according to the distortion samples of the multiple columns, apply data processing to the first distortion value based on a preset guided filtering algorithm to obtain a second distortion value, and correct the calibrated image according to the second distortion value to obtain a corrected image.

[0120] Figure 4 The data graph of the target camera provided by the embodiment of the present invention is as follows: Figure 4 As shown, the HJ-2A data (CCD1, CCD2, CCD4 and CCD5 cameras) used include 16 calibration data and 64 verification data. In the case of unknown strict imaging parameters, the geometric distortion of the linear array camera is successfully estimated and corrected using the method of the present invention; Figure 5 A residual image of a target camera distortion sample provided by an embodiment of the present invention, Figure 6 The geometric distortion diagram of the target camera in the vertical track direction and along the track direction provided by the embodiment of the present invention is as follows: Figure 5 , Figure 6 As shown, the systematic internal image error is basically eliminated. For the calibrated HJ-2A CCD1 image (taking CCD1 camera as an example), Figure 7 The error distribution diagram before and after the image calibration provided by the embodiment of the present invention is as follows: Figure 7As shown in the figure, using a small number of ground control points and the conventional image-space affine transformation compensation method, a high geometric correction accuracy can be obtained, especially the geometric accuracy of the image edge is effectively improved. In addition, 64 scene data distributed across the country are used for verification. Figure 8 The image geometric error distribution and error map provided by the embodiment of the present invention are as follows: Figure 8 As shown in the figure, the statistical results show that geometric calibration significantly reduces the distribution range of the geometric error of HJ-2A CCD1 images, and the circular probability error (CE90) with a confidence level of 90% is reduced to within 0.3 pixels. The average geometric accuracy (RMSE) of the four cameras after correction reaches 0.19, 0.20, 0.20 and 0.21 pixels respectively.

[0121] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image correction method for a line array camera, comprising: Acquire multiple calibration images of the line array camera for the geometric calibration field; Applying image-space affine transformation to the pre-acquired rational polynomial coefficient model for the geometric calibration field to obtain a first correction model; For each calibration image, a plurality of ground control points are determined; for each scan line of each calibration image, based on the ground control points, the exterior orientation parameters corresponding to each scan line in the first correction model are corrected to obtain a second correction model; For each calibration image, a plurality of ground check points are determined; for each scan line in each calibration image, a plurality of residuals between a plurality of observed image coordinates obtained through the plurality of ground check points and a plurality of predicted image coordinates obtained through the second correction model are determined, and distortion samples of a plurality of columns in the vertical track and along the track corresponding to the linear array camera are respectively confirmed according to the plurality of residuals; According to the distortion samples of the multiple columns, first distortion values ​​corresponding to the multiple columns are determined, data processing is performed on the first distortion values ​​based on a preset guided filtering algorithm to obtain second distortion values, and the calibration image is corrected according to the second distortion values ​​to obtain a corrected image.

2. The method according to claim 1, wherein: Determine multiple ground control points, including: Determine multiple ground control points based on pre-acquired global reference image GRI and / or Determining a plurality of ground checkpoints includes: determining a plurality of ground checkpoints based on a pre-acquired high-precision geometric calibration field.

3. The method according to claim 1, wherein: Determining first distortion values ​​corresponding to the plurality of columns according to the distortion samples of the plurality of columns, applying data processing to the first distortion values ​​based on a preset guided filtering algorithm to obtain second distortion values, and correcting the calibration image according to the second distortion values ​​to obtain a corrected image, including: Determine the number of distorted samples respectively possessed by each of the multiple columns, and determine known columns and unknown columns among the multiple columns, wherein the number of distorted samples possessed by the known columns is greater than or equal to a predetermined threshold, and the number of distorted samples possessed by the unknown columns is less than the predetermined threshold; determine the distortion value corresponding to the known columns according to the distorted samples possessed by the known columns; According to the distortion values ​​corresponding to the known columns, a thin plate spline function model for the distortion values ​​of the unknown columns is constructed, and the distortion values ​​of the unknown columns are obtained based on the thin plate spline function model; Inputting the first distortion value of each column of the sensor as a guidance signal into the guidance filter to obtain the second distortion value of each column of the sensor; The calibration image is corrected according to the second distortion value to obtain a corrected image.

4. The method according to claim 1, wherein: Applying image-space affine transformation to the pre-acquired rational polynomial coefficient model for the geometric calibration field can be expressed as: Where c and r are the normalized column and row coordinates of the image point on the image plane, respectively; (X, Y, Z) is the normalized object space coordinate, N c ,D c ,N R and D R is the polynomial term defined by the RPC model coefficients; a0, a1, a2, b0, b1, b2 are the affine transformation parameters.

5. The method according to claim 4, wherein: a0, a1, b0, b1 are estimated by the objective function Q, which can be expressed as: Among them, Q is the objective function, (c i ,r i ) and (X i ,Y i ,Z i ) represent the normalized coordinates of the ith control point in the image plane and object space, n is the total number of control points in the row, N c ,D c ,N r and D r is the polynomial term defined by the RPC model coefficients, and a0, a1, b0, b1 are the affine transformation parameters.

6. The method according to claim 1, wherein: According to the plurality of residuals, the corresponding plurality of distortion samples in the vertical track and along the track direction in the linear array camera are respectively confirmed, which can be expressed as: in, (c i ,r i ) and (X i ,Y i ,Z i ) represent the normalized coordinates of the ith control point in the image plane and object space, respectively, N c ,D c ,N r and D r is the polynomial term defined by the RPC model coefficients, a0, a1, b0, b1 are the affine transformation parameters, Δc i and Δr i Respectively represent the i-th ground checkpoint (c-th i The distortion samples in the vertical and along-track directions are shown in Figure 1.

7. The method according to claim 3, wherein: The thin plate spline function model can be expressed as: Among them, x represents the coordinates of the known column to be interpolated, x i represents the coordinates of the i-th ground control point, ψ(r) = r 2 logr is the radial basis kernel function, α i , β0 and β1 represent calculation coefficients, K n is the number of adjacent ground control points used for interpolation.

8. The method according to claim 3, wherein: The predetermined threshold is 20; determining the distortion value corresponding to the known column according to the distortion samples of the known column, including: determining the distortion value corresponding to the known column according to the median of the distortion samples of the known column.

9. The method according to claim 3, wherein: The guided filtering algorithm can be expressed as: Among them, f′(x) is the output signal, f(x) is the guidance signal, |ω x | represents the number of all windows containing x, b k =μ k -a k μ k ,μ k and They are respectively the guiding signal f(x) in the window ω k The mean and variance in , ∈ is the regularization parameter used to control the degree of smoothing, and ω is the window size.

10. An on-orbit calibration device for geometric distortion of a linear array camera, comprising: An acquisition unit is configured to acquire a plurality of calibration images of the linear array camera for a geometric calibration field; A first model correction unit is configured to apply an image-space affine transformation to a pre-acquired rational polynomial coefficient model for the geometric calibration field to obtain a first correction model; The second model correction unit is configured to determine a plurality of ground control points for each calibration image; for each scan line of each calibration image, based on the ground control points, correct the exterior orientation parameters corresponding to each scan line in the first correction model to obtain a second correction model; The processing unit is configured to determine, for each calibration image, a plurality of ground check points; for each scan line in each calibration image, determine a plurality of residuals between a plurality of observed image coordinates obtained through the plurality of ground check points and a plurality of predicted image coordinates obtained through the second correction model, and confirm, according to the plurality of residuals, the corresponding plurality of distortion samples in the vertical and along-track directions of the line array camera; The image correction unit is configured to determine first distortion values ​​corresponding to the multiple columns based on the distortion samples of the multiple columns, apply data processing to the first distortion value based on a preset guided filtering algorithm to obtain a second distortion value, and correct the calibrated image according to the second distortion value to obtain a corrected image.

Images