A method and apparatus for correcting internal geometric errors in images based on yaw imaging

Abstract

This invention proposes a method and apparatus for correcting internal geometric errors in images based on yaw imaging. The method includes: acquiring remote sensing images after a satellite platform performs a 90° yaw maneuver; detecting straight lines in the images using a straight line detection method to obtain the pixel coordinates of the start and end points of each straight line; detecting all straight lines in the yaw images obtained by each detector piece in the yaw images, obtaining the coordinates of the start and end points of each straight line, and calculating the angle of each straight line using these coordinates; fitting the angles of all straight lines obtained by each detector piece to obtain the optimal straight line angle value β of the yaw image of that detector piece, and calculating the angle difference β' between the actual and theoretical positions of the detector stitching; and performing geometric correction on the image based on the angle error of the detector stitching, using one of the detector pieces as a reference, to obtain an image that eliminates the inter-piece geometric errors caused by detector stitching.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing satellite image processing technology, specifically relating to a method and apparatus for internal geometric error correction of remote sensing images acquired by multi-piece stitched detectors. Background Technology

[0002] The geometric accuracy of satellite imagery is a key indicator of satellite remote sensing capabilities. With continuous improvements in spatial resolution, imaging swath width, and the number of spectral bands, a single detector is often insufficient. Currently, most remote sensing satellites employ optical or field-of-view stitching to combine multiple detectors. However, stitching errors exist between detectors, and distortions across the entire field of view in the optical system can also cause internal geometric distortions among multiple detectors.

[0003] Conventional geometric correction methods require high-precision ground control points and ground geometric calibration fields for satellite overflight, which is labor-intensive. Considering weather factors, it is difficult to perform high-frequency geometric correction. There is an urgent need for an on-orbit image geometric correction method that does not rely on ground control points. Summary of the Invention

[0004] In view of this, the present invention provides an image internal geometric error correction method based on yaw imaging, which can realize on-orbit image geometric correction independent of ground control points, including:

[0005] Step S1: After the satellite platform performs a 90° yaw maneuver, it acquires remote sensing images;

[0006] Step S2: Detect straight lines in the image using a line detection method and obtain the pixel coordinates of the start and end points of each line;

[0007] Step S3: Detect all straight lines in the yaw image obtained by each detector, obtain the coordinates of the start and end points of each straight line, and calculate the angle of each straight line using these coordinates; fit the angles of all the straight lines to obtain the optimal straight line angle value β of the yaw image of a certain detector.

[0008] Step S4: Based on the optimal straight-line angle value β of a certain detector image, the angle error β' between the actual position and the theoretical position of the detector splicing can be obtained;

[0009] Step S5: Based on the detector splicing angle error β' obtained in the previous step, take one of the detectors as a reference and perform geometric correction on the image to obtain an image that eliminates the inter-piece geometric error caused by detector splicing.

[0010] Specifically, step S2 includes: detecting straight lines in the image and obtaining the pixel coordinates of the start and end points of each straight line. The straight line detection algorithm can use either the Hough_line straight line detection algorithm or the LSD straight line detection algorithm. The Hough_line straight line detection algorithm includes the following steps:

[0011] Quantize the parameters (ρ, θ) and set a two-dimensional accumulation matrix (ρ). i ,θ i And initialize it;

[0012] For each pixel (x, y) in the image space, map it to the parameter space using the parametric expression: xcosθ + ysinθ = ρ, find the corresponding parameter position (ρ, θ), and accumulate N(ρ). i ,θ i )+=1;

[0013] For all N(ρ) i ,θ i Statistically analyze the values ​​of N(ρ) to find N(ρ) i ,θ i The parameter value of threshold is )>threshold, where threshold is a preset threshold value;

[0014] The coordinates of points in the parameter space that are greater than the threshold are the position parameters of the detected line in the original image.

[0015] The LSD line detection algorithm includes the following steps: reducing the input image to 80% of its original size; calculating the image gradient and sorting the calculated gradients; setting a gradient threshold and excluding points with gradients less than the threshold; merging pixels with approximately the same gradient direction to form a line support region; and enclosing the line support region composed of a series of discrete points within a rectangle, which is the candidate line to be detected.

[0016] Specifically, in step S3, obtaining the coordinates of the start and end points of each straight line, and calculating the angle of each straight line using these coordinates, includes: using a calculation formula.

[0017]

[0018] Where (x1,y1) and (x2,y2) are the coordinates of the starting and ending points of a certain line, respectively, and α is the angle of the line.

[0019] Specifically, in step S3, fitting the angles of all the straight lines to obtain the optimal straight-line angle value β for a certain detector yaw image specifically includes:

[0020] All straight lines from each detector are detected. To eliminate interference from stripe noise in the image, a straight line angle threshold is set, and straight lines with angles outside the threshold are discarded. For each detector, the acceptable straight lines are fitted with an optimal straight line angle using robust least squares. The robust least squares weight function is a function of the residuals. This residual function preserves the weights of normal observations, reduces the weights of abnormal but usable observations, and sets the weights of excessively large gross errors to zero. The robust least squares calculation formula is as follows:

[0021] β=(A T PA) -1 A T Pα

[0022] Where β is the optimal straight-line angle value of a certain detector yaw image, A is the coefficient matrix, P is the weight matrix, and α is the observed value, i.e., the angle of each detected straight line; P needs to be re-estimated based on the residuals of each calculation. The weight functions of the robust least squares method include Huber estimation function, Danish method function, IGG method function, Hampel triple-truncation estimation function, Andrews sine estimation function, and Tukey double-weight estimation function.

[0023] Specifically, in step S4, based on the straight-line angle β of the yaw image of a certain detector, the angular error β' between the actual and theoretical positions of the detector stitching can be calculated. The calculation formula is as follows:

[0024] β' = acos(cotβ).

[0025] Specifically, in step S5, based on the calculated detector stitching angle error β', using one of the detectors as a reference, geometric correction is performed on the image to obtain an image that eliminates the inter-piece geometric error caused by detector stitching; the calculation formula is:

[0026]

[0027] Where (x', y') are the corrected image coordinates, (x, y) are the original image coordinates, and Δβ' is the difference between the actual angle of the detector to be corrected and the actual angle of the reference detector.

[0028] The present invention also proposes an image internal geometric error correction device based on yaw imaging, comprising:

[0029] The yaw image acquisition module is used to acquire remote sensing images after the satellite platform yaws at a 90° angle.

[0030] The yaw image straight line detection module is used to detect straight lines in the image using a straight line detection method and obtain the pixel coordinates of the start and end points of each straight line.

[0031] The straight line angle fitting module is used to detect all straight lines in the yaw image obtained by each detector, obtain the coordinates of the start and end points of each straight line, calculate the angle of each straight line using these coordinates, and fit the angles of all the straight lines to obtain the optimal straight line angle value β of the yaw image of a certain detector.

[0032] The detector stitching angle calculation module is used to calculate the angle error β' between the actual position and the theoretical position of the detector stitching based on the optimal straight line angle value β of a certain detector image.

[0033] The detector splicing error correction module is used to perform geometric correction on the image based on the detector splicing angle error β' obtained in the previous step, using one of the detectors as a reference, to obtain an image that eliminates the inter-segment geometric error caused by detector splicing.

[0034] Beneficial effects:

[0035] 1) This invention can use on-orbit yaw data to correct inter-segment geometric errors caused by splicing multiple detectors, without relying on ground control point information, without special requirements for the imaging area, and can achieve high-frequency on-orbit geometric calibration.

[0036] 2) The straight line detection algorithm in this invention has high accuracy, and the inter-chip geometric error can be corrected with high precision.

[0037] 3) In this invention, the angles of all straight lines detected in the yaw image are fitted to obtain the optimal straight line angle value of a certain detector yaw image. Multiple methods can be selected for screening during the process to ensure the accurate calculation of the straight line angle.

[0038] 4) In this invention, based on the obtained angle error of the detector splicing, one of the detectors is used as a reference to perform geometric correction on the image, thereby obtaining an image that eliminates the inter-segment geometric error caused by detector splicing and avoiding internal geometric distortion between multiple detectors. Attached Figure Description

[0039] Figure 1 This is a flowchart of the technical solution in this invention;

[0040] Figure 2 This is a schematic diagram of a certain type of payload detector in the present invention, arranged in a triangular pattern.

[0041] Figures 3a-3b This is a schematic diagram of the yaw calibration principle in this invention;

[0042] Figure 4 This is an image of the on-orbit yaw in this invention;

[0043] Figure 5 This is a schematic diagram of yaw image straight line detection in this invention;

[0044] Figure 6 This describes the relationship between the yaw straight-line angle and the detector splicing angle in this invention. Detailed Implementation

[0045] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0046] This invention is a method for correcting internal geometric errors in images based on yaw imaging. The flowchart of the technical solution is as follows: Figure 1 As shown, the satellite performs a 90° yaw maneuver in the same attitude state. Remote sensing images are acquired under this state, and the geometric errors caused by detector stitching are corrected using the remote sensing images themselves. There are no special requirements for the ground features being photographed. Specific technical solutions include:

[0047] Assuming the detector uses four detector pieces arranged in a triangular pattern, with each detector piece containing 512 pixels, the arrangement is as follows: Figure 2 As shown. A detailed description of the inter-chip geometric error correction is as follows:

[0048] Step S1: Acquisition of yaw image

[0049] This method is based on satellite yaw imagery. The acquisition method involves maneuvering the satellite platform by 90° yaw, aligning the detector array direction with the imaging direction. Theoretically, during yaw calibration, since all pixels on the detector array image the same ground area, each detector acquires remote sensing data in this state. According to... Figures 3a-3b The yaw calibration principle shown is as follows: Figure 3a This is a normal imaging mode linear scan. Figure 3b This is a linear array scanning method for yaw calibration; by acquiring on-orbit yaw images, the obtained yaw images are as follows: Figure 4 As shown.

[0050] Step S2: Yaw image straight line detection

[0051] Images acquired during yaw typically consist of diagonal lines at approximately 45°. Line detection methods are used to detect these lines and obtain the pixel coordinates of their start and end points. Houghline and LSD line detection algorithms can be used for line detection.

[0052] Hough line detection algorithm steps:

[0053] Quantize the parameters (ρ, θ) and set a two-dimensional accumulation matrix (ρ). i ,θ i And initialize it;

[0054] For each pixel (x, y) in the image space, map it to the parameter space using the parametric expression: xcosθ + ysinθ = ρ, find the corresponding parameter position (ρ, θ), and accumulate N(ρ). i ,θ i )+=1;

[0055] For all N(ρ) i ,θ i Statistically analyze the values ​​of N(ρ) to find N(ρ) i ,θ i The threshold parameter value is the threshold size we set.

[0056] The coordinates of points in the parameter space that are greater than the threshold are the position parameters of the line we want to detect in the original image.

[0057] LSD line detection algorithm steps:

[0058] The input image is reduced to 80% of its original size, which reduces or even eliminates the jagged edges that appear in many images;

[0059] Calculate the image gradient;

[0060] Sort the calculated gradients;

[0061] Set a gradient threshold to exclude points where the gradient is less than the threshold;

[0062] Pixels with approximately the same gradient direction are merged to form a linear support region;

[0063] The support region for a straight line, consisting of a series of discrete points, is enclosed within a rectangle; this rectangle represents the candidate straight line for detection. In this embodiment, the LSD algorithm is used to monitor the straight lines present in each detector. Each detector can detect approximately 14,000 straight lines, some of which are as follows: Figure 5 As shown.

[0064] Step S3: Linear Angle Fitting

[0065] The previous step detected several straight lines in the yaw image, and obtained the coordinates of the start and end points of each line. The angle of each line was then calculated using these coordinates.

[0066] The calculation formula is:

[0067]

[0068] Where (x1,y1) and (x2,y2) are the coordinates of the starting and ending points of a certain line, respectively, and α is the angle of the line.

[0069] Ideally, the straight-line angles in the images from several detectors should be the same (approximately 45° to the horizontal). However, due to factors such as splicing errors between detectors, the straight-line angles in the yaw images from several detectors are not consistent.

[0070] Step S2 can be used to detect all straight lines from each detector. To eliminate interference from stripe noise in the image, a line angle threshold (e.g., 40-50°) is set, and lines with angles outside the threshold are discarded. For each detector, the lines that meet the requirements are fitted with an optimal line angle using robust least squares. The weights in robust least squares are functions of the residuals, and the weight function incorporates the concept of joint robustness. Normal observations are weighted, while abnormal but usable observations are weighted less. Gross errors with excessively large differences are eliminated by setting their weights to zero.

[0071] The robust least squares method calculation formula is as follows:

[0072] β=(A T PA) -1 A T Pα (2)

[0073] Where β is the optimal straight-line angle value for a certain detector, A is the coefficient matrix, P is the weight matrix, and α is the observed value, i.e., the angle of each detected straight line. P needs to be re-estimated based on the residuals of each calculation. Common weight functions for robust least squares methods include Huber estimation, Danish method, IGG method, Hampel triple-truncation estimation, Andrews sine estimation, and Tukey double-weight estimation.

[0074] In this embodiment, the robust least squares method is used to fit the angle of each detector. The threshold of the weight matrix is ​​set to 1°. If the residual between the observed value and the fitted angle exceeds 1°, the weight of the observed value is set to 0, and the weights of the rest are set to 1. The fitting calculation is performed again until the residual meets the requirements. The optimal angle of each detector is then obtained. The optimal angles of the straight line in the images of the four detectors are 48.47°, 48.4°, 48.44°, and 48.16°, respectively.

[0075] Step S4: Solving for the detector splicing angle

[0076] From the straight-line angle β of a certain detector yaw image obtained in step S3, the angle difference β' between the actual position and the theoretical position of the detector stitching can be calculated. The calculation formula is as follows:

[0077] β'=acos(cotβ) (3)

[0078] In this embodiment, the differences between the actual angle and the theoretical angle of each detector splicing are obtained by formula (3): 27.667°, 27.397°, 27.552°, and 26.445°.

[0079] Step S5: Correction of splicing error between detectors

[0080] Based on the angular error of the detector splicing obtained in the previous step, the image is geometrically corrected using one of the detectors as a reference, resulting in an image that eliminates the inter-segment geometric error caused by detector splicing.

[0081] The calculation formula is:

[0082]

[0083] Where (x', y') are the corrected image coordinates, (x, y) are the original image coordinates, and Δβ' is the difference between the actual angle of the detector to be corrected and the actual angle of the reference detector.

[0084] In this embodiment, the second detector is used as the reference, and the angle differences between the other three detectors and it are 0.27°, 0.155°, and -0.95°, respectively. According to formula (4), the correction matrix of the three detector images is:

[0085]

[0086]

[0087]

[0088] Based on the three correction matrices above, detectors 1, 3, and 4 are geometrically corrected. The relationship between the yaw straight-line angle and the detector splicing angle in this invention is as follows: Figure 6 As shown.

[0089] The present invention also proposes an image internal geometric error correction device based on yaw imaging, the device comprising:

[0090] The yaw image acquisition module is used to acquire remote sensing images after the satellite platform yaws at a 90° angle.

[0091] The yaw image straight line detection module is used to detect straight lines in the image using a straight line detection method and obtain the pixel coordinates of the start and end points of each straight line.

[0092] The straight line angle fitting module is used to detect all straight lines in the yaw image obtained by each detector, obtain the coordinates of the start and end points of each straight line, calculate the angle of each straight line using these coordinates, and fit the angles of all the straight lines to obtain the optimal straight line angle value β of the yaw image of a certain detector.

[0093] The detector stitching angle calculation module is used to calculate the angle difference β' between the actual position and the theoretical position of the detector stitching based on the optimal straight line angle value β of a certain detector image.

[0094] The detector splicing error correction module is used to perform geometric correction on the image based on the detector splicing angle error obtained in the previous step, using one of the detectors as a reference, to obtain an image that eliminates the inter-segment geometric error caused by detector splicing.

[0095] The contents defined in this device are similar to those defined in the previous method embodiment, so they will not be repeated here.

[0096] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0097] It will be apparent to those skilled in the art that the embodiments of the present invention are not limited to the details of the exemplary embodiments described above, and that the embodiments of the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the embodiments of the present invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the embodiments of the present invention is defined by the appended claims rather than the foregoing description. Therefore, all variations falling within the meaning and scope of equivalents of the claims are intended to be encompassed within the embodiments of the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units, modules, or devices recited in the system, apparatus, or terminal claims may also be implemented by the same unit, module, or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention and are not intended to limit them. Although the embodiments of the present invention have been described in detail with reference to the above preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the embodiments of the present invention should not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for correcting internal geometric errors in images based on yaw imaging, characterized in that, include: Step S1: After the satellite platform performs a 90° yaw maneuver, it acquires remote sensing images; Step S2: Detect straight lines in the image using a line detection method and obtain the pixel coordinates of the start and end points of each line; Step S3: Detect all straight lines in the yaw image obtained by each detector, obtain the coordinates of the start and end points of each line, and calculate the angle of each line using these coordinates; fit the angles of each line to obtain the optimal straight line angle value of the yaw image of a certain detector. ; Step S4: Based on the optimal straight-line angle value of the detector image. The angular error between the actual and theoretical positions of the detector splicing was calculated. In step S4, the optimal straight-line angle is determined based on the yaw image of a certain detector. The angular error between the actual and theoretical positions of the detector splicing was calculated. The calculation formula is: ； Step S5: Based on the angle error of the detector splicing obtained in the previous step... Using one of the detectors as a reference, geometric correction is performed on the image to obtain an image that eliminates the inter-segment geometric errors caused by detector splicing.

2. The image internal geometric error correction method based on yaw imaging as described in claim 1, characterized in that, Step S2 includes: detecting straight lines in the image and obtaining the pixel coordinates of the start and end points of each straight line. The line detection algorithm uses either the Hough_line line detection algorithm or the LSD line detection algorithm. The Hough_line line detection algorithm includes the following steps: Put the parameters ( , Quantization, setting a two-dimensional accumulation matrix ( , And initialize it; For each pixel point (x, y) in the image space, through the parameter expression: map it to the parameter space to find the corresponding parameter position( , ), and perform accumulation ; For all Statistically analyze the values ​​to find The threshold parameter value; threshold is a preset threshold value. The coordinates of points in the parameter space that are greater than the threshold are the position parameters of the detected line in the original image. The LSD line detection algorithm includes the following steps: reducing the input image to 80% of its original size; calculating the image gradient and sorting the calculated gradients; setting a gradient threshold and excluding points with gradients less than the threshold; merging pixels with approximately the same gradient direction to form a line support region; and enclosing the line support region composed of a series of discrete points within a rectangle, which is the candidate line to be detected.

3. The image internal geometric error correction method based on yaw imaging as described in claim 1 or 2, characterized in that, In step S3, the coordinates of the start and end points of each straight line are obtained, and the angle of each straight line is calculated using these coordinates, including by using the calculation formula. in, and These are the coordinates of the start and end points of a certain straight line. Let be the angle of the straight line.

4. The image internal geometric error correction method based on yaw imaging as described in claim 3, characterized in that, In step S3, the angles of all the straight lines are fitted to obtain the optimal straight line angle values ​​for a certain detector yaw image. Specifically, it includes: All straight lines from each detector are detected. To eliminate interference from stripe noise in the image, a straight line angle threshold is set, and straight lines with angles outside the threshold are discarded. For each detector, the acceptable straight lines are fitted with an optimal straight line angle using robust least squares. The robust least squares weight function is a function of the residuals. This residual function preserves the weights of normal observations, reduces the weights of abnormal but usable observations, and sets the weights of excessively large gross errors to zero. The robust least squares calculation formula is as follows: in, Let A be the optimal straight-line angle value for a certain detector yaw image, and let P be the coefficient matrix and P be the weight matrix. The angle of each detected straight line; P needs to be re-estimated based on the residuals of each calculation. The weight functions of the robust least squares method include Huber estimation function, Danish method function, IGG method function, Hampel triple-truncation estimation function, Andrews sine estimation function, and Tukey double-weight estimation function.

5. The image internal geometric error correction method based on yaw imaging as described in claim 4, characterized in that, In step S5, the obtained detector splicing angle error is used as a basis. Using one of the detectors as a reference, geometric correction is performed on the image to obtain an image that eliminates the inter-detector geometric errors caused by detector stitching; the calculation formula is: in,( , ) represents the corrected image coordinates, ( , () represents the image coordinates before correction. This represents the difference between the actual angle of the detector to be calibrated and the actual angle of the reference detector.

6. An image internal geometric error correction device based on yaw imaging, characterized in that, include: The yaw image acquisition module is used to acquire remote sensing images after the satellite platform yaws at a 90° angle. The yaw image straight line detection module is used to detect straight lines in the image using a straight line detection method and obtain the pixel coordinates of the start and end points of each straight line. The straight-line angle fitting module is used to detect all straight lines in the yaw image obtained by each detector, obtain the coordinates of the start and end points of each straight line, fit the angle of the straight line using these coordinates, and calculate the angle of each straight line; based on the fitting of the angles of all the straight lines, the optimal straight-line angle value of the yaw image of a certain detector is obtained. ; The detector stitching angle calculation module is used to determine the optimal straight-line angle value of a certain detector image. The angular error between the actual and theoretical positions of the detector splicing was calculated. Based on the optimal straight-line angle of the yaw image of a certain detector. The angular error between the actual and theoretical positions of the detector splicing was calculated. The calculation formula is: ； The detector splicing error correction module is used to correct the detector splicing angle error obtained in the previous step. Using one of the detectors as a reference, geometric correction is performed on the image to obtain an image that eliminates the inter-segment geometric errors caused by detector splicing.

Images