Geometric calibration method, positioning method and system for time-space error of swing-scanning satellite

Abstract

The present invention discloses a method for geometric calibration of the space-time error of a swing-scanning satellite, a positioning method and a system thereof, the method comprising: generating a simulated image of a swing-scanning camera image; extracting control points using the simulated image, and then determining the ground coordinates of the control points and the image point coordinates; using the ground coordinates of the control points and the image point coordinates for geometric calibration, wherein firstly using the control points of a single area to perform geometric calibration of the installation error and the camera parameter error to determine the installation parameters, the camera parameters, and the attitude parameters; then using the control points of a long orbit area to perform geometric calibration of the time synchronization error to determine the time synchronization parameters; finally repeating the geometric calibration of the installation error and the camera parameter error to update the installation parameters, the camera parameters, and the attitude parameters. The present invention realizes the calculation of the time synchronization error of the attitude determination system, the orbit determination system, and the camera system, eliminates the time positioning error that changes with the latitude, and realizes the accurate solution of the time synchronization error.

Description

Technical Field

[0001] The present invention belongs to the field of satellite image processing technology, and specifically relates to a method for geometrically calibrating the spatiotemporal errors of a swing-scanning satellite, a positioning method and a system thereof for preprocessing and payload monitoring applications of satellite imaging systems such as meteorology and oceanography, and realizes the calculation of time synchronization errors of a satellite-borne swing-scanning attitude determination system, an orbit determination system and a camera system. Background Art

[0002] Satellite-borne swing-scan cameras have a large field of view and high radiation accuracy, and are widely used in meteorological, oceanographic and other satellite systems. Satellite-borne swing-scan cameras use a narrow field of view, high-dynamic multi-scanning imaging mode, which results in multiple sources of image positioning errors and mutual coupling. This large distortion and discontinuous image makes it difficult to effectively correct the error, restricting its application level. Among them, the time synchronization error is usually easily coupled with the installation angle error and position error in positioning, which is difficult to effectively distinguish and causes a reduction in positioning accuracy, especially the introduction of latitude-related positioning errors. Summary of the invention

[0003] The purpose of the present invention is to solve the technical problem that the time synchronization error is usually easy to be coupled with the installation angle error and the position error in positioning, and it is difficult to effectively distinguish and cause the positioning accuracy to be reduced, and then provide a geometric calibration method, positioning method and system of the time and space error of a swing-scanning satellite. The method of the present invention takes into account the time synchronization error, installation error and camera parameter error at the same time, firstly performs geometric calibration of the installation error and the camera parameter error through the control points of a single area; then uses the control points of the long orbit area to perform geometric calibration of the time synchronization error; finally, performs geometric calibration of the installation error and the camera parameter error again, effectively solving the technical problem that the time synchronization error is usually easy to be coupled with the installation angle error and the position error in positioning, thereby effectively improving the positioning accuracy, realizing the calculation of the time synchronization error of the attitude determination system, the orbit determination system and the camera system, improving the positioning accuracy of the satellite image without ground control points, eliminating the time positioning error that changes with latitude, and realizing the accurate solution of the time synchronization error.

[0004] To this end, the present invention provides the following technical solutions:

[0005] On the one hand, the present invention provides a method for geometric calibration of space-time errors of a swing-scanning satellite, comprising the following steps:

[0006] Step 1: Generate a simulated image of a swing-scanning camera image;

[0007] Step 2: extracting control points using the simulated image, and then determining the ground coordinates of the control points and the corresponding image point coordinates on the original swing-scanning camera image;

[0008] Step 3: Determine the main error sources of the swing-scanning camera, and use the ground coordinates of the control points and the image point coordinates to perform geometric calibration and update the geometric calibration parameters, wherein the main error sources include at least time synchronization error, and also include installation error and / or camera parameter error;

[0009] Among them, first, the ground coordinates of the control points of a single area and the corresponding image point coordinates are used to perform geometric calibration of installation errors and / or camera parameter errors to determine the installation parameters and / or camera parameters and / or attitude parameters; then, the ground coordinates of the control points of the long track area and the corresponding image point coordinates are used to perform geometric calibration of time synchronization errors to determine the time synchronization parameters; after the time synchronization error is checked, the geometric calibration of the installation error and / or camera parameter error is repeated to update the installation parameters and / or camera parameters and / or attitude parameters.

[0010] The technical solution of the present invention takes into account the time synchronization error, installation error, and camera parameter error at the same time. First, the geometric calibration of the installation error and camera parameter error is performed through the control points of a single area to determine the geometric calibration parameters such as the installation parameters, or the camera parameters and / or the attitude parameters; on this basis, the control points of the long track area are used to perform the geometric calibration of the time synchronization error. It should be understood that the purpose of performing the geometric calibration of the time synchronization error is to eliminate / reduce the influence of the time synchronization error. During the time synchronization error calibration process, the attitude parameters will also be updated accordingly, or it can be understood that the time synchronization parameters are correlated with the installation parameters, the camera parameters, and the attitude parameters. When the time synchronization error is eliminated or reduced and the time synchronization parameters are updated, the attitude parameters change accordingly (as can be seen from the optical axis pointing formula (8) below). The attitude parameters are correlated with the installation parameters and the camera parameters (well known in the art). Therefore, after the time synchronization error is verified, the technical solution of the present invention finally performs the geometric calibration of the installation error and the camera parameter error again, thereby effectively solving the technical problem that the time synchronization error is usually easily coupled with the installation angle error and the position error in positioning.

[0011] Optionally, the process of using the ground coordinates of the control points of a single area and the corresponding image point coordinates to perform geometric calibration of the installation error and / or camera parameter error, and the process of using the ground coordinates of the control points of a long track area and the corresponding image point coordinates to perform geometric calibration of the time synchronization error are both solved by a parameter method, specifically:

[0012] Constructing the optical axis orientation of the image point under the corresponding error term in the body coordinate system, and then constructing the observation equation and its error equation based on the optical axis orientation, and then solving the error equation to determine the calibration parameters, wherein the calibration parameters are installation parameters and / or camera parameters and / or posture parameters and / or time synchronization parameters;

[0013] The observation equation is expressed as:

[0014]

[0015] In the above formula, q x ,q y is the virtual observation value corresponding to the x and y directions of the observation equation set; B ,Y B ,Z B The image point corresponds to the optical axis pointing u=[X B Y B Z B ] components in the x, y, and z directions;

[0016] The residual of the error equation is expressed as:

[0017]

[0018] satisfy:

[0019] In the above formula, v x ,v y is the residual in the x,y direction, is the object coordinate of the body coordinate system, is the installation angle of the swing-scan camera, p ECEF (r,c) is the ground point coordinate corresponding to the image point (r,c), r,c is the image point coordinate, is the Earth rotation matrix, is the posture matrix, s ECEF (t) is the position of the swing-scanning satellite in the Earth-centered Earth-fixed system, and s is the distance between the satellite and the Earth;

[0020] The error equation is expressed as: V = AX-L, P

[0021] In the formula, the error vector V = [v x ,v y ], A is the coefficient matrix, which is obtained by taking partial derivatives of the unknown parameters to be calibrated; L is the constant part, which is calculated using the initial values ​​of the calibration parameters; P is the weight matrix, and X is the unknown number vector, which represents the calibration parameter correction number.

[0022] Optionally, when the geometric calibration of the installation error and / or the camera parameter error is performed using the control points of a single region, the optical axis pointing u of the image point under the corresponding error term in the body coordinate system is s (θ,τ,α,β,x,y,f) is expressed as:

[0023]

[0024] In the above formula, μ represents the pixel size of the detector, f is the principal distance, (x, y) is the position of the pixel (r, c) in the focal plane, is the rotation matrix of the focal plane and the lens coordinate system, θ is the image scanning angle, α and β are the starting phase angle and pitch angle of the K mirror on the swing-scan satellite, τ is the 45° mirror axis angle on the swing-scan satellite, R M (θ,τ) is the reflection matrix of the 45° mirror, u c (r) is the light in the camera coordinate system, R K (θ, α, β) is the reflection matrix that takes into account the installation error of the K mirror.

[0025] Optionally, when the ground coordinates of the control points in the long track area and the corresponding image point coordinates are used to perform geometric calibration of the time synchronization error, the optical axis direction of the pixel under the corresponding error term in the body coordinate system is It is expressed as:

[0026]

[0027] In the above formula, m represents the proportional factor, is the installation angle of the swing-scanning camera, Δt3 is the time synchronization error between the attitude determination system and the camera system, is the angular velocity determined by the attitude determination system, Δt2 is the time synchronization error between the camera system and the earth system, is the angular velocity of the earth's rotation, Δt1 is the synchronization error between the navigation system and the camera system, s ECEF (t) is the position of the satellite in the Earth-centered fixed system, s is the distance between the satellite and the Earth, is the satellite platform speed, t is the time, ω,κ are the pitch angle, roll angle and yaw angle of the attitude respectively, p ECEF (r,c) is the ground point coordinate corresponding to the image point (r,c).

[0028] Optionally, the weight matrix P and the variance covariance matrix Q of the virtual observations ll The relationship is: The variance-covariance matrix Q ll It is expressed as:

[0029] Q ll =D·Q ii ·D T

[0030] Where D is the partial derivative matrix with respect to (x, y), Q ii is the variance-covariance matrix of the image point measurement coordinates converted to the focal plane coordinate system, and T is the matrix transpose symbol.

[0031] The technical solution of the present invention preferably takes into account the change in the weight of the control point, sets the above-mentioned weight matrix, takes into account the change in uncertainty caused by the difference in viewing angle, and realizes the accurate solution of the camera parameters.

[0032] Optionally, geometric calibration of installation error and / or camera parameter error is performed to determine installation parameters and / or camera parameters and / or posture parameters. After the calibration is completed, scanning instability modeling is performed to update the scanning angle θ and the 45° mirror axis angle τ, specifically:

[0033] The median value of the scanning error corresponding to the control point is used for denoising filtering, where the scanning error is expressed as:

[0034]

[0035] In the formula, Δp x ,Δp y is the deviation in the x and y directions in the homogeneous coordinate system of the focal plane, X C ,Y C ,Z C is the direction of the light in the camera coordinate system; f is the principal distance, (x, y) is the position of the pixel in the focal plane, and μ represents the pixel size of the detector;

[0036] The filtered scanning error is then converted into a function of the scanning angle θ and the 45° mirror axis angle τ:

[0037]

[0038]

[0039] In the formula, is the scanning error after filtering, Δτ(θ), Δθ(θ) is the correction value of the scanning angle θ and the 45° mirror axis angle τ, and the scanning angle θ and the 45° mirror axis angle τ are updated accordingly using the correction value.

[0040] Wherein, the installation error includes: installation angle, and / or K mirror installation error, corresponding to the starting phase angle and pitch angle of the K mirror, and / or 45° mirror installation error, corresponding to the 45° mirror axis angle;

[0041] The camera parameter errors include: principal point and / or principal distance f parameter errors;

[0042] The time synchronization error includes: the time synchronization error between the swing-scanning camera and the rotation of the earth, and / or the swing-scanning camera and the attitude fixing system, and / or the swing-scanning camera and the GNSS system.

[0043] Optionally, the process of extracting the control points is:

[0044] Calculate the gradient value of the entire scene image of the target area on the simulation image;

[0045] According to a certain window size and search interval, the size of the gradient in the matching window is calculated pixel by pixel;

[0046] Determine the point with the maximum gradient in the search matching window as the feature point, and the distance between adjacent feature points must exceed half of the matching window;

[0047] After extracting the feature points, it is necessary to match and search for points with the same name on the original scanning camera image to obtain sub-pixel level positioning, that is, the image point coordinates of the control points.

[0048] In another aspect, the present invention provides a positioning method based on geometric calibration of the space-time error of a swing-scanning satellite, comprising the following steps:

[0049] Step 1: Generate a simulated image of a swing-scanning camera image;

[0050] Step 2: extracting control points using the simulated image, and then determining the ground coordinates of the control points and the corresponding pixel coordinates on the original swing-scanning camera image;

[0051] Step 3: Determine the main error sources of the swing-scanning camera, and use the ground coordinates of the control points and the image point coordinates to perform geometric calibration and update the geometric calibration parameters, wherein the main error sources include at least time synchronization error, and also include installation error and / or camera parameter error;

[0052] Among them, first, the ground coordinates of the control points of a single area and the corresponding image point coordinates are used to perform geometric calibration of installation errors and / or camera parameter errors to determine the installation parameters and / or camera parameters and / or attitude parameters; then, the ground coordinates of the control points of the long track area and the corresponding image point coordinates are used to perform geometric calibration of time synchronization errors to determine the time synchronization parameters; after the time synchronization error is checked, the geometric calibration of the installation error and / or camera parameter error is repeated to update the installation parameters and / or camera parameters and / or attitude parameters.

[0053] Step 4: Based on the parameters of the swing-scan camera after geometric calibration, use global image data for positioning.

[0054] In a third aspect, the present invention provides an electronic terminal, comprising:

[0055] one or more processors;

[0056] and a memory storing one or more computer programs;

[0057] The processor calls the computer program to implement:

[0058] The steps of a method for geometrically calibrating the space-time errors of a swing-scanning satellite or the steps of a positioning method based on the geometric calibration of the space-time errors of a swing-scanning satellite.

[0059] In a fourth aspect, the present invention provides a readable storage medium storing a computer program, wherein the computer program is called by a processor to implement:

[0060] The steps of a method for geometrically calibrating the space-time errors of a swing-scanning satellite or the steps of a positioning method based on the geometric calibration of the space-time errors of a swing-scanning satellite.

[0061] Beneficial Effects

[0062] Compared with the existing method, the advantages of the present invention are:

[0063] 1. The geometric calibration method of the space-time error of the swing-scanning satellite proposed in the present invention first performs geometric calibration of the installation error and the camera parameter error through the control points of a single area; then uses the control points of the long orbit area to perform geometric calibration of the time synchronization error; finally, geometric calibration of the installation error and the camera parameter error is performed again, effectively solving the technical problem that the time synchronization error is usually easily coupled with the installation angle error and the position error in positioning, realizing the calculation method of the time synchronization error of the attitude determination system, the orbit determination system, and the camera system, improving the positioning accuracy of the satellite image without ground control points, and eliminating the time positioning error that changes with latitude. Among them, the long orbit data is used to achieve a more accurate solution to the time synchronization error.

[0064] 2. In the preferred embodiment of the present invention, the problem of discontinuous scanning of the swing-scanning imaging system is also taken into account, and then the scanning instability model is modeled to update the scanning angle θ and the 45° mirror axis angle τ, thereby expanding the scanning instability model at the pitch angle and improving the accuracy of the calibration.

[0065] 3. In the preferred embodiment of the present invention, a high-precision, high-density control point extraction method is proposed, which ensures the accuracy of the points through feature point extraction, realizes the extraction of high-precision, high-density control points, and further improves the geometric calibration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a schematic diagram of control points;

[0067] Figure 2 Schematic diagram of positioning residual after calibration;

[0068] Figure 3 It is a flow chart of a method for geometric calibration of space-time errors of a swing-scanning satellite provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The present invention aims at the problem of geometric calibration of satellite-borne swing-scanning cameras, and provides a geometric calibration method, positioning method and system for the spatiotemporal error of a swing-scanning satellite, with the purpose of realizing high-precision geometric calibration of the time synchronization error and camera parameters of satellite-borne swing-scanning cameras, and realizing high-precision positioning of satellite-borne swing-scanning cameras. Among them, on the one hand, the present invention takes into account the geometric characteristics of the imaging of the swing-scanning camera, and adopts a high-precision control point extraction method to extract dense ground control points; on the other hand, the installation error and camera parameter error are first eliminated to achieve accurate solution of the camera parameters; then the long orbit data is used to perform geometric calibration of the time synchronization error, determine and eliminate the time synchronization error, and finally update and eliminate the installation error and camera parameter error to update the camera parameters; on the third hand, the scanning model of the swing mirror is established for the errors that cannot be solved by the parameter method, that is, the scanning instability modeling is performed to update the scanning angle θ and the 45° mirror axis angle τ. In summary, the technical solution of the present invention can effectively improve the positioning accuracy of the swing-scanning satellite image and achieve global high-precision positioning.

[0070] The present invention will be further described below with reference to the embodiments.

[0071] Example 1

[0072] The embodiment of the present invention provides a method for geometric calibration of a sweeping satellite space-time error, comprising the following steps:

[0073] Step 1: Generate a simulated image of a panning camera image.

[0074] The embodiment of the present invention preferably selects image data with more land and no clouds (such as Australia, North Africa, the Middle East and other regions) and its attached or calculated geolocation lookup table (the geolocation of the swing-scan satellite image is the basic link of satellite image processing, which will generate the corresponding geolocation lookup table for the swing-scan satellite image, that is, establish a one-to-one correspondence between the swing-scan satellite image and the ground coordinates). Then, the position of each pixel in the near-infrared channel of the Sentinel-2 global reference base map (or Landsat-8) is calculated, and the point spread function of the swing-scan camera imaging is considered, and the gray value is calculated pixel by pixel to realize the construction of the simulated image.

[0075] Step 2: Extract control points using the simulated image, and then determine the ground coordinates of the control points and the corresponding image point coordinates on the original pan-scanning camera image.

[0076] The embodiment of the present invention preferably extracts feature points based on grayscale gradient changes on the simulated image, then matches and screens the feature points in the original image to obtain control points, and determines the ground coordinates of the control points and the corresponding image point coordinates on the original image.

[0077] The feature point extraction process is as follows:

[0078] 2-1: Calculate the gradient value of the entire scene image;

[0079] 2-2: According to a certain window size and search interval, the size of the gradient in the matching window is calculated pixel by pixel;

[0080] 2-3: Determine the point with the maximum gradient in the search window as the feature point, and the distance between the feature points must exceed half of the matching window.

[0081] It should be noted that the feature points preferably extracted in the embodiment of the present invention should also meet the following requirements: the gradient is the largest and is not a straight line, the matching window does not cross frames within the image scanning frame, the feature points are distributed as evenly as possible within the image range, and the matching window is smaller than the number of detectors.

[0082] 2-4: After extracting the feature points, it is necessary to perform matching and searching for points of the same name on the original image to obtain pixel-level positioning. In the embodiment of the present invention, the correlation coefficient method is preferably used to search for points of the same name on the original image.

[0083] In some embodiments, it is also preferred to use the least square method to optimize the point positions to achieve high-precision extraction of points of the same name. In the feature point matching search, a coarse-to-fine matching search strategy is adopted. Specifically, by constructing a pyramid image, the search range is preliminarily set, the initial estimation of the basic matching deviation is performed, and the coarse position is introduced as the initial value into the fine matching process, thereby reducing the search range and improving the extraction accuracy and reliability of the control points when large errors occur.

[0084] In some embodiments, it is preferred to use multiple points closest to high-precision points of the same name to calculate the topological geometric relationship of neighboring points of the same name and eliminate gross errors in matching points.

[0085] The ground coordinates of the control points are calculated using the initial geolocation lookup table, and the image point coordinates on the original image are the image coordinates. The ground coordinates of the control points are expressed as follows:

[0086]

[0087] In the formula, p ECEF (r,c) is the ground point coordinate corresponding to the image point (r,c), s ECEF (t) is the position of the satellite in the Earth-centered fixed system, s is the distance between the satellite and the Earth, is the Earth rotation matrix, is the posture matrix, is the camera installation angle; u s (r, c) is the optical axis direction of the image point (r, c) in this system, which is calculated according to formula 2.

[0088] Step 3: Determine the main error sources of the swing-scanning camera, and use the ground coordinates of the control points and the image point coordinates for geometric calibration, wherein the main error sources include at least time synchronization error, and also include installation error and / or camera parameter error. First, use the control points of a single area to perform geometric calibration of the installation error and / or camera parameter error to determine the installation parameters and / or camera parameters and / or attitude parameters; then use the control points of the long track area to perform geometric calibration of the time synchronization error to determine the time synchronization parameters; finally, repeat the geometric calibration of the installation error and / or camera parameter error to update the installation parameters and / or camera parameters and / or attitude parameters.

[0089] When using the parameter method to eliminate installation errors and camera parameter errors and performing geometric calibration to determine installation parameters, camera parameters, and attitude parameters, it is preferred to perform the following process:

[0090] Step 3-1: Screen the calibration data and select images of land areas with no or little cloud for control point extraction. The control points should be dense and evenly distributed within the image range to avoid the problem of fewer points in some areas due to large areas of water. It is also preferred that the attitude error in the image positioning error is relatively stable to avoid inaccurate camera parameter estimation due to point distribution and attitude errors.

[0091] Therefore, this step can be completed after step 1 and step 2, or the control points can be screened on the basis of step 1 and step 2.

[0092] Step 3-2: Use the pixel coordinates (r, c) of the control point to calculate the orientation u of the pixel in the body coordinate system s , and then construct the virtual observation value in the body coordinate system. Among them, the image row coordinates are used to determine the scanning frame, the image column coordinates are used to determine the image scanning angle θ, and other parameters are combined to calculate the optical axis direction in the corresponding body coordinate system. The optical axis direction u in the body coordinate system s (θ,τ,α,β,x,y,f) is expressed as:

[0093]

[0094] In the above formula, μ represents the pixel size of the detector, f is the principal distance, (x, y) is the position of the pixel (r, c) in the focal plane, is the rotation matrix of the focal plane and the lens coordinate system, θ is the image scanning angle, α and β are the starting phase angle and pitch angle of the K mirror on the swing-scan satellite, τ is the 45° mirror axis angle on the swing-scan satellite, R M (θ,τ) is the reflection matrix of the 45° mirror, u c (r) is the light in the camera coordinate system, R K (θ, α, β) is the reflection matrix that takes into account the installation error of the K mirror.

[0095] Step 3-3: Using the optical axis pointing in the body coordinate system, calculate the virtual observation value to construct the observation equation and its error equation.

[0096] The observation equation is expressed as:

[0097]

[0098] In the above formula, q x ,q y is the virtual observation value corresponding to the x and y directions of the observation equation set; B ,Y B ,Z B The image point corresponds to the optical axis pointing u=[X B Y B Z B ] components in the x, y, and z directions.

[0099] The embodiment of the present invention preferably uses the error propagation law to calculate the variance covariance matrix Q of the virtual observation value ll :

[0100] Q ll =D·Q ii ·D T (4)

[0101] Where D is the partial derivative matrix with respect to the pixel position (x, y) in the focal plane, which can be calculated by formula (2), Q ii is the variance-covariance matrix of the image point measurement coordinates converted to the focal plane coordinate system.

[0102] The residual of the error equation is expressed as:

[0103]

[0104] satisfy:

[0105] In the above formula, v x ,v y is the residual in the x,y direction, is the object coordinate of the body coordinate system, is the installation angle of the swing-scan camera, p ECEF (r,c) is the ground point coordinate corresponding to the image point (r,c), r,c is the image point coordinate, is the Earth rotation matrix, is the posture matrix, s ECEF (t) is the position of the swing-scanning satellite in the Earth-centered fixed system, and s is the distance between the satellite and the Earth.

[0106] Then the error equation is expressed as: V = AX-L, P

[0107] In the formula, the error vector V = [v x ,v y ], A is the coefficient matrix, obtained by taking partial derivatives of unknown parameters to be calibrated; L is the constant part, calculated using the initial values ​​of calibration parameters; P is the weight matrix, X is the unknown vector, representing the calibration parameter correction number. It should be noted that the error equation V = AX-L, P is a conventional expression in the industry, and the comma represents the interval. The former is a numerical model, that is, V = AX-L represents the error numerical model; the latter is a statistical model, that is, the weight matrix P. In addition, the expression and calculation of matrices A and L are conventional settings in this field, so they are not described in detail.

[0108] Step 3-4: Solve the error equation again to determine calibration parameters, where the calibration parameters are installation parameters and / or camera parameters and / or posture parameters and / or time synchronization parameters.

[0109] The embodiment of the present invention preferably adopts the weighted least squares method to calculate the calibration parameters, that is, by calculating the unknown vector X, the calibration parameters are updated; then according to formula 2, the direction of the optical axis in the corresponding body coordinate system is also updated, and the corresponding virtual observation quantity and error equation also change accordingly, and multiple iterations are performed until the parameters converge.

[0110] It should be noted that in this embodiment, the camera parameters and installation parameters, such as θ, τ, α, β, f defined above, will be calculated, and the attitude parameters (pitch angle, roll angle and yaw angle) will be further determined. The calculation process of the attitude parameters is an existing conventional means, and the present invention will not elaborate on this. It should also be noted that this embodiment takes into account both the installation error and the camera parameter error. In another embodiment, not considering one of the installation error or the camera error also falls within the protection scope of the present invention.

[0111] In addition, the present invention also preferably determines the installation parameters and / or camera parameters and / or posture parameters through geometric calibration of the installation error and / or camera parameter error, and after the calibration is completed, performs scanning instability modeling to update the scanning angle θ and the 45° mirror axis angle τ, specifically:

[0112] The median value of the scanning error corresponding to the control point is used for denoising filtering, where the scanning error is expressed as:

[0113]

[0114] In the formula, Δp x ,Δp y is the deviation in the x and y directions in the homogeneous coordinate system of the focal plane, X C ,Y C ,Z Cis the direction of the light in the camera coordinate system; f is the principal distance, (x, y) is the position of the pixel in the focal plane, and μ represents the pixel size of the detector;

[0115] The filtered scanning error is then converted into a function of the scanning angle θ and the 45° mirror axis angle τ:

[0116]

[0117] In the formula, is the scanning error after filtering, Δτ(θ), Δθ(θ) is the correction value of the scanning angle θ and the 45° mirror axis angle τ, and the scanning angle θ and the 45° mirror axis angle τ are updated accordingly using the correction value.

[0118] Regarding the use of parameter method to eliminate time synchronization error, the geometric calibration of time synchronization error is performed using the control points in the long track area as follows:

[0119] Step 4-1: Verify data screening, and give priority to selecting possibly long orbit data, such as data spanning Australia, China, and Russia in descending orbit data or long orbit data spanning Africa, the Middle East, and Russia in ascending orbit data. Select summer or autumn data in the northern hemisphere, extract control points from the long orbit data, and perform appropriate screening to ensure that the control points are evenly distributed on the orbit.

[0120] Therefore, this step can be completed after step 1 and step 2, or the control points can be screened on the basis of step 1 and step 2.

[0121] Step 4-2: The constructed time synchronization error calibration model takes the instrument time as the benchmark, constructs the time synchronization error between the instrument and the earth's rotation, the instrument and the attitude determination system, and the instrument and the GNSS system; when solving the parameters, the observation equation with orbit as the unit is constructed. When establishing the time synchronization error model, the optical axis of the object in the body coordinate system points to Defined as:

[0122]

[0123] In the above formula, m represents the proportional factor, is the installation angle of the swing-scanning camera, Δt3 is the time synchronization error between the attitude determination system and the camera system, is the angular velocity determined by the attitude determination system, Δt2 is the time synchronization error between the camera system and the earth system, is the angular velocity of the earth's rotation, Δt1 is the synchronization error between the navigation system and the camera system, s ECEF (t) is the position of the satellite in the Earth-centered fixed system, s is the distance between the satellite and the Earth, is the satellite platform speed, t is the time, ω,κ are the pitch angle, roll angle and yaw angle of the attitude respectively, p ECEF (r,c) is the ground point coordinate corresponding to the image point (r,c).

[0124] Step 4-3: Referring to step 3-3 to step 3-4, construct an error equation, and solve it to determine the calibration parameters, that is, obtain time synchronization parameters, such as time synchronization errors Δt1, Δt2, Δt3 or time synchronization parameters corrected based on time synchronization errors Δt1, Δt2, Δt3.

[0125] It should be understood that the present invention preferably updates the camera parameters, installation parameters, and posture parameters again after the geometric calibration of the time synchronization error, that is, executes steps 3-1 to 3-4; and updates the scanning model again.

[0126] Example 2

[0127] The embodiment of the present invention provides a positioning method based on geometric calibration of the space-time error of a swing-scanning satellite, comprising the following steps:

[0128] Step 1: Generate a simulated image of a swing-scanning camera image;

[0129] Step 2: extracting control points using the simulated image, and then determining the ground coordinates of the control points and the corresponding pixel coordinates on the original swing-scanning camera image;

[0130] Step 3: Determine the main error sources of the swing-scanning camera, and use the ground coordinates of the control points and the image point coordinates to perform geometric calibration and update the geometric calibration parameters, wherein the main error sources include at least time synchronization error, and also include installation error and / or camera parameter error;

[0131] Among them, first, the ground coordinates of the control points of a single area and the corresponding image point coordinates are used to perform geometric calibration of installation errors and / or camera parameter errors to determine the installation parameters and / or camera parameters and / or attitude parameters; then, the ground coordinates of the control points of the long track area and the corresponding image point coordinates are used to perform geometric calibration of time synchronization errors to determine the time synchronization parameters; after the time synchronization error is checked, the geometric calibration of the installation error and / or camera parameter error is repeated to update the installation parameters and / or camera parameters and / or attitude parameters.

[0132] Step 4: Based on the parameters of the swing-scanning camera after geometric calibration, use the global image data for positioning. It should be understood that the positioning process is achievable by the prior art in the art, so the present invention will not elaborate on it in detail.

[0133] In summary, the geometric calibration method for the space-time error of the swing-scanning satellite provided by the present invention is implemented as follows: Figure 1-3As shown in the figure, we first search for feature points on the simulated image, use the feature points to match on the original image, and remove gross errors. On this basis, we use the coordinates of the same-name points and the initial geographic positioning lookup table to calculate the corresponding ground coordinates and obtain dense ground control points, such as Figure 1 shown.

[0134] Then, the positioning error in the focal plane homogeneous coordinate system is calculated using the control points and the rigorous imaging geometry model, and the spatial and temporal characteristics of the error distribution are analyzed to determine the main error terms of the satellite-borne swing-scanning camera error. It should be understood that the error analysis methods in this field can be used to determine the error sources, such as installation error, camera parameter error, and time synchronization error.

[0135] Then, using dense ground control points and the imaging geometry model of the swing-scanning camera, the observation equation based on the body coordinate system is constructed, and based on the law of error propagation, the uncertainty of the control point in this system is determined; the partial derivatives of the geometric calibration elements are calculated, the error equation is constructed, and the calibration parameters are solved using weighted least squares. The execution details of steps 3-1 to 3-4 are as follows.

[0136] Secondly, based on the geometric calibration of the parameter method, the residual is calculated using dense control points, and the scanning instability model is constructed to update the scanning angle θ and the 45° mirror axis angle τ. The positioning residual after calibration is as follows: Figure 2 As shown;

[0137] Secondly, select long track data, extract dense control points, and ensure that the control points are evenly distributed in the track direction. On this basis, build a time synchronization error geometry calibration model to solve the time synchronization error and camera geometry calibration parameters. For example, the execution details of steps 4-1 to 4-3 above.

[0138] After completing the time synchronization error and camera parameter solution, the high-precision geometric calibration of the swing-scanning camera is once again carried out;

[0139] Finally, global image data is used to conduct positioning accuracy assessment and verification.

[0140] Example 3

[0141] The embodiment of the present invention provides a system for a geometric calibration method based on a sweeping satellite spatiotemporal error, comprising:

[0142] A simulation image generation module, used for generating a simulation image of a swing-scanning camera image;

[0143] A control point extraction module, used to extract control points using the simulated image, and then determine the ground coordinates of the control points and the corresponding image point coordinates on the original swing-scanning camera image;

[0144] A geometry calibration module is used to determine the main error sources of the swing-scanning camera and to perform geometry calibration and update geometry calibration parameters using the ground coordinates of the control points and the image point coordinates. The main error sources include at least time synchronization errors, and also installation errors and / or camera parameter errors.

[0145] Among them, first, the ground coordinates of the control points of a single area and the corresponding image point coordinates are used to perform geometric calibration of installation errors and / or camera parameter errors to determine the installation parameters and / or camera parameters and / or attitude parameters; then, the ground coordinates of the control points of the long track area and the corresponding image point coordinates are used to perform geometric calibration of time synchronization errors to determine the time synchronization parameters; after the time synchronization error is checked, the geometric calibration of the installation error and / or camera parameter error is repeated to update the installation parameters and / or camera parameters and / or attitude parameters.

[0146] It should be understood that the above module division is only an example. In some embodiments, the above functional modules are implemented in the form of hardware, and in other embodiments, the above functional modules are implemented in the form of software. The present invention does not make specific limitations on this.

[0147] In other embodiments, the system further includes a positioning module for performing positioning using global image data based on the parameters of the swing-scanning camera after geometric calibration.

[0148] Example 4

[0149] The present invention provides an electronic terminal, comprising:

[0150] one or more processors;

[0151] and a memory storing one or more computer programs;

[0152] The processor calls the computer program to implement:

[0153] The steps of a method for geometrically calibrating the space-time errors of a swing-scanning satellite or the steps of a positioning method based on the geometric calibration of the space-time errors of a swing-scanning satellite.

[0154] Specific implementation:

[0155] Step 1: Generate a simulated image of a swing-scanning camera image;

[0156] Step 2: extracting control points using the simulated image, and then determining the ground coordinates of the control points and the corresponding pixel coordinates on the original swing-scanning camera image;

[0157] Step 3: Determine the main error sources of the swing-scanning camera, and use the ground coordinates of the control points and the image point coordinates to perform geometric calibration and update the geometric calibration parameters, wherein the main error sources include at least time synchronization error, and also include installation error and / or camera parameter error;

[0158] Among them, first, the ground coordinates of the control points of a single area and the corresponding image point coordinates are used to perform geometric calibration of installation errors and / or camera parameter errors to determine the installation parameters and / or camera parameters and / or attitude parameters; then, the ground coordinates of the control points of the long track area and the corresponding image point coordinates are used to perform geometric calibration of time synchronization errors to determine the time synchronization parameters; after the time synchronization error is checked, the geometric calibration of the installation error and / or camera parameter error is repeated to update the installation parameters and / or camera parameters and / or attitude parameters.

[0159] Or specifically execute:

[0160] Step 1: Generate a simulated image of a swing-scanning camera image;

[0161] Step 2: extracting control points using the simulated image, and then determining the ground coordinates of the control points and the corresponding pixel coordinates on the original swing-scanning camera image;

[0162] Step 3: Determine the main error sources of the swing-scanning camera, and use the ground coordinates of the control points and the image point coordinates to perform geometric calibration and update the geometric calibration parameters, wherein the main error sources include at least time synchronization error, and also include installation error and / or camera parameter error;

[0163] Among them, first, the ground coordinates of the control points of a single area and the corresponding image point coordinates are used to perform geometric calibration of installation errors and / or camera parameter errors to determine the installation parameters and / or camera parameters and / or attitude parameters; then, the ground coordinates of the control points of the long track area and the corresponding image point coordinates are used to perform geometric calibration of time synchronization errors to determine the time synchronization parameters; after the time synchronization error is checked, the geometric calibration of the installation error and / or camera parameter error is repeated to update the installation parameters and / or camera parameters and / or attitude parameters.

[0164] Step 4: Based on the parameters of the swing-scan camera after geometric calibration, use global image data for positioning.

[0165] Example 5

[0166] An embodiment of the present invention provides a readable storage medium storing a computer program, wherein the computer program is called by a processor to implement:

[0167] The steps of a method for geometrically calibrating the space-time errors of a swing-scanning satellite or the steps of a positioning method based on the geometric calibration of the space-time errors of a swing-scanning satellite.

[0168] Specific implementation:

[0169] Step 1: Generate a simulated image of a swing-scanning camera image;

[0170] Step 2: extracting control points using the simulated image, and then determining the ground coordinates of the control points and the corresponding pixel coordinates on the original swing-scanning camera image;

[0171] Step 3: Determine the main error sources of the swing-scanning camera, and use the ground coordinates of the control points and the image point coordinates to perform geometric calibration and update the geometric calibration parameters, wherein the main error sources include at least time synchronization error, and also include installation error and / or camera parameter error;

[0172] Among them, first, the ground coordinates of the control points of a single area and the corresponding image point coordinates are used to perform geometric calibration of installation errors and / or camera parameter errors to determine the installation parameters and / or camera parameters and / or attitude parameters; then, the ground coordinates of the control points of the long track area and the corresponding image point coordinates are used to perform geometric calibration of time synchronization errors to determine the time synchronization parameters; after the time synchronization error is checked, the geometric calibration of the installation error and / or camera parameter error is repeated to update the installation parameters and / or camera parameters and / or attitude parameters.

[0173] Or specifically execute:

[0174] Step 1: Generate a simulated image of a swing-scanning camera image;

[0175] Step 2: extracting control points using the simulated image, and then determining the ground coordinates of the control points and the corresponding pixel coordinates on the original swing-scanning camera image;

[0176] Step 3: Determine the main error sources of the swing-scanning camera, and use the ground coordinates of the control points and the image point coordinates to perform geometric calibration and update the geometric calibration parameters, wherein the main error sources include at least time synchronization error, and also include installation error and / or camera parameter error;

[0177] Among them, first, the ground coordinates of the control points of a single area and the corresponding image point coordinates are used to perform geometric calibration of installation errors and / or camera parameter errors to determine the installation parameters and / or camera parameters and / or attitude parameters; then, the ground coordinates of the control points of the long track area and the corresponding image point coordinates are used to perform geometric calibration of time synchronization errors to determine the time synchronization parameters; after the time synchronization error is checked, the geometric calibration of the installation error and / or camera parameter error is repeated to update the installation parameters and / or camera parameters and / or attitude parameters.

[0178] Step 4: Based on the parameters of the swing-scan camera after geometric calibration, use global image data for positioning.

[0179] Similarly, the specific implementation process of each step can refer to the detailed process of the aforementioned method, and the present invention will not elaborate on this in detail.

[0180] It should be understood that in the embodiments of the present invention, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the type of device.

[0181] The readable storage medium is a computer-readable storage medium, which may be an internal storage unit of the controller described in any of the foregoing embodiments, such as a hard disk or memory of the controller. The readable storage medium may also be an external storage device of the controller, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the controller. Further, the readable storage medium may also include both an internal storage unit of the controller and an external storage device. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0182] Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned readable storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0183] It should be emphasized that the examples described in the present invention are illustrative rather than restrictive, and therefore the present invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solution of the present invention that do not depart from the purpose and scope of the present invention, whether modified or replaced, also fall within the scope of protection of the present invention.

Claims

1. A geometric calibration method for the space-time error of a swing-scanning satellite, characterized in that: The following steps are involved: Step 1: Generate a simulated image of a swing-scanning camera image; Step 2: extracting control points using the simulated image, and then determining the ground coordinates of the control points and the corresponding image point coordinates on the original swing-scanning camera image; Step 3: Determine the main error sources of the swing-scanning camera, and use the ground coordinates of the control points and the image point coordinates to perform geometric calibration and update the geometric calibration parameters, wherein the main error sources include at least time synchronization error, and also include installation error and / or camera parameter error; Among them, first, the ground coordinates of the control points of a single area and the corresponding image point coordinates are used to perform geometric calibration of installation errors and / or camera parameter errors to determine the installation parameters and / or camera parameters and / or attitude parameters; then, the ground coordinates of the control points of the long track area and the corresponding image point coordinates are used to perform geometric calibration of time synchronization errors to determine the time synchronization parameters; after the time synchronization error is checked, the geometric calibration of the installation error and / or camera parameter error is repeated to update the installation parameters and / or camera parameters and / or attitude parameters.

2. The method according to claim 1, characterized in that: The process of using the ground coordinates of the control points of a single area and the corresponding image point coordinates to perform geometric calibration of the installation error and / or camera parameter error, and the process of using the ground coordinates of the control points of a long track area and the corresponding image point coordinates to perform geometric calibration of the time synchronization error are both solved by the parameter method, specifically: Constructing the optical axis orientation of the image point under the corresponding error term in the body coordinate system, and then constructing the observation equation and its error equation based on the optical axis orientation, and then solving the error equation to determine the calibration parameters, wherein the calibration parameters are installation parameters and / or camera parameters and / or posture parameters and / or time synchronization parameters; The observation equation is expressed as: In the above formula, q x ,q y is the virtual observation value corresponding to the x and y directions of the observation equation set; B ,Y B ,Z B The image point corresponds to the optical axis pointing u=[X B Y B Z B ] components in the x, y, and z directions; The residual of the error equation is expressed as: satisfy: In the above formula, v x , v y is the residual in the x,y direction, is the object coordinate of the body coordinate system, is the installation angle of the swing-scan camera, p ECEF (r, c) is the ground point coordinate corresponding to the image point (r, c), r, c is the image point coordinate, is the Earth rotation matrix, is the posture matrix, s ECEF (t) is the position of the swing-scanning satellite in the Earth-centered Earth-fixed system, and s is the distance between the satellite and the Earth; The error equation is expressed as: V = AX-L, P In the formula, the error vector V = [v x , v y ], A is the coefficient matrix; L is the constant part, P is the weight matrix, and X is the unknown vector, which represents the correction number of the calibration parameter.

3. The method according to claim 2, characterized in that: When the geometric calibration of the installation error and / or the camera parameter error is performed using the ground coordinates of the control points of a single region and the corresponding image point coordinates, the optical axis pointing representation u of the image point under the corresponding error term in the body coordinate system is s (θ, τ, α, β, x, y, f) is: In the above formula, μ represents the pixel size of the detector, f is the principal distance, (x, y) is the position of the pixel (r, c) in the focal plane, is the rotation matrix of the focal plane and the lens coordinate system, θ is the image scanning angle, α and β are the starting phase angle and pitch angle of the K mirror on the swing-scan satellite, τ is the 45° mirror axis angle on the swing-scan satellite, R M (θ, τ) is the reflection matrix of the 45° mirror, u c (r) is the light in the camera coordinate system, R K (θ, α, β) is the reflection matrix that takes into account the installation error of the K mirror.

4. The method according to claim 2, characterized in that: When the control points in the long track area are used to perform geometric calibration of the time synchronization error, the optical axis direction of the pixel under the corresponding error item in the body coordinate system is It is expressed as: In the above formula, m represents the proportional factor, is the installation angle of the swing-scanning camera, Δt3 is the time synchronization error between the attitude determination system and the camera system, is the angular velocity determined by the attitude determination system, Δt2 is the time synchronization error between the camera system and the earth system, is the angular velocity of the earth's rotation, Δt1 is the synchronization error between the navigation system and the camera system, s ECEF (t) is the position of the satellite in the Earth-centered fixed system, s is the distance between the satellite and the Earth, is the satellite platform speed, t is the time, ω, κ are the pitch angle, roll angle and yaw angle of the attitude respectively, p ECEF (r, c) is the ground point coordinate corresponding to the image point (r, c).

5. The method according to claim 2, characterized in that: The weight matrix P and the variance covariance matrix Q of the virtual observations ll The relationship is: The variance-covariance matrix Q ll It is expressed as: Q ll =D·Q ii ·D T Where D is the partial derivative matrix with respect to the pixel position (x, y) in the focal plane, Q ii is the variance-covariance matrix of the image point measurement coordinates converted to the focal plane coordinate system, and T is the matrix transpose symbol.

6. The method according to claim 1, characterized in that: Perform geometric calibration of installation error and / or camera parameter error to determine installation parameters and / or camera parameters and / or posture parameters. After the calibration is completed, perform scanning instability modeling to update the scanning angle θ and the 45° mirror axis angle τ, specifically: The median value of the scanning error corresponding to the control point is used for denoising filtering, where the scanning error is expressed as: In the formula, Δp x , Δp y is the deviation in the x and y directions in the homogeneous coordinate system of the focal plane, X C , Y C , Z C is the direction of the light in the camera coordinate system; f is the principal distance, (x, y) is the position of the pixel in the focal plane, and μ represents the pixel size of the detector; The filtered scanning error is then converted into a function of the scanning angle θ and the 45° mirror axis angle τ: In the formula, is the filtered scanning error, Δτ(θ), Δθ(θ) is the correction value of the scanning angle θ and the 45° mirror axis angle τ, and the correction value is then used to update the scanning angle θ and the 45° mirror axis angle τ accordingly.

7. The method according to claim 1, characterized in that: The process of extracting the control points is as follows: Calculate the gradient value of the entire scene image of the target area on the simulation image; According to a certain window size and search interval, the size of the gradient in the matching window is calculated pixel by pixel; Determine the point with the maximum gradient in the search matching window as the feature point, and the distance between adjacent feature points must exceed half of the matching window; After extracting the feature points, it is necessary to match and search for points with the same name on the original scanning camera image to obtain sub-pixel level positioning, that is, the image point coordinates of the control points.

8. A positioning method based on geometric calibration of the space-time error of a swing-scanning satellite, characterized in that: The following steps are involved: Step 1: Generate a simulated image of a swing-scanning camera image; Step 2: extracting control points using the simulated image, and then determining the ground coordinates of the control points and the corresponding pixel coordinates on the original swing-scanning camera image; Step 3: Determine the main error sources of the swing-scanning camera, and use the ground coordinates of the control points and the image point coordinates to perform geometric calibration and update the geometric calibration parameters, wherein the main error sources include at least time synchronization error, and also include installation error and / or camera parameter error; Among them, firstly, the ground coordinates of the control points of the single area and the corresponding image point coordinates are used to perform geometric calibration of the installation error and / or camera parameter error to determine the installation parameters and / or camera parameters and / or attitude parameters; then, the ground coordinates of the control points of the long track area and the corresponding image point coordinates are used to perform geometric calibration of the time synchronization error to determine the time synchronization parameters; after the time synchronization error is checked, the geometric calibration of the installation error and / or camera parameter error is repeated to update the installation parameters and / or camera parameters and / or attitude parameters; Step 4: Based on the parameters of the swing-scan camera after geometric calibration, use global image data for positioning.

9. An electronic terminal, characterized in that: include: one or more processors; and a memory storing one or more computer programs; The processor calls the computer program to implement: The steps of a method for geometric calibration of space-time errors of a swing-scanning satellite as described in any one of claims 1-7 or the steps of a positioning method based on geometric calibration of space-time errors of a swing-scanning satellite as described in claim 8.

10. A readable storage medium, characterized in that: A computer program is stored, which is called by a processor to implement: The steps of a method for geometric calibration of space-time errors of a swing-scanning satellite as described in any one of claims 1-7 or the steps of a positioning method based on geometric calibration of space-time errors of a swing-scanning satellite as described in claim 8.

Images