Remote sensing satellite on-orbit automatic geometric calibration method and device, electronic equipment and medium

Abstract

The present application relates to the technical field of measurement, in particular to a kind of remote sensing satellite on-orbit automatic geometric calibration method, device, electronic equipment and medium, wherein, method includes: obtaining the three-dimensional point coordinates of building edge point from original point cloud data set, and project to the image space of satellite image, obtain point cloud two-dimensional point coordinates;Image two-dimensional point coordinates of building in image space are extracted from original image, to utilize point cloud two-dimensional point coordinates and image two-dimensional point coordinates are registered, obtain multiple sets of offset parameters;Carry out reliability evaluation of registration parameter, to obtain reliable image block and corresponding offset parameter to meet preset reliability condition, to establish virtual control point, and utilize virtual control point to solve satellite image geometric calibration model.Therefore, in the related art, it is dependent on professional technical personnel to process, the automation level is lower, it is difficult to adapt to the rapidly growing number of remote sensing satellites, so that its high-precision observation ability is difficult to be fully utilized Technical problems.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of measurement, in particular to a remote sensing satellite on-orbit automatic geometric calibration method, device, electronic equipment and medium. BACKGROUND

[0002] On-orbit calibration of remote sensing satellite sensors is crucial for producing high-precision and high-quality remote sensing products. Its tasks include geometric calibration, radiometric calibration and spectral calibration. For high-resolution satellite images, geometric calibration is the key to achieving meter-level direct positioning accuracy and sub-pixel-level internal accuracy. These accuracies are crucial for image product generation, stereo observation and target positioning tasks.

[0003] In the related art, geometric calibration is usually considered as a low-frequency task, and little consideration is given to the level of automation. With the reduction of satellite manufacturing and launch costs, the number of on-orbit remote sensing satellites has rapidly exceeded one thousand, and continues to grow at a rate of several hundred per year. The geometric calibration method in the related art, whether it relies on ground control points or not, is usually complex and requires professional technical personnel to complete. Tasks such as collecting control points from images, selecting appropriate multi-degree overlapping satellite images, and designing regional network adjustment schemes are usually handled by professional technical personnel, which makes it difficult to adapt to the rapid growth of the number of remote sensing satellites, making it difficult to fully utilize their high-precision observation capabilities, and improvement is urgently needed. SUMMARY

[0004] The present application provides a remote sensing satellite on-orbit automatic geometric calibration method, device, electronic equipment and medium to solve the technical problem that in the related art, professional technical personnel are relied on for processing, the level of automation is low, it is difficult to adapt to the rapid growth of the number of remote sensing satellites, and it is difficult to fully utilize their high-precision observation capabilities.

[0005] The first aspect of the present application provides a remote sensing satellite on-orbit automatic geometric calibration method, comprising the following steps: obtaining three-dimensional point coordinates of building edge points from an original point cloud data set, and projecting the three-dimensional point coordinates to the image space of a satellite image to obtain two-dimensional point coordinates of point clouds; extracting two-dimensional point coordinates of buildings in the image space from the original image to utilize the two-dimensional point coordinates of point clouds and the two-dimensional point coordinates of images for registration to obtain a plurality of sets of offset parameters; performing reliability evaluation of registration parameters based on the plurality of sets of offset parameters to obtain reliable image blocks and corresponding offset parameters that meet a preset reliability condition; establishing virtual control points using the reliable image blocks and corresponding offset parameters; and solving a satellite image geometric calibration model using the virtual control points to obtain model parameters of the satellite image geometric calibration model.

[0006] Optionally, in an embodiment of the present application, the method for obtaining the three-dimensional point coordinates of the building edge points from the original point cloud data set comprises: performing building extraction on the original point cloud data set to obtain a building point set; performing plane segmentation on the building point set to extract building edge points, and constructing a point cloud three-dimensional line feature point set by using the building edge points; and obtaining the three-dimensional point coordinates of the building edge lines from the point cloud three-dimensional line feature point set.

[0007] Optionally, in an embodiment of the present application, the method for obtaining the three-dimensional point coordinates of the building edge points from the original point cloud data set comprises: performing building extraction on the original point cloud data set to obtain a building point set; performing plane segmentation on the building point set to extract building edge points, and constructing a point cloud three-dimensional line feature point set by using the building edge points; and obtaining the three-dimensional point coordinates of the building edge lines from the point cloud three-dimensional line feature point set.

[0008] Optionally, in an embodiment of the present application, the method for obtaining the three-dimensional point coordinates of the building edge points from the original point cloud data set comprises: performing building extraction on the original point cloud data set to obtain a building point set; performing plane segmentation on the building point set to extract building edge points, and constructing a point cloud three-dimensional line feature point set by using the building edge points; and obtaining the three-dimensional point coordinates of the building edge lines from the point cloud three-dimensional line feature point set.

[0009] Optionally, in an embodiment of the present application, the method for obtaining the three-dimensional point coordinates of the building edge points from the original point cloud data set comprises: performing building extraction on the original point cloud data set to obtain a building point set; performing plane segmentation on the building point set to extract building edge points, and constructing a point cloud three-dimensional line feature point set by using the building edge points; and obtaining the three-dimensional point coordinates of the building edge lines from the point cloud three-dimensional line feature point set.

[0010] The second aspect embodiment of the present application provides an on-orbit automatic geometric calibration device of a remote sensing satellite, comprising: an acquisition module configured to acquire three-dimensional point coordinates of building edge points from an original point cloud data set, and project the three-dimensional point coordinates to an image space of a satellite image to obtain two-dimensional point coordinates of point clouds; a registration module configured to extract two-dimensional point coordinates of images of buildings in the image space from an original image, and perform registration using the two-dimensional point coordinates of point clouds and the two-dimensional point coordinates of images to obtain a plurality of sets of offset parameters; an evaluation module configured to perform reliability evaluation of registration parameters based on the plurality of sets of offset parameters to obtain reliable image blocks and corresponding offset parameters that satisfy a preset reliability condition; and a calculation module configured to establish virtual control points using the reliable image blocks and the corresponding offset parameters, and solve a satellite image geometric calibration model using the virtual control points to obtain model parameters of the satellite image geometric calibration model.

[0011] Optionally, in an embodiment of the present application, the acquisition module comprises: a first extraction unit configured to perform building extraction on the original point cloud data set to obtain a building point set; a construction unit configured to perform plane segmentation on the building point set to extract building edge points, and construct a point cloud three-dimensional line feature point set using the building edge points; and a second extraction unit configured to acquire three-dimensional point coordinates of building edge lines from the point cloud three-dimensional line feature point set.

[0012] Optionally, in an embodiment of the present application, the registration module comprises: a clipping unit configured to clip the original image into a plurality of image blocks based on the two-dimensional point coordinates of images; a first acquisition unit configured to acquire point cloud image blocks corresponding to the range of the plurality of image blocks based on the two-dimensional point coordinates of point clouds; and a first calculation unit configured to calculate offset parameters between each pair of image blocks and point cloud image blocks to obtain the plurality of sets of offset parameters.

[0013] Optionally, in an embodiment of the present application, the evaluation module comprises: a definition unit configured to define a virtual hypothesis that all building boundary points in the point cloud image block are meaningless points randomly distributed; a second acquisition unit configured to obtain corresponding basic statistics based on the plurality of image blocks and the point cloud image block; a second calculation unit configured to calculate a probability value under the virtual hypothesis using the basic statistics; a judgment unit configured to judge whether the virtual hypothesis is true or not using the probability value to obtain a judgment result; and an evaluation unit configured to perform reliability evaluation of registration parameters using the judgment result to screen reliable image blocks and corresponding offset parameters that satisfy the preset reliability condition from the plurality of sets of offset parameters.

[0014] Optionally, in one embodiment of the present application, the calculation module comprises: a correction unit configured to correct satellite sensor probe element pointing angle coefficients using the virtual control points to obtain an optimized coefficient vector; and a third calculation unit configured to solve the satellite image geometric rectification model using the optimized coefficient vector.

[0015] A third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the on-orbit automatic geometric rectification method of a remote sensing satellite as described in the above embodiments.

[0016] A fourth aspect of the present application provides a computer readable storage medium storing computer instructions for causing a computer to execute the on-orbit automatic geometric rectification method of a remote sensing satellite as described in the above embodiments.

[0017] A fifth aspect of the present application provides a computer program product comprising a computer program, which, when executed, implements the on-orbit automatic geometric rectification method of a remote sensing satellite as described above.

[0018] The embodiments of the present application can establish virtual control points through high-precision satellite image and point cloud data registration based on high-precision laser point cloud data, design a satellite image geometric rectification model based on the virtual control points, complete high-precision on-orbit geometric rectification of a remote sensing satellite, and maximize the geometric quality of image products. Thus, the technical problem that in the related art, professional technical personnel are relied on for processing, the automation level is low, and it is difficult to adapt to the rapid growth of the number of remote sensing satellites, so that the high-precision observation capability cannot be fully utilized, is solved.

[0019] Additional aspects and advantages of the present application will be made apparent by the following description and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0021] Figure 1 A flowchart of an on-orbit automatic geometric rectification method of a remote sensing satellite according to an embodiment of the present application is shown in FIG. 1;

[0022] Figure 2 A flowchart of an on-orbit automatic geometric rectification method of a remote sensing satellite according to an embodiment of the present application is shown in FIG. 1;

[0023] Figure 3This is a schematic diagram of the structure of an on-orbit automatic geometric calibration device for remote sensing satellites according to an embodiment of the present invention;

[0024] Figure 4 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present invention. Detailed Implementation

[0025] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0026] The following description, with reference to the accompanying drawings, outlines an embodiment of the present invention of an automatic on-orbit geometric calibration method, apparatus, electronic device, and medium for remote sensing satellites. Addressing the technical problems mentioned in the background art, such as reliance on skilled technicians, low automation levels, and difficulty in adapting to the rapidly increasing number of remote sensing satellites, thus hindering the full utilization of their high-precision observation capabilities, the present invention provides an automatic on-orbit geometric calibration method for remote sensing satellites. This method utilizes high-precision laser point cloud data, registers high-precision satellite imagery with the point cloud data, establishes virtual control points, designs a satellite image geometric calibration model based on these virtual control points, and completes high-precision on-orbit geometric calibration of the remote sensing satellite, maximizing the geometric quality of image products. This solves the technical problems of related technologies, such as reliance on skilled technicians, low automation levels, and difficulty in adapting to the rapidly increasing number of remote sensing satellites, thus hindering the full utilization of their high-precision observation capabilities.

[0027] Specifically, Figure 1 This is a flowchart illustrating an on-orbit automatic geometric calibration method for remote sensing satellites provided in an embodiment of the present invention.

[0028] like Figure 1 As shown, the on-orbit automatic geometric calibration method for remote sensing satellites includes the following steps:

[0029] In step S101, the three-dimensional point coordinates of the building edge points are obtained from the original point cloud data set, and the three-dimensional point coordinates are projected onto the image space of the satellite image to obtain the two-dimensional point coordinates of the point cloud.

[0030] In actual implementation, the embodiments of the present invention can obtain high-precision point cloud data as raw point cloud data, and obtain high-precision point cloud geometric three-dimensional line features from the raw point cloud data set. The three-dimensional point coordinates of the building edge lines are obtained through building segmentation, plane extraction, and other methods, denoted as... Projecting it onto the image space of the satellite image yields two-dimensional point coordinates, denoted as... .

[0031] Optionally, in an embodiment of the present application, the three-dimensional point coordinates of the building edge points are obtained from the original point cloud data set, comprising: performing building extraction on the original point cloud data set to obtain a building point set; performing plane segmentation on the building point set to extract building edge points, and constructing a point cloud three-dimensional line feature point set using the building edge points; and obtaining the three-dimensional point coordinates of the building edge line from the point cloud three-dimensional line feature point set.

[0032] In the embodiment of the present application, the original point cloud data point set can be , the point set after building extraction is , plane segmentation is performed to obtain a point set of a building roof , and then the edge points of the building roof are extracted to obtain a point cloud three-dimensional line feature point set .

[0033] In step S102, the image two-dimensional point coordinates of the building in the image space are extracted from the original image, so as to perform registration using the point cloud two-dimensional point coordinates and the image two-dimensional point coordinates to obtain a plurality of sets of offset parameters.

[0034] Further, in the embodiment of the present application, the image line feature extraction method can be used to obtain the two-dimensional point coordinates of the building edge line in the image space, denoted as ; the satellite image is cropped into a plurality of image blocks ; (for example, the image block size is 512*512), and the point cloud two-dimensional line feature image block corresponding to the range is obtained , and the high-precision point cloud and image registration is normalized to calculate the offset parameters dx and dy between each pair of image block and point cloud image block .

[0035] Optionally, in an embodiment of the present application, the point cloud two-dimensional point coordinates and the image two-dimensional point coordinates are used to perform registration to obtain a plurality of sets of offset parameters, comprising: based on the image two-dimensional point coordinates, the original image is cropped into a plurality of image blocks; based on the point cloud two-dimensional point coordinates, the point cloud image blocks corresponding to the ranges of the plurality of image blocks are obtained; and the offset parameters between each pair of image block and point cloud image block are calculated to obtain a plurality of sets of offset parameters.

[0036] In the embodiment of the present application, the offset parameter solving between the image block and the point cloud image block based on the distance transformation model is as follows:

[0037] The distance transform model (DT) is a 2D distance field and registration can be completed without explicitly constructing correspondences. First, the image patches are processed according to the following formula... Establish a distance field map:

[0038] ,

[0039] This distance field diagram With image blocks The pixels are of uniform size, and each pixel represents the distance of that pixel from the image patch. The distance to the nearest line feature point in the image. Distance calculation methods include, but are not limited to, Euclidean distance, checkerboard distance, Chebyshev distance, Manhattan distance, geodetic distance, or chamfer distance. A pair of image patches. With point cloud image blocks The offset parameters dx and dy between them are denoted as According to the maximum likelihood theory, when the distance At its minimum, the image offset parameter Closest to the truth:

[0040] ,

[0041] Furthermore, in this embodiment of the invention, robust parameter solving is based on asymptotic iterative weighting:

[0042] Transform the above problem into matrix form:

[0043] ,

[0044] in, for The number of feature points along the midline It is a constant. Represented as:

[0045] ,

[0046] in, j Let be the number of points, and be the number of rows in the matrix. Solving the above equation requires first linearization, and starting from the given initial values. Start iterative solution (can be set to 0), the linearized expression is:

[0047] ,

[0048] ,

[0049] in, Represents the distance vector relative to the unknown. Jacobi matrix (hereinafter abbreviated as) ), denotes the change value of the iteration process, is a weight matrix, representing the weight value of the i-th point in the j-th iteration, which is calculated according to the following formula: t j

[0050]

[0051] wherein, denotes the residual, which is actually the distance value until the last iteration; is a progressive adjustment hyperparameter in the robust solving model; is a fixed hyperparameter in the robust solving model, reflecting the priori building boundary detection error. The hyperparameter determines the robustness of the model. When = 2, the solving lacks robustness. When < 2, the smaller the value is, the stronger the robustness of the solving model is.

[0052] In step S103, the registration parameter reliability evaluation is performed based on the multiple sets of offset parameters, to obtain reliable image blocks and corresponding offset parameters that meet a preset reliability condition.

[0053] As a possible implementation manner, the embodiment of the present application can perform registration parameter reliability evaluation based on the opposite inference theory based on the multiple sets of offset parameters obtained from the pairs of satellite image blocks and point cloud image blocks, to screen out reliable image blocks and corresponding offset parameters.

[0054] Optionally, in an embodiment of the present application, the registration parameter reliability evaluation is performed based on the multiple sets of offset parameters, to obtain reliable image blocks and corresponding offset parameters that meet a preset reliability condition, including: defining a virtual hypothesis that all building boundary points in the point cloud image block are random distributed meaningless points; obtaining corresponding basic statistics based on the multiple image blocks and the point cloud image block; calculating a probability value under the virtual hypothesis by using the basic statistics; judging whether the virtual hypothesis is true or not by using the probability value, to obtain a judgment result; performing the registration parameter reliability evaluation by using the judgment result, to screen out reliable image blocks and corresponding offset parameters that meet the preset reliability condition from the multiple sets of offset parameters.

[0055] Each pair of image blocks obtained in step S102 of the embodiment of the present application and the point cloud image block has offset parameters dx and dy , but it is unable to judge whether the set of offset parameters is reliable or not.

[0056] ​​​​​This invention can propose a criterion for determining contradictory reasoning, first by defining a "null hypothesis". ": All point cloud building boundary points are meaningless, i.e., points... They are independent and randomly distributed at any location within the image area. The basic statistics can include... ( (Total number of building boundary points), N (image patch) (total number of pixels) and wait.

[0057] Satisfying the "null hypothesis" The expected number of visual events is called the "NFA (Number of False Alarms)". If the NFA is less than 1, this embodiment of the invention can determine that the false hypothesis has been rejected. It is meaningful (i.e., in) (This is unlikely to happen). conditions, The probability density function is only the distance field plot The histogram is denoted as The embodiments of the present invention can be derived from 0 to... d of Accumulate and easily calculate the probability. .

[0058] However, due to Image space sampling rules, The points should be unblocked, and the offset transformation estimated by the DT model will not change the unblocking rule. Therefore, it means... Under the null hypothesis Below, distance field diagram superior At least one k There are 1 point (denoted as point number 1). The distance value is less than The probability should follow a hypergeometric distribution rather than a binomial distribution, that is:

[0059] ,

[0060] Where N is the total number of pixels in the image patch; yes upper distance The number of pixels.

[0061] Assume the number of possible visual events is Then "in the null hypothesis" Down, At least one k Each point in the distance field map the distance value on the image patch is less than the number of false alarms (NFA) for this event should be When the NFA is fairly small (e.g., less than 1), embodiments of the present application can determine that the occurrence of the event is significant enough to reject the false assumption Thus and the registration is reliable enough. Otherwise, embodiments of the present application determine that it is not reliable.

[0062] In the image patch registration task, embodiments of the present application should strive to find the accurate registration result. The registration accuracy is judged according to the most significant distance threshold , which is calculated as follows:

[0063] ,

[0064] where is the upper bound of the registration error (e.g., can be set to 20 pixels). Since is a constant, embodiments of the present application only need to find the minimum value of for a certain image patch (the second is and ). is the tail of a hypergeometric distribution, which has extremely high computational complexity, so a relaxation function, i.e.,

[0065] ,

[0066] where ; and . Thus, the most significant distance threshold is calculated by converting

[0067] ,

[0068] After the registration of and on each image patch is completed, the most significant distance threshold is calculated, and only the image patches with are kept (the threshold of the registration accuracy is , e.g., can be set to 5 pixels).

[0069] In step S104, the virtual control points are established by using the reliable image patches and the corresponding offset parameters, so as to solve the geometric calibration model of the satellite image by using the virtual control points, and obtain the model parameters of the geometric calibration model of the satellite image.

[0070] According to the satellite image block, the point cloud image block and the offset parameter, a virtual control point is established, and the satellite image geometric calibration model is solved based on the virtual control point.

[0071] Firstly, the embodiment of the present application can establish a virtual control point based on the above steps for each pair of satellite image block and point cloud image block The object coordinates of the control point are the average coordinates of all three-dimensional points in the point cloud image block , and the image coordinates are the image coordinates of the center of the satellite image block plus the offset parameter, that is:

[0072] ,

[0073] Among them, represents the projection of the ground point to the satellite image space, represents the offset parameter of the point cloud image block and the satellite image block. Thus, the virtual control point is obtained.

[0074] Optionally, in an embodiment of the present application, the satellite image geometric calibration model is solved by using the virtual control point, which comprises: correcting the satellite sensor element pointing angle coefficient by using the virtual control point to obtain an optimized coefficient vector; and solving the satellite image geometric calibration model by using the optimized coefficient vector.

[0075] Further, the geometric calibration model of the remote sensing satellite can be as follows:

[0076] Taking the element pointing angle model as an example, the basic model of the traditional remote sensing satellite calibration is:

[0077] ,

[0078] Among them, represents the coordinates of the ground point in the WGS84 coordinate system, represents the coordinates of the GNSS antenna phase center in the WGS84 coordinate system. represents the rotation matrix from the J2000 coordinate system to the WGS84 coordinate system. represents the rotation matrix from the star sensor coordinate system to the J2000 coordinate system. represents the alignment matrix between the satellite body coordinate system and the star sensor coordinate system. represents the registration matrix between the camera coordinate system and the satellite body coordinate system. represents the viewing angle of each CCD detector in each direction. Based on the virtual control point constructed by the point cloud image block and the image block The following satellite sensor element pointing angle coefficient correction can be completed:

[0079] ​

[0080] wherein, and are balance coefficients; is set as the ratio of focal length to CCD pixel size; is set as the minimum pixel pointing angle of the corresponding sub-line array CCD. Based on the image point coordinates of the above virtual control points, the following can be obtained:

[0081]

[0082] wherein, is the non-distorted image space coordinate; is the pixel pointing angle calculated by the basic model calibrated by the traditional remote sensing satellite; is the integer row number calculated according to the acquisition time.

[0083] The reliable image point-control point combination is selected from the satellite image block The condition is:

[0084]

[0085] wherein, c is the threshold value of selecting reliable pixel observation value, which can be set as 5 pixels.

[0086] The pixel pointing angle coefficient optimization model based on the DT registration is:

[0087]

[0088] wherein is the normalized pixel pointing angle model optimization coefficient.

[0089] On this basis, the virtual control points are further brought into the global calibration model, and all calibration parameters are updated.

[0090] In combination with Figure 2 and Figure 3 , the working principle of the on-orbit automatic geometric calibration method of the remote sensing satellite according to an embodiment of the present application is described in detail.

[0091] As shown in Figure 2 , the embodiment of the present application can include the following steps:

[0092] Step S201, three-dimensional line feature extraction. The embodiment of the present application can extract high-precision point cloud geometric three-dimensional line features from laser point cloud data. The original point cloud data point set is , the point set after building extraction is , plane segmentation is performed to obtain the point set of the building roof ​​​​Then, the edge points of the building's roof are extracted to obtain a set of 3D line feature points in the point cloud. Projecting this onto the image space of the satellite image yields two-dimensional point coordinates, denoted as... .

[0093] Step S202, Image Line Feature Extraction. Using the image line feature extraction method, the two-dimensional point coordinates of the building edge lines in the image space are obtained, denoted as... .

[0094] Step S203: Image and point cloud registration. The satellite image is cropped into multiple image patches. (e.g., using an image patch size of 512*512), simultaneously acquire the corresponding range of point cloud 2D line feature image patches. High-precision point cloud and image registration is reduced to calculating each pair of image patches. With point cloud image blocks offset parameters between dx and dy .

[0095] Based on the theory of oppositional reasoning, the reliability assessment of registration parameters is performed, and the registration accuracy is determined according to the most meaningful distance threshold. This is done on each image patch. and After registration, the most meaningful distance threshold will be calculated. Then keep only The image patch and its corresponding offset parameters.

[0096] Step S204: Solve the calibration model. Based on the above steps, for each pair of satellite image blocks... and point cloud image blocks A virtual control point can be established, and the object coordinates of this control point are the point cloud image patch. The average coordinates of all three-dimensional points The image coordinates are the image coordinates of the center of the satellite image block plus the offset parameter.

[0097] Based on point cloud image patches With image blocks The virtual control points constructed through registration can be used to correct the pointing angle coefficient of satellite sensor elements.

[0098] Image point coordinates based on virtual control points, from satellite image blocks The reliable image point-control point combination conditions are selected to optimize the probe pointing angle coefficient model based on DT registration, thus obtaining the satellite image geometric calibration model.

[0099] In step S205, the calibration model parameters are output. The geometric calibration model of the satellite image is solved to obtain the normalized probe pointing angle model optimization coefficient, i.e., the calibration model parameters.

[0100] In summary, the embodiment of the present application can automatically extract high-precision point cloud line features and satellite image line features, realize high-precision registration, and based on the virtual control points obtained by registering the satellite image and the point cloud, design a satellite probe pointing angle coefficient optimization model, i.e., a satellite image geometric calibration model, to complete the high-precision geometric calibration of the remote sensing satellite in orbit.

[0101] The remote sensing satellite in-orbit automatic geometric calibration method according to the embodiment of the present application can be based on high-precision laser point cloud data, register high-precision satellite images and point cloud data, establish virtual control points, design a satellite image geometric calibration model based on the virtual control points, complete the high-precision in-orbit geometric calibration of the remote sensing satellite, and maximize the geometric quality of the image product. Thus, the technical problem that in the related art, the processing relies on professional technicians, the automation level is low, and it is difficult to adapt to the rapid growth of the number of remote sensing satellites, so that the high-precision observation capability cannot be fully utilized, is solved.

[0102] Secondly, the remote sensing satellite in-orbit automatic geometric calibration device according to the embodiment of the present application is described with reference to the accompanying drawings.

[0103] Figure 3 is a block schematic diagram of the remote sensing satellite in-orbit automatic geometric calibration device according to the embodiment of the present application.

[0104] As shown in Figure 3 , the remote sensing satellite in-orbit automatic geometric calibration device 10 comprises an acquisition module 100, a registration module 200, an evaluation module 300, and a calculation module 400.

[0105] Specifically, the acquisition module 100 is configured to acquire three-dimensional point coordinates of building edge points from an original point cloud data set, and project the three-dimensional point coordinates to an image space of a satellite image to obtain two-dimensional point coordinates of the point cloud.

[0106] The registration module 200 is configured to extract two-dimensional point coordinates of the building in the image space from an original image, and perform registration using the two-dimensional point coordinates of the point cloud and the two-dimensional point coordinates of the image to obtain a plurality of sets of offset parameters.

[0107] The evaluation module 300 is configured to perform registration parameter reliability evaluation based on the plurality of sets of offset parameters to obtain reliable image blocks and corresponding offset parameters that satisfy a preset reliability condition.

[0108] The computing module 400 is configured to establish a virtual control point by using the reliable image block and the corresponding offset parameter, and solve a satellite image geometric rectification model by using the virtual control point to obtain model parameters of the satellite image geometric rectification model.

[0109] Optionally, in an embodiment of the present application, the acquisition module 100 comprises a first extraction unit, a construction unit and a second extraction unit.

[0110] The first extraction unit is configured to perform building extraction on the original point cloud data set to obtain a building point set.

[0111] The construction unit is configured to perform plane segmentation on the building point set to extract building edge points, and construct a point cloud three-dimensional line feature point set by using the building edge points.

[0112] The second extraction unit is configured to acquire three-dimensional point coordinates of a building edge line from the point cloud three-dimensional line feature point set.

[0113] Optionally, in an embodiment of the present application, the registration module 200 comprises a clipping unit, a first acquisition unit and a first calculation unit.

[0114] The clipping unit is configured to clip the original image into a plurality of image blocks based on two-dimensional point coordinates of the image.

[0115] The first acquisition unit is configured to acquire point cloud image blocks corresponding to the ranges of the plurality of image blocks based on two-dimensional point coordinates of the point cloud.

[0116] The first calculation unit is configured to calculate offset parameters between each pair of image blocks and point cloud image blocks to obtain a plurality of groups of offset parameters.

[0117] Optionally, in an embodiment of the present application, the evaluation module 300 comprises a definition unit, a second acquisition unit, a second calculation unit, a judgment unit and an evaluation unit.

[0118] The definition unit is configured to define a virtual hypothesis that all building boundary points in the point cloud image block are random distribution meaningless points.

[0119] The second acquisition unit is configured to obtain corresponding basic statistics based on the plurality of image blocks and the point cloud image blocks.

[0120] The second calculation unit is configured to calculate a probability value under the virtual hypothesis by using the basic statistics.

[0121] The judgment unit is configured to judge whether the virtual hypothesis is established by using the probability value to obtain a judgment result.

[0122] The evaluation unit is configured to perform registration parameter reliability evaluation by using the judgment result, so as to screen reliable image blocks and corresponding offset parameters that meet a preset reliability condition from the plurality of sets of offset parameters.

[0123] Optionally, in an embodiment of the present application, the calculation module 400 comprises a correction unit and a third calculation unit.

[0124] The correction unit is configured to correct the satellite sensor probe pointing angle coefficient by using the virtual control point, so as to obtain an optimized coefficient vector.

[0125] The third calculation unit is configured to solve the satellite image geometric rectification model by using the optimized coefficient vector.

[0126] It should be noted that the above description of the remote sensing satellite on-orbit automatic geometric rectification method is also applicable to the remote sensing satellite on-orbit automatic geometric rectification device, which will not be described here.

[0127] The remote sensing satellite on-orbit automatic geometric rectification device provided by the embodiment of the present application can establish a virtual control point based on high-precision laser point cloud data by high-precision satellite image and point cloud data registration, design a satellite image geometric rectification model based on the virtual control point, complete high-precision on-orbit geometric rectification of the remote sensing satellite, and maximize the geometric quality of the image product. Thus, the technical problem that in the related art, professional technical personnel are relied on for processing, the automation level is low, and it is difficult to adapt to the rapid growth of the number of remote sensing satellites, so that the high-precision observation capability cannot be fully utilized is solved.

[0128] Figure 4 The electronic device provided by the embodiment of the present application is shown in the structure diagram. The electronic device can include:

[0129] The memory 401, the processor 402, and the computer program stored in the memory 401 and executable on the processor 402.

[0130] The processor 402 implements the remote sensing satellite on-orbit automatic geometric rectification method provided in the above embodiments when executing the program.

[0131] Further, the electronic device further includes:

[0132] The communication interface 403 is configured to communicate between the memory 401 and the processor 402.

[0133] The memory 401 is configured to store the computer program executable on the processor 402.

[0134] The memory 401 can include a high-speed RAM memory and can also include a non-volatile memory, such as at least one disk memory.

[0135] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the communication interface 403, the memory 401 and the processor 402 can be connected to each other through a bus and complete communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 4 Only one thick line is used in the figure to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0136] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can complete communication between each other through an internal interface.

[0137] The processor 402 can be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present application.

[0138] The embodiment also provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the above-mentioned on-orbit automatic geometric calibration method of a remote sensing satellite.

[0139] The embodiment of the present application also provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the on-orbit automatic geometric calibration method of a remote sensing satellite provided by the embodiment of the present application.

[0140] In the description of the application, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the application. The illustrative description of the above terms in the specification does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or N embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples, without contradiction.

[0141] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the application, the meaning of "N" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0142] Any process or method descriptions in flow charts or described elsewhere herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for performing specific logic functions or steps in the process, and the various embodiments of the application include additional implementations in which the order of steps can differ from those shown or discussed, including a step can occur at

[0143] The logic and / or steps represented in flow diagrams or otherwise described herein, for example, can be considered as a sequence of executable instructions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a product of the manufacturing and / or processing. The computer-readable medium can include, but is not limited to, the following: an electronic connection (an electronic device with one or N wires), a portable computer diskette (a magnetic device), a RAM (random access memory), a ROM (read-only memory), an EPROM (erasable programmable ROM) or a Flash memory, an optical fiber, and a portable CD ROM. In addition, the computer-readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via the optically scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and stored in the computer memory.

[0144] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As such, if implemented in hardware and in another embodiment, any of the following technologies, known in the art, or their combinations can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0145] Those of ordinary skill in the art can understand that all or part of the steps carried out by the above-mentioned embodiment methods can be completed by programs instructing relevant hardware, and the programs can be stored in a computer-readable storage medium. When the programs are executed, one or a combination of the steps of the method embodiments is included.

[0146] In addition, each function unit in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module. When the integrated module is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0147] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A method for automatic on-orbit geometric calibration of remote sensing satellites, characterized in that, Includes the following steps: The three-dimensional point coordinates of building edge points are obtained from the original point cloud data set, and the three-dimensional point coordinates are projected onto the image space of the satellite image to obtain the two-dimensional point coordinates of the point cloud; The image two-dimensional point coordinates of the building in the image space are extracted from the original image, and the two-dimensional point coordinates of the point cloud and the image are registered to obtain multiple sets of offset parameters. Based on the multiple sets of offset parameters, the reliability of the registration parameters is evaluated to obtain reliable image blocks and corresponding offset parameters that meet the preset reliability conditions. Virtual control points are established using the reliable image blocks and corresponding offset parameters. These virtual control points are then used to solve the satellite image geometric calibration model, thereby obtaining the model parameters of the satellite image geometric calibration model.

2. The method according to claim 1, characterized in that, The step of obtaining the three-dimensional point coordinates of the building edge points from the original point cloud data set includes: Buildings are extracted from the original point cloud dataset to obtain a set of building points; The set of building points is segmented into planes to extract building edge points, and the building edge points are used to construct a set of three-dimensional line feature points for the point cloud; The three-dimensional point coordinates of the building edge line are obtained from the set of feature points of the three-dimensional line of the point cloud.

3. The method according to claim 1, characterized in that, The registration process using the two-dimensional point coordinates of the point cloud and the two-dimensional point coordinates of the image yields multiple sets of offset parameters, including: Based on the two-dimensional point coordinates of the image, the original image is cropped into multiple image blocks; Based on the two-dimensional point coordinates of the point cloud, obtain point cloud image blocks within the range corresponding to the multiple image blocks; Calculate the offset parameters between each pair of image blocks and the point cloud image blocks to obtain the multiple sets of offset parameters.

4. The method according to claim 3, characterized in that, The process of performing registration parameter reliability evaluation based on the multiple sets of offset parameters to obtain reliable image blocks and corresponding offset parameters that meet preset reliability conditions includes: The null hypothesis is defined as all building boundary points in the point cloud image block being random, meaningless points. Based on the multiple image patches and the point cloud image patches, the corresponding basic statistics are obtained; Calculate the probability value under the null hypothesis using the basic statistics; The probability value is used to determine whether the null hypothesis is true, and the determination result is obtained. The determination result is used to evaluate the reliability of the registration parameters, so as to select reliable image blocks and corresponding offset parameters that meet the preset reliability conditions from the multiple sets of offset parameters.

5. The method according to claim 1, characterized in that, The process of using the virtual control points to solve for the geometric calibration model of satellite imagery includes: The virtual control point is used to correct the pointing angle coefficient of the satellite sensor element, resulting in an optimized coefficient vector. The optimized coefficient vector solution is used to obtain the geometric calibration model of the satellite image.

6. An on-orbit automatic geometric calibration device for remote sensing satellites, characterized in that, include: The acquisition module is used to obtain the three-dimensional point coordinates of building edge points from the original point cloud data set, and project the three-dimensional point coordinates onto the image space of the satellite image to obtain the two-dimensional point coordinates of the point cloud; The registration module is used to extract the two-dimensional point coordinates of buildings in the image space from the original image, and to register the two-dimensional point coordinates of the point cloud and the two-dimensional point coordinates of the image to obtain multiple sets of offset parameters. The evaluation module is used to evaluate the reliability of the registration parameters based on the multiple sets of offset parameters, so as to obtain reliable image blocks and corresponding offset parameters that meet the preset reliability conditions. The calculation module is used to establish virtual control points using the reliable image blocks and corresponding offset parameters, and to use the virtual control points to solve the satellite image geometric calibration model, thereby obtaining the model parameters of the satellite image geometric calibration model.

7. The apparatus according to claim 6, characterized in that, The acquisition module includes: The first extraction unit is used to extract buildings from the original point cloud data set to obtain a set of building points. The construction unit is used to perform planar segmentation on the set of building points to extract building edge points, and to construct a set of three-dimensional line feature points for the point cloud using the building edge points; The second extraction unit is used to obtain the three-dimensional point coordinates of the building edge line from the set of feature points of the three-dimensional line of the point cloud.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the on-orbit automatic geometric calibration method for remote sensing satellites as described in any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the on-orbit automatic geometric calibration method for remote sensing satellites as described in any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed, it is used to implement the on-orbit automatic geometric calibration method for remote sensing satellites as described in any one of claims 1-5.

Images