A multi-modal aerospace remote sensing collaborative mapping regional network refinement method

Abstract

The present invention provides a method for refining a multi-modal aerospace remote sensing collaborative mapping regional network, comprising the following steps: constructing a rigorous positioning model for aerial optical and SAR images in the geocentric rectangular coordinate system; extracting rational polynomial model parameters RPC of satellite optical and SAR images in the geocentric rectangular coordinate system; constructing a positioning model and a distance equation for satellite laser altimetry remote sensing targets; extracting the position of the spot energy center on the spot image and enhancing the clarity of the spot image; realizing the matching of homologous points of multi-modal remote sensing targets through image gradient feature matching, extracting homologous points between optical images through sift features or least squares matching, and realizing the matching between laser altimetry data and remote sensing images through the extraction of the spot energy center position and template matching; using SAR simulated image matching technology to extract homologous points between SAR and Lidar data; constructing a joint adjustment model for the multi-modal remote sensing regional network; and overall solving the refined parameters and solving the coordinates of encrypted points.

Description

Technical Field

[0001] The present invention relates to a method for processing surveying and mapping remote sensing data, and more particularly to a method for refining a multi-modal aerospace remote sensing collaborative surveying and mapping regional network. Background Art

[0002] Remote sensing surveying and mapping sensors are key devices for obtaining information on the Earth's surface. According to their working principles and application scopes, they can be classified into various types:

[0003] (1) Optical sensors: Optical sensors are sensors that use the energy of electromagnetic wave bands such as visible light, infrared light, and ultraviolet light to obtain information on the Earth's surface. Common optical sensors include cameras and imaging spectrometers carried on spacecraft. They can obtain information such as the color, texture, and morphology of the ground surface and are widely used in fields such as land cover classification, environmental monitoring, and urban planning. An optical panchromatic sensor is a sensor that can obtain high-resolution monochromatic images in the entire visible spectral range. Compared with multi-spectral sensors, optical panchromatic sensors only collect data within a single wavelength range, usually in the blue to near-infrared band (0.4 - 0.9 microns) of the visible spectrum.

[0004] (2) Synthetic Aperture Radar (SAR) sensors: SAR sensors use radar technology to obtain microwave reflection signals from the ground surface and have the ability to be independent of weather and lighting conditions. Therefore, they are suitable for surface monitoring and mapping in various environments. SAR sensors can obtain high-resolution images of the ground surface and provide ground object features such as altitude information and vegetation structure. Synthetic aperture radar generates high-resolution radar images by receiving radar signals reflected from the ground and using synthetic aperture technology. Compared with optical remote sensing, SAR remote sensing has the advantage of being unaffected by weather conditions such as clouds, rain, and fog and is suitable for monitoring and mapping of land surfaces, ocean surfaces, and ice and snow regions.

[0005] (3) LiDAR sensors: LiDAR sensors utilize the principle of laser beam emission and reception to measure the distance and elevation information of surface objects. LiDAR sensors feature high precision and high resolution, and can be used for the generation of digital elevation models (DEMs) and digital terrain models (DTMs), as well as applications such as topographic and geomorphic analysis and urban planning. The satellite laser altimetry method uses a laser radar system carried by a satellite to measure the elevation information of the earth's surface and terrain. By emitting laser pulses towards the ground and measuring the time required for the laser pulses to travel from emission to reception, the elevation information of surface points can be calculated. Laser altimetry data is characterized by high precision and high vertical resolution, and can be used for the construction of digital terrain models (DTMs) and digital surface models (DSMs), as well as the extraction of topographic features. Currently, multiple countries and regions have launched satellites equipped with laser altimetry systems, which provide high-resolution surface elevation data on a global scale, providing important data support for earth science research and applications.

[0006] According to the differences in the sensor carrier platforms, surveying and mapping remote sensing can be mainly divided into ground surveying and mapping remote sensing, aerial surveying and mapping remote sensing, and space surveying and mapping remote sensing. For example, the aerial aircraft platform using aerial surveying and mapping remote sensing and the space satellite platform using space surveying and mapping remote sensing are the most common remote sensing platforms for surveying and mapping remote sensing.

[0007] (1) Aerial surveying and mapping remote sensing uses remote sensors carried by aircraft or other aircraft to obtain remote sensing data of the earth's surface by flying within the atmosphere. Due to the relatively close distance of the aircraft to the ground, aerial surveying and mapping remote sensing can obtain remote sensing images with higher resolution, usually higher than satellite remote sensing data. Aerial surveying and mapping remote sensing is widely used in fields such as urban planning, land use monitoring, environmental protection, and resource exploration. Due to its high-resolution characteristics, aerial surveying and mapping remote sensing is particularly suitable for application scenarios with high requirements for small areas and details.

[0008] (2) Space surveying and mapping remote sensing uses remote sensors carried by satellites or other spacecraft to obtain remote sensing data of the earth's surface by orbiting the earth in space. The resolution of space surveying and mapping remote sensing is usually lower than that of aerial surveying and mapping remote sensing, but its coverage is wider, enabling global-scale earth observation. Space surveying and mapping remote sensing is widely used in global-scale application fields such as meteorological monitoring, environmental change monitoring, and disaster monitoring and assessment. Due to its advantages of wide coverage and continuous observation, space surveying and mapping remote sensing plays an important role in global environmental monitoring and climate change research.

[0009] Aerial surveying and mapping remote sensing and space surveying and mapping remote sensing are respectively applied to earth observation and remote sensing applications in different ranges and fields with different data acquisition platforms and resolution characteristics. Aerial surveying and mapping remote sensing is suitable for application scenarios with high resolution requirements for small areas, while space surveying and mapping remote sensing is suitable for large-scale global environmental monitoring and resource management.

[0010] Joint remote sensing mapping utilizes data from multiple sensors to complement and verify each other, aiming to improve the accuracy and reliability of remote sensing information extraction. By integrating multi-source data, the deficiencies of a single sensor in aspects such as spatial resolution, remote sensing information acquisition ability, and data acquisition cycle can be made up for, thereby enabling a more comprehensive and detailed description and analysis of surface features. Joint remote sensing mapping technology has broad application prospects in fields such as geological exploration, environmental monitoring, and disaster assessment. Further, in remote sensing mapping, achieving high-precision positioning is crucial, especially in cases where control points are scarce or non-existent. The control points are usually landmarks or features with known positions, used for map registration and image correction, so as to improve the geographical positioning accuracy of remote sensing data. However, in some remote and inaccessible areas or emerging development regions, control points may be extremely scarce or non-existent at all. In such cases, the use of multi-modal joint remote sensing technology can make up for this deficiency. Multi-modal joint remote sensing technology can achieve high-precision positioning in the absence or scarcity of control points. By means of data fusion, cross-positioning, etc., it can improve the geographical positioning accuracy of remote sensing data and provide reliable geographical information support for the field of remote sensing mapping applications.

[0011] In spaceborne mapping remote sensing, issues such as orbit refinement, attitude refinement, and sensor parameter refinement also need to be considered. The orbit refinement refers to further correcting and optimizing the satellite orbit parameters to improve the orbit positioning accuracy, which includes methods such as improving the orbit prediction model, using ground measurement data for orbit correction, and adopting precise satellite dynamics modeling. Through orbit refinement, the positioning error of the satellite in orbit can be reduced, thereby improving the geographical positioning accuracy of remote sensing images.

[0012] The attitude refinement refers to accurately estimating and correcting the attitude parameters of the satellite in space to ensure the accuracy of the sensor observation direction, which includes methods such as using on-board attitude sensor data for attitude solution, calibrating the parameters of the attitude control system, and adopting inter-satellite measurement technology for attitude correction. Through attitude refinement, the influence of attitude errors on the geometric positioning of remote sensing images can be reduced, improving the geographical positioning accuracy of the images.

[0013] The sensor parameter refinement refers to accurately calibrating and estimating the internal and external parameters of the remote sensing sensor to improve the accuracy and consistency of the sensor observation data, which includes methods such as accurately calibrating the camera internal parameters, external parameters, distortion parameters, etc. of the sensor, using a calibration board or special scene for sensor calibration, and adopting image registration technology for sensor external calibration. Through sensor parameter refinement, the systematic errors of the sensor can be eliminated, improving the geometric positioning accuracy of remote sensing images.

[0014] Multimodal remote sensing makes full use of the advantages of high-precision angular measurement of optical images, high-precision distance measurement of SAR and lidar, constructs a robust space-air ranging and angle-measuring triangulation network through the relationship between multi-source remote sensing, and improves the accuracy of remote sensing orbit, attitude and sensor parameters through refinement methods to achieve high-precision positioning of targets. Summary of the Invention

[0015] The present invention provides a refinement method for a multimodal space-air remote sensing collaborative mapping regional network, which solves the problem of realizing multimodal space-air remote sensing collaboration by using high-precision angular measurement of optical images, high-precision distance measurement of SAR and lidar, and the problem of improving the accuracy of orbit refinement, attitude refinement and sensor parameter refinement of spaceborne mapping remote sensing. The technical solution is as follows:

[0016] A refinement method for a multimodal space-air remote sensing collaborative mapping regional network includes the following steps:

[0017] S1; Construct a rigorous positioning model of airborne optical and SAR images in the geocentric rectangular coordinate system, including the collinearity equation of airborne optical images and the distance-coplanarity model of airborne SAR images;

[0018] S2; According to the satellite optical RFM geometric model and the satellite SAR RFM geometric model, extract the rational polynomial model parameters RPC of satellite optical and SAR images in the geocentric rectangular coordinate system;

[0019] S3: Construct a positioning model and a distance equation of satellite laser altimetry remote sensing targets in the geocentric rectangular coordinate system, including a laser altimetry positioning model and a laser altimetry distance equation;

[0020] S4: For the spot image of satellite laser, extract the position of the spot energy center on the spot image by a fitting method, and increase the clarity of the spot image by a spot suppression and image enhancement method;

[0021] S5: Achieve the matching of homologous points of multimodal remote sensing targets through image gradient feature matching, extract homologous points between optical images through sift features or least squares matching, and achieve the matching between laser altimetry data and remote sensing images through the extraction of the spot energy center position and template matching;

[0022] S6: Based on Lidar DSM data, indirectly achieve the matching and extraction of homologous points between Lidar and SAR images through SAR simulated image matching, or through template matching between Lidar DSM and InSAR DSM in the case of having InSAR DSM;

[0023] S7: Under the benchmark of a unified geocentric rectangular coordinate system, construct a joint adjustment model for the multi-modal remote sensing regional network. The joint adjustment model for the multi-modal remote sensing control network consists of an error equation system formed by multi-modal remote sensing data models;

[0024] S8: By means of error equation normal equation formation and ground point coordinate transformation, reduce the number of unknowns and solve for the refined parameters and encrypted point coordinates as a whole.

[0025] Furthermore, in step S1, under the geocentric rectangular coordinate system, the rigorous model of the aerial optical image is based on POS and attitude data, and through the transformation matrix between the tangent plane space rectangular coordinate system and the geocentric rectangular coordinate system, the collinearity equation of the aerial optical image under the geocentric rectangular coordinate system is constructed; under the geocentric rectangular coordinate system, the rigorous model of the aerial SAR image is a distance-coplanarity model constructed for the aerial SAR image based on the position, velocity, and attitude information of the SAR sensor in the terrain rectangular coordinate system.

[0026] Furthermore, in step S2, according to the satellite optical RFM geometric model and the satellite SAR RFM geometric model, extract the rational polynomial model parameters RPC of the satellite optical and SAR images under the geocentric rectangular coordinate system, including the following steps:

[0027] S11: The acquisition of the RPC of the satellite optical image in the geocentric rectangular coordinate system is constructed based on the geometric model of the pixel line-of-sight vector and the ellipsoid model of the earth with elevation parameters when the optical satellite image provides physical model parameters;

[0028] S12: The acquisition of the RPC of the satellite SAR image in the geocentric rectangular coordinate system is constructed based on the geometric model of range-Doppler (R-D) and the ellipsoid model of the earth with elevation parameters when the satellite SAR image provides physical model parameters;

[0029] S13: When the satellite optical image and the satellite SAR image provide the RPC in the geographic coordinate system, it is constructed based on the RFM geometric model in the geographic coordinate system and the ellipsoid model of the earth with elevation parameters;

[0030] S14: Divide grids in the image layer and elevation intervals in the elevation layer, use the above geometric model and the ellipsoid model with elevation parameters to calculate the spatial positioning coordinates of the image points of each grid point on different elevation layers, form virtual control points under the geocentric rectangular coordinate system, and use these virtual control points to fit and form the RPC parameters for the RFM geometric positioning of the satellite image under the geocentric rectangular coordinate system.

[0031] Further, in step S3, the laser altimetry positioning model is a positioning model for satellite laser altimetry remote sensing targets in the geocentric rectangular coordinate system. By means of the position and attitude of the sensor, the line-of-sight vector of the laser beam is obtained, and combined with the laser ranging length after atmospheric and tidal corrections, a laser altimetry remote sensing target positioning model in the geocentric rectangular coordinate system is constructed; the distance model is constructed based on the position coordinates of the ground point and the sensor position coordinates in the geocentric rectangular coordinate system.

[0032] Further, in step S5, the homologous point matching of multi-modal remote sensing targets is realized through image gradient feature matching, the extraction of homologous points between optical images is realized through sift features or least squares matching, and the matching between laser altimetry data and remote sensing images is realized through the extraction of the central position of the spot energy and template matching, including the following processes:

[0033] (1) The extraction of homologous points between optical images through sift features or least squares matching means that when there are only optical images in the overlapping area, the least squares or sift matching method can be used to realize the matching between multi-source optical images, and then the extraction of image homologous points;

[0034] (2) The homologous point matching of multi-modal remote sensing targets through image gradient feature matching means that when the overlapping area includes SAR images, the SAR image matching can be realized through the normalized local direction gradient histogram HOG features, as well as the multi-modal image matching including SAR images;

[0035] (3) The matching between laser altimetry data and remote sensing images through the extraction of the central position of the spot energy and template matching means that through the position of the spot energy center on the spot image, the position of the spot energy center on the remote sensing image is obtained, and combined with the mutual relationship obtained by the matching between the spot image and the remote sensing image, the matching between laser altimetry data and remote sensing images is realized, and the extraction of homologous points between laser altimetry data and remote sensing images is realized.

[0036] Further, in step S6, the simulated image matching refers to using the Lidar DSM data, simulating the imaging of this DSM data with the same imaging parameters as the SAR image, obtaining the relationship between the simulated SAR image and the real SAR image, then matching the simulated SAR and the real SAR image, and according to the geometric relationship between the simulated SAR image points and the DSM points, inverting the position of the simulated image homologous points on the Lidar DSM, realizing the construction of the homologous point relationship between the Lidar data and the SAR image, so as to obtain the homologous points between the SAR image and the Lidar DSM.

[0037] Further, in step S7, the adjustment model reconstructs different remote sensing geometric models in the geocentric rectangular coordinate system. For different sensor images, the error equations are established based on different multi-modal remote sensing data models:

[0038] For satellite optical and SAR remote sensing images, the error equation is constructed using the RFM model in the geocentric rectangular coordinate system;

[0039] For aerial optical images, the error equation is constructed using the collinear equation model in the geocentric rectangular coordinate system;

[0040] For aerial SAR images, the error equation is constructed using the distance coplanarity equation and the orbit attitude refinement model in the geocentric rectangular coordinate system;

[0041] For satellite laser altimetry data, the error equation is constructed using the laser point positioning model or the distance model in the geocentric rectangular coordinate system;

[0042] For airborne Lidar data, it is directly used as control data in the space-air-ground network;

[0043] For ground control points, they are converted to the geocentric rectangular coordinate system and used as known values:

[0044] For the model positioning of the RFM of optical and SAR images, a refinement model of image-plane affine transformation is adopted; for the sensor position and attitude refinement models of the rigorous models of images and laser altimetry data, a quadratic polynomial model of time is adopted.

[0045] Further, the sensor position and attitude refinement model is as follows:

[0046]

[0047] where ([[]] a i , b i , c i , e i , f i , g i )( i = 0, 1, 2) are the polynomial model coefficients of the systematic errors of the sensor position and attitude, t is the time parameter, (X 0 , Y 0 , Z 0 ) is the initial value of the sensor position, is the initial value of the sensor attitude, (X S , Y S , Z S ) is the refined value of the sensor position, is the refined value of the sensor attitude.

[0048] Furthermore, in step S8, before the overall solution of the multi-modal remote sensing refinement parameters, the error equations constructed by multi-modal remote sensing are legalized and the ground point coordinates are modified. Before the solution, the ground point coordinate unknowns are eliminated with the multi-modal remote sensing error equation set of the same homologous points as the unit. After obtaining the correction values of each remote sensing orientation parameter by solution, the ground point coordinates are obtained by intersection calculation to obtain the encrypted point information.

[0049] The multi-modal space-air remote sensing collaborative mapping regional network refinement method is suitable for the participation and processing of all or part of the above-mentioned types of remote sensing data such as satellite optical, satellite SAR, satellite laser altimetry, aerial optical, aerial SAR, and aerial Lidar.

[0050] The multi-modal space-air remote sensing collaborative mapping regional network refinement method provided by the present invention takes into account the main mapping remote sensing data of aerospace. In practical applications, not all the mentioned remote sensing data must participate simultaneously. Any combination of the mentioned different types of remote sensing data can be collaboratively processed within the framework and method designed by the present invention, providing an effective way for the collaborative mapping processing of multi-modal space-air remote sensing data. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is the overall process schematic diagram of the multi-modal space-air remote sensing collaborative mapping regional network refinement method;

[0052] Figure 2 is the working schematic diagram of the multi-modal space-air remote sensing collaborative mapping regional network refinement method. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] As Figure 1 and Figure 2 shown, the multi-modal space-air remote sensing collaborative mapping regional network refinement method includes the following steps:

[0054] Step S1: Construct a rigorous positioning model of aerial optical and SAR images in the geocentric rectangular coordinate system, including the collinearity equation of the aerial optical image and the distance-coplanarity model of the aerial SAR image;

[0055] In the geocentric rectangular coordinate system, the rigorous model of the aerial optical image is based on the POS and attitude data, and the collinearity equation of the aerial optical image in the geocentric rectangular coordinate system is constructed through the transformation matrix between the tangent plane space rectangular coordinate system and the geocentric rectangular coordinate system. In the geocentric rectangular coordinate system, the rigorous model of the aerial SAR image is for the aerial SAR image, and based on the position, velocity, and attitude information of the SAR sensor in the terrain rectangular coordinate system, the distance-coplanarity model of the aerial SAR image is constructed.

[0056] Specifically, for aerial optical images, the tangent plane space rectangular coordinate system refers to the coordinate system where the attitude parameters are obtained for the POS attitude. According to the transformation matrix between the tangent plane space rectangular coordinate system and the geocentric rectangular coordinate system R p E , the collinearity equation of the aerial optical image in the geocentric rectangular coordinate system can be constructed. Further, in the geocentric rectangular coordinate system, the line-of-sight vector of the CCD pixel of the satellite sensor is 。 Then, in the geocentric rectangular coordinate system:

[0057] Let \(R = R c E R' c E = m ij (i,j = 0,1,2), \(m ij is the matrix element, then there is the collinearity equation of the aerial optical image in the geocentric rectangular coordinate system (ECEF):

[0058]

[0059] In formula (1), \((X S ,Y S ,Z S ) is the refined value of the sensor position, abbreviated as the sensor position later, \((X,Y,Z)\) is the position of the ground target corresponding to the image point, \((x 0 ,y 0 ) is the coordinate of the principal point of the image, \((x,y)\) is the coordinate of the image point, f is the focal length.

[0060] For aerial SAR images, then in the geocentric rectangular coordinate system, a distance coplanarity model is constructed:

[0061]

[0062] In formula (2), the parameter \(R 0 is the initial slant range corresponding to the first column image coordinate of the aerial SAR image, \(M r is the image slant range resolution of the aerial SAR image, \(M a is the image azimuth resolution of the aerial SAR image. \((X S ,Y S ,Z S ) is the sensor position, \((X,Y,Z)\) is the position of the ground target corresponding to the image point. f D is the Doppler parameter, \([V Xs ,V Ys ,VZs T is the speed of the sensor, are attitude parameters in the geocentric rectangular coordinate system respectively, is the wavelength.

[0063] Step S2: According to the satellite optical RFM geometric model and the satellite SAR RFM geometric model, extract the rational polynomial model coefficients RPC of the satellite optical and SAR images in the geocentric rectangular coordinate system;

[0064] The geometric processing parameters attached to satellite remote sensing images usually include two categories. One is the satellite ephemeris attitude and sensor imaging parameters, including the physical model parameters provided by satellite optical images and the physical model parameters provided by satellite SAR images. The other is the RPC parameters in the geographic coordinate system. For the differences in the two types of sensor imaging parameters and sensor categories, the present invention classifies and gives the geometric positioning basic models and technical methods for extracting the rational polynomial (RFM) model parameters RPC of satellite optical and SAR images in the geocentric rectangular coordinate system, including the following steps:

[0065] S11: The acquisition of the RPC of the satellite optical image in the geocentric rectangular coordinate system is based on the geometric model of the line-of-sight vector of the CCD pixels of the sensor and the ellipsoid model of the earth with elevation parameters when the optical satellite image provides physical model parameters. Among them,

[0066] The geometric model of the pixel line-of-sight vector is as follows:

[0067]

[0068] In formula (3), [X, Y, Z] is the ground target position corresponding to the image point, [Xs, Ys, Zs] is the sensor position, The three parameters are respectively the transformation matrix from the orbital coordinate system to the geocentric rectangular coordinate system, the transformation matrix from the satellite's body coordinate system to the orbital coordinate system, and the transformation matrix from the sensor coordinate system to the satellite's body coordinate system, is the scale factor, (x, y) is the image point coordinate, f is the focal length.

[0069] The ellipsoid model of the earth with elevation parameters is as follows:

[0070]

[0071] In formula (4), H is the ground point elevation parameter, a , b are the major and minor axes of the earth ellipsoid.

[0072] Combining the above two formulas, the coordinates (X, Y, Z) of the ground point corresponding to the image point in the geocentric rectangular coordinate system can be obtained.​

[0073] S12: Obtaining the RPC of the satellite SAR image in the geocentric rectangular coordinate system is based on the range-Doppler (R-D) geometric model and the ellipsoid model of the earth with elevation parameters when the satellite SAR image provides the physical model parameters. Among them, the ellipsoid model of the earth with elevation parameters is the same as above.

[0074] The form of the R-D model is as follows:

[0075]

[0076] In formula (5), the first equation represents the range equation. The left side is the range extracted from the image information, and the parameter R 0 is the initial slant range corresponding to the first column image coordinate of the airborne SAR image, and M r is the image slant range resolution. The right side is the spatial distance between the sensor position (X S , Y S , Z S ) and the ground target position (X, Y, Z) corresponding to the image point.

[0077] The second equation represents the Doppler equation. The left side is the Doppler parameter used for imaging f D , and the right side is the Doppler value calculated according to the relationship between the velocity of the sensor [V Xs , V Ys , V Zs T , the relative position between the sensor and the target [X - X S , Y - Y S , Z - Z S T . Here, is the wavelength, and R represents the slant range.

[0078] Combining formula (5) with the ellipsoid model of the earth with elevation parameters can obtain the coordinates of the ground point corresponding to the image point in the geocentric rectangular coordinate system, that is, the ground target position (X, Y, Z) corresponding to the image point.

[0079] S13: When the satellite optical image and the satellite SAR image provide the RPC in the geographic coordinate system, it is constructed based on the RFM geometric model in the geographic coordinate system and the ellipsoid model of the earth with elevation parameters. The form of the RFM model is as follows:

[0080] When the elevation H is known, the longitude, latitude, and elevation coordinates (L, B, H) of the ground point corresponding to the image point in the geographic coordinate system can be calculated, and the geographic coordinates can be converted to the geocentric rectangular coordinate system to obtain the ground target position (X, Y, Z) corresponding to the image point. Among them, L represents longitude and B represents latitude.

[0081] S14: Divide grids in the image layer and elevation intervals in the elevation layer, and use the above geometric model and the ellipsoid model with elevation parameters to calculate the spatial positioning coordinates of the image points of each grid point on different elevation layers, form virtual control points in the geocentric rectangular coordinate system, and use these virtual control points to fit and form RPC parameters for the RFM geometric positioning of satellite images in the geocentric rectangular coordinate system.

[0082] Step S3: Construct a positioning model and a distance equation for satellite laser altimetry remote sensing targets in the geocentric rectangular coordinate system, including a laser altimetry positioning model and a laser altimetry distance equation;

[0083] The laser altimetry positioning model is a positioning model for satellite laser altimetry remote sensing targets in the geocentric rectangular coordinate system. Through the position and attitude of the sensor, the line-of-sight vector of the laser beam is obtained, and combined with the laser ranging length after atmospheric and tidal corrections, a laser altimetry remote sensing target positioning model in the geocentric rectangular coordinate system is constructed.

[0084] Let the distance after atmospheric correction and system correction of the laser ranging be L, then the positioning model of the laser altimetry spot center in the geocentric rectangular coordinate system is:

[0085]

[0086] In formula (6), [X, Y, Z] is the position of the ground target corresponding to the image point, are respectively the transformation matrix from the orbital coordinate system to the geocentric rectangular coordinate system, the transformation matrix from the body coordinate system to the orbital coordinate system, and the transformation matrix from the sensor coordinate system to the body coordinate system, and [Xs, Ys, Zs] is the sensor position;

[0087] The distance model is constructed based on the position coordinates of the ground point and the sensor position coordinates in the geocentric rectangular coordinate system.

[0088] In the geocentric rectangular coordinate system, the laser altimetry distance equation is:

[0089]

[0090] In formula (7), [X, Y, Z] is the position of the ground target corresponding to the image point, and [Xs, Ys, Zs] is the sensor position.

[0091] Step S4: For the spot image of the satellite laser, extract the position of the spot energy center on the spot image through a fitting method, and increase the clarity of the spot image through spot suppression and image enhancement methods;

[0092] According to the energy corresponding to the two-dimensional plane coordinates of the spot image, with a quadratic surface as the model, the exact coordinates of the spot energy center of the energy on the spot image are fitted. Optionally, by performing operations such as wavelet transform denoising, spot suppression, and image enhancement on the spot image, the clarity of the spot base image is improved, the spot image is enhanced, and a basis is provided for the high-quality matching of the spot image and the remote sensing image.

[0093] Step S5: Implement the matching of homologous points of multi-modal remote sensing targets through image gradient feature matching, extract homologous points between optical images through sift features or least squares matching, and implement the matching between lidar altimetry data and remote sensing images through the extraction of the spot energy center position and template matching;

[0094] Among them, extracting homologous points between optical images through sift features or least squares matching means that when there are only optical images in the overlapping area, the least squares or sift matching method can be used to implement the matching between multi-source optical images, and then the extraction of homologous points of the images is realized.

[0095] The gradient feature matching is mainly based on the normalized local direction gradient histogram HOG feature in the dense grid, and the matching and extraction of homologous points between SAR images and heterogeneous remote sensing images containing SAR images are realized through template matching. Implementing the matching of homologous points of multi-modal remote sensing targets through image gradient feature matching means that when the overlapping area includes SAR images, the matching of SAR images and the matching of multi-modal images including SAR images can be realized through the normalized local direction gradient histogram HOG feature.

[0096] Implementing the matching between lidar altimetry data and remote sensing images through the extraction of the spot energy center position and template matching means obtaining the position of the spot energy center on the remote sensing image through the position of the spot energy center on the spot image, and combining the mutual relationship obtained from the matching of the spot image and the remote sensing image to realize the matching between lidar altimetry data and remote sensing images and the extraction of homologous points between lidar altimetry data and remote sensing images.

[0097] Step S6: Based on the Lidar DSM data, indirectly implement the matching and extraction of homologous points between Lidar and SAR images through SAR simulated image matching, or through template matching between Lidar DSM and InSAR DSM in the case of having InSAR DSM;

[0098] The present invention provides two methods for matching airborne Lidar data and SAR image data. One is to use Lidar data to simulate SAR images with real SAR imaging parameters, and based on the matching results between the simulated SAR and the real SAR images, as well as the geometric relationship between the simulated SAR and the Lidar DSM, obtain the homologous points between the real SAR and the Lidar data.

[0099] Specifically, the simulated image matching refers to using the DSM data of Lidar to perform simulated imaging on this DSM data with the same imaging parameters as the SAR image. By establishing the mapping relationship between the ground coordinate system and the slant range coordinate system using the DSM, SAR image simulation is carried out, and then the relationship between the simulated SAR image and the real SAR image is obtained; then, the simulated SAR is matched with the real SAR image, and according to the geometric relationship between the simulated SAR image points and the DSM points, the position of the homologous points of the simulated image on the Lidar DSM is inverted, realizing the construction of the relationship between the homologous points of the Lidar data and the SAR image, thereby obtaining the homologous points between the SAR image and the Lidar DSM.

[0100] The other is to obtain the homologous points between the Lidar data and the SAR image through the matching results between the Lidar DSM and the InSAR DSM, as well as the geometric relationship between the SAR and the InSAR DSM.

[0101] Specifically, the matching and extraction of homologous points between the Lidar and the SAR image refer to using the three-dimensional coordinate homologous points obtained by the matching between the Lidar DSM and the InSAR DSM, converting the InSAR DSM points to the SAR image according to the SAR imaging geometry, and extracting the homologous points according to the corresponding relationship between the three-dimensional points of the Lidar DSM and the two-dimensional image points of the SAR image.

[0102] Step S7: Under the benchmark of a unified geocentric rectangular coordinate system, construct a joint adjustment model for the multi-modal remote sensing regional network. The joint adjustment model of the multi-modal remote sensing control network consists of an error equation system formed by multi-modal remote sensing data models.

[0103] The adjustment model reconstructs different remote sensing geometric models in the geocentric rectangular coordinate system. The geocentric rectangular coordinate system not only provides a unified reference support for multi-source remote sensing geometric models but also is not restricted by the adjustment range. For images of different sensors, the error equations are established based on different multi-modal remote sensing data models: for satellite optical and SAR remote sensing images, the RFM model in the geocentric rectangular coordinate system is used to construct the error equations; for aerial optical images, the collinearity equation model in the geocentric rectangular coordinate system is used to construct the error equations; for aerial SAR images, the distance coplanarity equation and orbit attitude refinement model in the geocentric rectangular coordinate system are used to construct the error equations; for satellite altimetry data, the laser point positioning model or distance model in the geocentric rectangular coordinate system is used to construct the error equations; for airborne Lidar data, usually with high positioning accuracy, it can be directly used as control data in the space-air-ground network; for ground control points, they are converted to the geocentric rectangular coordinate system and used as known values.

[0104] In the embodiment, different multi-modal remote sensing data models are specifically described as follows:

[0105] For satellite optical and SAR images, based on the RFM model of the geocentric rectangular coordinate system extracted in step S1, it is constructed with the coefficients of the image-side affine transformation model as unknowns. For different scenes within a long orbit, a set of common image-side translation parameters a 0 , b 0 ;

[0106] For aerial optical images, based on the collinearity equation and orbit attitude refinement model of the geocentric rectangular coordinate system extracted in step S2, it is constructed with the parameters of the sensor orbit attitude refinement function and the sensor refinement parameters as unknowns;

[0107] For aerial SAR images, based on the geocentric rectangular distance coplanarity equation and orbit attitude refinement model extracted in step S2, it is constructed with the parameters of the sensor orbit attitude refinement function and the sensor refinement parameters as unknowns;

[0108] For satellite altimetry data, based on the laser altimetry positioning model and orbit attitude refinement model of the geocentric rectangular coordinate system constructed in step S3, it is constructed with the parameters of the sensor orbit refinement function, the attitude refinement function parameters related to the positioning model, and the sensor refinement parameters as unknowns; in the case of low accuracy of the laser altimetry positioning plane, it is based on the distance equation and the sensor position refinement model, and constructed with the parameters of the sensor orbit refinement function and the sensor refinement parameters as unknowns;

[0109] For airborne Lidar data, due to its high accuracy, usually the extracted corresponding points are used as control points and no further refinement is performed on it.

[0110] The system parameters to be solved in the joint adjustment model of the multi-modal remote sensing control network. For the RFM models of optical and SAR images, an affine transformation six-parameter model with image-side correction is adopted; for the sensor position and attitude refinement models of the rigorous models of images and laser altimetry data, a low-order polynomial of time is used to express, and there are:

[0111]

[0112] Among them ([[]] a i , b i , c i , e i , f i , g i )( i = 0, 1, 2) are the polynomial model coefficients of the systematic errors of the sensor position and attitude, t is the time parameter, (X 0 , Y 0 , Z 0 ) are the initial values of the sensor position, is the initial value of the sensor attitude, (X S , Y S , Z S ) are the refined values of the sensor position, is the refined value of the sensor attitude.

[0113] The above error equations are constructed based on homologous points. All homologous points are used to construct error equation systems, forming the basis of the overall adjustment model for the collaborative processing of multi-modal remote sensing data.

[0114] Step S8: By means of error equation normal equation formation and ground point coordinate transformation, reduce the number of unknowns, and overall solve the refined parameters and solve the coordinates of the encrypted points.

[0115] In order to reduce the pressure on the scale of the normal equation matrix caused by the overall solution of large-scale remote sensing data, before the overall solution of the multi-modal remote sensing refined parameters, the error equations constructed by multi-modal remote sensing are normal equation formed and ground point coordinates are transformed. Before the solution, the ground point coordinate unknowns are eliminated with the multi-modal remote sensing error equation systems of the same homologous points as the unit. After obtaining the correction values of each remote sensing orientation parameter by intersection calculation, the ground point coordinates are obtained, and the encrypted point information is acquired. Finally, the results are output.

[0116] The multi-modal aerospace remote sensing collaborative mapping regional network refinement method provided by the present invention takes into account the main mapping remote sensing data of aerospace. In practical applications, not all of the mentioned remote sensing data must participate simultaneously. Any combination of different types of the mentioned remote sensing data can be collaboratively processed within the framework and method designed by the present invention, providing an effective way for the collaborative mapping processing of multi-modal aerospace remote sensing data.

Claims

1. A method for refining a multi-modal air-space remote sensing collaborative mapping regional network, comprising the following steps: S1; Construct a rigorous positioning model for aero-optical and SAR images in a geocentric rectangular coordinate system, including the collinearity equation for aero-optical images and the range-coplanarity model for aero-SAR images; S2; extract the rational polynomial model parameters RPC of satellite optical and SAR images in the geocentric rectangular coordinate system according to the satellite optical RFM geometric model and the satellite SARRFM geometric model; S3: Construct the positioning model and distance equation of satellite laser altimetry remote sensing targets in the geocentric rectangular coordinate system, including the laser altimetry positioning model and the laser altimetry distance equation; S4: For the spot image of satellite laser, the position of the spot energy center on the spot image is extracted by fitting method, and the clarity of the spot image is increased by spot suppression and image enhancement methods; S5: Matching of homonymous points of multimodal remote sensing targets is achieved through image gradient feature matching, extraction of homonymous points between optical images is achieved through SIFT features or least squares matching, and matching between laser altimetry data and remote sensing images is achieved through extraction of spot energy center position and template matching; S6: Based on Lidar DSM data, matching of Lidar and SAR images and extraction of same-name points are achieved indirectly through SAR simulated image matching, or through template matching between Lidar DSM and InSAR DSM when InSAR DSM is available; S7: constructing a multi-modal remote sensing regional network joint adjustment model under a unified geocentric rectangular coordinate system benchmark, wherein the multi-modal remote sensing control network joint adjustment model is composed of an error equation group established by a multi-modal remote sensing data model; S8: Reduce the number of unknowns through error equation formalization and ground point coordinate modification, and solve the refined parameters and encrypted point coordinates as a whole.

2. The method for refining a multi-modal air-space remote sensing collaborative mapping regional network according to claim 1, characterized in that: In step S1, in the geocentric rectangular coordinate system, the rigorous model of the aerial optical image is based on the POS and attitude data, and the colinearity equation of the aerial optical image in the geocentric rectangular coordinate system is constructed through the conversion matrix between the section space rectangular coordinate system and the geocentric rectangular coordinate system; in the geocentric rectangular coordinate system, the rigorous model of the aerial SAR image is for the aerial SAR image, and the distance-coplanarity model of the aerial SAR image is constructed based on the position, velocity and attitude information of the SAR sensor in the terrain rectangular coordinate system.

3. The method for refining a multi-modal air-space remote sensing collaborative mapping regional network according to claim 1, characterized in that: In step S2, according to the satellite optical RFM geometric model and the satellite SARRFM geometric model, the rational polynomial model parameters RPC of the satellite optical and SAR images in the geocentric rectangular coordinate system are extracted, including the following steps: S11: The acquisition of satellite optical image RPC in the geocentric rectangular coordinate system is constructed based on the geometric model of pixel sight vector and the earth ellipsoid model with elevation parameters when the optical satellite image provides the physical model parameters; S12: The acquisition of RPC of satellite SAR images in the geocentric rectangular coordinate system is constructed based on the geometric model of range Doppler and the earth ellipsoid model with elevation parameters when the satellite SAR image provides the physical model parameters; S13: When satellite optical images and satellite SAR images provide RPC in geographic coordinate system, they are constructed based on the RFM geometric model in geographic coordinate system and the earth ellipsoid model with elevation parameters; S14: Divide the image layer into grids and the elevation layer into elevation intervals, and use the above-mentioned geometric model and the ellipsoid model with elevation parameters to calculate the spatial positioning coordinates of the image point coordinates of each grid point on different elevation layers to form virtual control points in the geocentric rectangular coordinate system, and use these virtual control points to fit the RPC parameters of the satellite image RFM geometric positioning in the geocentric rectangular coordinate system.

4. The method for refining a multi-modal air-space remote sensing collaborative mapping regional network according to claim 1, characterized in that: In step S3, the laser altimetry positioning model is a positioning model of a satellite laser altimetry remote sensing target in a geocentric rectangular coordinate system. The positioning model obtains the line of sight vector of the laser beam through the sensor position and attitude, and combines the laser ranging length after atmospheric and tidal correction to construct a laser altimetry remote sensing target positioning model in a geocentric rectangular coordinate system; the distance model is constructed according to the ground point position coordinates and the sensor position coordinates in the geocentric rectangular coordinate system.

5. The method for refining a multi-modal air-space remote sensing collaborative mapping regional network according to claim 1, characterized in that: In step S5, the matching of the same-name points of the multimodal remote sensing target is achieved through image gradient feature matching, the extraction of the same-name points between the optical images is achieved through SIFT features or least squares matching, and the matching between the laser altimetry data and the remote sensing image is achieved through spot energy center position extraction and template matching, including the following processing: (1) Extracting the same-name points between optical images through SIFT features or least squares matching. When there are only optical images in the overlapping area, the least squares or SIFT matching method can be used to match the multi-source optical images, thereby extracting the same-name points between the images. (2) The matching of same-name points of multimodal remote sensing targets is achieved through image gradient feature matching. When the overlapping area includes SAR images, the SAR image matching can be achieved through the normalized local directional gradient histogram HOG feature, as well as the matching of multimodal images including SAR images; (3) The matching between laser altimeter data and remote sensing images is achieved through the extraction of the spot energy center position and template matching. The position of the spot energy center on the remote sensing image is obtained through the position of the spot energy center on the spot image. The relationship between the spot image and the remote sensing image is combined to achieve the matching between the laser altimeter data and the remote sensing image, and the extraction of the same-name points between the laser altimeter data and the remote sensing image is achieved.

6. The method for refining a multi-modal air-space remote sensing collaborative mapping regional network according to claim 1, characterized in that: In step S6, the simulated image matching refers to using the DSM data of the Lidar to simulate imaging of the DSM data with the same imaging parameters as the SAR image, obtaining the relationship between the simulated SAR image and the real SAR image, and then matching the simulated SAR with the real SAR image, and inverting the positions of the same-name points of the simulated image on the LidarDSM according to the geometric relationship between the simulated SAR image points and the DSM points, thereby realizing the construction of the relationship between the same-name points of the Lidar data and the SAR image, thereby obtaining the same-name points between the SAR image and the Lidar DSM.

7. The method for refining a multi-modal space-air remote sensing collaborative mapping regional network according to claim 1, characterized in that: In step S7, the adjustment model reconstructs different remote sensing geometric models in the geocentric rectangular coordinate system. For different sensor images, the error equation is established based on different multimodal remote sensing data models: For satellite optical and SAR remote sensing images, the error equations are constructed using the RFM model in the geocentric rectangular coordinate system; For aerial optical images, the error equations are constructed using the collinearity equation model in the geocentric rectangular coordinate system; For aerial SAR images, the error equation is constructed using the range coplanarity equation in the geocentric rectangular coordinate system and the orbit attitude refinement model; For satellite laser altimetry data, the error equation is constructed using the laser point positioning model or distance model in the geocentric rectangular coordinate system; For airborne Lidar data, it is directly used as control data in the space network; For ground control points, they are converted to the geocentric rectangular coordinate system as known values: The RFM model positioning of optical and SAR images adopts the refined model of image-space affine transformation; the sensor position and attitude refined model of the image and laser altimetry data rigorous model adopts the quadratic polynomial model of time.

8. The method for refining a multi-modal air-space remote sensing collaborative mapping regional network according to claim 7, characterized in that: The sensor position and attitude refinement model is as follows: ， in( a i , b i , c i , e i , f i , g i )( i =0,1,2) are the polynomial model coefficients of the sensor position and attitude system error, t is the time parameter, (X0,Y0,Z0) are the initial values ​​of the sensor position, is the initial value of the sensor attitude, (X S ,Y S ,Z S ) is the refined value of the sensor position, It is the refined value of sensor attitude.

9. The method for refining a multi-modal space-air remote sensing collaborative mapping regional network according to claim 1, characterized in that: In step S8, before the overall solution of the multimodal remote sensing refined parameters, the error equation constructed by the multimodal remote sensing is normalized and the ground point coordinates are modified. Before solving, the multimodal remote sensing error equation group of the same points with the same name is used as a unit, and the unknown variables of the ground point coordinates are eliminated. After the correction values ​​of each remote sensing orientation parameter are obtained, the ground point coordinates are obtained by intersection calculation to obtain the encrypted point information.

Images