A large-area fast geometric positioning method based on remote sensing images

Abstract

The application discloses a large-area fast geometric positioning method based on remote sensing images, which combines sensor model positioning theory with deep learning, and can realize fast geometric positioning of a region of interest. The application comprises the following steps: step S1, obtaining a large number of remote sensing images covering the region of interest; step S2, constructing a space virtual grid by using a sensor model to obtain a plurality of groups of ground point coordinates and corresponding image point coordinates, and extracting four-to-longitude and latitude coordinates and image width and height data of each image as training sample data; step S3, converting the obtained training sample data into a standard data set; step S4, building two geometric positioning models of direct calculation and inverse calculation according to the relationship between input and output data; step S5, training and testing the positioning model by using sample data; and step S6, integrating and applying the model trained in step S5. The application is applied to the technical field of aerospace remote sensing images.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of photogrammetry, and particularly relates to a large-area fast geometric positioning method based on remote sensing images. BACKGROUND

[0002] Precise positioning of a target using a remote sensing image is an important task of space remote sensing observation. The use of a space sensor positioning model mainly includes on-orbit use and ground station use. Current satellites are mostly based on a rigorous imaging model for geometric positioning. In order to facilitate use, a rational function model is also generated for use. The establishment of the rigorous imaging model involves imaging methods and sensor parameters, and describes the real imaging relationship. The model is based on a collinear condition equation, and constructs a rigorous geometric relationship between image point coordinates and ground point geodetic coordinates. Due to technical secrecy and other reasons, image suppliers usually provide users with a rational function model. The rational function model is a general model that does not require inner and outer orientation elements, and establishes the conversion relationship between the image space and the target space through a mathematical expression. For each image, the supplier generally attaches a file used to construct the rational function model. In actual application, coordinate calculation based on the traditional positioning model requires iterative calculation and consumes a lot of time and needs data caching. Especially for on-board platforms, this reduces the efficiency of the on-board processing system, which is not conducive to subsequent data processing. The existing method replaces the traditional rational function model with a deep learning model to perform coordinate positioning, but needs to train a corresponding model for each image. In the ground station, one training model or rational function model is generally used for one image, and the universality is not strong. Especially for satellite platforms, due to limited on-board computing resources, on-orbit model training requires high chip requirements, which cannot meet the requirements of on-orbit intelligent real-time operation. The positioning calculation of an aerial sensor involves camera parameters, coordinate system conversion, attitude data, and the like, and needs a certain technical background. If the coordinate conversion process is simplified and the technical threshold is reduced, it will help to improve the coordinate calculation efficiency. Therefore, it is of great significance to research a large-area positioning model with strong universality, wide positioning range, high calculation efficiency, and good real-time performance, to realize fast acquisition of coordinates of a region of interest. SUMMARY

[0003] The present application aims to solve the problems of 1, the traditional sensor model coordinate calculation needs iterative operation, which consumes time and resources, especially for satellite on-orbit intelligent operation, data caching has a more obvious impact, and is limited by on-board computing resources, which cannot be implemented on-orbit model training; 2, not limited to one image corresponding to one sensor positioning model or neural network model, one positioning model can be used in the region of interest; 3, which can be applied to aerospace remote sensing images; In view of the above situation, a large-area fast geometric positioning method based on remote sensing image is proposed, which combines sensor model positioning theory and deep learning to realize fast geometric positioning of the region of interest.

[0004] The technical scheme adopted by the present application is as follows:

[0005] Step S1: obtaining a large number of remote sensing images covering the region of interest; Specifically, the range of the region of interest is determined, and multiple remote sensing images completely covering the target area are obtained, and the selected images should be consistent with the quality of the applied remote sensing images as much as possible, and when the data is insufficient, multi-source data can be used for data supplement and regional network adjustment processing;

[0006] Step S2: using a sensor model to construct a virtual grid in space to obtain a plurality of ground point coordinates and their corresponding image point coordinates, and extracting the four to latitude and longitude coordinates and image width and height data of each image as training sample data;

[0007] Step S3: converting the obtained training sample data into a standard data set;

[0008] Step S4: building two geometric positioning models of forward calculation and inverse calculation according to the relationship between input and output data, wherein the forward calculation positioning model takes the image four to latitude and longitude coordinates, image width and height value, and grid point geodetic coordinates as input data, and the grid point image coordinate as output data for neural network training; the inverse calculation positioning model takes the four to latitude and longitude coordinates, image width and height value, grid point image coordinate and elevation value as input data, and the grid point latitude and longitude coordinates as output data for inverse calculation positioning model training, and the model needs to be called according to the actual situation when used;

[0009] Step S5: training and testing the positioning model using sample data;

[0010] Step S6: integrating and applying the model trained in step S5.

[0011] Further, the specific steps of step S2 are as follows: the ground point geodetic coordinates and image point coordinates are obtained by constructing a virtual grid in space through a sensor model, and the four to latitude and longitude coordinates, image width and height data, and one-to-one corresponding object coordinates and image coordinates of each remote sensing image are extracted; wherein the one-to-one corresponding object coordinates and image coordinates are obtained according to the following steps:

[0012] Step S21: Divide the entire image into N×N grids by uniformly dividing the image into grids in the horizontal direction;

[0013] Step S22: Then, based on the elevation range of the area where the image is located, the elevation is evenly divided into K layers;

[0014] The values ​​of N and K can be adjusted according to requirements.

[0015] Step S23: Combine the uniform grid divided in the plane with the elevation values ​​to construct a spatial three-dimensional grid covering the target area;

[0016] Step S24: Using the image point coordinates (r, c) and elevation value (H) of each grid point, substitute them into the sensor model inverse projection to calculate the corresponding object latitude and longitude coordinates (P, L), and then obtain all the coordinates of all grid points. Use all the coordinates of all grid points as training sample data.

[0017] Furthermore, step S3 includes the following sub-steps:

[0018] Step S31: Divide the training sample data into a training set and a test set, with the training set and test set being divided equally in a ratio of 8:2;

[0019] Step S32: Normalize all training data using deviation standardization, mapping the data to 0 and 1 to reduce bias caused by large data differences and improve model prediction accuracy. This involves normalizing the ground point geodetic coordinates (P, L, H), image width (W), image height (V), and the four cardinal coordinates (latitude and longitude). , After normalizing the image point coordinates (r, c), we obtain the normalized ground point geodetic coordinates (X, Y, Z), image width (wn), image height (vn), and boundary latitude and longitude coordinates. , The image point coordinates (l, s) are normalized using deviation standardization, and the minimum value of each data set is used. and maximum value The data, and the relationship, are as follows:

[0020] , where data is the original data after denormalization, and Data is the normalized data.

[0021] Furthermore, step S4 includes the following sub-steps:

[0022] Step S41: Forward positioning model training using normalized ground point geodetic coordinates (X, Y, Z), image width (wn), image height (vn), and boundary latitude and longitude coordinates (…). , The normalized image point coordinates (l, s) are used as input data, and the normalized image point coordinates (l, s) are used as output data.

[0023] Step S42: Inversely calculate the positioning model using the normalized ground point elevation values ​​(Z), image width (wn), image height (vn), and boundary latitude and longitude coordinates (…). , The image point coordinates (l, s) are used as input data, and the normalized ground point latitude and longitude coordinates (X, Y) are used as output data.

[0024] Step S43: Establish a multi-layer neural network;

[0025] Step S44: Train two localization models using a neural network.

[0026] Furthermore, step S5 includes the following sub-steps:

[0027] Step S51: Train the model accordingly based on the input and output data of the training set;

[0028] Step S52: Use the output data predicted by the neural network and output the actual truth value The mean squared error loss is used as the loss function, and backpropagation automatically solves for the Jacobian matrix, accelerating the convergence speed of the training model; the mean squared error regression loss is:

[0029]

[0030] in, For the sample size, Input data;

[0031] Step S53: Normalize the input data of the test set, use the test set data to test the prediction effect of the neural network model, denormalize the prediction result of the neural network, compare it with the output data of the test set, and use the mean absolute error as the evaluation index. The denormalization formula is expressed as:

[0032] ;

[0033] Step S54: Adjust the optimizer, hidden layers, activation functions, and hyperparameters of the model to continuously improve the model's predictive performance and effectiveness;

[0034] Step S55: Stop training when the evaluation metric is less than the threshold.

[0035] Further, if the model trained in step S5 is applied to the satellite in orbit in step S6, the trained model needs to be embedded into the satellite chip in advance. When the satellite is in orbit, the four-to-longitude and latitude coordinates are calculated by using the rigorous imaging model, the width and height data of the image are obtained, the coordinates to be calculated are determined, the data is normalized, the corresponding positioning model is called, the coordinates of the to-be-solved point can be predicted, and the obtained data is de-normalized, which is the coordinate value to be solved. If the model trained in step S5 is applied to the ground station in step S5, the model is directly called to predict the coordinates, and the four-to-longitude and latitude coordinates are obtained through the image metadata file.

[0036] The beneficial effects of the present application are: 1. The coordinate calculation speed based on the neural network model is fast, without iterative calculation, and the generalization is good, and the time required is less than that of the traditional sensor model; 2. The application range of the present application is wide, which can be applied to aerospace remote sensing images, and one positioning model can be used in a large area on the ground station, without the need for one model corresponding to one image; the satellite in orbit can quickly perform geometric positioning on the region of interest, and the resource occupation of calculation power is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is the flowchart of the present application. DETAILED DESCRIPTION

[0038] As shown in Figure 1 In the present embodiment, the present application comprises the following steps: S1: obtaining a large number of remote sensing images covering the region of interest; specifically, data preparation is performed to obtain multiple remote sensing images covering the region of interest, and a certain image overlap rate is ensured. The selected images should be consistent with the quality of the applied remote sensing images as much as possible. When the data is insufficient, multiple source data can be used for data supplement. At this time, the images in the region should be subjected to regional network adjustment processing, and control points can be added for control network adjustment under the condition to improve the geometric positioning accuracy of each image;

[0039] Step S2: obtaining a large number of remote sensing images covering the region of interest; specifically, data preparation is performed to obtain multiple remote sensing images covering the region of interest, and a certain image overlap rate is ensured. The selected images should be consistent with the quality of the applied remote sensing images as much as possible. When the data is insufficient, multiple source data can be used for data supplement. At this time, the images in the region should be subjected to regional network adjustment processing, and control points can be added for control network adjustment under the condition to improve the geometric positioning accuracy of each image;

[0040] Step S21: dividing the grid in the horizontal direction of the image uniformly, and dividing the whole image into N*N grids;

[0041] Step S22: then according to the height range of the image area, the height is uniformly divided into K layers;

[0042] wherein the values of N and K are adjusted according to the requirements.

[0043] Step S23: Combine the uniform grid divided in the plane with the elevation values ​​to construct a spatial three-dimensional grid covering the target area;

[0044] Step S24: Using the image point coordinates (r, c) and elevation value (H) of each grid point, substitute them into the sensor model inverse projection to calculate the corresponding object latitude and longitude coordinates (P, L), and then obtain all the coordinates of all grid points. Use all the coordinates of all grid points as training sample data.

[0045] Step S3: Convert the acquired training sample data into a standard dataset. The conversion into a standard dataset is implemented according to the following steps:

[0046] Step S31: Divide the training sample data into a training set and a test set, with the training set and test set being divided equally in a ratio of 8:2;

[0047] Step S32: Normalize all training data using deviation standardization, mapping the data to 0 and 1 to reduce bias caused by large data differences and improve model prediction accuracy. This involves normalizing the ground point geodetic coordinates (P, L, H), image width (W), image height (V), and the four cardinal coordinates (latitude and longitude). , After normalizing the image point coordinates (r, c), we obtain the normalized ground point geodetic coordinates (X, Y, Z), image width (wn), image height (vn), and boundary latitude and longitude coordinates. , The image point coordinates (l, s) are normalized using deviation standardization, and the minimum value of each data set is used. and maximum value The data, and the relationship, are as follows:

[0048] , where data is the original data after denormalization, and Data is the normalized data;

[0049] Step S4: Based on the relationship between input and output data, construct two geometric positioning models: forward and inverse. The forward positioning model uses the image's latitude and longitude coordinates, image width and height values, and grid point geodetic coordinates as input data, and the grid point image-side coordinates as output data for neural network training. The inverse positioning model uses the image's latitude and longitude coordinates, image width and height values, grid point image-side coordinates, and elevation values ​​as input data, and the grid point latitude and longitude coordinates as output data for inverse positioning model training. The model needs to be called according to the actual situation during use. Specifically, implement the two geometric positioning models according to the following steps:

[0050] Step S41: The direct positioning model is trained with the normalized ground point geodetic coordinates (X, Y, Z), the image width (wn), the image height (vn) and the four-vertex longitude and latitude coordinates (,,, ) as input data, and the normalized image point coordinates (l, s) as output data; ,

[0051] Step S42: The inverse positioning model is trained with the normalized ground point elevation value (Z), the image width (wn), the image height (vn), the four-vertex longitude and latitude coordinates (,,,, ) and the image point coordinates (l, s) as input data, and the normalized ground point longitude and latitude coordinates (X, Y) as output data; ,

[0052] Step S43: A multi-layer neural network is established;

[0053] Step S44: The two geometric positioning models are trained by using the neural network;

[0054] Step S5: The positioning model is trained and tested by using sample data; the training and testing are implemented according to the following steps:

[0055] Step S51: The input and output data are set, and the corresponding model training is performed according to the input and output data of the training set;

[0056] Step S52: The input data after the neural network is used The mean square error loss between the predicted output data value and the actual true value of the output is used as the loss function, the automatic solution of the Jacobian matrix is realized by back propagation, the convergence speed of the training model is accelerated, and the mean square error regression loss is represented as:

[0057]

[0058] wherein, is the number of training samples;

[0059] Step S53: The input data of the test set is normalized, the prediction effect of the neural network model is tested by using the test set data, the prediction result of the neural network is de-normalized, compared with the output data of the test set, and the mean absolute error is used as an evaluation index, wherein the de-normalization formula is represented as:

[0060]

[0061] Step S54: The optimizer, hidden layer, activation function and hyperparameters of the model are adjusted, and the prediction performance and effect of the model are continuously improved.

[0062] ​​Step S55: Stop training when the evaluation metric is less than the threshold;

[0063] Step S6: Integrate and apply the model trained in step S5, specifically by following these steps:

[0064] Step S61: If the method described in this invention is applied to aerospace remote sensing images, the model trained in step S5 can be directly called when used at the ground station. When used in orbit, the positioning model needs to be integrated into the satellite chip, eliminating the need for in-orbit model training and saving on-board computing resources.

[0065] Step S62: When the satellite is in orbit, firstly, the four cardinal coordinates (latitude and longitude) of the image are calculated using a rigorous imaging geometry model to determine the area covered by the image. Then, the width and height data of the image are acquired to determine whether the object-side coordinates or image-side coordinates need to be calculated. Taking the calculation of image-side coordinates as an example, the ground point geodetic coordinates (P, L, H), image width (W), image height (V), and four cardinal coordinates (latitude and longitude) are input. , The data is normalized, the forward geometric positioning model is called, and the normalized image point coordinates are output. The image point coordinates are then reverse-normalized to obtain the image point coordinates corresponding to the object point in the local image.

[0066] Step S63: When using the ground station, obtain the latitude and longitude coordinates from the metadata file to get the width and height data of the image. Then, call the corresponding positioning model based on the coordinate data obtained.

[0067] Leveraging the advantages of neural network data analysis and GPU acceleration, the trained model is used to calculate geometric positioning coordinates, saving the time spent on coordinate iteration calculations. This is highly beneficial for rapid geometric positioning of a large number of target points over a wide area.

[0068] Although the embodiments of the present invention are described with reference to actual solutions, they do not constitute a limitation on the meaning of the present invention. Modifications to the embodiments and combinations with other solutions based on this specification will be obvious to those skilled in the art.

Claims

1. A method for rapid geometric localization of large areas based on remote sensing imagery, characterized in that, It includes the following steps: Step S1: Acquire a large number of remote sensing images covering the region of interest; Step S2: Construct a spatial virtual grid using the sensor model to obtain multiple sets of ground point coordinates and their corresponding image point coordinates, and extract the latitude and longitude coordinates and image width and height data of each image as training sample data; Step S3: Convert the acquired training sample data into a standard dataset; Step S4: Based on the relationship between the input and output data, construct two geometric positioning models: forward calculation and inverse calculation. Step S5: Train and test the localization model using sample data; Step S6: Integrate and apply the model trained in step S5; Step S2 specifically involves: constructing a spatial virtual grid using the sensor model, and extracting the latitude and longitude coordinates, width and height data, and corresponding object-space and image-space coordinates for each remote sensing image. The acquisition of these corresponding object-space and image-space coordinates is implemented according to the following steps: Step S21: Divide the entire image into N×N grids by uniformly dividing the image into grids in the horizontal direction; Step S22: Then, based on the elevation range of the area where the image is located, the elevation is evenly divided into K layers; The values ​​of N and K can be adjusted according to requirements. Step S23: Combine the uniform grid divided in the plane with the elevation values ​​to construct a spatial three-dimensional grid covering the target area; Step S24: Using the image point coordinates (r, c) and elevation value (H) of each grid point, substitute them into the sensor model inverse projection to calculate the corresponding object latitude and longitude coordinates (P, L), and then obtain all the coordinates of all grid points. Use all the coordinates of all grid points as training sample data. Step S3 includes the following sub-steps: Step S31: Divide the training sample data into a training set and a test set, with the training set and test set being divided equally in a ratio of 8:2; Step S32: Normalize all training data, mapping the data to between 0 and 1, and normalize the ground point geodetic coordinates (P, L, H), image width (W), image height (V), and boundary latitude and longitude coordinates (…). , After normalizing the image point coordinates (r, c), we obtain the normalized ground point geodetic coordinates (X, Y, Z), image width (wn), image height (vn), and boundary latitude and longitude coordinates. , The image point coordinates (l, s) are normalized using deviation standardization, and the minimum value of each data set is used. and maximum value The data, and the relationship, are as follows: ， Where data is the original data after denormalization, and Data is the normalized data; Step S4 includes the following sub-steps: Step S41: Forward positioning model training using normalized ground point geodetic coordinates (X, Y, Z), image width (wn), image height (vn), and boundary latitude and longitude coordinates (…). , The image point coordinates (l, s) are used as input data, and the normalized image point coordinates are used as output data. Step S42: Inversely calculate the positioning model using the normalized ground point elevation values ​​(Z), image width (wn), image height (vn), and boundary latitude and longitude coordinates (…). , The image point coordinates (l, s) are used as input data, and the normalized ground point latitude and longitude coordinates (X, Y) are used as output data. Step S43: Establish a multi-layer neural network; Step S44: Train two localization models using a neural network; In step S6, if the model trained in step S5 is integrated and applied to an on-orbit satellite, the trained model needs to be embedded into the satellite chip in advance. When the satellite is in orbit, the latitude and longitude coordinates are calculated using the rigorous imaging model, the width and height data of the image are obtained, the coordinates to be calculated are determined, the data is normalized, and the corresponding positioning model is called to predict the coordinates of the point to be determined. The obtained data is then inversely normalized to obtain the coordinate values ​​to be determined. If the model trained in step S5 is integrated and applied to ground stations, the model can be directly called to predict coordinates, and the latitude and longitude coordinates can be obtained through the image metadata file.

2. The method for rapid geometric localization of large areas based on remote sensing imagery according to claim 1, characterized in that: The specific steps of step S1 are as follows: determine the range of the region of interest, acquire multiple remote sensing images that fully cover the target area, and perform regional network adjustment on the multiple remote sensing images.

3. The method for rapid geometric localization of large areas based on remote sensing imagery according to claim 1, characterized in that: Step S5 includes the following sub-steps: Step S51: Train the model accordingly based on the input and output data of the training set; Step S52: Use the output data predicted by the neural network and output the actual truth value The mean squared error loss is used as the loss function, and backpropagation automatically solves for the Jacobian matrix, accelerating the convergence speed of the training model; the mean squared error regression loss is: in, For the sample size, Input data; Step S53: Normalize the input data of the test set, use the test set data to test the prediction effect of the neural network model, denormalize the prediction result of the neural network, compare it with the output data of the test set, and use the mean absolute error as the evaluation index. The denormalization formula is expressed as: ； Step S54: Adjust the optimizer, hidden layers, activation functions, and hyperparameters of the model to continuously improve the model's predictive performance and effectiveness; Step S55: Stop training when the evaluation metric is less than the threshold.

Images