Large-area rapid geometric positioning method based on remote sensing image

Abstract

The invention discloses a large-area rapid geometric positioning method based on a remote sensing image, and the method combines a sensor model positioning theory with deep learning, and can achieve the rapid geometric positioning of an interested area. The method comprises the following steps: S1, obtaining a large number of remote sensing images covering a region of interest; s2, constructing a space virtual grid by using a sensor model to obtain multiple groups of ground point coordinates and corresponding image point coordinates, and extracting four-dimensional coordinates and image width and height data of each image as training sample data; s3, converting the obtained training sample data into a standard data set; s4, two geometric positioning models of forward calculation and inverse calculation are built according to the relation between the input data and the output data; s5, training and testing a positioning model by using the sample data; and S6, carrying out integrated application on the model trained in the step S5. The method is applied to the technical field of aerospace remote sensing images.

Description

Technical Field

[0001] The present invention is applied to the technical field of photogrammetry, and particularly relates to a large-area rapid geometric positioning method based on remote sensing images. Background Art

[0002] Using remote sensing images to accurately locate targets is an important task of aerospace remote sensing earth observation. The use of aerospace sensor positioning models is mainly divided into two aspects: on-orbit use and ground station use. Currently, most satellites in orbit are based on rigorous imaging models for geometric positioning. In order to facilitate use, rational function models are also generated for use. The establishment of a rigorous imaging model involves imaging methods and sensor parameters, etc., which describes the real imaging relationship. The model is based on the collinear condition equation to construct a strict geometric relationship between the image point coordinates and the ground point geodetic coordinates. Due to technical confidentiality and other reasons, image suppliers usually provide users with rational function models. Rational function models are a general model that does not require internal and external orientation elements and establishes the conversion relationship between image space and target space through mathematical expressions. For each image, the general supplier will provide a file for constructing a rational function model. In practical applications, iterative calculations are required to calculate coordinates based on traditional positioning models, which takes a lot of time and requires data caching. Especially for on-board platforms, this reduces the efficiency of the on-board processing system and is not conducive to subsequent data processing. At present, deep learning models are used to replace traditional rational function models for coordinate positioning, but each image needs to be trained to generate a corresponding model. When used in ground stations, one image corresponds to one training model or rational function model, which is not very versatile. Especially for satellite platform applications, due to the limited computing power resources on the satellite, the chip requirements for on-orbit model training are relatively high, and it cannot meet the real-time intelligent operation of satellites on orbit. The positioning calculation of aviation sensors involves camera parameters, coordinate system conversion, attitude data, etc., which requires a certain technical background. If the coordinate conversion process is simplified and the technical threshold is lowered, it will help improve the efficiency of coordinate calculation. Therefore, it is of great significance to study a large-area positioning model with strong versatility, wide positioning range, high computing efficiency, and good real-time performance to achieve rapid acquisition of coordinates of the area of ​​interest. Summary of the invention

[0003] The purpose of the present invention is to solve the following problems: 1. The traditional sensor model coordinate calculation requires iterative operations, which consumes time and resources. Especially for the satellite's on-orbit intelligent operation, the data cache has a more obvious impact, and due to the limitation of on-board computing resources, on-orbit model training cannot be achieved; 2. It is not limited to one sensor positioning model or neural network model corresponding to one scene image, and one positioning model can be used for the area of ​​interest; 3. It can be applied to aerospace remote sensing images; In view of the above situation, a large-area rapid geometric positioning method based on remote sensing images is proposed. This method combines sensor model positioning theory with deep learning, and can achieve rapid geometric positioning of the area of ​​interest.

[0004] The technical solution adopted by the present invention is: the present invention comprises the following steps:

[0005] Step S1: Acquire a large number of remote sensing images covering the area of ​​interest; specifically, to determine the scope of the area of ​​interest, obtain multi-view remote sensing images that completely cover the target area. The selected images should be as consistent as possible with the quality of the applied remote sensing images. When the data is insufficient, multi-source data can be used for data supplementation and regional network adjustment processing can be performed;

[0006] Step S2: Use the sensor model to construct a spatial virtual grid to obtain multiple sets of ground point coordinates and their corresponding image point coordinates, and extract the four-dimensional coordinates and image width and height data of each scene image as training sample data;

[0007] Step S3: converting the acquired training sample data into a standard data set;

[0008] Step S4: construct two geometric positioning models, namely, forward calculation and inverse calculation, according to the relationship between input and output data. The forward positioning model uses the four-dimensional longitude and latitude coordinates of the image, the image width and height values, and the geodetic coordinates of the grid points as input data, and the image coordinates of the grid points as output data for neural network training; the inverse positioning model uses the four-dimensional longitude and latitude coordinates, the image width and height values, the image coordinates and elevation values ​​of the grid points as input data, and the latitude and longitude coordinates of the grid points as output data for inverse positioning model training. When using, the model needs to be called according to the actual situation;

[0009] Step S5: Using sample data to train and test the positioning model;

[0010] Step S6: Integrate and apply the model trained in step S5.

[0011] Further, the specific steps of step S2 are: the acquisition of the geodetic coordinates of the ground points and the coordinates of the image points is to construct a spatial virtual grid through the sensor model, extract the longitude and latitude coordinates of each remote sensing image, the width and height data of the image, and the one-to-one corresponding object coordinates and image coordinates; wherein the acquisition of the one-to-one corresponding object coordinates and image coordinates is specifically implemented according to the following steps:

[0012] Step S21: The entire image is evenly divided into N×N grids by evenly dividing the grids in the horizontal direction of the image;

[0013] Step S22: Then, the elevation is evenly divided into K layers according to the elevation range of the area where the image is located;

[0014] Among them, the values ​​of N and K are adjusted autonomously according to demand;

[0015] Step S23: Combining the uniform grid divided by the plane and the elevation value, a spatial three-dimensional grid covering the target area is constructed;

[0016] Step S24: Use the image point coordinates (r, c) and elevation value (H) of each grid point, substitute them into the sensor geometric positioning model for reverse projection to calculate the corresponding object space longitude and latitude coordinates (P, L), and you can get all the coordinates of all the grid points, and use all the coordinates of all the grid points as training sample data.

[0017] Furthermore, step S3 includes the following sub-steps:

[0018] Step S31: Divide the training sample data into a training set and a test set, wherein the training set and the test set are equally divided in a ratio of 8:2;

[0019] Step S32: normalize all training data. The normalization method uses the deviation standardization method to map the data between 0 and 1, reduce the deviation caused by large data differences, and improve the prediction accuracy of the model. The ground point geodetic coordinates (P, L, H), image width (W), image height (H), four-dimensional longitude and latitude coordinates (lon4, lat4), and image point coordinates (r, c) are normalized to obtain the normalized ground point geodetic coordinates (X, Y, Z), image width (wn), image height (hn), four-dimensional longitude and latitude coordinates (lon 4_n ,lat 4_n ), image point coordinates (l, s), the method uses deviation standardization for normalization, using the minimum value data of each group of data min and maximum value data max Data, the relationship is as follows:

[0020] Where data is the original data after denormalization, and Data is the normalized data.

[0021] Furthermore, step S4 comprises the following sub-steps:

[0022] Step S41: Calculate the normalized ground point geodetic coordinates (X, Y, Z), image width (wn), image height (hn) and four-dimensional longitude and latitude coordinates (lon 4_n ,lat 4_n ) as input data, and the normalized image pixel coordinates (l, s) as output data;

[0023] Step S42: Inversely calculate the positioning model using the normalized ground point elevation value (Z), image width (wn), image height (hn), and four-dimensional longitude and latitude coordinates (lon 4_n ,lat 4_n ), the image point coordinates (l, s) are used as input data, and the normalized ground point longitude and latitude coordinates (X, Y) are used as output data;

[0024] Step S43: Establish a multi-layer neural network;

[0025] Step S44: using a neural network to train two positioning models.

[0026] Furthermore, step S5 includes the following sub-steps:

[0027] Step S51: Perform corresponding model training according to the input and output data of the training set;

[0028] Step S52: Output data Y predicted using the neural network pre And output the actual true value Y train The mean square error loss between is used as the loss function, and the back propagation realizes the automatic solution of the Jacobian matrix to speed up the convergence of the training model; the mean square error regression loss is:

[0029]

[0030] Where n is the number of samples, X train For input data;

[0031] Step S53: normalize the input data of the test set, use the test set data to test the prediction effect of the neural network model, denormalize the prediction result of the neural network, compare it with the output data of the test set, and use the mean absolute error as the evaluation index, where the denormalization formula is expressed as:

[0032] data=Data(data max -data min )+data min ;

[0033] Step S54: adjusting the optimizer, hidden layer, activation function and hyperparameters of the model to continuously improve the prediction performance and effect of the model;

[0034] Step S55: When the evaluation index is less than the threshold, stop training.

[0035] Furthermore, in step S6, if the model trained in step S5 is integrated and applied to an on-orbit satellite, the trained model needs to be embedded in the satellite chip in advance. When the satellite is in orbit, the four-dimensional longitude and latitude coordinates are calculated using a rigorous imaging model, the width and height data of the image are obtained, the coordinates to be calculated are determined, the data is normalized, and the corresponding positioning model is called to predict the coordinates of the point to be calculated. The obtained data is denormalized to obtain the required coordinate value; if the model trained in step S5 is integrated and applied to a ground station, the model can be directly called for coordinate prediction, and the four-dimensional coordinates are obtained through the image metadata file.

[0036] The beneficial effects of the present invention are: 1. The coordinate calculation speed of the present invention based on the neural network model is fast, no iterative calculation is required, it can use less time and has good versatility compared with the traditional sensor model; 2. The present invention has a wide range of applications and can be applied to aerospace remote sensing images. At the ground station, a positioning model can be used in a large area, and there is no need for one model to correspond to one scene of the image; the use of satellites in orbit can quickly perform geometric positioning of the area of ​​interest and reduce the occupation of computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0038] like Figure 1 As shown, in this embodiment, the present invention includes step S1: obtaining a large number of remote sensing images covering the area of ​​interest; specifically, data preparation, obtaining multiple remote sensing images covering the area of ​​interest, and ensuring a certain image overlap rate, and the selected images should be as consistent as possible with the quality of the applied remote sensing images. When the data is insufficient, multi-source data can be used to supplement the data. At this time, the images in the area should be processed by regional network adjustment, and control points can be added to perform control network adjustment if conditions permit, so as to improve the geometric positioning accuracy of each scene image;

[0039] Step S2: Use the sensor model to construct a spatial virtual grid to obtain multiple sets of ground point coordinates and their corresponding image point coordinates, extract the four-dimensional coordinates, image width and height data, and the one-to-one corresponding object coordinates and image coordinates of each scene image, wherein the one-to-one corresponding object coordinates and image coordinates are obtained in accordance with the following steps:

[0040] Step S21: The entire image is evenly divided into N×N grids by evenly dividing the grids in the horizontal direction of the image;

[0041] Step S22: Then, the elevation is evenly divided into K layers according to the elevation range of the area where the image is located;

[0042] Among them, the values ​​of N and K are adjusted autonomously according to demand;

[0043] Step S23: Combining the uniform grid divided by the plane and the elevation value, a spatial three-dimensional grid covering the target area is constructed;

[0044] Step S24: Use the image point coordinates (r, c) and elevation value (H) of each grid point, substitute them into the sensor geometric positioning model for reverse projection to calculate the corresponding object space longitude and latitude coordinates (P, L), and you can get all the coordinates of all the grid points, and use all the coordinates of all the grid points as training sample data.

[0045] Step S3: convert the acquired training sample data into a standard data set. The conversion into a standard data set is specifically implemented by the following steps:

[0046] Step S31: Divide the training sample data into a training set and a test set, wherein the training set and the test set are equally divided in a ratio of 8:2;

[0047] Step S32: normalize all training data. The normalization method uses the deviation standardization method to map the data between 0 and 1, reduce the deviation caused by large data differences, and improve the prediction accuracy of the model. The ground point geodetic coordinates (P, L, H), image width (W), image height (H), four-dimensional longitude and latitude coordinates (lon4, lat4), and image point coordinates (r, c) are normalized to obtain the normalized ground point geodetic coordinates (X, Y, Z), image width (wn), image height (hn), four-dimensional longitude and latitude coordinates (lon 4_n ,lat 4_n ), image point coordinates (l, s), the method uses deviation standardization for normalization, using the minimum value data of each group of data min and maximum value data max Data, the relationship is as follows:

[0048] Where data is the original data after denormalization, and Data is the normalized data;

[0049] Step S4: construct two geometric positioning models, namely, forward calculation and inverse calculation, according to the relationship between input and output data. The forward positioning model uses the four-dimensional longitude and latitude coordinates of the image, the image width and height values, and the geodetic coordinates of the grid points as input data, and the image coordinates of the grid points as output data for neural network training; the inverse positioning model uses the four-dimensional longitude and latitude coordinates, the image width and height values, the image coordinates and elevation values ​​of the grid points as input data, and the latitude and longitude coordinates of the grid points as output data for inverse positioning model training. When using, the model needs to be called according to the actual situation; specifically, the following steps are followed to implement the two geometric positioning models:

[0050] Step S41: Calculate the normalized ground point geodetic coordinates (X, Y, Z), image width (wn), image height (hn) and four-dimensional longitude and latitude coordinates (lon 4_n , lan 4_n ) as input data, and the normalized image pixel coordinates (l, s) as output data;

[0051] Step S42: Inversely calculate the positioning model using the normalized ground point elevation value (Z), image width (wn), image height (hn), and four-dimensional longitude and latitude coordinates (lon 4_n ,lat 4_n ), the image point coordinates (l, s) are used as input data, and the normalized ground point longitude and latitude coordinates (X, Y) are used as output data;

[0052] Step S43: Establish a multi-layer neural network;

[0053] Step S44: using a neural network to train two geometric positioning models;

[0054] Step S5: Use sample data to train and test the positioning model; this is specifically achieved by following the steps below:

[0055] Step S51: setting input and output data, and performing corresponding model training according to the input and output data of the training set;

[0056] Step S52: Input data X train The predicted output data value Y after the neural network pre The actual output value Y train The mean square error loss between is used as the loss function, and the Jacobian matrix is ​​automatically solved by back propagation to speed up the convergence of the training model. The mean square error regression loss is expressed as:

[0057]

[0058] Where n is the number of training samples;

[0059] Step S53: normalize the input data of the test set, use the test set data to test the prediction effect of the neural network model, denormalize the prediction result of the neural network, compare it with the output data of the test set, and use the mean absolute error as the evaluation index, where the denormalization formula is expressed as:

[0060] data=Data(data max -data min )+data min

[0061] Step S54: Adjust the optimizer, hidden layer, activation function and hyperparameters of the model to continuously improve the prediction performance and effect of the model.

[0062] Step S55: When the evaluation index is less than the threshold, stop training;

[0063] Step S6: The model trained in step S5 is integrated and applied, which is implemented by the following steps:

[0064] Step S61: If the method described in the present invention is applied to aerospace remote sensing images, the model trained in step S5 can be directly called when used at the ground station. When used in orbit, the positioning model needs to be integrated into the satellite chip, and there is no need to perform on-orbit model training, thus saving on-board computing resources.

[0065] Step S62: When the satellite is in orbit, the four-dimensional longitude and latitude coordinates of the image are first calculated using a rigorous imaging geometry model to determine the range of image coverage, and then the width and height data of the image are obtained to determine whether the object coordinates or the image coordinates need to be calculated; taking the calculation of the image coordinates as an example, the geodetic coordinates of the ground point (P, L, H), image width (W), image height (H), and four-dimensional longitude and latitude coordinates (lon4, lat4) are input, the data is normalized, the forward geometric positioning model is called, the normalized value of the image point coordinates is output, and the image point coordinates are denormalized to obtain the image point coordinate value corresponding to the object point in the current scene image.

[0066] Step S63: When used at the ground station, the longitude and latitude coordinates are obtained according to the metadata file to obtain the width and height data of the image, and the corresponding positioning model is called according to the coordinate data that needs to be obtained.

[0067] Based on the advantages of neural network data analysis and GPU acceleration, the trained model is used to calculate the geometric positioning coordinates, which saves the time spent on coordinate iterative calculation, and is very beneficial for the rapid geometric positioning of a large range and a large number of target points.

[0068] Although the embodiments of the present invention are described with practical solutions, they do not constitute limitations on the meaning of the present invention. For those skilled in the art, it is obvious to modify the implementation scheme and combine it with other solutions based on this description.

Claims

1. A large area rapid geometric positioning method based on remote sensing images, characterized in that: It includes the following steps: Step S1: Acquire a large number of remote sensing images covering the area of ​​interest; Step S2: Use the sensor model to construct a spatial virtual grid to obtain multiple sets of ground point coordinates and their corresponding image point coordinates, and extract the four-dimensional coordinates and image width and height data of each scene image as training sample data; Step S3: converting the acquired training sample data into a standard data set; Step S4: constructing two geometric positioning models, forward calculation and reverse calculation, according to the relationship between input and output data; Step S5: Using sample data to train and test the positioning model; Step S6: Integrate and apply the model trained in step S5.

2. The large-area rapid geometric positioning method based on remote sensing images according to claim 1 is characterized by: The specific steps of step S1 are: determining the scope of the region of interest, obtaining multi-view remote sensing images that completely cover the target area, and performing regional network adjustment processing on the multi-view remote sensing images.

3. The large-area rapid geometric positioning method based on remote sensing images according to claim 1 is characterized by: The specific steps of step S2 are: constructing a spatial virtual grid through the sensor model, extracting the longitude and latitude coordinates, width and height data of each remote sensing image, and the one-to-one corresponding object coordinates and image coordinates; wherein the one-to-one corresponding object coordinates and image coordinates are obtained specifically according to the following steps: Step S21: The entire image is evenly divided into N×N grids by evenly dividing the grids in the horizontal direction of the image; Step S22: Then, the elevation is evenly divided into K layers according to the elevation range of the area where the image is located; Among them, the values ​​of N and K are adjusted autonomously according to demand; Step S23: Combining the uniform grid divided by the plane and the elevation value, a spatial three-dimensional grid covering the target area is constructed; Step S24: Use the image point coordinates (r, c) and elevation value (H) of each grid point, substitute them into the sensor geometric positioning model for reverse projection to calculate the corresponding object space longitude and latitude coordinates (P, L), and you can get all the coordinates of all the grid points, and use all the coordinates of all the grid points as training sample data.

4. The large-area rapid geometric positioning method based on remote sensing images according to claim 1 is characterized by: Step S3 includes the following sub-steps: Step S31: Divide the training sample data into a training set and a test set, wherein the training set and the test set are equally divided in a ratio of 8:2; Step S32: Normalize all training data, map the data to between 0 and 1, and normalize the ground point geodetic coordinates (P, L, H), image width (W), image height (H), four-dimensional longitude and latitude coordinates (lon4, lat4), and image point coordinates (r, c) to obtain the normalized ground point geodetic coordinates (X, Y, Z), image width (wn), image height (hn), four-dimensional longitude and latitude coordinates (lon 4_n ,lat 4_n ), image point coordinates (l, s), the method uses deviation standardization for normalization, using the minimum value data of each group of data min and maximum value data max Data, the relationship is as follows: Where data is the original data after denormalization, and Data is the normalized data.

5. The large-area rapid geometric positioning method based on remote sensing images according to claim 1 is characterized by: The step S4 comprises the following sub-steps: Step S41: Calculate the normalized ground point geodetic coordinates (X, Y, Z), image width (wn), image height (hn) and four-dimensional longitude and latitude coordinates (lon 4_n ,lat 4_n ) as input data, and the normalized image pixel coordinates (l, s) as output data; Step S42: Inversely calculate the positioning model using the normalized ground point elevation value (Z), image width (wn), image height (hn), and four-dimensional longitude and latitude coordinates (lon 4_n ,lat 4_n ), the image point coordinates (l, s) are used as input data, and the normalized ground point longitude and latitude coordinates (X, Y) are used as output data; Step S43: Establish a multi-layer neural network; Step S44: using a neural network to train two positioning models.

6. The large-area rapid geometric positioning method based on remote sensing images according to claim 1 is characterized by: Step S5 includes the following sub-steps: Step S51: Perform corresponding model training according to the input and output data of the training set; Step S52: Output data Y predicted using the neural network pre And output the actual true value Y train The mean square error loss between is used as the loss function, and the back propagation realizes the automatic solution of the Jacobian matrix to speed up the convergence of the training model; the mean square error regression loss is: Where n is the number of samples, X train For input data; Step S53: normalize the input data of the test set, use the test set data to test the prediction effect of the neural network model, denormalize the prediction result of the neural network, compare it with the output data of the test set, and use the mean absolute error as the evaluation index, where the denormalization formula is expressed as: date=Date(date max -date min )+date min ; Step S54: adjusting the optimizer, hidden layer, activation function and hyperparameters of the model to continuously improve the prediction performance and effect of the model; Step S55: When the evaluation index is less than the threshold, stop training.

7. The large-area rapid geometric positioning method based on remote sensing images according to claim 1 is characterized by: In step S6, if the model trained in step S5 is integrated and applied to an on-orbit satellite, the trained model needs to be embedded in the satellite chip in advance. When the satellite is in orbit, the four-dimensional longitude and latitude coordinates are calculated using a rigorous imaging model, the width and height data of the image are obtained, the coordinates to be calculated are determined, the data are normalized, and the corresponding positioning model is called to predict the coordinates of the point to be calculated. The obtained data is denormalized to obtain the desired coordinate value; If the model trained in step S5 is integrated and applied to the ground station, the model can be directly called for coordinate prediction, and the four-axis coordinates are obtained through the image metadata file.

Images