Neural-radiance-field-based satellite image rational polynomial coefficient optimization method and device

Abstract

A satellite image rational polynomial coefficient optimization method and device based on neural radiation field, comprising: preprocessing satellite images to improve data quality; using a self-attention mechanism to establish a long-distance dependence relationship model between images to enhance the accuracy of three-dimensional reconstruction; point sampling the RPC model of the image to generate initial RPC parameters; applying NeRF technology to combine multiple images for three-dimensional reconstruction to restore the geometric relationship between images and generate the radiation value of the three-dimensional scene; based on the point cloud data obtained by reconstruction, an optimization objective function is constructed and the RPC coefficient is iteratively updated to optimize the RPC parameters of the image. This method does not need to rely on ground control points, effectively overcomes the problem of difficult matching of large geometric distortion images, improves the accuracy of the recovery of the spatial relationship between images, and provides a robust and flexible solution for high-precision geometric positioning of remote sensing images.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of satellite remote sensing data processing, and in particular to a satellite image rational polynomial coefficient optimization method and device based on neural radiation field. BACKGROUND

[0002] In satellite remote sensing image geometric positioning, Rational Polynomial Coefficients (RPC) parameters, as a widely used geometric model, approximate the geometric deformation of the image through a set of rational polynomial coefficients. In the prior art, the optimization of RPC parameters mainly relies on control points, which provide accurate geographic coordinates to help optimize the geometric accuracy of the image.

[0003] Existing RPC parameter optimization algorithms are generally divided into least squares adjustment methods based on control points and optimization methods based on regional network adjustment. Traditional RPC optimization methods mainly rely on the observation values of control points to calculate the optimization solution of RPC parameters, thereby improving the geometric accuracy of the image. However, with the continuous development and progress of satellite remote sensing technology, the imaging system of remote sensing satellites is gradually becoming more complex, and the application of satellite images is gradually becoming more diversified, which poses higher challenges to the accurate positioning of satellite images. The applicability and robustness of traditional RPC parameter optimization methods are insufficient in new applications, especially for satellite images with large geometric distortion, which makes it difficult for these traditional methods to meet the high precision requirements.

[0004] Existing optimization methods based on control points have certain limitations, especially in the case of large distortion images. When the target image cannot obtain enough control points, the optimization of RPC parameters becomes difficult. At this time, how to efficiently process satellite images, overcome the matching difficulties caused by the geometric differences between images, and achieve the RPC precision improvement of satellite images has become a technical difficulty that needs to be solved.

[0005] Traditional RPC optimization methods usually rely on ground control points (GCPs) for parameter optimization to improve geometric positioning accuracy. The RPC parameter optimization of the prior art generally follows the following basic process:

[0006] (1) Control point extraction and matching: compare the image with known points (control points) in the ground coordinate system to extract internal and external geometric information of the image.

[0007] (2) RPC parameter optimization: based on the extracted control points, solve the RPC parameters through least squares method or other adjustment algorithms.

[0008] (3) Geometric correction: perform geometric correction on the image through the optimized RPC parameters to improve the positioning accuracy.

[0009] The existing RPC parameter optimization method has the following shortcomings:

[0010] Strong dependence on control points: Traditional RPC optimization methods require a large number of ground control points. Some satellite images cover a large area and may cover both land and ocean regions, requiring more control points and making it more difficult to obtain them.

[0011] Matching accuracy problem: Traditional RPC optimization methods rely on matching satellite remote sensing images with control points. However, most satellite images today have large angles and large distortions, and a single image often cannot match enough control points, making RPC parameter optimization difficult.

[0012] Insufficient recovery accuracy of spatial relationships between images: Based on the original satellite image RPC parameters, it is often difficult to accurately recover the spatial relationships between multiple images. The spatial relationship between images serves as the initial value input in the traditional RPC optimization process. When the accuracy of this original value is too low, the traditional RPC optimization algorithm may have over-parameterization during iteration and is unstable.

[0013] The RPC parameter optimization method in the prior art, although widely used in satellite image geometric positioning and correction, has significant limitations and defects when dealing with new imaging modes. In particular, for satellite images with large geometric distortion, traditional methods cannot provide sufficient accuracy and robustness. The shortcomings of the prior art are analyzed as follows:

[0014] 1. Strong dependence on control points

[0015] Existing RPC optimization methods mainly rely on ground control points (GCPs) for image geometric correction and positioning accuracy improvement. However, with the development of satellite remote sensing technology, the coverage area of a single satellite image is becoming larger and larger, and it may cover different regions such as land and sea. This situation makes the distribution of control points very uneven, especially in the ocean or remote areas, making it extremely difficult to obtain high-precision ground control points. In addition, traditional methods require a large number of control points, and if the number and quality of control points are insufficient, the accuracy and effectiveness of RPC optimization will be greatly reduced. Therefore, when there is a lack of sufficient control points, the optimization results of the prior art often cannot meet the requirements of high-precision remote sensing positioning.

[0016] 2. Matching accuracy problem

[0017] Geometric distortion is a persistent problem in the process of satellite image geometric positioning. The vibration and attitude change of remote sensing platform on the satellite will cause geometric distortion of the image, especially in the edge area of the image, the distortion is more serious. The geometric characteristics of the image make it very difficult for the traditional RPC optimization method to rely on the matching between the image and the control points. The traditional method usually relies on the matching of feature points in the image and the control points for geometric positioning, but in practice, when the satellite image has large inclination and distortion, the feature points in the image are often difficult to accurately match with the ground control points. Therefore, using the traditional matching method on these images will cause a significant reduction in matching accuracy, which will affect the calculation of RPC parameters in the optimization process and further exacerbate the geometric positioning error.

[0018] 3. Insufficient recovery accuracy of spatial relationship between images

[0019] The existing RPC optimization method usually assumes that the spatial relationship between images can be accurately recovered from the control points. However, due to the complex geometric distortion of satellite images, the accuracy of the spatial relationship between images recovered based on traditional methods is often not high. This makes the initial spatial relationship error directly affect the final RPC parameter solution when the traditional RPC optimization method is iteratively calculated. In the iteration process, if the accuracy of the initial spatial relationship is low, the traditional method often leads to the phenomenon of "over-parameterization", that is, the solution process is unstable and the solution does not converge, so that an effective optimization solution cannot be obtained. The inaccurate recovery of the spatial relationship between images further affects the optimization effect of the RPC parameters, resulting in the optimized RPC parameters failing to meet the demand of high-precision geometric positioning.

[0020] It should be noted that the information disclosed in the above background section is only for understanding the background of the present application, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0021] The main purpose of the present application is to overcome the defects existing in the background art, and to provide a satellite image rational polynomial coefficient optimization method and device based on neural radiation field.

[0022] To achieve the above purpose, the present application adopts the following technical solutions:

[0023] A satellite image rational polynomial coefficient optimization method based on neural radiation field, comprising the following steps:

[0024] S1, image preprocessing: pre-processing the satellite multi-image to obtain image data with improved quality;

[0025] S2, self-attention mechanism modeling: using the self-attention mechanism to capture the long-distance dependence relationship between the preprocessed images, capturing long-distance geometric information by calculating the dependence relationship between points in the image, and improving the accuracy of three-dimensional reconstruction;

[0026] S3, point sampling of the RPC model of the satellite image: using the dependence relationship obtained in step S2, replacing the RPC model of each input view with a simplified pinhole camera matrix to generate the initialized RPC parameters, providing basic data for three-dimensional space reconstruction;

[0027] S4, restoring the three-dimensional relationship between images based on neural radiation field: using the three-dimensional coordinate data obtained by point sampling of the RPC model of the satellite image in step S3, combining the neural radiation field (NeRF) technology, restoring the geometric relationship between images through three-dimensional reconstruction of multiple images, and generating the radiation value of the three-dimensional scene through volume rendering method;

[0028] S5, RPC parameter optimization solution: based on the dense point cloud data obtained by the neural radiation field reconstruction in step S4, providing accurate three-dimensional information for RPC parameter optimization solution, constructing an optimization objective function, updating the RPC coefficients through least square method iteration, optimizing the rational polynomial coefficients of the image, and improving the geometric positioning accuracy.

[0029] A computer readable storage medium stores a computer program, the computer program is executed by a processor to realize the rational polynomial coefficient optimization method of satellite image based on neural radiation field.

[0030] A computer program product includes a computer program, the computer program is executed by a processor to realize the rational polynomial coefficient optimization method of satellite image based on neural radiation field.

[0031] A rational polynomial coefficient optimization device of satellite image based on neural radiation field, comprising:

[0032] A memory for storing a computer program;

[0033] A processor for executing the computer program to realize the rational polynomial coefficient optimization method of satellite image based on neural radiation field.

[0034] The present application has the following advantages:

[0035] The application provides a satellite image rational polynomial coefficient optimization method based on a neural radiation field, effectively reduces the dependence on ground control points, improves image matching accuracy, and overcomes the precision bottleneck of traditional RPC optimization methods in satellite images. In the traditional RPC optimization method, the high dependence on ground control points limits its application in large areas or remote areas where control points are difficult to obtain. The application effectively overcomes this limitation by introducing an optimization method based on the three-dimensional geometric relationship between images, enabling high-precision RPC parameter optimization even in the absence of control points. The method not only improves the stability of the optimization process, but also provides reliable optimization results without sufficient control points, making it highly valuable for satellite images covering a wide area. In the application, the combination of deep learning and neural radiation field (NeRF) technology significantly improves the spatial relationship recovery quality between images and the robustness and adaptability of the algorithm. Traditional RPC optimization methods often result in unstable optimization results when dealing with satellite images with large geometric distortion due to inaccurate initial spatial relationships. The application achieves joint optimization between multiple images through high-precision three-dimensional reconstruction technology, significantly improving the accuracy of spatial relationship recovery. This method not only improves the effectiveness of RPC parameter optimization, but also generates accurate three-dimensional point cloud information through neural networks, enabling the algorithm to achieve high geometric accuracy even without sufficient control points, thereby enhancing the robustness and adaptability of the algorithm when facing various complex images.

[0036] Overall, compared with the prior art, the application has the following significant technical advantages:

[0037] Reduced dependence on control points: Traditional RPC optimization methods highly depend on ground control points, making it difficult to effectively optimize in the absence of control points. The application eliminates the strong dependence on traditional control points by introducing the geometric relationship between images, which is of great practical value for satellite images covering large areas and difficult to obtain control points.

[0038] Higher spatial relationship recovery quality: Traditional methods have initial error problems in spatial relationship recovery between images, which may lead to non-convergent optimization results. Through high-precision three-dimensional reconstruction technology based on neural radiation field, joint optimization between multiple images is achieved, significantly improving the accuracy of spatial relationship recovery and improving the effectiveness of RPC parameter optimization.

[0039] Enhanced robustness and adaptability of the algorithm: In the face of geometric distortion of satellite images, the application combines joint processing of multiple images to generate accurate three-dimensional point cloud information using neural networks, achieving high geometric accuracy even without sufficient control points, thereby improving the robustness and adaptability of the algorithm on various complex images.

[0040] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a general flowchart of optimizing rational polynomial coefficients of satellite images based on neural radiation fields according to an embodiment of the present invention.

[0042] Figure 2 Schematic diagram of three-dimensional reconstruction of satellite images based on neural radiation fields according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope of the present invention and its application.

[0044] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, connection can be used for both fixing and coupling or communication.

[0045] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention.

[0046] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0047] In view of the above shortcomings in the prior art, the present invention aims to solve the following technical problems by introducing new technical means:

[0048] 1. Eliminate the dependence on control points: traditional RPC optimization methods rely heavily on ground control points, especially in the absence of control points, the optimization process is difficult to stabilize. The present application introduces an optimization method based on the three-dimensional geometric relationship between images, eliminating the dependence on control points, so that high-precision RPC parameter optimization can still be achieved when there are not enough control points.

[0049] 2. Improve the accuracy of image registration and matching: traditional methods have great difficulty in the registration and matching of large distortion images, especially in complex satellite imaging modes. The present application introduces an image registration method based on deep learning, combined with self-attention mechanism and neural radiation field (NeRF) technology, which can more accurately restore the spatial relationship between images, thereby improving the geometric matching accuracy between images and overcoming the influence of large distortion.

[0050] 3. Improve the accuracy of spatial relationship recovery between images: in the process of traditional method of spatial relationship recovery between images, due to the inaccuracy of initial spatial relationship, the optimization result is unstable. The present application improves the accuracy of spatial relationship recovery between images by high-precision three-dimensional reconstruction method based on the joint optimization of spatial relationship of multiple images, which provides more reliable initial value for subsequent RPC optimization.

[0051] Reference Figure 1 , the embodiment of the present application provides a satellite image rational polynomial coefficient optimization method based on neural radiation field, comprising the following steps:

[0052] Step S1, image preprocessing: pre-processing satellite multi-image to obtain image data with improved quality; providing high-quality input data for self-attention mechanism modeling.

[0053] In some embodiments, step S1 specifically comprises:

[0054] Noise removal: by applying filtering algorithm to reduce random noise in image and improve signal-to-noise ratio of image;

[0055] Geometric correction: using transformation matrix to perform geometric transformation on image to correct geometric distortion of image;

[0056] Radiometric correction: through standardization processing method, uniform the radiation brightness between different images to eliminate the brightness inconsistency caused by sensor difference and climate change.

[0057] Step S2, self-attention mechanism modeling: using self-attention mechanism to capture the long-distance dependence relationship between the pre-processed images, calculating the dependence relationship between each point in the image to capture long-distance geometric information, and improving the accuracy of three-dimensional reconstruction.

[0058] In some embodiments, step S2 specifically comprises: performing feature extraction on the input images to generate feature maps representing key information in the images; generating a query matrix Q and a key matrix K based on the feature maps for calculating the dependency relationship between points in the images; obtaining attention weights A reflecting the relevance between features by calculating the dot product of the query matrix Q and the key matrix K and applying a softmax function; and weighting the feature maps using the attention weights A to highlight important regions in the images and improve the accuracy of three-dimensional reconstruction.

[0059] Step S3, point sampling of the RPC model of the satellite image: using the dependency relationship obtained in step S2, replacing the RPC model of each input view with a simplified pinhole camera matrix to generate initialized RPC parameters to provide basic data for three-dimensional space reconstruction.

[0060] In some embodiments, step S3 specifically comprises: replacing the RPC model of each input view with a simplified pinhole camera matrix to adapt to the input requirements of the neural radiance field model; defining the minimum and maximum height of the scene to provide height limits for point sampling; determining the position of the pixel points in the image in the three-dimensional space using the RPC positioning function to convert two-dimensional pixel coordinates into three-dimensional coordinates; and converting the three-dimensional points obtained through the positioning function into a fixed coordinate system centered on the Earth to provide standardized coordinate data for subsequent three-dimensional space reconstruction.

[0061] Step S4, restoring the three-dimensional relationship between images based on the neural radiance field: using the three-dimensional coordinate data obtained by point sampling of the RPC model of the satellite image in step S3, combining the neural radiance field (NeRF) technology, and through three-dimensional reconstruction of multiple images, the geometric relationship between the images is restored, and the radiance values of the three-dimensional scene are generated through volume rendering method.

[0062] In some embodiments, step S4 specifically comprises: using the neural radiance field (NeRF) technology to integrate multi-view information of multiple images to generate a three-dimensional point cloud to restore the geometric relationship between the images; using a volume rendering method to generate radiance values of a three-dimensional scene, representing a 3D scene's radiance field through a deep neural network, given camera poses and pixel ray directions, generating radiance values of a three-dimensional scene; adding shadows by darkening albedo, calculating shadow scalars to simulate the effect of environmental irradiance on shadows; adding a regularization term for the sun's rays to minimize loss, using the geometric rules encoded by transmittance and opacity to supervise the learning of shadow scalars, ensuring that non-occluded and non-shadow regions are mainly explained by albedo.

[0063] Figure 2The structure of the NeRF network in the preferred embodiment is shown, in which the input point coordinates and ray directions are processed through multiple fully connected layers to predict the volume density, albedo and shadow scalar, which are then used in the volume rendering process to generate the radiance values of the three-dimensional scene.

[0064] Step S5, RPC parameter optimization solving: based on the dense point cloud data obtained by the neural radiance field reconstruction in step S4, accurate three-dimensional information is provided for RPC parameter optimization solving, an optimization objective function is constructed, RPC coefficients are iteratively updated by least squares method, rational polynomial coefficients of the image are optimized, and the geometric positioning accuracy is improved.

[0065] In some embodiments, step S5 specifically comprises: using the geometric distortion information of the image and the dense point cloud data obtained by the neural radiance field reconstruction, constructing an optimization objective function containing image geometric error and three-dimensional point cloud and image matching error; by minimizing the error in the optimization objective function, the geometric positioning accuracy of the image is improved; RPC coefficients are iteratively updated by least squares method to better fit the geometric deformation of the image; the optimization objective is to minimize the projection error so that the result of projecting a point in three-dimensional space to the image plane through the RPC model matches the actual pixel position in the image as much as possible.

[0066] Referring to Figure 1 , in further preferred embodiments, the method further comprises the following steps:

[0067] Step S6, error evaluation: error evaluation is performed on the optimized RPC parameters to verify the accuracy and robustness of the optimization result.

[0068] The satellite image rational polynomial coefficient optimization method based on neural radiance field provided by the present application uses the neural radiance field method to process multiple satellite images, optimizes the rational polynomial coefficients thereof, and improves the positioning accuracy of the images. By using the neural radiance field method based on deep learning and introducing the self-attention mechanism, the dependence on traditional control points is eliminated, the image matching accuracy is improved, and the precision bottleneck of the traditional RPC optimization method in satellite images is overcome, thereby providing a more robust and flexible solution for high-precision geometric positioning of remote sensing images.

[0069] The following further describes specific embodiments of the present application and examples of algorithm implementation thereof.

[0070] As Figure 1 shown, a satellite image rational polynomial coefficient optimization method based on neural radiance field comprises the following specific steps:

[0071] 1. Image preprocessing;

[0072] Satellite images are processed for noise removal, geometric correction and radiometric correction. Satellite images often contain stray noise caused by sensor noise, climate change, etc., which can affect the accuracy of subsequent image processing. Therefore, denoising is the first step to ensure image quality. Common denoising methods include Gaussian filtering, wavelet transform, etc. The denoising method used in the present application is Gaussian filtering, and the formula for Gaussian filtering is:

[0073]

[0074] wherein, represents the original image, is the Gaussian filter kernel, is the standard deviation, is the filter radius.

[0075] Geometric correction aims to eliminate the geometric distortion of the image caused by changes in the attitude of the satellite platform, rotation of the scanning mirror and other factors. The geometric correction method usually uses image registration technology, which matches the pixels in the image with the ground coordinates to correct the geometric distortion of the image. If the image pixel coordinates are and the three-dimensional coordinates of the ground control points are , the basic transformation of geometric correction is:

[0076]

[0077] wherein, is the transformation matrix, which usually uses affine transformation or perspective transformation to correct the image geometry.

[0078] The purpose of radiometric correction is to eliminate the inconsistency of brightness between different images caused by sensor differences, climate changes, etc. Common radiometric correction methods include brightness normalization and radiative transfer model correction. Since the biggest difficulty affecting satellite image matching is the inconsistency of radiometric differences, the present application mainly performs brightness normalization on it:

[0079]

[0080] wherein, and are the mean and standard deviation of the image to be corrected, and are the mean and standard deviation of the reference image.

[0081] 2. Self-attention mechanism modeling;

[0082] The self-attention mechanism can effectively capture the long-distance dependency relationship between images, especially when the geometric distortion of the image is large, it can improve the accuracy of three-dimensional reconstruction by focusing on important areas in the image.

[0083] The self-attention mechanism captures long-range geometric information by calculating the dependencies between points in the image. Feature map , the self-attention mechanism generates two matrices: the query matrix and bond matrix , the calculation formula is:

[0084]

[0085] in, is the parameter matrix obtained through learning.

[0086] By calculating the dot product of the query and the key, we can get the output of the self-attention mechanism, that is, the attention weight :

[0087]

[0088] in, The self-attention mechanism learns the correlation between features in this way, and then weights the image features in the subsequent process of restoring the 3D relationship between images, thereby improving the accuracy of geometric structure restoration.

[0089] 3. Perform point sampling on the RPC model of satellite images;

[0090] To use the RPC model of satellite imagery in Nerf, the RPC model of each input view is replaced by a simplified pinhole camera matrix. The minimum and maximum heights of the scene are represented as and . Go through the scene and RPC positioning function of an image Pixel Positioned in and Obtained from:

[0091]

[0092] Among them, the sub-index Represents the positioning function The returned 3D points are transformed into a fixed coordinate system centered on the Earth.

[0093] 4. Restore the three-dimensional relationship between images based on neural radiation fields;

[0094] Neural Radiance Fields (Nerf) is based on convolutional neural network and deep learning framework, which uses multi-view information of images to generate three-dimensional point cloud, so as to restore the geometric relationship between images. The core idea of Nerf is to link the two-dimensional projection of each image with the three-dimensional structure of the scene, and through the three-dimensional reconstruction of multiple images, a dense three-dimensional point cloud can be obtained. Nerf uses a deep neural network to represent the radiance field of a 3D scene. Given the pose of the camera and the ray direction of each pixel, Nerf network generates the radiance value of the three-dimensional scene through volume rendering method. The rendering formula of Nerf is:

[0095]

[0096] where, is a three-dimensional vector that saves the coordinates of the points in the scene voxel;

[0097] is a three-dimensional vector that encodes the direction of the sun ray. For the input remote sensing image, can be extracted from the azimuth and elevation angles, which represents the position of the sun in the remote sensing image metadata;

[0098] is the function learned from the image index The goal of dimensional embedding vector, is to express the transient elements in the th view;

[0099] is the albedo;

[0100] is the shadow scalar, which takes value (0, 1), and adds shadow by darkening the albedo;

[0101] represents the irradiance of the environment, which is related to and specific date conditions (such as weather, seasonal changes).

[0102] In satellite remote sensing images, the invisible direction of the sun ray will produce false images for the shadow scalar Therefore, an additional regularization term for the sun ray is needed to minimize the loss:

[0103]

[0104] where, is the second batch of sun corrected rays, the rays in follow the direction of the sun ray , while in the color loss main term of Nerf, they follow the observation direction of the camera; is the transmittance, for opacity, i.e., the aforementioned shadow scalar.

[0105] The regularization term for the sun rays uses geometric rules encoded by transmittance and opacity to further supervise the learning of the shadow scalar. The first part enforces that the predicted by the model at the point of each ray in should be close to i.e., high values before reaching the visible surface and low values after reaching the visible surface (both and take values between 0 and 1). The second part encourages the integral to be close to 1, because non-occluded and non-shadowed areas must be mainly explained by albedo.

[0106] 5. Based on the three-dimensional relationship between images, the RPC parameters are optimized;

[0107] The RPC parameter optimization is to adjust the rational polynomial coefficients of the image to improve the geometric positioning accuracy.

[0108] Firstly, according to the geometric distortion of the image and the dense point cloud data obtained by the neural radiation field reconstruction, an optimization objective function is constructed, which includes the image geometric error and the matching error between the three-dimensional point cloud and the image, the purpose is to minimize these errors to improve the positioning accuracy. Through the least square method, the RPC coefficients are iteratively updated, so that the final RPC parameters can better fit the geometric deformation of the image. The optimization goal is to minimize the projection error, that is, to project the points in the three-dimensional space to the image plane through the RPC model, so that the projection result matches the actual pixel position in the image as much as possible.

[0109] 6. The optimization result is verified, the error is evaluated, and the optimized RPC parameters are obtained.

[0110] In summary, the present application designs a satellite image RPC optimization method based on neural radiation field, which solves the challenges faced by traditional RPC optimization methods under new satellite imaging modes, especially when dealing with satellite images with large geometric distortion, significantly improving its adaptability and robustness. The present application proposes a RPC parameter optimization method without control points, which is more suitable for satellite images with more complex imaging characteristics. In the case of fewer available and matched control points, the present application uses deep learning technology, combined with self-attention mechanism and neural radiation field (NeRF) technology, to realize RPC parameter optimization without control points.

[0111] Compared with the prior art, the present application has the following obvious technical advantages:

[0112] Reduce dependence on control points: Traditional RPC optimization methods highly depend on ground control points, making it difficult to perform effective optimization in the case of insufficient control points. By introducing inter-image geometric relationships, the present application eliminates the strong dependence on traditional control points, which is of great practical value for satellite images covering large areas and difficult to obtain control points.

[0113] Higher spatial relationship recovery quality: Traditional methods have the problem of initial error in the recovery of spatial relationship between images, which may lead to non-convergence of the optimization results. By high-precision neural radiation field-based three-dimensional reconstruction technology, joint optimization between multiple images is realized, which greatly improves the accuracy of spatial relationship recovery and improves the effect of RPC parameter optimization.

[0114] Enhance the robustness and adaptability of the algorithm: In the face of geometric distortion of satellite images, the present application combines the joint processing of multiple images to generate accurate three-dimensional point cloud information using neural networks, achieving high geometric accuracy without sufficient control points, thereby improving the robustness and adaptability of the algorithm on various complex images.

[0115] The embodiment of the present application also provides a storage medium for storing a computer program, which is executed to perform at least the method described above.

[0116] The embodiment of the present application also provides a control device, which includes a processor and a storage medium for storing a computer program; wherein the processor is used to execute the computer program to perform at least the method described above.

[0117] The embodiment of the present application also provides a processor, which executes a computer program to perform at least the method described above.

[0118] The storage medium can be implemented by any type of nonvolatile storage device, or a combination thereof. The nonvolatile memory can be a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Ferromagnetic Random Access Memory (FRAM), a Flash memory, a magnetic surface storage, an optical disc, or a Compact Disc Read-Only Memory (CD-ROM). The magnetic surface storage can be a magnetic disc memory or a magnetic tape memory. The storage medium described in the embodiments of the present application is intended to include, but is not limited to, these and any other suitable type of memory.

[0119] In several embodiments provided by the present application, it should be understood that the disclosed system and method can be implemented in other manners. The described device embodiments are merely schematic, and the division of the units is merely a logical function division. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling, or direct coupling or communication connection between the components can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or in other forms.

[0120] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0121] In addition, each functional unit in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in the form of hardware, or in the form of hardware plus software functional units.

[0122] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program performs the steps of the above-mentioned method embodiments when executed; and the foregoing storage medium includes a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disc or an optical disc and various storage medium capable of storing program codes.

[0123] Alternatively, the integrated unit of the present application can be stored in a computer readable storage medium if it is realized in the form of a software function module and sold or used as an independent product. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in the embodiments of the present application. The foregoing storage medium includes a mobile storage device, a ROM, a RAM, a magnetic disc or an optical disc and various storage medium capable of storing program codes.

[0124] The methods disclosed in the several method embodiments of the present application can be combined arbitrarily without conflict to obtain new method embodiments.

[0125] The features disclosed in the several product embodiments of the present application can be combined arbitrarily without conflict to obtain new product embodiments.

[0126] The features disclosed in the several method or device embodiments of the present application can be combined arbitrarily without conflict to obtain new method or device embodiments.

[0127] The above is a further detailed description of the present application in combination with specific preferred embodiments, and the specific implementation of the present application cannot be limited to these descriptions. For those skilled in the art, without departing from the concept of the present application, a number of equivalent substitutions or obvious modifications can be made, and the performance or use is the same, which should be regarded as belonging to the protection scope of the present application.

Claims

1. A method for optimizing satellite image rational polynomial coefficients based on neural radiation field, characterized in that: The following steps are involved: S1. Image preprocessing: Preprocess multiple satellite images to obtain image data with improved quality; S2. Self-attention mechanism modeling: The self-attention mechanism is used to capture long-range dependencies between pre-processed images. By calculating the dependencies between points in the image, long-range geometric information is captured to improve the accuracy of 3D reconstruction. S3. Perform point sampling on the rational polynomial coefficient RPC model of the satellite image: Using the dependency relationship modeled in step S2, the RPC model of each input view is replaced with a simplified pinhole camera matrix to generate initialized RPC parameters, providing basic data for 3D space reconstruction. S4. Restoring the 3D relationship between images based on Neural Radiance Field: Using the 3D coordinate data obtained by point sampling the RPC model of the satellite image in step S3, combined with Neural Radiance Field (NeRF) technology, the geometric relationship between the images is restored through 3D reconstruction of multiple images, and the radiance value of the 3D scene is generated through volume rendering. S5, RPC parameter optimization solution: Based on the dense point cloud data obtained by neural radiation field reconstruction in step S4, provide accurate three-dimensional information for RPC parameter optimization solution, construct the optimization objective function, iteratively update the RPC coefficients through the least squares method, optimize the rational polynomial coefficients of the image, and improve the geometric positioning accuracy; Step S4 specifically includes: Neural Radiance Field (NeRF) technology is used to integrate multi-view information from multiple images to generate a 3D point cloud to restore the geometric relationship between images. The volume rendering method is used to generate the radiation value of the 3D scene. The radiation field of the 3D scene is represented by a deep neural network. Given the camera pose and pixel ray direction, the radiation value of the 3D scene is generated. Add shadows by darkening the albedo and calculating the shadow scalar to simulate the effect of ambient irradiance on shadows; To minimize the loss, a regularization term for sunlight is added, and the learning of the shadow scalar is supervised using geometric rules encoded in transmittance and opacity to ensure that non-occluded and non-shadowed areas are explained by albedo; Step S5 specifically includes: Using the geometric distortion information of the image and the dense point cloud data obtained by neural radiation field reconstruction, an optimization objective function is constructed that includes the image geometric error and the matching error between the 3D point cloud and the image; Improve the geometric positioning accuracy of the image by minimizing the error in the optimization objective function; The RPC coefficients are iteratively updated using the least squares method to better fit the geometric deformation of the image; The goal of optimization is to minimize the projection error so that the result of projecting a point in three-dimensional space onto the image plane through the RPC model matches the actual pixel position in the image as much as possible.

2. The method according to claim 1, characterized in that The following steps are also included: S6. Error evaluation: Perform error evaluation on the optimized RPC parameters to verify the accuracy and robustness of the optimization results.

3. The method according to claim 1, characterized in that Step S1 specifically includes: Noise removal: By applying filtering algorithms to reduce random noise in the image, the signal-to-noise ratio of the image can be improved; Geometric correction: Use the transformation matrix to perform geometric transformation on the image to correct the geometric distortion of the image; Radiometric correction: Through standardized processing methods, the radiometric brightness between different images is unified to eliminate brightness inconsistencies caused by sensor differences and climate change.

4. The method according to claim 1, wherein Step S2 specifically includes: Extract features from the input image and generate a feature map to represent the key information in the image; Generate query matrix Q and key matrix K based on feature map to calculate the dependency relationship between points in the image; By calculating the dot product of the query matrix Q and the key matrix K and applying the softmax function, we get the attention weight A, which reflects the correlation between features. The feature map is weighted using the attention weight A to highlight important areas in the image and improve the accuracy of 3D reconstruction.

5. The method according to claim 1, wherein Step S3 specifically includes: The RPC model of each input view is replaced with a simplified pinhole camera matrix to adapt to the input requirements of the neural radiance field model; Define the minimum and maximum height of the scene to provide height limits for point sampling; Use the RPC positioning function to determine the position of the pixel points in the image in three-dimensional space and convert the two-dimensional pixel coordinates into three-dimensional coordinates; The three-dimensional points obtained by the positioning function are converted into a fixed coordinate system centered on the earth, providing standardized coordinate data for subsequent three-dimensional space reconstruction.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for optimizing satellite image rational polynomial coefficients based on neural radiation field according to any one of claims 1 to 5 is implemented.

7. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for optimizing satellite image rational polynomial coefficients based on neural radiation field according to any one of claims 1 to 5 is implemented.

8. A device for optimizing satellite image rational polynomial coefficients based on neural radiation field, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the method for optimizing satellite image rational polynomial coefficients based on neural radiation fields as described in any one of claims 1 to 5 when executing the computer program.

Images