Satellite image rational polynomial coefficient optimization method and device based on neural radiation field

Abstract

The invention discloses a neural radiation field-based satellite image rational polynomial coefficient optimization method and device. The method comprises the following steps of: preprocessing a satellite image to improve data quality; a long-distance dependency relationship model between images is established by using a self-attention mechanism, and the accuracy of three-dimensional reconstruction is enhanced; performing point sampling on the RPC model of the image to generate an initialized RPC parameter; performing three-dimensional reconstruction by combining a NeRF technology with a plurality of images, recovering a geometrical relationship among the images, and generating a radiation value of a three-dimensional scene; and based on the reconstructed point cloud data, optimizing the RPC parameters of the image by constructing an optimization objective function and iteratively updating the RPC coefficient. The method does not need to depend on ground control points, the problem that large geometric distortion images are difficult to match is effectively solved, the precision of spatial relation recovery between the images is improved, and a robust and flexible solution is provided for high-precision geometric positioning of remote sensing images.

Description

Technical Field

[0001] The present invention relates to the field of satellite remote sensing data processing, and in particular to a method and device for optimizing satellite image rational polynomial coefficients based on neural radiation fields. Background Art

[0002] In the geometric positioning of satellite remote sensing images, rational polynomial coefficients (RPC) parameters are widely used as a geometric model to approximate the geometric deformation of images through a set of rational polynomial coefficients. In the existing technology, the optimization of RPC parameters mainly depends on control points, which provide precise geographic coordinates to help optimize the geometric accuracy of images.

[0003] The existing RPC parameter optimization algorithms are generally divided into the least squares adjustment method based on control points and the optimization method based on regional network adjustment. The traditional RPC optimization method mainly relies on the observation values of control points to calculate the optimal solution of RPC parameters, thereby improving the geometric accuracy of the image. However, with the continuous development and progress of satellite remote sensing technology, the imaging system of remote sensing satellites has gradually become more complex, and the application of satellite images has gradually diversified, which has posed higher challenges to the precise positioning of satellite images. The applicability and robustness of traditional RPC parameter optimization methods are insufficient in new applications, especially for satellite images with large geometric distortion. These traditional methods are difficult to meet high-precision requirements.

[0004] Existing optimization methods based on control points have certain limitations, especially in the case of large distortion images. When the target image cannot obtain enough control points, the optimization of RPC parameters becomes difficult. At this time, how to efficiently process satellite images, overcome the matching difficulties caused by geometric differences between images, and improve the RPC accuracy of satellite images has become a technical difficulty that needs to be solved urgently.

[0005] Traditional RPC optimization methods usually rely on ground control points (GCPs) for parameter optimization to improve geometric positioning accuracy. The existing RPC parameter optimization generally follows the following basic process: (1) Control point extraction and matching: The internal and external geometric information of the image is extracted by comparing the image with the known points (control points) in the ground coordinate system.

[0006] (2) RPC parameter optimization: Based on the extracted control points, the RPC parameters are solved using the least squares method or other adjustment algorithms.

[0007] (3) Geometric correction: The image is geometrically corrected through optimized RPC parameters to improve positioning accuracy.

[0008] The existing RPC parameter optimization methods have the following deficiencies: Strong dependence on control points: Traditional RPC optimization methods require a large number of ground control points. For some satellite images, the coverage area is large and may cover both land and ocean areas simultaneously, which requires an even larger number of control points and makes it more difficult to obtain them.

[0009] Matching accuracy problem: Traditional RPC optimization methods rely on the matching of satellite remote sensing images and control points. Nowadays, most satellite images have the characteristics of large inclination and large distortion, and a single image often cannot match enough control points, resulting in difficulties in optimizing RPC parameters.

[0010] Insufficient accuracy in restoring the spatial relationship between images: Based on the RPC parameters of the original satellite images, it is often difficult to accurately restore the spatial relationship between multiple images. The spatial relationship between images is used as the initial value input in the traditional RPC optimization process. When this original value has too low accuracy, the traditional RPC optimization algorithm will exhibit over-parameterization during the iteration process and has strong solution instability.

[0011] Although the RPC parameter optimization methods in the existing technology are widely used in the geometric positioning and correction of satellite images, there are significant limitations and defects when dealing with new imaging modes. Especially for satellite images with large geometric distortions, traditional methods cannot provide sufficient accuracy and robustness. The specific analysis of the disadvantages of the existing technology is as follows: 1. Strong dependence on control points The existing RPC optimization methods mainly rely on ground control points (GCPs) to improve the geometric correction and positioning accuracy of images. However, with the development of satellite remote sensing technology, the ground coverage area of a single satellite image is getting larger and may cover different regions such as land and ocean simultaneously. This situation will make the distribution of control points very uneven. Especially in ocean or remote areas, it becomes extremely difficult to obtain high-precision ground control points. In addition, traditional methods have a high demand for the number of control points. If the number and quality of control points are insufficient, the accuracy and effect of RPC optimization will be greatly reduced. Therefore, when there are insufficient control points in the existing technology, its optimization results often fail to meet the requirements of high-precision remote sensing positioning.

[0012] 2. Matching accuracy problem Geometric distortion is a persistent problem in the geometric positioning process of satellite images. Vibrations and attitude-orbit changes of the remote sensing platform on the satellite can all lead to geometric distortion of the images, especially in the edge regions of the images, where the distortion is more severe. This geometric feature of the images makes it very difficult to match the images with control points, which the traditional RPC optimization method relies on. Traditional methods usually rely on feature points in the images that match the control points for geometric positioning. However, in actual situations, when the inclination and distortion of satellite images are large, it is often very difficult to accurately match the feature points in the images with ground control points. Therefore, using traditional matching methods on these images will lead to a significant reduction in the matching accuracy, which will affect the parameter calculation in the RPC optimization process and further exacerbate the geometric positioning error.

[0013] 3. Insufficient accuracy in restoring the spatial relationship between images Existing RPC optimization methods usually assume that the spatial relationship between images can be accurately restored from control points. However, due to the often complex geometric distortion in satellite images, the accuracy of the restored spatial relationship between images based on traditional methods is often not high. This makes the initial spatial relationship error directly affect the final RPC parameter solution when traditional RPC optimization methods perform iterative calculations. During the iterative process, if the accuracy of the initial spatial relationship is low, traditional methods often lead to the phenomenon of "over-parameterization", that is, the solution process is unstable and the solution does not converge, resulting in an ineffective optimization solution. The inaccurate restoration of the spatial relationship between images further affects the effect of RPC parameter optimization, resulting in the optimized RPC parameters not meeting the requirements of high-precision geometric positioning.

[0014] It should be noted that the information disclosed in the above background art section is only used for understanding the background of the present application, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0015] The main object of the present invention is to overcome the defects existing in the above background art, and provide a rational polynomial coefficient optimization method and device for satellite images based on neural radiance fields.

[0016] To achieve the above object, the present invention adopts the following technical solutions: A rational polynomial coefficient optimization method for satellite images based on neural radiance fields, comprising the following steps: S1. Image preprocessing: Preprocess multiple satellite images to obtain image data with improved quality; S2. Self-attention mechanism modeling: Use the self-attention mechanism to capture the long-range dependence relationship between the preprocessed images, and capture the long-range geometric information by calculating the dependence relationship between points in the images, so as to improve the accuracy of three-dimensional reconstruction; S3. Perform point sampling on the RPC model of satellite images: Utilize the dependency relationships obtained in the modeling in step S2 to replace the RPC model of each input view with a simplified pinhole camera matrix, generate the initialized RPC parameters, and provide basic data for 3D space reconstruction; S4. Restore the 3D relationships between images based on the neural radiance field: Utilize the 3D coordinate data obtained by performing point sampling on the RPC model of satellite images in step S3, combine with the neural radiance field (NeRF) technology, and through the 3D reconstruction of multiple images, restore the geometric relationships between images, and generate the radiance values of the 3D scene through volume rendering methods; S5. Optimize and solve the RPC parameters: Based on the dense point cloud data reconstructed by the neural radiance field in step S4, provide accurate 3D information for optimizing and solving the RPC parameters, construct an optimization objective function, iteratively update the RPC coefficients through the least squares method, optimize the rational polynomial coefficients of the images, and improve the geometric positioning accuracy.

[0017] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the method for optimizing the rational polynomial coefficients of satellite images based on the neural radiance field.

[0018] A computer program product includes a computer program, and when the computer program is executed by a processor, it implements the method for optimizing the rational polynomial coefficients of satellite images based on the neural radiance field.

[0019] An apparatus for optimizing the rational polynomial coefficients of satellite images based on the neural radiance field includes: A memory for storing a computer program; A processor for implementing the method for optimizing the rational polynomial coefficients of satellite images based on the neural radiance field when executing the computer program.

[0020] The present invention has the following beneficial effects: The present invention proposes a rational polynomial coefficient optimization method for satellite images based on neural radiance fields, which effectively reduces the dependence on ground control points, improves the image matching accuracy, and overcomes the accuracy bottleneck of traditional RPC optimization methods in satellite images. In traditional RPC optimization methods, the high dependence on ground control points limits their application in large areas or remote regions where it is difficult to obtain control points. By introducing an optimization method based on the three-dimensional geometric relationship between images, the present invention effectively overcomes this limitation, enabling high-precision RPC parameter optimization even in the case of insufficient control points. The method of the present invention not only improves the stability of the optimization process but also provides reliable optimization results in the absence of sufficient control points, which has important practical value for satellite images covering a wide area. In the present invention, by combining deep learning and neural radiance field (NeRF) technology, the quality of the spatial relationship recovery between images and the robustness and adaptability of the algorithm are significantly improved. When dealing with satellite images with large geometric distortions, traditional RPC optimization methods often result in unstable optimization results due to inaccurate initial spatial relationships. The present invention realizes joint optimization between multiple images through high-precision three-dimensional reconstruction technology, thereby greatly improving the accuracy of spatial relationship recovery. This method not only improves the effect of RPC parameter optimization but also enables the algorithm to achieve high geometric accuracy in the absence of sufficient control points through the precise three-dimensional point cloud information generated by the neural network, thus enhancing the robustness and adaptability of the algorithm in the face of various complex images.

[0021] Generally speaking, compared with the prior art, the present invention has the following significant technical advantages: Reduced dependence on control points: Traditional RPC optimization methods highly rely on ground control points, making it difficult to conduct effective optimization in the case of insufficient control points. However, by introducing the geometric relationship between images, the present invention eliminates the strong dependence on traditional control points, which has important practical value for satellite images covering a large area and where it is difficult to obtain control points.

[0022] Higher quality of spatial relationship recovery: Traditional methods have problems with initial errors in the recovery of spatial relationships between images, which may lead to non-convergent optimization results. Through high-precision three-dimensional reconstruction technology based on neural radiance fields, joint optimization between multiple images is realized, greatly improving the accuracy of spatial relationship recovery, and thus improving the effect of RPC parameter optimization.

[0023] Enhanced robustness and adaptability of the algorithm: In the face of geometric distortions of satellite images, the present invention combines joint processing of multiple images, uses neural networks to generate precise three-dimensional point cloud information, and can achieve high geometric accuracy even in the absence of sufficient control points, thereby improving the robustness and adaptability of the algorithm on various complex images.

[0024] Other beneficial effects in the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is the overall flowchart of the rational polynomial coefficient optimization of satellite images based on neural radiance fields in the embodiments of the present invention.

[0026] Figure 2 It is a schematic diagram of the 3D reconstruction of satellite images based on neural radiance fields in the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] The following makes a detailed description of the embodiments of the present invention. It should be emphasized that the following description is merely exemplary and not intended to limit the scope of the present invention and its applications.

[0028] It should be noted that when an element is referred to as "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, the connection can be for a fixing function or for a coupling or communicating function.

[0029] It should be understood that the orientation or positional relationship indicated by terms such as "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the embodiments of the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be understood as a limitation to the present invention.

[0030] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present invention, "a plurality" means two or more unless otherwise specifically defined.

[0031] Aiming at the above-mentioned drawbacks in the prior art, the present invention aims to solve the following technical problems by introducing new technical means: 1. Eliminate the dependence on control points: Traditional RPC optimization methods rely heavily on ground control points. Especially in the case of a lack of control points, it is difficult to stably carry out the optimization process. The present invention eliminates the dependence on control points by introducing an optimization method based on the three-dimensional geometric relationship between images, so that high-precision RPC parameter optimization can still be achieved when there are not enough control points.

[0032] 2. Improve the accuracy of image registration and matching: Traditional methods have great difficulties in registering and matching large-distortion images, especially in complex satellite imaging modes. By introducing a deep learning-based image registration method and combining self-attention mechanism and Neural Radiance Field (NeRF) technology, the present invention can more accurately restore the spatial relationship between images, thereby improving the geometric matching accuracy between images and overcoming the influence of large distortions.

[0033] 3. Improve the accuracy of restoring the spatial relationship between images: In the process of restoring the spatial relationship between images by traditional methods, due to inaccurate initial spatial relationships, the optimization results are unstable. The present invention significantly improves the accuracy of restoring the spatial relationship between images through a high-precision 3D reconstruction method and jointly optimizes based on the spatial relationships of multiple images, providing a more reliable initial value for subsequent RPC optimization.

[0034] Refer to Figure 1 , an embodiment of the present invention provides a rational polynomial coefficient optimization method for satellite images based on neural radiance field, including the following steps: Step S1, Image preprocessing: Preprocess multiple satellite images to obtain image data with improved quality; provide high-quality input data for self-attention mechanism modeling.

[0035] In some embodiments, step S1 specifically includes: Noise removal: Apply a filtering algorithm to reduce random noise in the image and improve the signal-to-noise ratio of the image; Geometric correction: Use a transformation matrix to perform geometric transformation on the image to correct geometric distortion of the image; Radiometric correction: Through a normalization processing method, unify the radiance between different images to eliminate brightness inconsistencies caused by sensor differences and climate changes.

[0036] Step S2, Self-attention mechanism modeling: Use the self-attention mechanism to capture long-range dependence relationships between preprocessed images, capture long-range geometric information by calculating the dependence relationships between points in the image, and improve the accuracy of 3D reconstruction.

[0037] In some embodiments, step S2 specifically includes: Extract features from the input image to generate a feature map to represent key information in the image; Generate a query matrix Q and a key matrix K based on the feature map for calculating the dependence relationships between points in the image; Calculate the dot product of the query matrix Q and the key matrix K and apply the softmax function to obtain the attention weight A, which reflects the correlation between features; Use the attention weight A to weight the feature map to highlight important regions in the image and improve the accuracy of 3D reconstruction.

[0038] Step S3: Perform point sampling on the RPC model of the satellite image: Utilize the dependency relationships obtained from the modeling in Step S2 to replace the RPC model of each input view with a simplified pinhole camera matrix, generate the initialized RPC parameters, and provide the basic data for 3D space reconstruction.

[0039] In some embodiments, Step S3 specifically includes: replacing the RPC model of each input view with a simplified pinhole camera matrix to meet the input requirements of the neural radiance field model; defining the minimum and maximum heights of the scene to provide height bounds for point sampling; using the RPC positioning function to determine the positions of pixel points in the image in 3D space and convert the 2D pixel coordinates into 3D coordinates; and converting the 3D points obtained through the positioning function into a fixed coordinate system centered on the Earth to provide standardized coordinate data for subsequent 3D space reconstruction.

[0040] Step S4: Restore the 3D relationships between images based on the neural radiance field: Utilize the 3D coordinate data obtained from point sampling the RPC model of the satellite image in Step S3, combine with the neural radiance field (NeRF) technology, and through the 3D reconstruction of multiple images, restore the geometric relationships between the images, and generate the radiance values of the 3D scene through the volume rendering method.

[0041] In some embodiments, Step S4 specifically includes: integrating the multi-view information of multiple images using the neural radiance field (NeRF) technology to generate a 3D point cloud to restore the geometric relationships between the images; adopting the volume rendering method to generate the radiance values of the 3D scene, representing the radiance field of the 3D scene through a deep neural network, and given the camera pose and pixel ray direction, generating the radiance values of the 3D scene; adding shadows by darkening the albedo, calculating the shadow scalar to simulate the influence of environmental irradiance on the shadows; and to minimize the loss, adding a regularization term for the sun rays, using the geometric rules encoded by the transmittance and opacity to supervise the learning of the shadow scalar, ensuring that the non-occluded and non-shadow areas are mainly explained by the albedo.

[0042] Figure 2 Shows the structure of the NeRF network in the preferred embodiment, where the input point coordinates and ray directions are processed through multiple fully connected layers to predict the volume density, albedo, and shadow scalar, and these predicted values are then used in the volume rendering process to generate the radiance values of the 3D scene.

[0043] Step S5: Optimize and solve the RPC parameters: Based on the dense point cloud data reconstructed by the neural radiance field in Step S4, provide accurate 3D information for optimizing and solving the RPC parameters, construct an optimization objective function, and iteratively update the RPC coefficients through the least squares method to optimize the rational polynomial coefficients of the image and improve the geometric positioning accuracy.

[0044] In some embodiments, step S5 specifically includes: using the geometric distortion information of the image and the dense point cloud data reconstructed by the neural radiance field to construct an optimization objective function that includes the geometric error of the image and the matching error between the 3D point cloud and the image; by minimizing the error in the optimization objective function, improving the geometric positioning accuracy of the image; using the least squares method to iteratively update the RPC coefficients to better fit the geometric deformation of the image; the optimization goal is to minimize the projection error so that the result of projecting points in 3D space onto the image plane through the RPC model matches the actual pixel positions in the image as much as possible.

[0045] Referring to Figure 1 , in a further preferred embodiment, the method further includes the following steps: Step S6, error evaluation: perform error evaluation on the optimized RPC parameters to verify the accuracy and robustness of the optimization result.

[0046] The rational polynomial coefficient optimization method for satellite images based on neural radiance fields of the present invention uses the method of neural radiance fields to process multiple satellite images, optimizes their rational polynomial coefficients, and improves the positioning accuracy of the images. By adopting the neural radiance field method based on deep learning and introducing the self-attention mechanism, it eliminates the dependence on traditional control points, improves the image matching accuracy, and overcomes the accuracy bottleneck of traditional RPC optimization methods in satellite images, providing a more robust and flexible solution for high-precision geometric positioning of remote sensing images.

[0047] The following further describes specific embodiments of the present invention and examples of its algorithm implementation.

[0048] As Figure 1 shown, a rational polynomial coefficient optimization method for satellite images based on neural radiance fields specifically includes the following steps: 1. Image preprocessing; Perform noise removal, geometric correction, and radiometric correction on the satellite image. Satellite images often contain stray noise caused by sensor noise, climate change, etc., which will affect the accuracy of subsequent image processing. Therefore, denoising is the first step to ensure image quality. Common denoising methods include Gaussian filtering, wavelet transform, etc. The denoising method used in the present invention is Gaussian filtering, and the formula for Gaussian filtering is:

[0049] Among them, represents the original image, is the Gaussian filtering kernel, is the standard deviation, is the filtering radius.

[0050] Geometric correction aims to eliminate the geometric distortion of images caused by factors such as satellite platform attitude changes and scanning mirror rotation. Geometric correction methods usually adopt image registration technology to correct the geometric distortion of images by matching the pixels in the image with the ground coordinates. If the pixel coordinates of the image are , and the three-dimensional coordinates of the ground control point are , then the basic transformation of geometric correction is:

[0051] where, is the transformation matrix, and affine transformation or perspective transformation is usually used to correct the image geometry.

[0052] The purpose of radiometric correction is to eliminate the brightness inconsistency between different images caused by sensor differences, climate changes, etc. Common radiometric correction methods include brightness normalization and radiometric transfer model correction. Since the biggest difficulty in affecting satellite image matching is the problem of inconsistent radiometric differences, the present invention mainly performs brightness normalization on it:

[0053] where, and are the mean and standard deviation of the image to be corrected respectively, and are the mean and standard deviation of the reference image.

[0054] 2. Self-attention mechanism modeling; The self-attention mechanism can effectively capture the long-range dependence relationships between images. Especially when the geometric distortion of the image is large, it can improve the accuracy of 3D reconstruction by focusing on the important regions in the image.

[0055] The self-attention mechanism captures long-range geometric information by calculating the dependence relationships between points in the image. Given a feature map of an image , the self-attention mechanism will generate two matrices: a query matrix and a key matrix , and the calculation formula is:

[0056] where, is the parameter matrix obtained by learning.

[0057] By calculating the dot product of the query and the key, the output of the self-attention mechanism, that is, the attention weight can be obtained:

[0058] where, is the dimension of the key. The self-attention mechanism learns the correlation between features in this way, weights the image features during the subsequent restoration process of the three-dimensional relationship between images, and improves the accuracy of geometric structure restoration.

[0059] 3. Perform point sampling on the RPC model of the satellite image; For using the RPC model of the satellite image in Nerf, the RPC model of each input view is replaced by a simplified pinhole camera matrix. Denote the minimum and maximum heights of the scene as and . Pass through the scene and with the RPC positioning function of the th image locate the pixel at and to obtain:

[0060] where the sub-index represents that the 3D point returned by the positioning function is converted to a fixed coordinate system centered on the earth.

[0061] 4. Restore the three-dimensional relationship between images based on the neural radiance field; The neural radiance field (Nerf) is based on a convolutional neural network and a deep learning framework, uses the multi-view information of the images to generate a three-dimensional point cloud, so as to achieve the purpose of restoring the geometric relationship between images. The core idea of Nerf is to connect the two-dimensional projection of each image with the three-dimensional structure of the scene. Through the three-dimensional reconstruction of multiple images, a dense three-dimensional point cloud can be obtained. Nerf uses a deep neural network to represent the radiance field of the 3D scene. Given the pose of the camera and the ray direction of each pixel, the Nerf network generates the radiance value of the three-dimensional scene through the volume rendering method. The rendering formula of Nerf is:

[0062] where, is the three-dimensional vector of the coordinates of the points saved in the scene voxels; is the three-dimensional vector encoding the direction of the sun's rays. For the input remote sensing image, can be extracted from the azimuth angle and elevation angle, representing the position of the sun in the metadata of the remote sensing image; is the -dimensional embedding vector learned from the function of the image index , and the goal of is to express the transient elements in the th view; is the albedo; is the shadow scalar, with a value range of (0, 1), and it adds shadows by darkening the albedo; represents the irradiance of the environment, which is related to and specific date conditions (such as weather, seasonal changes).

[0063] In satellite remote sensing images, the direction of the sun's rays that cannot be seen will generate incorrect images for the shadow scalar Therefore, an additional regularization term for the sun's rays is needed to minimize the loss:

[0064] where is the second batch of solar correction rays. The rays in follow the direction of the sun's rays while following the viewing direction of the camera in the main term of the color loss of Nerf; is the transmittance, is the opacity, is the above-mentioned shadow scalar.

[0065] The regularization term of the sun's rays uses the geometric rules encoded by the transmittance and opacity to further supervise the learning of the shadow scalar. The first part enforces that for each ray in , the predicted at the point should approximate , that is, high values before reaching the visible surface and low values after reaching the visible surface ( and both take values between 0 and 1). The second part encourages the integral to be close to 1 because non-occluded and non-shadow areas must be mainly explained by the albedo.

[0066] 5. Optimize the RPC parameters based on the three-dimensional relationship between images; The optimization of RPC parameters is to improve the geometric positioning accuracy by adjusting the rational polynomial coefficients of the images.

[0067] First, based on the geometric distortion of the images and the dense point cloud data reconstructed by the neural radiance field, an optimization objective function is constructed. This objective function includes the geometric error of the images and the matching error between the three-dimensional point cloud and the images, aiming to minimize these errors to improve the positioning accuracy. Through the least squares method, the RPC coefficients are iteratively updated so that the final RPC parameters can better fit the geometric deformation of the images. The optimization goal is to minimize the projection error, that is, to project the points in the three-dimensional space onto the image plane through the RPC model, making the projection results match the actual pixel positions in the images as much as possible.

[0068] 6. Verify the optimization results, conduct error evaluation, and obtain the optimized RPC parameters.

[0069] In summary, the present invention designs a method for optimizing RPC of satellite images based on neural radiance fields, which solves the challenges faced by traditional RPC optimization methods in new satellite imaging modes. Especially when dealing with satellite images with large geometric distortions, it significantly improves their adaptability and robustness. The present invention proposes an RPC parameter optimization method without control points. For satellite images with more complex imaging characteristics and fewer available and matchable control points, the present invention utilizes deep learning technology, combines self-attention mechanism and neural radiance field (NeRF) technology to achieve RPC parameter optimization without control points.

[0070] Compared with the prior art, the present invention has the following remarkable technical advantages: Reduce the dependence on control points: Traditional RPC optimization methods highly rely on ground control points, making it difficult to conduct effective optimization in the case of insufficient control points. However, the present invention eliminates the strong dependence on traditional control points by introducing the geometric relationship between images, which has important practical value for satellite images covering large areas and difficult to obtain control points.

[0071] Higher quality of restoring spatial relationships: Traditional methods have problems with initial errors in restoring the spatial relationship between images, which may lead to non-convergence of the optimization results. Through the high-precision three-dimensional reconstruction technology based on neural radiance fields, joint optimization among multiple images is realized, greatly improving the accuracy of restoring spatial relationships, and thus improving the effect of RPC parameter optimization.

[0072] Enhance the robustness and adaptability of the algorithm: In the face of geometric distortions of satellite images, the present invention realizes high geometric accuracy by combining the joint processing of multiple images and using neural networks to generate accurate three-dimensional point cloud information. Even in the case of insufficient control points, the robustness and adaptability of the algorithm on various complex images can be improved.

[0073] The embodiment of the present invention also provides a storage medium for storing a computer program, which when executed, at least executes the method described above.

[0074] The embodiment of the present invention also provides a control device, including a processor and a storage medium for storing a computer program; wherein, the processor is used to execute the computer program to at least execute the method described above.

[0075] The embodiment of the present invention also provides a processor, which executes a computer program and at least executes the method described above.

[0076] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The storage medium described in the embodiments of the present invention is intended to include, but not limited to, these and any other suitable types of memories.

[0077] In several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the couplings, or direct couplings, or communication connections between the components shown or discussed with each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be electrical, mechanical, or other forms.

[0078] The units described above as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0079] In addition, each functional unit in the embodiments of the present invention can be all integrated in a processing unit, or each unit can be separately used as a unit, or two or more units can be integrated in a unit; the above integrated units can be implemented in the form of hardware, or in the form of hardware plus software functional units.

[0080] Those of ordinary skill in the art can understand that all or part of the steps to implement the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including those of the above method embodiments; and the foregoing storage medium includes: various media such as removable storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0081] Alternatively, if the above integrated units are implemented in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of the embodiments of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. And the foregoing storage medium includes: various media such as removable storage devices, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0082] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0083] The features disclosed in the several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0084] The features disclosed in the several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0085] The above content is a further detailed description of the present invention in combination with specific preferred implementation manners. It cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those skilled in the technical field to which the present invention belongs, without departing from the concept of the present invention, several equivalent substitutions or obvious variations can be made, and as long as the performance or use is the same, they should all be regarded as falling within the protection scope of the present invention.

Claims

1. A rational polynomial coefficient optimization method for satellite images based on neural radiance fields, characterized in that, It includes the following steps: S1. Image preprocessing: Preprocess multi - satellite images to obtain image data with improved quality; S2. Self - attention mechanism modeling: Use the self - attention mechanism to capture long - range dependence relationships between preprocessed images, and capture long - range geometric information by calculating the dependence relationships between points in the images, improving the accuracy of 3D reconstruction; S3. Point sampling of the RPC model of satellite images: Use the dependence relationships modeled in step S2 to replace the RPC model of each input view with a simplified pinhole camera matrix, generate initialized RPC parameters, and provide basic data for 3D space reconstruction; S4. Restore the 3D relationship between images based on neural radiance fields: Use the 3D coordinate data obtained by point sampling the RPC model of satellite images in step S3, combine with neural radiance field (NeRF) technology, and through 3D reconstruction of multiple images, restore the geometric relationship between images, and generate the radiance value of the 3D scene through volume rendering method; S5. Optimization and solution of RPC parameters: Based on the dense point cloud data reconstructed by the neural radiance field in step S4, provide accurate 3D information for the optimization and solution of RPC parameters, construct an optimization objective function, and iteratively update the RPC coefficients by the least - squares method to optimize the rational polynomial coefficients of the images and improve the geometric positioning accuracy.

2. The method according to claim 1, wherein It also includes the following steps: S6. Error evaluation: Evaluate the error of the optimized RPC parameters to verify the accuracy and robustness of the optimization results.

3. The method according to claim 1, wherein Step S1 specifically includes: Noise removal: Apply a filtering algorithm to reduce random noise in the images and improve the signal - to - noise ratio of the images; Geometric correction: Use a transformation matrix to perform geometric transformation on the images to correct geometric distortion of the images; Radiometric correction: Through a normalization processing method, unify the radiance between different images to eliminate brightness inconsistencies caused by sensor differences and climate changes.

4. The method according to claim 1, wherein Step S2 specifically includes: Extract features from the input images to generate feature maps to represent key information in the images; Generate a query matrix Q and a key matrix K based on the feature maps for calculating the dependence relationships between points in the images; Calculate the dot product of the query matrix Q and the key matrix K, and apply the softmax function to obtain the attention weight A, which reflects the correlation between features; Weight the feature maps with the attention weight A to highlight important regions in the images and improve the accuracy of 3D reconstruction.

5. The method according to claim 1, wherein Step S3 specifically includes: Replace the RPC model of each input view with a simplified pinhole camera matrix to meet the input requirements of the neural radiance field model; Define the minimum and maximum heights of the scene to provide height boundaries for point sampling; Use the RPC positioning function to determine the positions of pixel points in the images in 3D space and convert 2D pixel coordinates to 3D coordinates; Convert the 3D points obtained by the positioning function to a fixed coordinate system centered on the earth to provide standardized coordinate data for subsequent 3D space reconstruction.

6. The method according to claim 1, wherein Step S4 specifically includes: Use neural radiance field (NeRF) technology to integrate multi - view information of multiple images to generate 3D point clouds to restore the geometric relationship between images; Generate the radiance value of a 3D scene using volume rendering method, represent the radiance field of the 3D scene through a deep neural network, and given the camera pose and pixel ray direction, generate the radiance value of the 3D scene; Add shadows by darkening the albedo, calculate the shadow scalar to simulate the influence of environmental irradiance on shadows; To minimize the loss, add a regularization term for the sun rays, and use the geometric rules encoded by the transmittance and opacity to supervise the learning of the shadow scalar, ensuring that the non-occluded and non-shadow regions are mainly explained by the albedo.

7. The method according to claim 1, characterized in that, Step S5 specifically includes: Utilize the geometric distortion information of the image and the dense point cloud data reconstructed from the neural radiance field to construct an optimization objective function that includes the geometric error of the image and the matching error between the 3D point cloud and the image; Improve the geometric positioning accuracy of the image by minimizing the error in the optimization objective function; Adopt the least squares method to iteratively update the RPC coefficients to better fit the geometric deformation of the image; The optimization goal is to minimize the projection error so that the result of projecting points in 3D space onto the image plane through the RPC model matches the actual pixel positions in the image as much as possible.

8. A computer-readable storage medium storing a computer program, characterized in that, When executed by a processor, the computer program implements the rational polynomial coefficient optimization method for satellite images based on neural radiance fields according to any one of claims 1 to 7.

9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the rational polynomial coefficient optimization method for satellite images based on neural radiance fields according to any one of claims 1 to 7.

10. A rational polynomial coefficient optimization device for satellite images based on neural radiance fields, characterized in that, Comprising: A memory for storing a computer program; A processor for implementing the rational polynomial coefficient optimization method for satellite images based on neural radiance fields according to any one of claims 1 to 7 when executing the computer program.

Images