A method for reconstructing an extraterrestrial planet surface environment based on a neural radiation field

Abstract

This invention discloses a method for reconstructing the surface environment of exoplanets based on a neural radiation field. The method includes: acquiring images of the exoplanet's surface within a visible area using a navigation camera on an exoplanet rover; calculating the camera's intrinsic and extrinsic parameters during imaging based on the common viewpoint matching relationship of the images; constructing a neural radiation field model using the images and the camera's intrinsic and extrinsic parameters as input to reconstruct the exoplanet's surface environment structure; and using the intrinsic and extrinsic parameters corresponding to the new perspective image to be synthesized as input to the neural radiation field model to synthesize a new perspective image or virtual reality image of the exoplanet's surface environment. This method overcomes the disadvantages of exoplanet surface images, such as single and repetitive texture features, sparse and dispersed viewpoints, and low overlap areas, achieving a reconstructed exoplanet surface environment with excellent visual effects and correct geometric structure.

Description

Technical Field

[0001] This invention belongs to the field of deep space exploration technology, specifically relating to a method for reconstructing the surface environment of exoplanets based on neural radiation sites. Background Technology

[0002] Exoplanetary surface exploration missions, exemplified by exploration of the Mars and Moon, are the most direct means of conducting scientific research on exoplanetary surfaces. According to my country's overall deep space exploration plan, before 2030, my country will continue with the fourth phase of its lunar exploration program and the second phase of its Mars exploration program, and plans to conduct unmanned, long-distance exploration experiments on exoplanetary surfaces. This places new demands on the efficiency of ground-based planning for these exploration missions.

[0003] Taking the "Zhurong" Mars rover exploration mission as an example, to ensure the rover's safe operation, the "Move-Wait" control method was mainly adopted: the rover used a fixed-point positioning image acquisition mode, using a binocular navigation camera to acquire images and transmit them to the ground. At the ground-based remote operation center, terrain reconstruction, visual positioning, and path planning were performed, generating a command sequence which was then uploaded to the rover to control its movement. The remote operation process involves numerous planning and decision-making steps, requiring the joint participation of scientists, engineers, and operators in interpreting the navigation camera images. The work cycle often lasts for several hours, which is a major factor limiting the efficiency of the exploration mission.

[0004] Introducing virtual reality technology into teleoperation processes to reconstruct the surface environment of exoplanets for ground teams is an important way and development direction to improve the efficiency of perception and decision-making in teleoperation. A key technology in virtual reality teleoperation is to construct an exoplanet surface scene that supports observation from any perspective while meeting the accuracy requirements for teleoperation planning and decision-making by fusing images acquired by the rover's navigation camera from limited viewpoints.

[0005] Reconstruction of exoplanet surface scenes mainly includes two types of methods: dense 3D reconstruction combined with graphics rendering and image-based novel perspective synthesis. Due to limitations such as narrow interplanetary communication bandwidth and barren exoplanet surfaces, images transmitted by navigation cameras from rovers often exhibit unfavorable characteristics, including repetitive texture features, sparse and dispersed viewpoints, and low overlap in the field of view. Dense 3D reconstruction methods, which rely on local data similarity, are ill-suited to the stringent conditions of repetitive textures and low-overlapping fields of view, thus failing to meet the accuracy and visual quality requirements of remote operation teams during interpretation, planning, and decision-making. In contrast, image-based novel perspective synthesis methods, without constructing 3D models, directly utilize the projection relationships between images from different viewpoints to synthesize high-quality visual images. Representative methods include panoramic view methods, viewpoint deformation methods, and concentric mosaic methods. However, these methods also require tight alignment of the viewpoints and texture features of the dense input images; otherwise, the synthesized images may suffer from texture blurring, abrupt changes, and image distortion.

[0006] With the deepening integration of deep learning methods with computer vision and computer graphics theories, novel perspective synthesis methods based on differentiable rendering, represented by neural radiation fields, have been proposed. Neural radiation fields use a multilayer perceptron neural network model to represent the opacity of each point in 3D space and the color distribution under different viewpoints, combining this with volume rendering methods to synthesize images from novel perspectives. Because neural networks provide a smooth fit to the 3D field distribution, they offer advantages such as good texture continuity and high accuracy, achieving the highest quality novel view synthesis results to date. However, the neural radiation field model only considers scenarios with bounded or unbounded front-view scenes and relatively dense input viewpoints. For images of exoplanet surfaces captured by rover navigation cameras, which have unbounded surrounding views and sparse perspectives, the neural radiation field model still faces the following problems and challenges that need to be addressed.

[0007] First, there's the issue of parameterizing 3D coordinates in boundless, all-around scenes. Because unparalleled data distributions have large upper and lower bounds, model parameter convergence is difficult. Therefore, the neural radiation field represents spatial positions in bounded and front-view boundless scenes using Euclidean and normalized device coordinate systems, respectively, to ensure the neural network input has upper and lower bounds. However, the outward-facing image acquisition mode of the rover's navigation camera is a typical example of a boundless, all-around scene. Regardless of whether Euclidean or normalized device coordinate system is used for parameterization, the neural network input will approach infinity, leading to gradient instability and ultimately model collapse.

[0008] Secondly, there is the problem of lacking geometric consistency constraints for sparse perspective input. The neural radiation field model assumes that the color of objects changes with the viewing angle and is trained solely based on photometric reconstruction errors generated by volume rendering. The training process relies entirely on the photometric information of the scene, which may lead to incorrect estimations of the scene's geometry, especially when the input perspective is sparse, causing the model to produce overly smooth estimates of the scene's geometry. Due to bandwidth limitations in space-to-ground communication, my country's rover exploration uses a fixed-point positioning image acquisition mode. The viewing angles of two images acquired by the binocular navigation camera differ by 30°, and typically only 6-24 images are transmitted per station, representing a very sparse input perspective. Therefore, it is necessary to design a regularization constraint method based on scene geometric consistency to avoid incorrect estimations of geometric structure by the model and improve the quality and accuracy of reconstructed exoplanet surface environment scenes. Summary of the Invention

[0009] For exoplanet surface images captured by rover navigation cameras, which exhibit boundless and sparse viewpoints, existing methods suffer from low quality and accuracy in scene reconstruction. To address these issues, this invention provides an exoplanet surface environment reconstruction method based on neural radiation fields. This method overcomes the limitations of current approaches in reconstructing boundless scenes and lacking geometric consistency constraints. Furthermore, with the aim of supporting future remote operation modes incorporating virtual reality technology, a workflow for exoplanet surface environment reconstruction and virtual reality image synthesis with excellent visual effects is designed.

[0010] To achieve the above objectives, the present invention provides a method for reconstructing the surface environment of an exoplanet based on a neural radiation field, which may include the following steps:

[0011] The first step is to use the exoplanet rover's navigation camera to acquire images of the exoplanet's surface within the visible area;

[0012] The second step is to calculate the first internal and external parameters of the exoplanet rover's navigation camera during imaging based on the common viewpoint matching relationship of the exoplanet surface images.

[0013] The third step involves using the images of the exoplanet's surface and the first internal and external parameters of the exoplanet rover's navigation camera during imaging as inputs to construct a neural radiation field model to reconstruct the exoplanet's surface environment structure.

[0014] The fourth step involves using the second internal and external parameters corresponding to the new perspective image to be synthesized as input to the neural radiation field model to achieve the synthesis of the new perspective image or virtual reality image.

[0015] In some optional embodiments of the present invention, the second step includes:

[0016] Identifying co-visual feature points with matching relationships in the images of the exoplanet's surface can include feature points of types such as SIFT, SUFT, ORB, and SuperPoint;

[0017] The first intrinsic and extrinsic parameters are calculated using bundle adjustment.

[0018] In some optional embodiments of the present invention, the neural radiation field model in the third step is constructed by the following method:

[0019] Based on the input image of the exoplanet's surface and the first internal and external parameters, calculate the first ray equation corresponding to the pixels in the image of the exoplanet's surface.

[0020] The first ray equation is sampled to obtain sampling points. The three-dimensional coordinates of the sampling points in the Euclidean coordinate system are mapped to the inverted spherical coordinate system with upper and lower bounds to obtain the parameterized coordinates of the sampling points.

[0021] The parameterized coordinates of the sampling point of the first ray are used as input, and the color value of the corresponding pixel is used as supervision. The backpropagation algorithm is used to adjust the parameters of the neural radiation field model.

[0022] By utilizing the parallel principal optical axis of the navigation camera imaging, the expected value of the termination position of the first ray is calculated as supervision, and the parameters of the neural radiation field model are adjusted using the backpropagation algorithm.

[0023] In some optional embodiments of the present invention, the fourth step includes:

[0024] The second ray equation is calculated based on the input second intrinsic and extrinsic parameters to determine the second ray equation corresponding to the pixels contained in the new perspective image to be synthesized.

[0025] Based on the second ray equation, the color and opacity distribution on the second ray are queried in the neural radiation field model, and the pixel color of the new perspective image is calculated using the volume rendering formula;

[0026] The third internal and external parameters are calculated based on the input second internal and external parameters and the preset observer interpupillary distance, and the third ray equation corresponding to the pixels contained in the virtual reality image to be synthesized is calculated.

[0027] Based on the third ray equation, the color and opacity distribution on the third ray are queried in the neural radiation field model, and the pixel color of the virtual reality image is calculated using the volume rendering formula.

[0028] The above-mentioned technical solution of the present invention overcomes the disadvantages of repetitive and monotonous texture features, sparse and scattered viewpoints, and low overlap area of ​​exoplanet surface images, and achieves the reconstruction of exoplanet surface environment with excellent visual effects and correct geometric structure, and has the following beneficial technical effects:

[0029] (1) By limiting the input range of the neural radiation field through parameterization of the inverted spherical coordinate system, the stability of the model parameters during training is enhanced, and the reconstruction effect of the area near the rover, which is of great interest in the surface environment of exoplanets, is improved.

[0030] (2) A regularization method was designed for the imaging relationship of parallel principal optical axes of the left and right visual navigation cameras of the patrol vehicle, making full use of the geometric consistency of the scene as a constraint for the training process of neural radiation field parameters. Attached Figure Description

[0031] Figure 1 This is a flowchart of the overall process for reconstructing the surface environment of an exoplanet based on neural radiation sites, as described in this invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0033] The accompanying drawings illustrate a layer structure according to an embodiment of the present invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0034] Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0035] In the description of this invention, it should be noted that the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0036] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0037] The method for reconstructing the surface environment of an extraterrestrial planet provided by the present invention will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0038] like Figure 1 As shown, this embodiment of the invention provides a method for reconstructing the surface environment of an exoplanet based on a neural radiation field, which may include the following steps:

[0039] S110: Use the exoplanet rover's navigation camera to acquire images of the exoplanet surface within the visible area;

[0040] S120: Calculate the first internal and external parameters of the exoplanet rover's navigation camera during imaging based on the common viewpoint matching relationship of the exoplanet surface images;

[0041] S130: Using the images of the exoplanet's surface and the first internal and external parameters of the exoplanet rover's navigation camera during imaging as input, a neural radiation field model is constructed to reconstruct the exoplanet's surface environment structure.

[0042] S140: The second internal and external parameters corresponding to the new perspective image to be synthesized are used as inputs to the neural radiation field model to achieve the synthesis of the new perspective image or virtual reality image.

[0043] The proposed method is an exoplanet surface environment reconstruction approach based on neural radiation fields. Targeting the characteristics of fixed-point positioning and binocular navigation camera panoramic imaging modes, it proposes a parameterized method for coordinates of boundless panoramic scenes and geometric consistency constraints for narrow baseline imaging by binocular cameras. This addresses the limitations of current methods in reconstructing boundless panoramic scenes and lacking geometric consistency constraints. The proposed method aims to support future remote operation modes incorporating virtual reality technology, and designs a visually appealing workflow for exoplanet surface environment reconstruction and virtual reality image synthesis. Experimental results using data from the Zhurong Mars rover demonstrate that the proposed method significantly improves the visual quality and geometric accuracy of exoplanet surface environment reconstruction compared to existing methods.

[0044] In some embodiments, S120 includes:

[0045] Identify the surface image of the exoplanet I l,i ,I r,i Commonly viewed feature points with matching relationships in (i = 1, 2, 3, ..., N), wherein the exoplanet surface image I l,i ,I r,i (i = 1, 2, 3, ..., N) is a group of images taken by the probe's binocular navigation camera at the same site, where I l,i and I r,i The images are respectively taken by the left and right visual navigation cameras of the rovers, where l represents the left and r represents the right. The feature points can include feature points of the types such as scale-invariant feature transform (SIFT) and speed-up robust features (SURF).

[0046] The first intrinsic and extrinsic parameters are calculated using bundle adjustment. The first intrinsic parameter is denoted as K, and the first extrinsic parameter is denoted as (R,t). l,i Or (R,t) r,i The subscript indicates the corresponding image. Where (R,t) l,i and (R,t) r,i The following constraints must be satisfied:

[0047] Let B be the baseline length between the left and right navigation cameras of the rover. Since the principal optical axes of the left and right visual navigation cameras of the rover are parallel, then:

[0048]

[0049] In some embodiments, in S130, the neural radiation field is constructed by the following method:

[0050] Based on the input image of the exoplanet surface I l,i ,I r,i (i = 1, 2, 3, ..., N) and the first intrinsic parameter K and the first extrinsic parameter (R, t) l,i Or (R,t) r,i Calculate the first ray equation r(t) = o + td for pixel (u, v) in an image of an exoplanet's surface, where all points (x, y, z) on the ray satisfy:

[0051]

[0052] Among them, z c Let be the z-coordinate of the point in the camera coordinate system, K and (R,t) be the first intrinsic and extrinsic parameters, R be the rotation matrix, t be the translation vector, o be the camera optical center, and d be the ray direction. In the above formula, the letter l represents left, and the number 1 represents the number 1.

[0053] The sampling points are obtained by sampling the first ray equation. The three-dimensional coordinates of the sampling points in the Euclidean coordinate system are then mapped to the inverted spherical coordinate system with upper and lower bounds to obtain the parameterized coordinates of the sampling points. The specific implementation is as follows:

[0054] Based on the characteristics of measured panoramic images, a northeast-eastern coordinate system (NED coordinate system) with its origin at the intersection of the fixed axis and the principal optical axis of the binocular camera is defined as the world coordinate system for the surface environment of exoplanets. This world coordinate system is a Euclidean coordinate system. A parametric method using a reciprocal spherical surface is proposed to map the three-dimensional coordinates (x, y, z) in this world coordinate system to... Where the radial distance r, azimuth angle θ, and polar angle are... According to the general definition of a spherical coordinate system, that is:

[0055]

[0056] Satisfying 1 / r∈(0,1 / r) camsystem ],θ∈[0,π], In acquiring sampling points, a sampling strategy with uniform distribution along the inverse depth is adopted, taking advantage of the "closer density and farther sparser density" of the reciprocal spherical coordinate system. That is, for the first ray r(t) = o + td traveling along the ray direction d from the camera's optical center o in the Euclidean coordinate system, if n sampling points are collected, the inverse depth of the i-th sampling point on the ray is 1 / r. i In the interval ((i-1) / nr) camsystem ,i / nr camsystem The probability density within ] is 1 / nr camsystem Where rcamsystem This represents the closest distance at which the camera system consisting of the binocular navigation cameras can image the image. Its value can be taken as 1 / 2 of the length of the line connecting the optical centers of the left and right navigation cameras.

[0057] For inverse depth 1 / r i The method for determining t in the first ray equation of the sampling point is to calculate the first ray r(t) = o + td and the ray with the origin as the center of the sphere, r i Intersection of spheres with radius:

[0058]

[0059] In a reciprocated spherical coordinate system, when the polar angle θ = 0 or θ = π, the azimuth angle is... Failure, i.e. regardless The value of always corresponds to (0,0,1 / r) in the Euclidean coordinate system. However, under the outward-facing sampling strategy, the binocular navigation camera shoots around a fixed axis. Due to the limited field of view of the camera, the polar angle θ = 0 or θ = π will not occur. Therefore, in the proposed parameterization method, there will be no one-to-many mapping relationship in the transformation from the reciprocal coordinate system to the Euclidean coordinate system, and the parameterized coordinates are always bounded. In addition, the parameterization method of the reciprocal coordinate system utilizes the inverse depth of the scene, which has the characteristic of "near dense and far sparse", improving the problems of difficult convergence of neural radiation field models, blurred distant scenery, and angle-limited reconstruction areas in the outward-facing boundless scene.

[0060] Using the parameterized coordinates of the sampling point of the first ray as input and the color value of the corresponding pixel as supervision, the backpropagation algorithm is used to adjust the parameters of the neural radiation field model. The specific implementation method is as follows:

[0061] The neural radiation field model uses a multilayer perceptron F with weight parameter Θ. Θ The observed color c = (r, g, b) and volume density σ are obtained at each three-dimensional position x = (x, y, z) along the first ray direction d = (θ, φ) of the fitted exoplanet surface environment. Here, θ and φ represent the azimuth and polar angles of the ray direction in the coordinate system, and r, g, and b represent the color intensity in the red (R), green (G), and blue (B) channels. Multilayer perceptron F... Θ It can be written as:

[0062] (r,g,b,σ)=F Θ (x,y,z,θ,φ) (5)

[0063] The neural radiation field model uses the color values ​​of an exoplanet surface image with known first intrinsic and extrinsic parameters as supervisory information to optimize the multilayer perceptron F. Θ The weight parameters Θ make the multilayer perceptron F ΘIt can accurately represent the mapping relationship between the 3D position x and the light direction d, the observed color c, and the volume density σ. To establish the correspondence between the observed values ​​of spatial point color and volume density and the color values ​​on the image, the neural radiation field model uses volume rendering methods, combined with a multilayer perceptron F... Θ Generates an estimate of the pixel color.

[0064] In volume rendering theory, the first ray r(t) = o + td originates from o and travels along the ray direction d. The volume density σ at the 3D position x represents the probability that the first ray r(t) terminates at that location, and thus has a near-boundary t n and far boundary t f The observed color C(r) corresponding to the first ray r(t) is:

[0065]

[0066] Where T(t) represents the first ray r(t) originating from point o+t near the boundary. f The cumulative transmittance between point d and point o+td in three dimensions, i.e., the transmittance of light from point o+t on the near boundary. f The probability that d travels to the 3D point o+td without stopping is expressed as:

[0067]

[0068] Here, exp() represents the logarithm with base e, r(s) is the integral variable s, and the integrand is the first ray r(t). Due to the multilayer perceptron F... Θ The query retrieves the color and transparency at discrete x(x,y,z), as shown in formula (5). The neural radiation field estimates the color C(r) using a sampling method. This is achieved by approaching the boundary t... n and far boundary t f Divide the data into N intervals, and randomly select sampling points t in each interval. i (i = 1, 2, 3, ..., N), use these samples to generate color estimates.

[0069]

[0070] Wherein, δ in formula (8) i =t i+1 -t i , is the distance between two adjacent sampling points, c i Indicates sampling point t i The lookup value for the color. T i For the light to travel from near the boundary to the sampling point t i Estimation of cumulative transmittance:

[0071]

[0072] Where j represents the value of the summation variable in the summation operation. The parameter tuning process of the neural radiation field model adopts backpropagation and stochastic gradient descent. A batch of pixels is randomly selected from the input training set image, and the first ray set R corresponding to these pixels is calculated. The color estimate of the pixels corresponding to these first ray sets is calculated according to formulas (5)(8)(9). The second moment of the residual between the real pixel color C(r) and the actual pixel color C(r) is used as the multilayer perceptron F of the MLP network. Θ Loss function:

[0073]

[0074] in, This represents the calculation of the L2 norm. Utilizing the parallel principal optical axes of the navigation camera's imaging, the expected value of the termination position of the first ray is calculated as supervision. The backpropagation algorithm is then used to adjust the parameters of the neural radiation field model. The specific implementation is as follows:

[0075] For the binocular camera images of the roving device that have the characteristics of parallel principal optical axes and identical camera intrinsic parameters, the geometric consistency constraint based on binocular disparity estimation can be easily introduced into the neural radiation field construction process as prior knowledge of the neural radiation field geometry:

[0076] For a pair of rovers with the same matrix of baseline length B and first intrinsic parameter K, and their left and right navigation cameras, let the origin of the coordinate system be the midpoint of the line connecting the optical centers of the left and right cameras. For the matching point pair (u,v) and (u+Δu,v) in the images of the exoplanet surface captured by the left and right navigation cameras, their corresponding three-dimensional point (x,y,z) is:

[0077]

[0078] Where Δu represents the disparity of the matched point pair, c x c y Let represent the coordinates of the optical center in the intrinsic parameter matrix, and f represent the focal length in the intrinsic parameter matrix. When the feature points are only within the left and right eye images, it is impossible to estimate the uncertainty of disparity and depth by calculating the reprojection error. For the visual system composed of the left and right eye cameras of the patrol vehicle, which has the characteristic of parallel principal optical axes, the analytical solution for the standard deviation of the error value based on the optical system transfer function is:

[0079]

[0080] Where, ω1=arctan[(ux c ) / f],ω2=arctan[(u+Δu-x c) / f], where Δx, Δy, and Δz represent the components of the standard deviation of the error values ​​in the x, y, and z axes, respectively. Since the physical meaning of opacity in the neural radiation field is the probability that a ray terminates at that point, it is the same as the probability density distribution of the depth corresponding to that pixel. Therefore, when the error in the depth estimation result is considered to be a Gaussian distribution, the condition for the neural radiation field model to recover the correct geometric structure of the scene is that the KL divergence between these two distributions is minimized, i.e., for all sampling depths t corresponding to a pixel during the sampling process... i (i = 1, 2, 3, ..., N) Minimize the loss function:

[0081]

[0082] In formula (13), Indicates t i The calculated value of opacity.

[0083] In some embodiments, in S140, the result of reconstructing the surface environment of an exoplanet uses the second intrinsic and extrinsic parameters corresponding to the new perspective image to be synthesized as input to the neural radiation field model to achieve the synthesis of the new perspective image or virtual reality image. The method for achieving this includes:

[0084] The second ray equation corresponding to the pixels contained in the new perspective image to be synthesized is calculated based on the input second intrinsic and extrinsic parameters, and the specific implementation is as follows;

[0085] Input the second intrinsic and extrinsic parameters K2 and (R2,t2), and calculate the second ray equation r2(t) = o2 + td2 corresponding to pixel (u,v) in the second image of the exoplanet's surface, where all points (x,y,z) on the second ray satisfy:

[0086]

[0087] Based on the second ray equation, the color and opacity distribution on the second ray are queried in the neural radiation field model. The pixel colors of the new perspective image are then calculated using the volume rendering formula. The specific implementation is as follows:

[0088] For a second ray r2(t) = o2 + td2 traveling along direction d2 from the camera's optical center o2 in Euclidean coordinates, and n sampling points are collected, the inverse depth 1 / r of the i-th sampling point on the ray is... i In the interval ((i-1) / nr) camsystem ,i / nr camsystem The probability density within ] is 1 / nr camsystem .

[0089] For inverse depth 1 / r iThe method for determining t in the second ray equation of the sampling point is to calculate the second ray r2(t) = o2 + td2 and the ray with the origin as the center of the sphere, r i Intersection of spheres with radius:

[0090]

[0091] Using the multilayer perceptron F Θ Query the observed color c = (r, g, b) and volume density σ observed at the sampling point along the direction d2 = (θ, φ) of the second ray, and use the sampling point to generate an estimate of the color of the second ray.

[0092]

[0093] The third internal and external parameters are calculated based on the input second internal and external parameters and the preset observer pupillary distance, as follows:

[0094] Input the second intrinsic and extrinsic parameters K2 and (R2,t2), with the observer's pupillary distance preset to L. eye The third intrinsic and extrinsic parameters K3 and (R3,t3) are:

[0095]

[0096] The method of calculating the third ray equation corresponding to the pixels contained in the virtual reality image to be synthesized, and querying the color and opacity distribution on the third ray in the neural radiation field model according to the third ray equation, and calculating the pixel color of the virtual reality image using the volume rendering formula is the same as the method of calculating the second ray equation and synthesizing a new perspective image according to the second ray equation.

[0097] The above embodiments utilize the navigation camera of an exoplanet rover to acquire images of the exoplanet's surface within the visible area; calculate the camera's first intrinsic and extrinsic parameters during imaging based on the co-viewpoint matching relationship of the exoplanet surface images; use the exoplanet surface images and the camera's first intrinsic and extrinsic parameters as input to construct a neural radiation field model to reconstruct the exoplanet's surface environment structure; and use the second intrinsic and extrinsic parameters corresponding to the new perspective image to be synthesized as input to the neural radiation field model to achieve the synthesis of a new perspective image or virtual reality image of the exoplanet's surface environment. This method overcomes the disadvantages of exoplanet surface images, such as single and repetitive texture features, sparse and scattered perspectives, and low overlap areas, achieving excellent visual effects and geometrically correct reconstruction of the exoplanet's surface environment.

[0098] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of the present invention is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0099] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0100] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

[0101] The above embodiments are provided merely for the purpose of describing the present invention and are not intended to limit the scope of the invention. The scope of the invention is defined by the appended claims. Various equivalent substitutions and modifications made without departing from the spirit and principles of the invention should be covered within the scope of the invention.

Claims

1. A method for reconstructing the surface environment of an exoplanet based on neural radiation fields, characterized in that, Includes the following steps: The first step is to use the exoplanet rover's navigation camera to acquire images of the exoplanet's surface within the visible area; The second step is to calculate the first internal and external parameters of the exoplanet rover's navigation camera during imaging based on the common viewpoint matching relationship of the exoplanet surface images. The third step involves using the images of the exoplanet's surface and the first internal and external parameters of the exoplanet rover's navigation camera during imaging as input to construct a neural radiation field model to reconstruct the exoplanet's surface environment structure, including: Based on the input image of the exoplanet's surface and its camera pose, calculate the first ray equation corresponding to the pixels in the image of the exoplanet's surface. The first ray equation is sampled to obtain sampling points. The three-dimensional coordinates of the sampling points in the Euclidean coordinate system are mapped to the inverted spherical coordinate system with upper and lower bounds to obtain the parameterized coordinates of the sampling points. The parameterized coordinates of the sampling point of the first ray are used as input, and the color value of the corresponding pixel is used as supervision. The backpropagation algorithm is used to adjust the parameters of the neural radiation field model. Utilizing the parallel principal optical axis characteristic of the navigation camera imaging, the expected value of the termination position of the first ray is calculated as supervision, and the parameters of the neural radiation field model are adjusted using the backpropagation algorithm; The fourth step is to use the second internal and external parameters corresponding to the new perspective image to be synthesized as input to the neural radiation field model to achieve the synthesis of the new perspective image or virtual reality image.

2. The method for reconstructing the surface environment of an exoplanet based on neural radiation fields according to claim 1, characterized in that: The fourth step includes: The second ray equation is calculated based on the input second intrinsic and extrinsic parameters to determine the second ray equation corresponding to the pixels contained in the new perspective image to be synthesized. Based on the second ray equation, the color and opacity distribution on the second ray are queried in the neural radiation field model, and the pixel color of the new perspective image is calculated using the volume rendering formula; The third internal and external parameters are calculated based on the input second internal and external parameters and the preset observer interpupillary distance, and the third ray equation corresponding to the pixels contained in the virtual reality image to be synthesized is calculated. Based on the third ray equation, the color and opacity distribution on the third ray are queried in the neural radiation field model, and the pixel color of the virtual reality image is calculated using the volume rendering formula.

Images