A remote sensing image neural rendering method, an electronic device, and a storage medium

Abstract

The application provides a remote sensing image neural rendering method, an electronic device and a storage medium, which can be applied to the technical field of remote sensing image rendering. The method comprises the following steps: performing attention-aware feature extraction on a remote sensing image in stages to obtain a plurality of target feature maps with different scales; performing rational polynomial deformation on the plurality of target feature maps with different scales in stages to obtain a plurality of single-scale cost volumes, and aggregating the plurality of single-scale cost volumes into a feature map cost volume; performing 3D convolution processing on the feature map cost volume to obtain a neural encoding feature volume, and performing shadow radiation processing on the neural encoding feature volume and a ray sampling point of the remote sensing image to obtain regression rendering attribute parameters of the remote sensing image; and performing image neural rendering on the remote sensing image by using the regression rendering attribute parameters to obtain a rendering result of the remote sensing image.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image rendering technology, specifically to a remote sensing image neural rendering method, electronic device, and storage medium. Background Technology

[0002] Remote sensing image rendering refers to the process of using multi-view image data acquired through remote sensing technology to create a 3D model and render the Earth's surface through a series of algorithms and steps. Remote sensing image 3D reconstruction technology can not only intuitively and accurately assess geospatial information, but also accurately model the 3D information of surface scenes. The methods for remote sensing image 3D reconstruction are mainly divided into two categories: one is based on Multiple View Stereo (MVS) methods, and the other is based on Neural Radiance Field (NeRF) methods.

[0003] However, traditional Neural Radiation Field (NeRF) methods suffer from several significant drawbacks in 3D reconstruction of remote sensing images. First, NeRF training typically requires substantial computational resources and time, especially when processing high-resolution imagery, where the training speed can be extremely slow, limiting its feasibility in real-time applications. Second, NeRF's dependence on large amounts of input data makes it difficult to implement in certain scenarios. In remote sensing applications, acquiring multi-view images can be limited, which is crucial for obtaining high-quality reconstruction results. Furthermore, traditional NeRF lacks effective geometric constraints, which may prevent the full utilization of geometric features in the remote sensing data during reconstruction, leading to insufficient accuracy. MVS-based methods require highly accurate and up-to-date ground depth maps for training, which are difficult to obtain in satellite environments. Moreover, MVS methods establish correspondences between pixels in different images by calculating patch-based similarity metrics; insufficient surface feature extraction can lead to inconsistent scene modeling, making it difficult to handle images lacking texture, discontinuous images, or images with non-Lambertian surfaces. Summary of the Invention

[0004] In view of the above problems, the present invention provides a remote sensing image neural rendering method, electronic device and storage medium, which are used to solve at least one of the problems of the prior art.

[0005] According to a first aspect of the present invention, a method for neural rendering of remote sensing images is provided, comprising: performing attention-aware feature extraction on the remote sensing image in stages to obtain multiple target feature maps of different scales; performing rational polynomial deformation on the multiple target feature maps of different scales in stages to obtain multiple single-scale cost volumes, and aggregating the multiple single-scale cost volumes into a feature map cost volume; performing 3D convolution processing on the feature map cost volume to obtain a neural coding feature volume, and performing shadow radiative processing on the neural coding feature volume and the light sampling points of the remote sensing image to obtain regression rendering attribute parameters of the remote sensing image; and performing image neural rendering on the remote sensing image using the regression rendering attribute parameters to obtain a rendering result of the remote sensing image.

[0006] According to an embodiment of the present invention, the above-mentioned phased attention-aware feature extraction of remote sensing images to obtain multiple target feature maps of different scales includes: in the initial feature extraction stage, performing convolution processing on the remote sensing image to obtain an initial feature map with a preset scale, and performing feature map dilation processing with different dilation rates on the initial feature map to obtain multiple initial sub-feature maps; connecting the multiple initial sub-feature maps by performing multiple dilation convolution processing on the multiple initial sub-feature maps to obtain an initial intermediate feature map; randomly obtaining initial attention weights using a preset activation function, and performing calculations between the initial intermediate feature map and the initial attention weights to obtain the target feature map of the initial feature extraction stage.

[0007] According to an embodiment of the present invention, the above-mentioned phased attention-aware feature extraction of remote sensing images to obtain multiple target feature maps of different scales further includes: in the subsequent feature extraction stage, performing convolution processing on the target feature maps obtained in the preceding feature extraction stage to obtain subsequent feature maps with different scales than those in the preceding feature extraction stage, and performing feature map dilation processing with different dilation rates on the subsequent feature maps to obtain multiple subsequent sub-feature maps; connecting the multiple subsequent sub-feature maps by performing multiple dilation convolution processing on the multiple subsequent sub-feature maps to obtain subsequent intermediate feature maps; randomly obtaining subsequent attention weights using a preset activation function, and performing calculations between the subsequent intermediate feature maps and the subsequent attention weights to obtain the target feature map of the subsequent feature extraction stage.

[0008] According to an embodiment of the present invention, the above-described rational polynomial deformation of multiple target feature maps of different scales in stages to obtain multiple single-scale cost volumes includes: in each rational polynomial deformation stage, performing point normalization processing on the current target feature map based on the currently given height plane, and constructing the current tensor using the result of the current point normalization processing; performing inverse rational polynomial deformation on the current tensor to obtain the current intermediate tensor, and obtaining the normalized feature map of the current reference view by performing forward rational polynomial deformation on the current intermediate tensor; performing point inverse normalization processing on the normalized feature map of the current reference view, and performing differentiable resampling on the result of the current point inverse normalization processing to distort the current target feature map from the source view to the reference view, thereby obtaining the single-scale cost volume of the current rational polynomial deformation stage.

[0009] According to an embodiment of the present invention, the above-mentioned aggregation of multiple single-scale cost volumes into a feature map cost volume includes: calculating the voxel features of all points in each single-scale cost volume, and using the voxel features of all points in each single-scale cost volume to obtain the variance cost feature vector of each single-scale cost volume; and aggregating all single-scale cost volumes based on the variance cost feature vectors of all single-scale cost volumes to obtain the feature map cost volume.

[0010] According to an embodiment of the present invention, the above-described 3D convolution processing of the feature map cost volume to obtain the neural coding feature volume includes: performing 3D convolution processing on the feature map cost volume using a pre-trained 3D deep convolutional neural network to reconstruct the neural coding feature volume from the feature map cost volume, wherein the pre-trained 3D deep convolutional neural network includes downsampled 3D convolutional layers and upsampled 3D convolutional layers that are skip-connected to the downsampled 3D convolutional layers.

[0011] According to an embodiment of the present invention, the above-mentioned shadow radiative processing of the neural coding feature volume and the light sampling points of the remote sensing image to obtain the regression rendering attribute parameters of the remote sensing image includes: performing point parameterization of the neural coding feature volume in the normalized device coordinate space of the reference viewpoint, and performing trilinear interpolation on the point parameterized neural coding feature volume to obtain the trilinear interpolation feature; using the light sampling points to obtain the angle information of the incident sunlight when generating the remote sensing image, and using the light sampling points to obtain the transient embedding vector of the remote sensing image; using a pre-trained multilayer perceptron to perform shadow radiative processing on the neural coding feature volume, the trilinear interpolation feature, the angle information of the incident sunlight, the transient embedding vector, and the unit vector in the reference viewpoint coordinate system to obtain the regression rendering attribute parameters, wherein the regression rendering attribute parameters include volume density, reflected light, shadow gray value, ambient light, and uncertainty coefficient.

[0012] According to an embodiment of the present invention, the above-mentioned image neural rendering of remote sensing images using regression rendering attribute parameters to obtain the rendering result of remote sensing images includes: determining the distance information of sampling points on the sampling ray using volume density, and correcting the reflected light using ambient light; performing color rendering on the points on the sampling ray using the corrected emitted light and the distance information of the sampling points, and calculating the opacity and transmittance of the points in the remote sensing image using volume density, wherein the sampling ray is jointly determined by camera parameters and the pixel observation direction when acquiring the remote sensing image; supervising the color rendering process of the points on the sampling ray using shadow grayscale values ​​and the distance information of the points on the sampling ray, and supervising the contribution of the transient embedding vector using uncertainty coefficients to complete the rendering of the remote sensing image and obtain the rendering result.

[0013] A second aspect of the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0014] A third aspect of the present invention also provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0015] The remote sensing image neural rendering method provided by this invention can focus on and enhance key regions in remote sensing images by performing multi-scale feature extraction based on attention perception. Moreover, multi-scale feature extraction can capture information at different scales in remote sensing images and improve the representation of detailed features. At the same time, the remote sensing image neural rendering method provided by this invention constructs a neurally encoded feature volume by combining features with the shadow radiation field. The neurally encoded feature volume is aggregated from the feature map of the input remote sensing image. That is, the image features contained in the neurally encoded feature volume enrich the feature information from the original shadow radiation field that only comes from the sampling points, which can help the original shadow radiation field better learn the appearance and structural information of the scene. Attached Figure Description

[0016] The above-described features, other objects, and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0017] Figure 1 This is an application scenario diagram of the remote sensing image neural rendering method according to an embodiment of the present invention.

[0018] Figure 2 This is a flowchart of a remote sensing image neural rendering method according to an embodiment of the present invention.

[0019] Figure 3This is an architecture diagram of a remote sensing image neural rendering method based on multi-view stereo RPC deformation according to an embodiment of the present invention.

[0020] Figure 4 This is a schematic diagram of the multi-scale feature extraction process of multi-stage attention perception according to an embodiment of the present invention.

[0021] Figure 5 This is a block diagram of an electronic device suitable for implementing a neural rendering method for remote sensing images according to an embodiment of the present invention. Detailed Implementation

[0022] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0024] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0025] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0026] Multiple View Stereo (MVS) is a classic computer vision problem that aims to achieve dense geometric reconstruction using images captured from multiple viewpoints, and has been extensively explored by various traditional methods. MVS techniques extract and match feature points from each image, estimate the position and orientation of each camera, compute a sparse 3D point cloud using triangulation, perform pixel-level dense matching to construct a depth map, and finally generate a dense 3D model based on the depth map and camera parameters. Recently, deep learning techniques have been introduced to solve the MVS problem. MVSNet applies 3D CNNs for depth estimation on the planar scan cost volume of the reference views, achieving high-quality 3D reconstruction with performance superior to classic traditional methods. To address the computational overhead of MVSNet, which processes all depth values ​​at once during cost volume construction, CasMVSNet (Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching) employs a multi-stage refinement strategy. The network is divided into multiple stages, each generating a depth map. As the stages progress, the resolution and accuracy of the depth maps improve. PA-MVSNet (Sparse-to-Dense Multi-View Stereo With Pyramid Attention) introduces a pyramid attention network. Utilizing a multi-scale feature pyramid attention mechanism, it introduces a scale-agnostic attention module to capture long-distance feature correspondences along the top-down path, extracting richer feature information. LANet (Local Attention Embedding to Improve the Semantic Segmentation of Remote Sensing Images) introduces a remote attention network to capture the remote correlations between pixels to enhance image features and gather more information to measure the similarity between images. MVSTR (Multi-View Stereo with Transformer) designs a global context Transformer and a 3D geometry Transformer module to extract dense features with global context, achieve 3D consistency of features, and promote information interaction between views.ACINR-MVSNet (Implicit neural refinement based multi-view stereo network with adaptive correlation) designs a voxel-view weight calculation network for adaptive aggregation, utilizing average group correlation similarity metric to measure matching cost and adaptively measuring total cost, effectively suppressing the negative impact of invalid information. CDS-MVSNet (Curvature-guided dynamic scale networks for multi-view stereo) uses curvature information to estimate pixel visibility; normal curvature can implicitly provide hierarchical detail information of the surface, improving the quality of matching cost by removing mismatched pixels. Subsequent works extend this technique through cyclic planar scanning, point-based densification, confidence-based aggregation, and multiple cost volumes, improving reconstruction quality. A major drawback of these works is that they often struggle with images lacking texture, discontinuous images, or images with non-Lambertian surfaces, such as forest canopies or ice surfaces.

[0027] Recently, neural radiance field technology has significantly advanced the field of 3D reconstruction of remote sensing images. Neural radiance field for 3D reconstruction of remote sensing images involves collecting remote sensing images and their pose parameters from different perspectives, then performing ray sampling and high-dimensional feature encoding on each sampling point in 3D space based on its position, orientation, and the direction of light rays. The density and color of each sampling point are estimated through volume rendering, and the scene representation is optimized by minimizing the difference between the reconstructed image and the original image. Finally, a neural radiance field model capable of synthesizing realistic images from different perspectives is generated, achieving 3D reconstruction. DietNeRF (Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis) introduces an auxiliary semantic loss to maximize the similarity between high-level features, rather than RGB (Red, Green, Blue) color. Mip-NeRF (Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields) prevents blurring and aliasing in image sets of different resolutions. PixelNeRF (PixelNeRF: Neural Radiance Fields from One or Few Images) describes a framework that trains across multiple scenes and learns priors that can generalize to unseen scenes with a limited number of available images. NeRF-W (NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections) achieves robustness to radiance variations and transient objects by learning to separate transient phenomena from static scenes. S-NeRF (S-NeRF: Neural Radiance Fields for Street Views) demonstrates the benefits of simultaneously utilizing the direction of sunlight to learn the amount of sunlight reaching each point in the scene for geometric estimation. DS-NeRF (Depth-supervised NeRF: Fewer Views and Faster Training for Free) incorporates a depth supervision term into the loss function to accelerate learning and reduce the number of input images.Sat-NeRF (Learning Multi-View Satellite Photogrammetry With Transient Objects and Shadow Modeling Using RPC Cameras) improves upon S-NeRF by directly employing RPC (Rational Polynomial Coefficients) camera models from satellite imagery and adding an auxiliary head to handle transient objects. EO-NeRF (Multi-Date Earth Observation NeRF: The Detail Is in the Shadows) does not predict shadows for multi-date satellite imagery but instead renders shadows based on the geometry and position of the sun. Geometrically consistent shadows provide cues that allow for geometry improvements, thus enhancing the shadow quality. However, a major drawback of these works is the requirement for very long per-scene optimization processes to obtain high-quality radiation fields, which is both expensive and severely limits their practicality.

[0028] To address the issue of insufficient surface feature extraction leading to inconsistent scene modeling in the MVS method, which establishes correspondences between pixels in different images by calculating patch-based similarity metrics, this invention proposes an attention-aware multi-scale feature extraction module. This module enhances feature representation capabilities and information connections across different input images by focusing on key regions and capturing information at different scales. Furthermore, RPCwarpping (rational polynomial coefficient packs) is used to construct the costvolume for remote sensing images, and a 3D CNN (3D-Convolutional Neural Networks) is used to convert the costvolume into a neural feature volume, which is then input into the shadow radiation model. Simultaneously, light sampling is performed to decode rendering parameters along with the features. Compared to existing MVS methods, this achieves differentiable neural rendering, allows training without 3D supervision, and further improves rendering quality. Compared to existing NeRF work, an MVS-like architecture naturally performs cross-view correspondence inference, promotes generalization to unseen test scenes, and delivers better neural scene reconstruction and rendering.

[0029] Figure 1 This is an application scenario diagram of the remote sensing image neural rendering method according to an embodiment of the present invention.

[0030] like Figure 1As shown, the application scenario 100 according to this embodiment may include scenarios such as agricultural monitoring, environmental monitoring, and urban planning. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0031] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0032] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0033] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0034] It should be noted that the remote sensing image neural rendering method provided in this embodiment of the invention can generally be executed by server 105. Accordingly, the remote sensing image neural rendering method provided in this embodiment of the invention can also be executed by a server or server cluster that is different from server 105 and is capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or server 105.

[0035] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0036] The following will be based on Figure 1 The described scene, through Figures 2-3 The remote sensing image neural rendering method of the disclosed embodiments is described in detail.

[0037] Figure 2This is a flowchart of a remote sensing image neural rendering method according to an embodiment of the present invention.

[0038] like Figure 2 As shown, the remote sensing image neural rendering method of this embodiment includes operations S210 to S240.

[0039] In operation S210, attention-based feature extraction is performed on remote sensing images in stages to obtain target feature maps at multiple scales.

[0040] When extracting features from remote sensing images, an attention-aware multi-scale feature extraction module is used to extract features in stages, resulting in target feature maps at multiple scales.

[0041] In operation S220, rational polynomial deformation is performed in stages on target feature maps of different scales to obtain multiple single-scale cost volumes, and the multiple single-scale cost volumes are aggregated into feature map cost volume.

[0042] In operation S230, 3D convolution processing is performed on the feature map cost volume to obtain the neural coding feature volume, and shadow radiation processing is performed on the neural coding feature volume and the light sampling points of the remote sensing image to obtain the regression rendering attribute parameters of the remote sensing image.

[0043] The aforementioned neural coding feature volume is also called the neural coding feature cost volume.

[0044] In operation S240, image neural rendering is performed on the remote sensing image using regression rendering attribute parameters to obtain the rendering result of the remote sensing image.

[0045] The remote sensing image neural rendering method provided by this invention can focus on and enhance key regions in remote sensing images by performing multi-scale feature extraction based on attention perception. Moreover, multi-scale feature extraction can capture information at different scales in remote sensing images and improve the representation of detailed features. At the same time, the remote sensing image neural rendering method provided by this invention constructs a neurally encoded feature volume by combining features with the shadow radiation field. The neurally encoded feature volume is aggregated from the feature map of the input remote sensing image. That is, the image features contained in the neurally encoded feature volume enrich the feature information from the original shadow radiation field that only comes from the sampling points, which can help the original shadow radiation field better learn the appearance and structural information of the scene.

[0046] The following specific embodiments, in conjunction with the appendix, demonstrate this process. Figure 3 The remote sensing image neural rendering method provided by this invention will be described in further detail.

[0047] Figure 3This is an architecture diagram of a remote sensing image neural rendering method based on multi-view stereo RPC deformation according to an embodiment of the present invention.

[0048] The remote sensing image neural rendering method provided by this invention mainly includes three parts: an attention-aware multi-scale feature extraction module, a cost volume and neural coding feature volume (or neural coding feature cost volume) construction module, and a feature-joint shadow radiation field. Figure 3 As shown, Input image The data is sent to the multi-scale feature extraction module. After feature extraction, the multi-scale feature map... The multi-scale feature maps are input into the cost volume construction stage in three stages. After undergoing differentiable RPC deformation, the multi-scale feature maps are aggregated into the feature map cost volume. (or cost volume), and then a 3D convolutional neural network is used. Cost volume of features from the original 2D image A neural coding feature volume was reconstructed from the data. Finally, the neural coding feature volume is... The features and light sampling points in the image are simultaneously input into the shadow radiation field. Figure 3 Taking a multilayer perceptron as an example, those skilled in the art can obtain regression rendering attribute parameters from other shadow radiation field models according to actual needs, and then use the regression rendering attribute parameters to perform graph neural rendering on the target remote sensing image. Here, gt refers to the real color data of the pixels in the remote sensing image sample, which is the benchmark used to calculate the rendering error (loss value) during model training. The blue square next to gt represents the rendered color.

[0049] This invention provides a neural rendering method for remote sensing images based on multi-view stereo RPC deformation. By combining features with a shadow radiance field, it introduces a neurally encoded feature volume transformed from the feature map cost volume, building upon a shadow irradiance model that only uses light sampling point information. The shadow radiance field can learn meaningful scene geometry and appearance information encoded in the neural features. Sampling points and neural feature volumes jointly construct scene information. Furthermore, the method proposed in this invention uses remote sensing images to construct the neurally encoded feature volume. RPC deformation is used to project feature maps extracted from multiple remote sensing images onto the same view space through high-order tensor transformation to construct the volume. A 3D CNN is then used to construct the neurally encoded feature volume, providing input features for the feature-shading radiance field.

[0050] According to an embodiment of the present invention, the above-mentioned phased attention-aware feature extraction of remote sensing images to obtain multiple target feature maps of different scales includes: in the initial feature extraction stage, performing convolution processing on the remote sensing image to obtain an initial feature map with a preset scale, and performing feature map dilation processing with different dilation rates on the initial feature map to obtain multiple initial sub-feature maps; connecting the multiple initial sub-feature maps by performing multiple dilation convolution processing on the multiple initial sub-feature maps to obtain an initial intermediate feature map; randomly obtaining initial attention weights using a preset activation function, and performing calculations between the initial intermediate feature map and the initial attention weights to obtain the target feature map of the initial feature extraction stage.

[0051] According to an embodiment of the present invention, the above-mentioned phased attention-aware feature extraction of remote sensing images to obtain multiple target feature maps of different scales further includes: in the subsequent feature extraction stage, performing convolution processing on the target feature maps obtained in the preceding feature extraction stage to obtain subsequent feature maps with different scales than those in the preceding feature extraction stage, and performing feature map dilation processing with different dilation rates on the subsequent feature maps to obtain multiple subsequent sub-feature maps; connecting the multiple subsequent sub-feature maps by performing multiple dilation convolution processing on the multiple subsequent sub-feature maps to obtain subsequent intermediate feature maps; randomly obtaining subsequent attention weights using a preset activation function, and performing calculations between the subsequent intermediate feature maps and the subsequent attention weights to obtain the target feature map of the subsequent feature extraction stage.

[0052] The following specific embodiments, in conjunction with the appendix, demonstrate this process. Figure 4 The process of multi-scale feature extraction for multi-stage attention perception provided by this invention will be further described in detail.

[0053] Figure 4 This is a schematic diagram of the multi-scale feature extraction process of multi-stage attention perception according to an embodiment of the present invention.

[0054] like Figure 4 As shown in the embodiment of the present invention, the multi-scale feature extraction of multi-stage attention perception includes a first stage, a second stage, and a third stage. Through the above three stages, the multi-scale feature extraction of multi-stage attention perception is... Figure 4 The input image shown undergoes multi-stage feature extraction. This invention employs an attention-aware weight-sharing multi-scale feature extractor, consisting of an encoder and decoder with skip connections, for feature extraction. The encoder-decoder structure with skip connections offers significant advantages over ordinary image feature extractors in image feature extraction. This structure, by preserving low-level detail information, fusing multi-scale features, improving accuracy, accelerating network convergence, enhancing generalization ability, and reducing the number of parameters, can more effectively capture details and contextual information in images during image feature extraction tasks. Firstly, using… Features are extracted using convolutional layers with kernel sizes of 2, 3, and 4. The feature maps are then fed into three different dilated convolutional layers with dilation rates of 2, 3, and 4. Finally, all three output feature maps are fed into a convolutional layer with an attention module. Convolutional layers. Finally, the three output feature maps are concatenated to generate a new feature map, which serves as the final feature map. The triple dilated convolution is shown in Equation (1):

[0055] (1).

[0056] in, Indicates multiplication operation. Represents element-wise product. Indicates the first The weights of the dilated convolution, Indicates the first The feature maps input to each branch Represents the identity matrix. [*] represents the intermediate feature map, and [*] represents the operator that concatenates multiple branches.

[0057] The input feature map is defined as Using a core as Two convolutional layers are used to generate further features. Then, the sigmoid function is used to obtain the result defined as... Attention weights. The final output is the target feature map. As shown in formula (2):

[0058] (2).

[0059] The attention-aware multi-scale feature map extraction method adopted in this invention achieves pyramid-shaped feature capture through staged processing. The initial stage extracts basic textures, and the subsequent stages gradually focus on abstract features, which meets the "global-local" analysis requirements of remote sensing images. At the same time, the receptive field is effectively expanded by processing different dilation rates, avoiding the problem of scale uniformity in traditional convolution. In addition, dynamic weight allocation solves the problem of feature oversmoothing caused by fixed convolution kernels in traditional methods.

[0060] According to an embodiment of the present invention, the above-described rational polynomial deformation of multiple target feature maps of different scales in stages to obtain multiple single-scale cost volumes includes: in each rational polynomial deformation stage, performing point normalization processing on the current target feature map based on the currently given height plane, and constructing the current tensor using the result of the current point normalization processing; performing inverse rational polynomial deformation on the current tensor to obtain the current intermediate tensor, and obtaining the normalized feature map of the current reference view by performing forward rational polynomial deformation on the current intermediate tensor; performing point inverse normalization processing on the normalized feature map of the current reference view, and performing differentiable resampling on the result of the current point inverse normalization processing to distort the current target feature map from the source view to the reference view, thereby obtaining the single-scale cost volume of the current rational polynomial deformation stage.

[0061] The multi-stage RPC deformation process involved in this invention will be further described in detail below through specific embodiments.

[0062] The RPC model is one of the most commonly used camera models for high-resolution satellite imagery. It uses cubic rational polynomial coefficients to connect image points to their corresponding world coordinate points. As shown in equation (3), a 3D point in the normalized world coordinate system (latitude, longitude, altitude) is represented as... The coordinates are transformed to the image space to obtain normalized image coordinates. , which correspond to the coordinates along the array direction and along the track direction, respectively. and All are cubic polynomials, as shown in formula (5), where the integers are... , and The sum does not exceed 3.

[0063] (3).

[0064] (4).

[0065] (5).

[0066] in, This represents the coordinates "along the array direction" in the normalized image coordinates. Represents the coordinates "along the orbital direction" in the normalized image coordinates; Indicates the first A forward RPC cubic polynomial, where the superscript fwd indicates "forward", which is the transformation from world coordinates to image coordinates; These represent the latitude, longitude, and altitude values ​​in the normalized world coordinate system, respectively.

[0067] Indicates the first A reverse RPC cubic polynomial, with the superscript inv indicating "inverse", which is the transformation from image coordinates to world coordinates.

[0068] This represents a cubic ternary polynomial, used to describe the mapping relationship between input and output in RPC transformation; The three variables representing the polynomial refer to different physical quantities depending on the "forward / backward" scenario; in formula (3), In formula (4), . The coefficients of the polynomial determine the specific mapping relationship of the transformation; , , Polynomial order restrictions; Indicates the summation index, corresponding to respectively powers of.

[0069] The RPC model is a general geometric model, not a physical camera model. However, existing research has shown that the RPC model can achieve accuracy comparable to the rigorous Rigorous Sensor Model (RSM), ensuring its successful application in all high-resolution optical satellite imagery. In the pinhole camera model, the geometric transformation between corresponding pixels in a stereo image can be expressed as a simple transformation given a depth. Homography matrix. Almost all state-of-the-art multi-view stereo methods warp the source view to the reference view using a homography matrix and a set of assumed orthogonal planes of the reference views. In contrast, the RPC model is much more complex, and its warping cannot be described solely by matrices. This invention employs a rigorous and efficient RPC warping module, which is essentially a high-order tensor transformation. This module projects image features from different views to the reference view through a set of assumed height planes in the world coordinate system, rather than the orthogonal planes of the reference view, because there are no explicit physical parameters in the RPC model to define the front of the camera.

[0070] This invention first transforms the cubic ternary polynomial in formula (5) into a quaternion cubic homogeneous polynomial. ,in and By setting and , Become the numerator or denominator of the forward form of RPC, by setting Become the numerator or denominator of the inverse form. Then the four variables... Represented as a tensor of rank 1 The polynomial coefficients are expressed as rank 3 and shape . The tensor . The numerator and denominator of the RPC model can be represented in tensor form, as shown in equation (6):

[0071] (6).

[0072] in, Represents a tensor. and The relationship between the elements in the middle is: when and When they are equal, ; and Only when two are equal ;when and When none of them are equal, Finally, the left side of equations (3) and (4) can be obtained by dividing the numerator and denominator element by element. Batch calculation and RPC transformation results for a set of points. As shown in formula (7):

[0073] (7).

[0074] in, Indicates the first The first batch The quaternion tensor of point n is the first The values ​​of each variable component. Indicates the first The first batch The quaternion tensor of point n is the first The values ​​of each variable component. Indicates the first The first batch The quaternion tensor of point n is the first The values ​​of each variable component. Indicates the first The first batch The, the The and the first A coefficient tensor.

[0075] Specifically, for a point in the source image Given a plane, obtain the normalized form based on the available normalization parameters. And construct tensors Then, it is twisted onto the reference view in the following way, as shown in formula (8):

[0076] (8).

[0077] in, This represents the normalized form of the coordinates of a point in the source image along the array direction, with the subscript... Represents the source image. Indicates normalization, This represents the normalized form of the coordinates of a point in the source image along the orbital direction. This represents the normalized height value. Indicates to Proceed to the first Next forward RPC, This represents the quaternion tensor used for the forward RPC transformation. This represents element-wise division. Represents tensor Proceed to the first Secondary reverse RPC. After denormalization, the point... Distorted to the corresponding point The RPC warping is accomplished using differentiable resampling (e.g., bilinear interpolation).

[0078] According to an embodiment of the present invention, the above-mentioned aggregation of multiple single-scale cost volumes into a feature map cost volume includes: calculating the voxel features of all points in each single-scale cost volume, and using the voxel features of all points in each single-scale cost volume to obtain the variance cost feature vector of each single-scale cost volume; and aggregating all single-scale cost volumes based on the variance cost feature vectors of all single-scale cost volumes to obtain the feature map cost volume.

[0079] According to an embodiment of the present invention, the above-described 3D convolution processing of the feature map cost volume to obtain the neural coding feature volume includes: performing 3D convolution processing on the feature map cost volume using a pre-trained 3D deep convolutional neural network to reconstruct the neural coding feature volume from the feature map cost volume, wherein the pre-trained 3D deep convolutional neural network includes downsampled 3D convolutional layers and upsampled 3D convolutional layers that are skip-connected to the downsampled 3D convolutional layers.

[0080] The following detailed description of the construction process of the feature map cost volume and neural coding feature volume provided by the present invention will be further illustrated through specific embodiments.

[0081] Feature map cost volume It is constructed based on the feature map after PRC deformation. This invention utilizes a variance-based metric to calculate the cost. Specifically, for China and Israel For each voxel centered at the center, its cost feature vector Calculated using formula (9):

[0082] (9).

[0083] in, Calculate the variance across M views. Represents a view of The dimension feature pixel value is the first In the feature map of each view, the corresponding depth / height dimension Pixel coordinates , where For view indexing (covering M input views). Matching the depth / height of voxels, The two-dimensional location of the feature map is the fundamental data for calculating the variance (Var). The variance is calculated using this feature value across M views, thus obtaining... .

[0084] Variance-based cost volume encodes image appearance variations between different input views, explaining appearance variations caused by scene geometry and view-related shadow effects. This invention utilizes a deep neural network to efficiently transform the constructed cost volume into radiation field reconstruction in realistic view synthesis. Specifically, this invention employs a pre-trained 3D deep convolutional neural network. Cost volume of features from the original 2D image A neural coding volume was reconstructed from the data. , It contains features for each voxel, which encode the geometry and appearance of the local scene.

[0085] Previous multi-view stereo (MVS) methods typically predict depth probabilities directly from the cost volume, but depth probabilities only represent the scene's geometry. To achieve high-quality rendering, this invention aims to predict depth probabilities from the feature map cost volume. This allows for the inference of more appearance-related information. Therefore, this invention trains a 3D deep convolutional neural network. It can transform the constructed image feature cost volume into a new neural coding feature volume. The feature space is learned and discovered by the network itself and used for subsequent volume attribute regression. This process can be represented by formula (10):

[0086] (10).

[0087] 3D Deep Convolutional Neural Network It is a 3D U-Net architecture that combines downsampling and upsampling convolutional layers and features skip connections, enabling efficient inference and propagation of scene appearance information to generate meaningful scene encoding volumes. It is worth noting that this encoded volume is predicted in an unsupervised manner and inferred during end-to-end volume rendering training. The network of this invention is able to learn how to encode meaningful scene geometry and appearance in the neural features of each voxel; these features are then continuously interpolated and converted into volume density and viewpoint-dependent radiosity.

[0088] According to an embodiment of the present invention, the above-mentioned shadow radiative processing of the neural coding feature volume and the light sampling points of the remote sensing image to obtain the regression rendering attribute parameters of the remote sensing image includes: performing point parameterization of the neural coding feature volume in the normalized device coordinate space of the reference viewpoint, and performing trilinear interpolation on the point parameterized neural coding feature volume to obtain the trilinear interpolation feature; using the light sampling points to obtain the angle information of the incident sunlight when generating the remote sensing image, and using the light sampling points to obtain the transient embedding vector of the remote sensing image; using a pre-trained multilayer perceptron to perform shadow radiative processing on the neural coding feature volume, the trilinear interpolation feature, the angle information of the incident sunlight, the transient embedding vector, and the unit vector in the reference viewpoint coordinate system to obtain the regression rendering attribute parameters, wherein the regression rendering attribute parameters include volume density, reflected light, shadow gray value, ambient light, and uncertainty coefficient.

[0089] According to an embodiment of the present invention, the above-mentioned image neural rendering of remote sensing images using regression rendering attribute parameters to obtain the rendering result of remote sensing images includes: determining the distance information of sampling points on the sampling ray using volume density, and correcting the reflected light using ambient light; performing color rendering on the points on the sampling ray using the corrected emitted light and the distance information of the sampling points, and calculating the opacity and transmittance of the points in the remote sensing image using volume density, wherein the sampling ray is jointly determined by camera parameters and the pixel observation direction when acquiring the remote sensing image; supervising the color rendering process of the points on the sampling ray using shadow grayscale values ​​and the distance information of the points on the sampling ray, and supervising the contribution of the transient embedding vector using uncertainty coefficients to complete the rendering of the remote sensing image and obtain the rendering result.

[0090] The following detailed description, through specific embodiments, further illustrates the process of obtaining regression rendering attribute parameters and the neural rendering process of remote sensing images provided by the present invention.

[0091] Because satellite imagery is captured over multiple dates, it exhibits significant variations in appearance, primarily in shadows and transient objects (cars, vegetation). Therefore, a shadow-perceived irradiance model and uncertainty weighting are used to handle transient phenomena that cannot be explained by the sun's position. The scene is represented as a static surface with albedo color, i.e., the inherent color of static objects. The model learns to simultaneously predict geometry and albedo color through a set of additional outputs that attempt to explain transient phenomena observed in the input image without causing changes in scene geometry. Given any 3D location... and observation direction and incident sunlight and the Embedded vectors This invention uses MLP From neural coding volume Regression to the corresponding volume density and reflected light and shadow grayscale value and ambient light Uncertainty coefficient The implicit functions encoding a 3D scene can be expressed by formulas (11) and (12):

[0092] (11). (12). Among them, From volume In the middle position The characteristics of trilinear interpolation. Specifically, Parameterization is performed in the NDC (Normalized Device Coordinates) space of the reference viewpoint, and The coordinates are represented by unit vectors in the reference viewpoint's coordinate system. Using the NDC space effectively normalizes the scene scale across different data sources, contributing to the improved generalization ability of the method. Furthermore, position encoding is applied to the position and orientation vectors, which further enhances the high-frequency details in the results.

[0093] For a ray of light, let its origin be... , direction is Then the ray can be denoted as The first light Points Recorded as , Represents the first The distance between each point and the point from which the light ray originates. The color value at that location is recorded as The color that this ray of light appears in a two-dimensional view. The weighted sum of the colors of all three-dimensional points on it is shown in formula (13):

[0094] (13). A ray of light is split into At each of the several points, the color value and volume density of the neural radiation field are calculated, and the volume density is used to calculate the first... Opacity at each point and transmittance As shown in formula (14):

[0095] (14). Scene shadows The direction of sunlight not present in the training data will produce incorrect results. Therefore, this invention minimizes the light error by adding a solar correction term loss, as shown in formula (15):

[0096] (15). in These are the second batch of solar correction rays. The rays in the sun follow the solar rays. The direction. For Each ray in In the Point prediction The higher the value before reaching the visible surface, the lower the value thereafter. Indicates the first Opacity of each point Indicates the first Transmittance at each point This indicates the number of the second batch of corrective solar rays.

[0097] Transient objects typically refer to objects or phenomena in an image that change rapidly or appear briefly, and these changes cannot be easily explained by a static background or known metadata (such as the location of the light source). Uncertainty coefficient. This is typically used to weight the contribution of each ray in the image to the final output. When A higher value indicates greater uncertainty in the ray, thus reducing its contribution; conversely, a lower value indicates less uncertainty. A lower value indicates less uncertainty in the ray, and its contribution will be increased. For each pixel... or rays This invention can calculate the MSE between its predicted color and the true color, and use... The value is used to weight this MSE, that is In this way, the contribution of pixels or rays with higher uncertainty to the final MSE will be reduced, thereby improving robustness to transient targets, as shown in Equation (16):

[0098] (16). in, , representing the minimum uncertainty coefficient and its relationship with the ray Related uncertainty coefficients The sum. Use and To avoid negative values ​​in the logarithm, This indicates the adjustment parameter. The logarithm is... The function of the middle is to prevent Converging to infinity Indicates ray The relevant uncertainty coefficient, This represents the minimum value of the uncertainty coefficient. Represents rays (or light) predicted color, Indicates ray The true color (or light).

[0099] Among them, with rays Related uncertainty coefficients Through the The uncertainty coefficients of N points are obtained by integrating, as shown in formula (17):

[0100] (17).

[0101] in, Is The The uncertainty coefficient of the point prediction. Based on the scenario's... spatial coordinates and the first of each input training image Transient embedding vectors To learn the uncertainty coefficient of each point in the prediction scenario , Indicates the first Opacity of each point Indicates the first Transmittance at each point.

[0102] Figure 5 This is a block diagram of an electronic device suitable for implementing a neural rendering method for remote sensing images according to an embodiment of the present invention.

[0103] like Figure 5As shown, an electronic device 500 according to an embodiment of the present invention includes a processor 501, which can perform various appropriate actions and processes according to a program stored in ROM 502 (Read-Only Memory) or a program loaded from storage portion 508 into RAM 503 (Random Access Memory). The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0104] RAM 503 stores various programs and data required for the operation of electronic device 500. Processor 501, ROM 502, and RAM 503 are interconnected via bus 504. Processor 501 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 502 and / or RAM 503. It should be noted that the programs may also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0105] According to an embodiment of the present invention, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to a bus 504. The electronic device 500 may also include one or more of the following components connected to the input / output (I / O) interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 510 as needed so that computer programs read from it can be installed into the storage section 508 as needed.

[0106] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0107] According to embodiments of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of the present invention, the computer-readable storage medium may include ROM 502 and / or RAM 503 and / or one or more memories other than ROM 502 and RAM 503 described above.

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0110] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A method for neural rendering of remote sensing images, characterized in that, The method includes: Attention-based feature extraction is performed on remote sensing images in stages to obtain target feature maps at multiple scales. Rational polynomial deformation is performed on the target feature maps of different scales in stages to obtain multiple single-scale cost volumes, and the multiple single-scale cost volumes are aggregated into a feature map cost volume; The feature map cost volume is subjected to 3D convolution processing to obtain a neural coding feature volume. Shadow radiative processing is then performed on the neural coding feature volume and the ray sampling points of the remote sensing image to obtain regression rendering attribute parameters of the remote sensing image. This includes: performing point parameterization on the neural coding feature volume in the normalized device coordinate space of the reference viewpoint, and performing trilinear interpolation on the point parameterized neural coding feature volume to obtain trilinear interpolation features; using the ray sampling points to obtain the angle information of the incident sunlight when the remote sensing image was generated, and using the ray sampling points to obtain the transient embedding vector of the remote sensing image; and using a pre-trained multilayer perceptron to perform shadow radiative processing on the neural coding feature volume, the trilinear interpolation features, the angle information of the incident sunlight, the transient embedding vector, and the unit vector in the reference viewpoint coordinate system to obtain the regression rendering attribute parameters, wherein the regression rendering attribute parameters include volume density, reflected light, shadow grayscale value, ambient light, and uncertainty coefficient. The remote sensing image is rendered using the regression rendering attribute parameters to obtain the rendering result of the remote sensing image.

2. The method according to claim 1, characterized in that, Attention-based feature extraction is performed on remote sensing images in stages to obtain target feature maps at multiple scales, including: In the initial feature extraction stage, convolution processing is performed on the remote sensing image to obtain an initial feature map with a preset scale, and feature map dilation processing with different dilation rates is performed on the initial feature map to obtain multiple initial sub-feature maps; An initial intermediate feature map is obtained by concatenating multiple initial sub-feature maps through multiple dilated convolution processes. Initial attention weights are randomly obtained using a preset activation function, and the initial intermediate feature map is then processed with the initial attention weights to obtain the target feature map for the initial feature extraction stage.

3. The method according to claim 2, characterized in that, Also includes: In the subsequent feature extraction stage, the target feature map obtained in the previous feature extraction stage is subjected to convolution processing to obtain a subsequent feature map with a different scale from the previous feature extraction stage, and the subsequent feature map is subjected to feature map dilation processing with different dilation rates to obtain multiple subsequent sub-feature maps. The subsequent intermediate feature maps are obtained by concatenating the multiple subsequent sub-feature maps through multiple dilated convolution processes. The subsequent attention weights are randomly obtained using the preset activation function, and the subsequent intermediate feature map is calculated with the subsequent attention weights to obtain the target feature map for the subsequent feature extraction stage.

4. The method according to claim 1, characterized in that, Rational polynomial deformation is performed in stages on the target feature maps of different scales to obtain multiple single-scale cost volumes, including: In each rational polynomial deformation stage, point normalization is performed on the current target feature map based on the currently given height plane, and the current tensor is constructed using the result of the current point normalization. Perform inverse rational polynomial transformation on the current tensor to obtain the current intermediate tensor, and perform forward rational polynomial transformation on the current intermediate tensor to obtain the normalized feature map of the current reference view. Perform point denormalization processing on the normalized feature map of the current reference view, and perform differentiable resampling on the result of the current point denormalization processing to distort the current target feature map from the source view to the reference view, thereby obtaining the single-scale cost volume of the current rational polynomial deformation stage.

5. The method according to claim 4, characterized in that, Aggregating multiple single-scale cost volumes into a feature map cost volume includes: Calculate the voxel features of all points in each single-scale cost volume, and use the voxel features of all points in each single-scale cost volume to obtain the variance cost feature vector of each single-scale cost volume. The feature map cost volume is obtained by aggregating all the single-scale cost volumes based on the variance cost feature vectors of all the single-scale cost volumes.

6. The method according to claim 1, characterized in that, Performing 3D convolution on the feature map cost volume yields the neural coding feature volume, which includes: The neural encoded feature volume is reconstructed from the feature map cost volume by performing 3D convolution processing on the feature map cost volume using a pre-trained 3D deep convolutional neural network, wherein the pre-trained 3D deep convolutional neural network includes downsampled 3D convolutional layers and upsampled 3D convolutional layers that are skip-connected to the downsampled 3D convolutional layers.

7. The method according to claim 1, characterized in that, Using the regression rendering attribute parameters to perform image neural rendering on the remote sensing image, the rendering result of the remote sensing image includes: The distance information of the sampling points on the sampling light is determined using the volume density, and the reflected light is corrected using ambient light; Color rendering is performed on the points on the sampling ray using the corrected emitted light and the distance information of the sampling points, and the opacity and transmittance of the points in the remote sensing image are calculated using the volume density. The sampling ray is determined by the camera parameters and the pixel observation direction when the remote sensing image is acquired. The color rendering process of the points on the sampling ray is supervised by using the shadow grayscale value and the distance information of the points on the sampling ray, and the contribution of the transient embedding vector is supervised by using the uncertainty coefficient, thereby completing the rendering of the remote sensing image and obtaining the rendering result.

8. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images