Neural radiation field rendering method and system combining multi-view feature body and variance annealing mechanism

Abstract

The invention relates to a neural radiation field rendering method and system combining a multi-view feature body and a variance annealing mechanism, and belongs to the technical field of computer vision. The method comprises the following steps: S1, constructing a multi-view feature body; s2, decoding depth priori and confidence; s3, designing a variance annealing scheduling mechanism; s4, executing depth-guided adaptive sampling; and S5, executing feature enhancement body rendering. According to the method, the spatial geometric features and the confidence information thereof are extracted by constructing the multi-view feature body, and the global variance annealing sampling strategy dynamically adjusted along with the training process is designed, so that the number of sampling points is greatly reduced, the calculation cost is reduced, and the rendering quality and the depth map precision are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology and relates to a neural radiation field rendering method and system that combines multi-view features and variance annealing mechanism. Background Technology

[0002] 3D reconstruction and novel viewpoint synthesis technologies have been widely applied in augmented reality, filmmaking, and autonomous driving. In recent years, methods based on implicit neural representations have achieved groundbreaking progress. Among them, neural radiation fields, as a representative technique, represent scenes as continuous volume functions using a multilayer perceptron. By leveraging volumetric rendering techniques to learn the geometry and appearance of 3D scenes from 2D images, photorealistic rendering effects are achieved.

[0003] Mildenhall et al. first proposed a novel viewpoint synthesis method based on neural radiation fields. This method takes spatial coordinates and the viewing direction as input, outputs the color values ​​and volume density of sampled points through a fully connected neural network, and then performs layered sampling along the camera ray and integrates to obtain the pixel color. To address the problem of low rendering quality in sparse views, Roessle et al. proposed a method utilizing dense depth priors. This approach first uses sparse point clouds generated by motion recovery structures, then generates dense depth maps and uncertainty maps through a depth completion network, and uses the depth map to guide sampling. In addition, another class of techniques attempts to construct multi-view feature volumes and extract geometric features through 3D convolutional networks to assist in the inference of neural radiation fields, trying to combine the generalization ability of multi-view stereo matching with the rendering capabilities of neural radiation fields.

[0004] The original neural radiation field requires dense sampling along each ray during rendering, resulting in a huge computational burden and extremely slow training and inference speeds. In addition, using positional encoding to directly map the original low-dimensional features of the input to high-dimensional features is prone to ignoring low-frequency signals. In areas with weak texture or complex boundaries, "shape-radial ambiguity" can easily occur, leading to large-scale distortion and noise in the generated depth map, making it impossible to infer accurate surface geometry.

[0005] Existing methods that utilize depth maps for guided sampling typically employ static sampling strategies. The model often over-relies on these erroneous depth values, leading to sampling points clustering in incorrect locations. This mechanism lacks "exploratory" capabilities, is prone to getting trapped in local minima, and cannot automatically correct geometric errors during training. Despite incorporating prior depth information, rendering quality remains low in details such as complex geometric surfaces, and depth errors remain significant, making high-precision geometric reconstruction difficult.

[0006] Therefore, given the problems of high training computation and the tendency of depth-guided sampling to get trapped in local minima, which limits rendering and geometric quality in the existing technologies, there is an urgent need for a low-cost, high-quality neural radiation field reconstruction method. This invention utilizes an efficient processing method for multi-view stereo geometric features and feature constraints to resolve the correspondence ambiguity problem. This invention also investigates an adaptive spatial awareness learning mechanism to ensure depth continuity and image photometric edge alignment. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a neural radiation field rendering method and system that combines multi-view feature volume and variance annealing mechanism. By constructing multi-view feature volume to extract spatial geometric features and their confidence information, and designing a global variance annealing sampling strategy that dynamically adjusts with the training process, the rendering quality and depth map accuracy are significantly improved while greatly reducing the number of sampling points and reducing computational costs.

[0008] To achieve the above objectives, the present invention provides the following technical solution: A neural radiation field rendering method combining multi-view feature volume and variance annealing mechanism, the method specifically includes the following steps: S1. Constructing a multi-view feature body: Extract features from the input multi-view image, align the multi-view features to the reference view space through homography transformation, construct a multi-view cost body, and use a 3D convolutional network for regularization to obtain a regularized multi-view feature body containing geometric information. S2. Decoding Depth Prior and Confidence: Decode the coarse depth map and its corresponding confidence map from the regularized multi-view feature volume. The confidence map is used to quantify the reliability of depth prediction. S3. Design a variance annealing scheduling mechanism: dynamically calculate the global annealing coefficient based on the number of training iterations, and calculate the final sampling variance that decays over time by combining the basic variance generated by the confidence map. S4. Perform depth-guided adaptive sampling: Based on the coarse depth map, confidence map and final sampling variance, generate a mixed set of sampling points on each ray, including uniform sampling points and depth-guided sampling points based on Gaussian distribution; S5. Perform feature enhancement volume rendering: Extract geometric features from the multi-view feature volume for each sampling point, combine the position encoding and view direction input to the neural network to predict color and volume density, and synthesize the final image and depth map through volume rendering.

[0009] Furthermore, step S1 specifically includes: S11. Extract multi-scale feature maps from the reference image and the source image; S12. Divide the reference camera's view frustum space into multiple depth assumption planes; S13. Project the source view features onto the reference view space through homography transformation; S14. Calculate the variance between the reference feature and the projected source feature as the matching cost, and construct the original cost body; S15. Regularize the original cost volume using the 3D U-Net structure to obtain the regularized multi-view feature volume.

[0010] Furthermore, step S2 specifically includes: S21. Perform a Softmax operation on the multi-view feature volume along the depth dimension to obtain the depth probability distribution; S22. Calculate the expected value using Soft-argmax to create a rough depth map; S23. Extract the peak value of the depth probability distribution as the confidence map.

[0011] Furthermore, step S3 specifically includes: S31. Map the confidence plot to the basic variance using an exponential decay model; S32. Calculate the global annealing coefficient based on the number of training iterations; S33. Based on the basic variance and global annealing coefficient, the final sampling variance is calculated using the root mean square formula.

[0012] Furthermore, step S4 specifically includes: S41. Construct a Gaussian distribution sampling interval based on the roughness depth map as the mean and the final sampling variance as the standard deviation. S42. Generate depth-guided sampling points within this interval and supplement with a small number of uniform sampling points; S43, the total number of sampling points is significantly less than that of the traditional uniform sampling strategy.

[0013] Furthermore, step S5 specifically includes: S51. Project the coordinates of the sampling points onto the reference viewpoint coordinate system, and extract geometric features from the multi-view feature volume using trilinear interpolation. S52. Input the geometric features along with the position-encoded coordinates and view direction into a lightweight MLP to predict color and volume density. S53, Color output fusion of direct projection color features and MLP prediction residuals; S54. The final pixel color and depth are obtained by integrating along the light rays using the volume rendering formula.

[0014] Furthermore, this method also includes a training optimization step, which uses color loss and depth loss for end-to-end optimization. The loss functions include photometric reconstruction loss and depth consistency loss.

[0015] The present invention also provides a neural radiation field rendering system that combines multi-view feature volume with variance annealing mechanism.

[0016] The beneficial effects of this invention are as follows: 1. Significant improvement in training efficiency Traditional neural radiation fields rely on uniformly or hierarchically sampling a large number of points along the light rays, resulting in a significant consumption of computational resources in open or transparent areas, with computational costs typically on the order of billions of floating-point operations (FLOPs). This scheme concentrates ray sampling within an adaptive interval near the prediction depth, reducing computational costs to the order of tens of millions of FLOPs. This efficient sampling strategy significantly reduces the number of effective 3D sampling points involved in each iteration, thereby lowering the overall computational burden for training. Compared to uniform sampling, it achieves faster training convergence and a significantly shorter total training time while maintaining the same rendering quality.

[0017] 2. Reliable enhancement of geometric accuracy The variance annealing scheduling mechanism introduced in this invention effectively solves the problem of traditional depth-guided methods easily getting trapped in local optima, improving the reliability and geometric accuracy of the model. Furthermore, by fusing multi-view geometric features as input priors, the learning space of volume density is greatly constrained, resulting in a lower root mean square error in the final rendered depth map. Especially at object edges and in areas with weak texture, the rendered geometry is clearer and more accurate, effectively suppressing the "floating-point artifact" phenomenon in traditional neural radiation fields.

[0018] 3. Robust improvement in rendering quality Because volume density and geometry are modeled more accurately under the combined constraints of geometric prior and variance annealing mechanism, the final pixel color is significantly improved in objective metrics such as peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and locally learnable perceptual blocks (LPIPS), achieving a comprehensive enhancement of cross-view color consistency and visual fidelity.

[0019] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1This diagram illustrates the input and output of the present invention compared to other methods. "Source Image1", "Source Image2", and "Source Image3" represent the original viewpoints input to the network. "Predicted Image" and "Depth Map" are the RGB image and corresponding depth map output by the network, respectively. "Reference Image" is the ground truth image corresponding to "Predicted Image", and "RGB Error" represents the error between "Predicted Image" and "Reference Image".

[0021] Figure 2 This is a schematic diagram of the multi-scene results of the present invention, where “Predicted RGB” and “Predicted Depth” are the RGB image and the depth map corresponding to the RGB image output by the network, respectively. The two Ground Truths are the RGB ground truth and the depth ground truth, respectively. “Depth Error” is the error between “Predicted Depth” and the depth ground truth “Ground Truth”.

[0022] Figure 3 This is a schematic diagram of the geometric prior network based on multi-view stereo geometric features of the present invention.

[0023] Figure 4 This is a schematic diagram of the neural radiation field network of the present invention. Detailed Implementation

[0024] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0025] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0026] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0027] The purpose of this invention is to solve the technical defects in existing three-dimensional reconstruction methods of neural radiation fields, such as low training convergence efficiency, fuzzy geometric surface description, and unreliable depth estimation in weakly textured regions.

[0028] This invention proposes a neural radiative field rendering method that combines multi-view feature volume with a variance annealing mechanism. This method utilizes multi-view stereo vision to provide accurate geometric priors and innovatively designs a variance annealing scheduling mechanism to dynamically adjust the ray sampling strategy. This mechanism maintains high uncertainty in the early stages of training to promote geometric exploration, and gradually converges to the vicinity of high-confidence predictions for multi-view stereo matching in the later stages, achieving accurate utilization. Through this approach, this invention can effectively improve the efficiency and robustness of the volume rendering process, ultimately generating renderings with clearer geometric details and cross-view accuracy. Figure 1 Achieve greater consistency in scene depth and color representation.

[0029] See Figures 1-4 The following detailed description of the solution is provided in conjunction with specific embodiments.

[0030] 1. Technical Solution Overview This invention proposes a novel 3D reconstruction and rendering method that combines multi-view stereoscopic vision geometric priors with neural radiation field volume rendering. The core of this method lies in constructing a multi-view feature volume containing scene geometric information and designing a variance annealing mechanism and depth-guided sampling strategy that dynamically adjusts with the training process. This approach solves the problems of low training efficiency, blurred geometric edges, and depth distortion in weakly textured regions associated with traditional neural radiation field rendering.

[0031] 2. System Structure Composition The system described in this invention mainly includes the following seven core functional modules.

[0032] The first is the feature extraction module, which is used to extract deep features from the input reference image and source image.

[0033] The second is the multi-view feature volume construction module, which mainly uses homography transformation to achieve feature alignment and calculates the multi-view matching cost.

[0034] Third is the feature volume regularization module, which uses a 3D convolutional neural network to smooth the feature volume and aggregate the context.

[0035] Fourth is the depth prior and confidence decoding module, which calculates the coarse depth map and pixel-level confidence map from the regularized feature volume.

[0036] Fifth, the variance annealing scheduling mechanism calculates the global annealing coefficient based on the training process and dynamically adjusts the sampling uncertainty.

[0037] Sixth, the depth-guided adaptive sampling module combines depth prior, confidence level, and annealing coefficient to determine adaptive sampling points in the sampling space.

[0038] The seventh is the feature enhancement volume rendering module, which uses sampling point features and neural radiation networks to predict color and density, and uses volume rendering to synthesize the final image.

[0039] 3. System module relationships 3.1 Spatial Coordinate System When processing spatial relationships, the current viewpoint is selected as the reference viewpoint, and a normalized device coordinate system is constructed within the view frustum. The remaining adjacent viewpoints are used as source viewpoints. Feature maps of all source viewpoints are transformed into the camera frustum space of the reference viewpoint through a projection matrix. The final sampling point coordinates, depth map, and confidence map are all defined in the coordinate system of the reference viewpoint.

[0040] 3.2 Data Flow The data flow of this method reflects a collaborative optimization process of multi-view geometric priors and neural radiation field volume rendering. The original image and camera pose serve as the initial input to the system. First, the feature extraction module generates 2D feature maps with semantic information. Subsequently, the 2D features are processed by the multi-view feature volume construction module, achieving precise geometric alignment through homography transformation, and then calculating the multi-view... Figure 1 After the consistency cost, it is merged into the initial 3D cost body.

[0041] The cost volume is processed by the feature volume regularization module to aggregate spatial context information, and then undergoes two-branch processing: a depth prior branch, which is input to the depth and confidence decoding module to calculate a coarse depth geometric prior in real time. and its uncertainty confidence level Feature branches, regularized multi-view feature volumes It is retained as a feature index library for the volume rendering module, for subsequent sampling points to perform feature queries.

[0042] During the light sampling stage, based on the decoded geometric prior data , The global annealing coefficients are calculated by the variance annealing scheduling module based on the current training process. The common input is fed into the depth-guided adaptive sampling module. The underlying variance of multi-view stereo geometry prediction is utilized. and global annealing variance Determine the final sampling uncertainty. And guide the light rays in a Gaussian distribution. Determine efficient 3D sampling point intervals within the specified range.

[0043] Ultimately, the generated 3D sampling point coordinates were used in the inverse index feature index library. Geometric features are extracted using trilinear interpolation. These extracted features, along with position-encoded spatial coordinates and viewpoint orientation, are input into a feature-enhanced volume rendering module to predict the color and volume density at each sampling point. The system ultimately performs volume integration along the ray and outputs the rendered image. and rendering depth .

[0044] 4. Working principle and technical steps 4.1 Multi-view feature alignment and cost volume construction The construction of the multi-view cost matrix aims to map 2D images from different perspectives onto a unified 3D geometric space. First, feature extraction is performed using a 2D CNN network similar to FPN on the reference image. and N source images Processing is performed to extract feature maps at a 1 / 4 scale. and .

[0045] Secondly, the depth assumption plane is partitioned, within the reference camera's view frustum space, based on the nearest plane of the scene. and farthest plane Divide the plane into D depth hypothetical planes.

[0046] Then, homography transformation is performed using a planar scanning algorithm, for each depth assumption. Construct the homography matrix:

[0047] In the formula It refers to the first Feature map and depth of each view Homography transformation between reference feature maps at the location; , , They refer to the first The camera intrinsics, rotation and translation matrices for each view, when When, is the reference image, and n is the main axis of the reference camera.

[0048] Source perspective features Projection transformation to reference view space:

[0049] This process enables the geometric alignment of multi-view features in 3D space.

[0050] Finally, the cost volume at a 1 / 4 scale and 3D regularization are generated. The variance between the reference features and the transformed source features is calculated and used as the matching cost to construct the original cost volume. :

[0051] Simultaneously, the RGB color information of the reference image and the transformed source image are concatenated into the cost body to preserve the appearance information.

[0052] CostRegNet uses a 3D U-Net structure. Perform regularization to output the final multi-view features. .

[0053] 4.2 Decoding Deep Priors and Uncertainty To achieve efficient and robust depth-guided sampling, the regularized multi-view feature volume must be... The implicit geometric information in the data is transformed into explicit depth priors. and its corresponding uncertainty indicators .

[0054] First of all, Softmax normalization is performed along the depth dimension for specific channels, transforming the matching cost into a depth probability distribution. This distribution It describes the probability that each pixel is a real surface point on all D hypothetical depth planes from the reference viewpoint.

[0055] Secondly, by calculating the expectation value of this depth probability distribution, i.e., performing a soft-argmax operation, a coarse depth prediction map is obtained. :

[0056] This expected value calculation method ensures that the calculated depth map is continuous and differentiable, enabling the entire calculation process to participate in the overall end-to-end optimization.

[0057] Finally, the confidence level of the prediction is quantified by analyzing the sharpness of the probability distribution. Typically, the peak value in the probability distribution is selected as the confidence level for a pixel. A sharper distribution indicates higher consistency in multi-view matching at that location, and a more reliable depth prediction; conversely, a flat distribution indicates higher geometric ambiguity in that region, such as weak texture, occluded, or repetitive texture areas, resulting in lower confidence. The calculated... and As a geometric prior, it guides ray spatial sampling, serving as a geometric prior for subsequent variance annealing mechanisms and depth-guided sampling modules.

[0058] 4.3 Variance Annealing Depth-Guided Sampling Mechanism The variance annealing depth-guided sampling mechanism aims to dynamically balance geometric exploration and utilization to solve the local convergence problem caused by multi-view geometric depth prior errors in static depth-guided sampling.

[0059] First, a "confidence-variance" mapping is performed. This involves mapping the confidence scores from the multi-view output of the depth decoding module. Transformed into basic spatial variance A negative correlation mapping is established using an exponential decay model:

[0060] This mapping transforms the network’s uncertainty in geometric estimation into a quantified, pixel-based sampling range, ensuring that low-confidence regions have a larger initial sampling space.

[0061] Secondly, a global variance annealing schedule is introduced. This schedule varies with the number of training iterations. Variation of global annealing coefficient It was used to modulate the basic exploration variance. Effective mixed standard deviation used for sampling Calculated using the root mean square formula:

[0062] During the exploration phase, Approaching 1, the global variance dominates, making The larger the area, the more the light sampling covers, allowing the neural radiation network to correct for potentially coarse errors in multi-view stereo networks. As training progresses into the utilization phase, Gradually approaching 0, the global term decays. It gradually converges to the level determined by the confidence level of the multi-view stereo network regression. This allows sampling to be concentrated near high-confidence geometric surfaces, enabling refined rendering.

[0063] Finally, mixed sampling is performed. The sample set consists of two parts: a small proportion of uniformly sampled points. This ensures global coverage and avoids local optima, as well as depth-guided sampling points based on geometric priors. The sampling points are concentrated in the depth. Nearby Within the range. This adaptive hybrid sampling strategy significantly improves volumetric rendering efficiency and concentrates the computational budget on the spatial region most likely to contain scene geometry information.

[0064] 4.4 Feature Enhancement Volume Rendering and Optimization Feature enhancement aims to leverage the sparse, efficient set of sampling points obtained through depth-guided sampling, and combine it with multi-view feature volumes. This achieves robust feature-enhanced volume rendering and is optimized using color and depth loss functions.

[0065] First, regarding light Each 3D sampling point generated above Perform voxel feature indexing. Specifically, sampling points The world coordinates are back-projected to the normalized device coordinate space under the reference viewpoint, and then trilinear interpolation is used to obtain the normalized multi-view feature volume. The corresponding geometric feature vectors are extracted. This method of explicitly introducing geometric features provides powerful spatial prior information for the inference of neural radiation fields, greatly constraining its free learning space of geometry.

[0066] The extracted geometric feature vectors, along with the position-encoded point coordinates and the ray's viewpoint direction, are then fed into a lightweight, fully connected neural network for inference learning. This network's role is to predict sampling points. volume density and RGB colors To further enhance rendering quality and detail, color prediction values... A fusion strategy was adopted, which combines color features obtained directly from image projection with residuals predicted by multilayer perceptrons to ensure that texture details of multiview stereo geometry estimation are preserved while allowing neural radiation fields to optimize cross-view color consistency and illumination effects.

[0067] After obtaining the volume density of all sampling points on the light ray... and color Then, standard volume integration is performed to transform the discrete attributes into the final pixel color. and rendering depth :

[0068]

[0069] in

[0070]

[0071]

[0072] Transmittance, For the first Opacity of each sampling point The distance between adjacent sampling points. This represents the depth value of the sampling point.

[0073] This embodiment is fully validated through comparison with existing methods on key metrics. Through quantitative and qualitative analysis of the DTU dataset, this embodiment elucidates in detail the advantages of the depth-guided neural radiation field rendering method of this invention in terms of geometric accuracy and rendering quality.

[0074] As shown in Table 1 and Figure 1 As shown, this invention demonstrates powerful rendering capabilities in image generation. Compared to some other deep learning methods, this invention outperforms other methods in metrics such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Locally Learnable Perceptual Blocks (LPIPS).

[0075] As shown in Table 2 and Figure 2 As shown, this invention demonstrates precise computational capabilities in depth inference. Conventional neural radiation field methods often focus only on image rendering quality while neglecting depth inference and depth evaluation. Since there are no sufficient relevant depth reference comparison methods available for this invention, the superior MVSNeRF method is selected for comparison. Figure 1 As shown, the present invention has high consistency across views. Compared with MVSNeRF, the present invention has efficient modeling capabilities, strong recovery capabilities for high-frequency detail textures, and can effectively distinguish weak texture areas, enabling effective separation of foreground and background.

[0076] Table 1

[0077] Table 2

[0078] This invention constructs a multi-view feature volume based on variance cost. Utilizing homography transformation and 3D convolutional regularization, the geometric consistency of multiple views is encoded into an explicit volumetric representation, providing a more robust geometric context compared to 2D feature extraction. A global variance annealing mechanism is proposed. An innovative dynamic training strategy, shifting from "wide-area exploration" to "local utilization," is designed. By using time-decreasing global noise, the problem of traditional depth-guided sampling easily getting trapped in local minima is solved. Furthermore, efficient sampling with uncertainty awareness is achieved. The confidence of multi-view stereo geometry is transformed into the sampling variance of physical space, and an adaptive sampling strategy is adopted in conjunction with the availability of ground truth depth, which not only significantly reduces the total number of sampling points and computational load but also improves rendering quality.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A neural radiation field rendering method combining multi-view feature volume and variance annealing mechanism, characterized in that, The method specifically includes the following steps: S1. Constructing a multi-view feature body: Extract features from the input multi-view image, align the multi-view features to the reference view space through homography transformation, construct a multi-view cost body, and use a 3D convolutional network for regularization to obtain a regularized multi-view feature body containing geometric information. S2. Decoding Depth Prior and Confidence: Decode the coarse depth map and its corresponding confidence map from the regularized multi-view feature volume. The confidence map is used to quantify the reliability of depth prediction. S3. Design a variance annealing scheduling mechanism: dynamically calculate the global annealing coefficient based on the number of training iterations, and calculate the final sampling variance that decays over time by combining the basic variance generated by the confidence map. S4. Perform depth-guided adaptive sampling: Based on the coarse depth map, confidence map and final sampling variance, generate a mixed set of sampling points on each ray, including uniform sampling points and depth-guided sampling points based on Gaussian distribution; S5. Perform feature enhancement volume rendering: Extract geometric features from the multi-view feature volume for each sampling point, combine the position encoding and view direction input to the neural network to predict color and volume density, and synthesize the final image and depth map through volume rendering.

2. The neural radiation field rendering method combining multi-view feature volume and variance annealing mechanism according to claim 1, characterized in that, Step S1 specifically includes: S11. Extract multi-scale feature maps from the reference image and the source image; S12. Divide the reference camera's view frustum space into multiple depth assumption planes; S13. Project the source view features onto the reference view space through homography transformation; S14. Calculate the variance between the reference feature and the projected source feature as the matching cost, and construct the original cost body; S15. Regularize the original cost volume using the 3D U-Net structure to obtain the regularized multi-view feature volume.

3. The neural radiation field rendering method combining multi-view feature volume and variance annealing mechanism according to claim 2, characterized in that, Step S2 specifically includes: S21. Perform a Softmax operation on the multi-view feature volume along the depth dimension to obtain the depth probability distribution; S22. Calculate the expected value using Soft-argmax to create a rough depth map; S23. Extract the peak value of the depth probability distribution as the confidence map.

4. The neural radiation field rendering method combining multi-view feature volume and variance annealing mechanism according to claim 3, characterized in that, Step S3 specifically includes: S31. Map the confidence plot to the basic variance using an exponential decay model; S32. Calculate the global annealing coefficient based on the number of training iterations; S33. Based on the basic variance and global annealing coefficient, the final sampling variance is calculated using the root mean square formula.

5. The neural radiation field rendering method combining multi-view feature volume and variance annealing mechanism according to claim 4, characterized in that, Step S4 specifically includes: S41. Construct a Gaussian distribution sampling interval based on the roughness depth map as the mean and the final sampling variance as the standard deviation. S42. Generate depth-guided sampling points within this interval and supplement with a small number of uniform sampling points; S43, the total number of sampling points is significantly less than that of the traditional uniform sampling strategy.

6. The neural radiation field rendering method combining multi-view feature volume and variance annealing mechanism according to claim 5, characterized in that, Step S5 specifically includes: S51. Project the coordinates of the sampling points onto the reference viewpoint coordinate system, and extract geometric features from the multi-view feature volume using trilinear interpolation. S52. Input the geometric features along with the position-encoded coordinates and view direction into a lightweight MLP to predict color and volume density. S53, Color output fusion of direct projection color features and MLP prediction residuals; S54. The final pixel color and depth are obtained by integrating along the light rays using the volume rendering formula.

7. The neural radiation field rendering method combining multi-view feature volume and variance annealing mechanism according to claim 6, characterized in that, This method also includes a training optimization step, which uses color loss and depth loss for end-to-end optimization. The loss functions include photometric reconstruction loss and depth consistency loss.

8. A neural radiation field rendering system combining multi-view feature volume and variance annealing mechanism, characterized in that, The system employs the method described in any one of claims 1 to 7.

Images