Bayesian nerve radiation field modeling method and system based on uncertainty perception and dynamic importance sampling

Abstract

The invention provides a Bayesian nerve radiation field modeling method and system based on uncertainty perception and dynamic importance sampling. The method comprises the following steps: replacing a full connection layer in a multi-layer perceptron with a Bayesian linear layer to obtain a Bayesian neural radiation field BN-NeRF model; acquiring a data set containing house source photos of different viewing angles and corresponding camera positions, and training a BN-NeRF model by using the data set; performing preliminary coarse sampling on each light passing through the house source scene to obtain a coarse sampling point set, and performing uncertainty evaluation on each sampling point in the coarse sampling point set by adopting a trained BN-NeRF model; according to the uncertainty evaluation result corresponding to the preliminary coarse sampling, performing secondary sampling on each light passing through the housing resource scene to obtain a fine sampling point set; and integrating the coarse sampling point set and the fine sampling point set to generate a final sampling point set, calculating the color and volume density of each sampling point in the final sampling point set by adopting a trained BN-NeRF model so as to carry out volume rendering, and generating a final house viewing picture.

Description

Technical Field

[0001] This invention relates to the fields of three-dimensional computer graphics and virtual display technology, and in particular to a Bayesian neural radiation field modeling method and system based on uncertainty perception and dynamic importance sampling. Background Technology

[0002] Traditional property listings suffer from serious deficiencies: traditional two-dimensional photos and linear videos fail to provide potential tenants with an immersive sense of space and trust, often leading to discrepancies between online information and the actual offline situation.

[0003] The existing virtual property viewing solutions present a dilemma between immersion and cost: 360° panoramic images are essentially spherical photos stitched together from fixed points, limiting users to "jumping" between preset points and preventing them from freely walking and observing within the space, thus restricting immersion. High-precision 3D modeling, which enables true spatial roaming, heavily relies on specialized equipment such as LiDAR scanners and complex processing procedures. Its high hardware, time, and labor costs prevent it from being a universally accessible technology applied to the thousands of properties on a platform.

[0004] The shortcomings of Neural Radiation Field (NeRF) technology in real-world housing scenarios: The emergence of NeRF technology has brought hope for generating high-quality 3D scenes from ordinary multi-view images. However, when the standard NeRF algorithm is directly applied to the real-world business scenarios of online rental platforms, its performance and robustness face severe challenges. The housing photos on the platform are usually taken under complex conditions and generally have the following problems: (1) Low data source quality: incomplete view coverage, poor lighting conditions, a large number of high-frequency details (such as furniture and decorations) in the scene, and highly reflective or transparent materials such as mirrors, glass, and metal. (2) Unreliable rendering results: Traditional NeRF is a deterministic model and cannot assess the confidence of its own predictions. When faced with the above "uncertain" data, the single density prediction it relies on is very likely to be inaccurate. This leads to visual defects such as blurring, distortion, and floating artifacts that seriously affect the viewing experience in key areas, undermining the realism and credibility of virtual house viewing. (3) Poor rendering efficiency and difficulty in commercialization: The existing NeRF importance sampling strategies (such as HVS) are relatively basic and static. It cannot identify and focus on areas important to user decisions (such as details and materials of furniture), nor can it specifically repair "difficult areas" in the rendering. This sampling method results in a large waste of computing resources on low-information areas such as empty walls, causing long rendering times. This makes it an impossible task to quickly and cost-effectively generate high-quality virtual models for a massive number of properties while meeting business timeliness requirements (e.g., the need for rapid online listing of property information).

[0005] In summary, existing technologies cannot provide a virtual house viewing solution that can balance low cost, high efficiency, high fidelity, and strong robustness to non-field data sources. This constitutes a major technical obstacle for online rental platforms to improve their core user experience. Summary of the Invention

[0006] To address the issues of high costs, inadequate performance of traditional 2D photos and linear videos in providing a satisfactory viewing experience for tenants, or low fidelity of generated 3D property scenes, this invention provides a Bayesian neural radiation field modeling method and system based on uncertainty perception and dynamic importance sampling.

[0007] In a first aspect, the present invention provides a Bayesian neural radiation field modeling method based on uncertainty perception and dynamic importance sampling, comprising:

[0008] The fully connected layers in the multilayer perceptron are replaced with Bayesian linear layers, and a Bayesian neural radiation field (BN-NeRF) model is constructed based on the replaced multilayer perceptron.

[0009] A dataset containing photos of properties from different perspectives and their corresponding camera positions is obtained, and the BN-NeRF model is trained using this dataset. The BN-NeRF model takes the position x and viewing direction d of a point in 3D space as input and outputs the volume density σ and color c of the property scene; where x, d, c ∈ R 3 R 3 Represents the three-dimensional real number space;

[0010] Each ray of light passing through the property scene is initially coarsely sampled to obtain a coarsely sampled point set, and the uncertainty of each sampled point in the coarsely sampled point set is evaluated using a trained BN-NeRF model;

[0011] Based on the uncertainty assessment results corresponding to the initial coarse sampling, each ray of light passing through the housing scene is sampled a second time to obtain a fine sampling point set;

[0012] The coarse and fine sampling point sets are integrated to generate the final sampling point set. The trained BN-NeRF model is used to calculate the color and volume density of each sampling point in the final sampling point set. Volume rendering is performed based on the color and volume density of each sampling point to generate the final house viewing image.

[0013] Furthermore, during the training process, the weight matrix and bias vector in the network are parameterized as probability distributions, and the corresponding objective function is:

[0014] L(θ)=E q(w|θ) [logp(D|w)-KL[q(w|θ)‖p(w)]]

[0015] Where D is the dataset, w represents the set of all weight matrices and bias vectors in the BN-NeRF model, θ represents the parameters of the probability distribution to which all parameters in w are followed; q(w|θ) represents the learned variational distribution, which is an approximation of the posterior probability distribution of w; and p(w) represents the prior distribution of w.

[0016] Furthermore, the trained BN-NeRF model is used to evaluate the uncertainty of each sampling point in the coarse sampling point set, specifically including:

[0017] Monte Carlo sampling is performed on the learned variational distribution q(w|θ) to extract K sets of network weight samples.

[0018] Set coarse sampling point set For each coarse sampling point In each group of weighted samples w (k) Below, K density prediction values ​​are obtained through K forward propagations using the BN-NeRF model.

[0019] For each coarse sampling point Calculate the average density prediction value based on K density prediction values. Uncertainty in density prediction

[0020] Furthermore, based on the uncertainty assessment results corresponding to the initial coarse sampling, each ray of light passing through the housing scene is sampled a second time to obtain a fine sampling point set, specifically including:

[0021] Predicted density based on the average density value of each coarse sampling point and uncertainty Define the importance weight of each coarse sampling point. f is a preset mapping function; the importance weight is used to balance high-density regions and high-uncertainty regions;

[0022] Weight the importance of all coarse sampling points Normalization yields the probability density function PDF along the ray, and the corresponding cumulative distribution function CDF is constructed based on the PDF; N c Indicates the number of coarse sampling points;

[0023] Based on the constructed CDF, the inverse transform sampling method is used to perform non-uniform secondary sampling on each ray passing through the housing scene to obtain a fine sampling point set. N f This indicates the number of fine sampling points.

[0024] Furthermore, the importance weight α≥0 is a hyperparameter.

[0025] Furthermore, the non-uniform secondary sampling specifically includes:

[0026] The spatial distribution density of sampling points is adjusted by CDF to make them denser in important areas;

[0027] Based on the uncertainty index assessed along the entire ray, adjust the number of fine sampling points N. f The uncertainty index is the average uncertainty or the peak uncertainty.

[0028] Furthermore, the process of performing volume rendering based on the color and density of each sampling point to generate the final house viewing image specifically includes: performing volume rendering according to the following formula and fusing it into the final pixel color C;

[0029]

[0030] Where, N final c represents the final number of sampling points. l and σ l The color and density of sampling point l are respectively, δ l This represents the optical path length at sampling point l.

[0031] Secondly, the present invention provides a Bayesian neural radiation field modeling system based on uncertainty perception and dynamic importance sampling, comprising:

[0032] The network construction module is used to replace the fully connected layers in the multilayer perceptron with Bayesian linear layers, thereby constructing a Bayesian neural radiation field (BN-NeRF) model based on the replaced multilayer perceptron.

[0033] The training module is used to acquire a dataset containing photos of properties from different viewpoints and their corresponding camera positions, and to train the BN-NeRF model using this dataset. The BN-NeRF model takes the position x and viewing direction d of a point in 3D space as input, and outputs the volume density σ and color c of the property scene; where x, d, c ∈ R 3 R 3 Represents the three-dimensional real number space;

[0034] The coarse sampling module is used to perform preliminary coarse sampling on each ray of light passing through the housing scene to obtain a coarse sampling point set, and to use a trained BN-NeRF model to evaluate the uncertainty of each sampling point in the coarse sampling point set;

[0035] The fine sampling module is used to perform secondary sampling on each ray of light passing through the housing scene based on the uncertainty assessment results corresponding to the initial coarse sampling, so as to obtain a fine sampling point set;

[0036] The volume rendering module integrates the coarse and fine sampling point sets to generate the final sampling point set. It uses a trained BN-NeRF model to calculate the color and density of each sampling point in the final sampling point set, and performs volume rendering based on the color and density of each sampling point to generate the final house viewing image.

[0037] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as described in the first aspect.

[0038] Fourthly, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect.

[0039] The beneficial effects of this invention are as follows:

[0040] (1) This invention can provide a scene representation with confidence. The system can not only learn the geometric appearance of the property scene, but also provide a quantifiable estimate of the uncertainty of the prediction for each point. This information is crucial for evaluating the reliability of the model's predictions when dealing with unprofessional, unevenly lit photos, and helps to make more informed decisions in subsequent processing, such as automatically identifying rendering areas that need further optimization, or marking virtual viewing areas for users where the model may not be clear enough.

[0041] (2) This invention can realize intelligent and adaptive focusing of computing resources, thereby reducing costs. The dynamic importance strategy designed in this invention combines density and uncertainty, and adaptively adjusts the number and distribution of sampling points according to the uncertainty level of light. The system can devote more computing power to areas that are crucial to the decision of potential tenants (such as kitchen countertop material and bathroom tile details) and "difficult areas" where model prediction is ambiguous (such as specular reflection and window highlights), avoiding wasting resources in simple or known areas. This significantly improves the utilization efficiency of computing resources and lowers the cost threshold for large-scale deployment of virtual house viewing functions on the platform.

[0042] (3) This invention can achieve a synergistic improvement in rendering quality and speed, optimizing the user experience. With the same rendering time or sampling point budget, this invention can capture richer scene details, reduce image blur, noise, or artifacts, and obtain rendering results with better visual effects and higher fidelity, providing users with a near-realistic immersive house-viewing experience. To achieve the same level of rendering quality, this invention typically requires fewer total sampling points, thereby shortening rendering time, reducing computational demands, making high-quality virtual house-viewing services more efficient and feasible, and reducing user waiting time.

[0043] (4) This invention enhances the ability to handle complex scenes and "difficult areas," ensuring the final viewing experience. The explicit secondary sampling step and the prioritization of highly uncertain "regions of interest" enable this invention to more effectively handle complex situations where traditional methods may fail when processing non-professional property photos. Its closed-loop feedback sampling and refinement process actively explores and refines areas that the model previously failed to accurately grasp, improving the robustness of the overall modeling and the ability to capture scene details, providing a more reliable and flawless scene representation for the interactive virtual tour ultimately presented to the user. Attached Figure Description

[0044] Figure 1 A flowchart illustrating the Bayesian neural radiation field modeling method based on uncertainty perception and dynamic importance sampling provided in this embodiment of the invention;

[0045] Figure 2 A schematic diagram of the structure of a Bayesian neural radiation field modeling system based on uncertainty perception and dynamic importance sampling provided in an embodiment of the present invention;

[0046] Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0048] like Figure 1 As shown, this embodiment of the invention provides a Bayesian neural radiation field modeling method based on uncertainty perception and dynamic importance sampling, including the following steps:

[0049] S101: Replace the fully connected layers in the multilayer perceptron (MLP) with Bayesian linear layers, and then construct the Bayesian neural radiation field (BN-NeRF) model based on the replaced multilayer perceptron.

[0050] Specifically, a network architecture based on a multilayer perceptron (MLP) is used as the backbone network to learn the 3D set and appearance of housing scenes. The linear layers (or fully connected layers) in the MLP are replaced with Bayesian linear layers, which parameterize the network weights as a probability distribution. In practical applications, this Bayesian linear layer can be implemented using the probabilistic programming library in the existing deep learning framework PyTorch or by creating a custom implementation.

[0051] S102: Obtain a dataset containing photos of properties from different perspectives and their corresponding camera positions, and train the BN-NeRF model using the dataset; the BN-NeRF model takes the position x and viewing direction d of a point in three-dimensional space as input, and outputs the volume density σ and color c of the property scene; where x, d, c ∈ R 3 R 3 This represents a three-dimensional real space. It can be understood that the input three-dimensional coordinates x∈R 3 and direction d∈R 3 Typically, it is necessary to first map the location to a higher-dimensional space, which is more conducive to capturing high-frequency details (such as floor texture and wallpaper texture).

[0052] S103: Perform preliminary coarse sampling on each ray of light passing through the housing scene to obtain a coarse sampling point set, and use a trained BN-NeRF model to evaluate the uncertainty of each sampling point in the coarse sampling point set;

[0053] Specifically, the initial coarse sampling can adopt a stratified sampling strategy, and the distribution of coarse samples is relatively uniform compared to the subsequent secondary sampling process.

[0054] S104: Based on the uncertainty assessment results corresponding to the initial coarse sampling, each ray of light passing through the housing scene is sampled a second time to obtain a fine sampling point set;

[0055] Specifically, steps S103 and S104 constitute the uncertainty-based two-stage dynamic importance sampling mechanism of the present invention. This mechanism aims to achieve the dynamic coupling of uncertainty perception and sampling strategy in the visual inspection process of "quickly browsing first and then carefully observing details" during actual house viewing.

[0056] S105: Integrate the coarse sampling point set and the fine sampling point set to generate the final sampling point set. Use the trained BN-NeRF model to calculate the color and volume density of each sampling point in the final sampling point set. Perform volume rendering based on the color and volume density of each sampling point to generate the final house viewing image.

[0057] The Bayesian Neural Radiation Field (BN-NeRF) modeling method based on uncertainty perception and dynamic importance sampling provided in this invention constructs a BN-NeRF to probabilistically model the volume density and color of a scene. This provides uncertainty information about the prediction results for each spatial point during the rendering process. Furthermore, based on this uncertainty information and density values, an importance sampling index is constructed, guiding sampling points not only to high-density regions but also to the regions where the model is most uncertain. This improves the information gain and robustness of sampling, overcoming the limitations of existing methods that rely solely on density values ​​for importance sampling. Moreover, by constructing a closed-loop optimization sampling framework of "coarse sampling-feedback-fine sampling," the problem of traditional unidirectional sampling processes being susceptible to initial prediction errors is solved. This closed-loop optimization sampling framework utilizes the uncertainty map obtained from the first-stage Bayesian inference to explicitly guide the sampling decisions in the second stage, thus forming an adaptive optimization closed loop that improves model convergence speed and final rendering quality.

[0058] In one embodiment, unlike the traditional NeRF model, this embodiment of the invention parameterizes the weight matrix W and bias vector b in the above-mentioned MLP network as probability distributions, rather than traditional deterministic point estimation. Let w be the set of all weights and biases, then its parameterized distribution is q(w|θ), where θ contains the parameters of all distributions. For example, it can be assumed that each weight follows an independent Gaussian distribution, then θ contains the mean μ and variance σ of each Gaussian distribution. 2 In other words, each weight w ij and bias b j Instead of being a scalar, it is a Gaussian distribution N(μ,σ) described by a pair of parameters (e.g., mean μ and standard deviation σ, or their stable representation such as logσ). 2 These (μ,σ) parameters are the variational parameters θ that the model needs to learn.

[0059] The improvements described above in this embodiment enable the BN-NeRF model to express uncertain information such as blurriness, reflection, and vignetting commonly found in property photos in a probabilistic manner, rather than giving a mandatory and incorrect judgment.

[0060] Correspondingly, this invention trains the BN-NeRF model by maximizing the variational evidence lower bound (ELBO), using the following objective function:

[0061] L(θ)=E q(w|θ) [logp(D|w)-KL[q(w|θ)‖p(w)]]

[0062] Where D is the dataset, w represents the set of all weight matrices and bias vectors in the BN-NeRF model, θ represents the parameters of the probability distribution to which all parameters in w are followed; q(w|θ) represents the learned variational distribution, which is an approximation of the posterior probability distribution of w; and p(w) represents the prior distribution of w.

[0063] Specifically, the goal of training is to learn the posterior probability distribution of the weights and biases (approximate by the variational distribution q(w|θ)), which enables the model not only to predict the volume density and color at any point in the room, but also to provide a confidence (i.e. uncertainty) estimate of its predictions.

[0064] The aforementioned loss function mainly consists of two parts: the expected log-likelihood term and the KL divergence term. Specifically, the expected log-likelihood term is approximated using the Monte Carlo method. In each training step: the reparameterization technique is used to sample K network weights from the current variational distribution q(w|θ) (allowing gradient propagation); these K sets of network weights are used for forward propagation and volume rendering respectively, yielding K predicted colors c(k); the predicted colors and the true color C of the actual property image are calculated. gt The losses between them (such as MSE) are averaged (or, as mentioned above, the average forecast is calculated). The loss is then recalculated as an estimate of this term. On the other hand, q and p, based on the Gaussian distribution, are directly calculated from the current variational parameter θ as a regularization penalty.

[0065] During optimization, reparameterization techniques are employed for differentiable sampling, and the variational parameter θ is optimized using dataset D and stochastic gradient descent (e.g., Adam). Training is performed on a single housing scenario until the model converges on the validation set. This training process ensures the model possesses the crucial capability required for subsequent uncertainty-aware sampling. Furthermore, it is understandable that when setting the prior distribution p(w) for the network weights w, an easily tractable distribution is typically chosen, such as the standard isotropic Gaussian distribution p(q,b) = N(0,I). This serves a regularization function, enabling the model to make smoother and more reasonable guesses when information is insufficient.

[0066] In one embodiment, a trained BN-NeRF model is used to evaluate the uncertainty of each sampling point in the coarse sampling point set, specifically including:

[0067] S201: Perform Monte Carlo sampling on the learned variational distribution q(w|θ) to extract K sets of network weight samples. It should be noted that the value of K can be different in the training phase and the inference phase; in the inference phase, K is generally 10 to 20.

[0068] S202: Set the coarse sampling point set For each coarse sampling point In each group of weighted samples w (j) Below, K density prediction values ​​are obtained through K forward propagations using the BN-NeRF model.

[0069] S203: For each coarse sampling point Calculate the average density prediction value based on K density prediction values. Uncertainty in density prediction

[0070] Specifically, the average density prediction value This represents the most likely physical density of the coarse sampling point, with uncertainty. This represents the "hesitation" level of the BN-NeRF model regarding the prediction result. Generally speaking, in a housing scenario, a point located inside a mirror, in a window pane, or in a dimly lit corner... The value will be significantly higher; conversely, when A higher value indicates that the model has doubts about its understanding of that spatial point (such as a specular reflection or a dark corner region).

[0071] In one embodiment, based on the uncertainty assessment results corresponding to the initial coarse sampling, each ray of light passing through the property scene is sampled a second time to obtain a fine sampling point set, specifically including:

[0072] S301: Predicted density value based on the average density of each coarse sampling point and uncertainty Define the importance weight of each coarse sampling point. f is a preset mapping function; the importance weight is used to balance high-density regions and high-uncertainty regions, that is, to balance between rendering the scene surface (corresponding to high-density regions) and repairing potential rendering defects (corresponding to high-uncertainty regions);

[0073] Specifically, this mapping function integrates the volume density mean predicted by BN-NeRF with quantified prediction uncertainty information, thereby simultaneously addressing the interior surfaces of the building scene and rendering challenges caused by factors such as lighting and reflection. In this embodiment, the importance weight is defined as: α≥0 controls the model's focus on exploring uncertain regions and is an adjustable hyperparameter.

[0074] S302: Weight the importance of all coarse sampling points Normalization yields the probability density function PDF along the ray, and the corresponding cumulative distribution function CDF is constructed based on the PDF; N c Indicates the number of coarse sampling points;

[0075] Specifically, based on this dynamic importance weight, the system is able to reconstruct the probability density function (PDF) and cumulative distribution function (CDF) for each ray in real time.

[0076] S303: Based on the constructed CDF, the inverse transform sampling method is used to perform non-uniform secondary sampling on each ray passing through the housing scene to obtain a fine sampling point set. N f This indicates the number of fine-sampling points. It's understandable that the fine-sampling point set generated by the secondary sampling decision focuses more on important regions (such as high-density areas like furniture details and wall textures, or "regions of interest" with high uncertainty like mirrors and glass), achieving the goal of prioritizing sampling of "regions of interest" where model prediction is uncertain.

[0077] Specifically, the dynamically reconstructed CDF guides non-uniform sampling, enabling sampling points to intelligently cover high-density and high-uncertainty areas. The dynamically reconstructed CDF is equivalent to planning a "key area map" for the rendering task of the corresponding light, where areas with larger values ​​need more attention, significantly improving the modeling fidelity of complex indoor scenes.

[0078] The non-uniform secondary sampling specifically includes: adjusting the spatial distribution density of sampling points using CDF to make them denser in important regions; and adjusting the number N of fine sampling points based on the uncertainty index evaluated across the entire ray. f The uncertainty index is the average uncertainty. Or uncertainty peak Specifically, for light with high uncertainty, N is automatically increased. f To enhance exploration; for light with low uncertainty, N can be reduced. f This saves computation. The dual adaptive adjustment of the number and distribution density of sampling points at the light level enables intelligent optimization of computational sampling resources, significantly improving the quality-efficiency ratio in virtual house viewing applications.

[0079] To address the limitations of existing methods that rely solely on density values ​​for sampling, this invention proposes an importance sampling index driven by both uncertainty and density. This index creatively combines the average density value predicted by the model with its corresponding uncertainty to construct a new sampling probability density function (PDF). This allows sampling points to be directed not only to high-density regions but also to regions where the model is most uncertain, thereby improving the information gain and robustness of the sampling.

[0080] Furthermore, this embodiment establishes an adaptive adjustment mechanism for sampling resource allocation, overcoming the drawback of fixed sampling numbers in existing sampling strategies. Specifically, it can dynamically adjust the number and spatial distribution of sampling points based on local characteristics along the light rays. By assessing the degree of uncertainty in local regions, this mechanism can more efficiently allocate limited computational resources (i.e., sampling points) to areas that contribute the most to the final rendering result or require further exploration.

[0081] Based on the above embodiments, step S105 specifically includes the following sub-steps:

[0082] S401: Integrate the final optimized sample set. This involves integrating the coarse sample point set... and the fine-grained sampling point set after uncertainty guidance and adaptive quantity adjustment The samples are merged and sorted by depth along the light rays to form the final, highly optimized set of final sampling points. N final =N c +N f .

[0083] S402: Calculate the color and volume density of the final sample point set. For each point x in the final sample point set... l The color c is calculated using a trained BN-NeRF model. l and volume density σ l That is, the mean parameter μ of the variational distribution obtained through training can be used directly. w ,μ b BN-NeRF is computed once as the network weights and biases. This avoids the computational overhead of multiple samplings during the rendering stage and typically provides stable and smooth rendering results for virtual house viewing experiences.

[0084] S403: Performs efficient volume rendering integration. Utilizing classic volume rendering formulas, it fuses the final sample point set and its corresponding color and volume density information into the final pixel color C.

[0085]

[0086] Where, δ l This represents the optical path length at sampling point l. Since the sampling point set has been optimized, this rendering step can maintain or improve image quality while using fewer total sampling points N. final , or in the same N final To achieve higher accuracy.

[0087] In summary, addressing the shortcomings of existing NeRF models in importance sampling, such as reliance on single density prediction, static sampling strategies, lack of adaptability, and failure to utilize model uncertainty information, this invention proposes an efficient rendering method integrating Bayesian neural radiation field uncertainty quantification and dynamic adaptive importance sampling. This introduces uncertainty quantification capabilities into the NeRF model, enabling it to predict uncertainties when processing non-professional property photos. Utilizing this uncertainty information, a dual-index dynamic sampling strategy is designed. Through the organic combination of the aforementioned Bayesian modeling, dual-index dynamic sampling strategy, adaptive resource allocation, and closed-loop optimization framework, while ensuring or improving rendered image quality (e.g., reducing blur, enhancing details, and eliminating artifacts), the total number of sampling points required for model convergence is significantly reduced, lowering computational complexity. This achieves a better balance between rendering quality and computational efficiency, ultimately realizing efficient and high-quality 3D scene rendering of property scenes.

[0088] Based on the same inventive concept, such as Figure 2 As shown, this embodiment of the invention provides a Bayesian neural radiation field modeling system based on uncertainty perception and dynamic importance sampling, including a network construction module, a training module, a coarse sampling module, a fine sampling module, and a volume rendering module.

[0089] The network construction module replaces the fully connected layers in the multilayer perceptron with Bayesian linear layers, thereby constructing a Bayesian neural radiation field (BN-NeRF) model based on the replaced multilayer perceptron. The training module acquires a dataset containing photos of properties from different viewpoints and their corresponding camera positions, and uses the dataset to train the BN-NeRF model. The BN-NeRF model takes the position x and viewing direction d of a point in three-dimensional space as input, and outputs the volume density σ and color c of the property scene; where x, d, c ∈ R. 3 R 3 The system represents a three-dimensional real space. A coarse sampling module performs preliminary coarse sampling on each ray of light passing through the property scene to obtain a coarse sampling point set. A trained BN-NeRF model is then used to evaluate the uncertainty of each sampling point in the coarse sampling point set. A fine sampling module performs secondary sampling on each ray of light passing through the property scene based on the uncertainty evaluation results of the preliminary coarse sampling to obtain a fine sampling point set. A volume rendering module integrates the coarse and fine sampling point sets to generate a final sampling point set. A trained BN-NeRF model is used to calculate the color and density of each sampling point in the final sampling point set. Volume rendering is then performed based on the color and density of each sampling point to generate the final property viewing image.

[0090] The system provided in this invention endows the model with uncertainty perception capabilities through BN-NeRF, dynamically adjusts sampling weights and distribution using uncertainty information, and adaptively allocates optimal computing resources to each ray. Ultimately, the entire system can significantly improve the efficiency and realism of generating high-quality virtual house viewing experiences using non-professional photos. Furthermore, the system includes a two-stage dynamic sampling process of "coarse sampling - uncertainty assessment and feedback - fine sampling." The Bayesian uncertainty obtained in the first stage serves as a key decision signal, explicitly guiding and triggering the fine sampling in the second stage. The second-stage sampling is designed to prioritize "regions of interest" (such as dark corners, highlight areas, or complex textured surfaces) where the model's prediction is uncertain. Through this targeted secondary sampling and closed-loop feedback mechanism, the rendering difficulties of complex scenes are effectively overcome, ensuring sufficient exploration of important details and uncertain areas, and guaranteeing the realism and reliability of the final virtual house viewing experience.

[0091] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include: a processor 301, a communication interface 302, a memory 303, and a communication bus 304. The processor 301, communication interface 302, and memory 303 communicate with each other via the communication bus 304. The processor 301 can call logical instructions in the memory 303 to execute a Bayesian neural radiation field modeling method based on uncertainty perception and dynamic importance sampling. This method includes: replacing the fully connected layers in a multilayer perceptron with Bayesian linear layers, thereby constructing a Bayesian neural radiation field (BN-NeRF) model based on the replaced multilayer perceptron; acquiring a dataset containing photos of buildings from different perspectives and their corresponding camera positions, and training the BN-NeRF model using the dataset; the BN-NeRF model takes the position x and viewing direction d of a three-dimensional point as input, and outputs the volume density σ and color c of the building scene; where x, d, c ∈ R. 3 R 3 The system represents a three-dimensional real space. Each ray of light passing through the property scene is initially coarsely sampled to obtain a coarse sample point set. A trained BN-NeRF model is then used to evaluate the uncertainty of each sample point in the coarse sample point set. Based on the uncertainty evaluation results of the initial coarse sampling, each ray of light passing through the property scene is secondarily sampled to obtain a fine sample point set. The coarse and fine sample point sets are then integrated to generate a final sample point set. A trained BN-NeRF model is used to calculate the color and volume density of each sample point in the final sample point set. Volume rendering is then performed based on the color and volume density of each sample point to generate the final property viewing image.

[0092] Furthermore, when the logical instructions in the aforementioned memory 303 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0093] This invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the Bayesian neural radiation field modeling method based on uncertainty perception and dynamic importance sampling provided in the above-described method embodiments.

[0094] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the Bayesian neural radiation field modeling method based on uncertainty perception and dynamic importance sampling provided in the above-described method embodiments.

[0095] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A Bayesian neural radiation field modeling method based on uncertainty perception and dynamic importance sampling, characterized in that, include: The fully connected layers in the multilayer perceptron are replaced with Bayesian linear layers, and a Bayesian neural radiation field (BN-NeRF) model is constructed based on the replaced multilayer perceptron. A dataset containing photos of properties from different perspectives and their corresponding camera positions is obtained, and the BN-NeRF model is trained using this dataset. The BN-NeRF model takes the position x and viewing direction d of a point in 3D space as input and outputs the volume density σ and color c of the property scene; where x, d, c ∈ R 3 R 3 Represents the three-dimensional real number space; Each ray of light passing through the property scene is initially coarsely sampled to obtain a coarsely sampled point set, and the uncertainty of each sampled point in the coarsely sampled point set is evaluated using a trained BN-NeRF model; Based on the uncertainty assessment results corresponding to the initial coarse sampling, each ray of light passing through the housing scene is sampled a second time to obtain a fine sampling point set; The coarse and fine sampling point sets are integrated to generate the final sampling point set. The trained BN-NeRF model is used to calculate the color and volume density of each sampling point in the final sampling point set. Volume rendering is performed based on the color and volume density of each sampling point to generate the final house viewing image.

2. The Bayesian neural radiation field modeling method based on uncertainty perception and dynamic importance sampling according to claim 1, characterized in that, During the training process, the weight matrix and bias vector in the network are parameterized as probability distributions, and the corresponding objective function is: L(θ)=E q(w|θ) [logp(D|w)-KL[q(w|θ)‖p(w)]] Where D is the dataset, w represents the set of all weight matrices and bias vectors in the BN-NeRF model, θ represents the parameters of the probability distribution to which all parameters in w are followed; q(w|θ) represents the learned variational distribution, which is an approximation of the posterior probability distribution of w; and p(w) represents the prior distribution of w.

3. The Bayesian neural radiation field modeling method based on uncertainty perception and dynamic importance sampling according to claim 2, characterized in that, The uncertainty of each sampling point in the coarse sampling point set is evaluated using a pre-trained BN-NeRF model, specifically including: Monte Carlo sampling is performed on the learned variational distribution q(w|θ) to extract K sets of network weight samples. Set coarse sampling point set For each coarse sampling point In each group of weighted samples w (k) Below, K density prediction values ​​are obtained through K forward propagations using the BN-NeRF model. For each coarse sampling point Calculate the average density prediction value based on K density prediction values. Uncertainty in density prediction 4. The Bayesian neural radiation field modeling method based on uncertainty perception and dynamic importance sampling according to claim 3, characterized in that, Based on the uncertainty assessment results corresponding to the initial coarse sampling, each ray of light passing through the property scene is sampled a second time to obtain a fine sampling point set, specifically including: Predicted density based on the average density value of each coarse sampling point and uncertainty Define the importance weight of each coarse sampling point. f is a preset mapping function; the importance weight is used to balance high-density regions and high-uncertainty regions; Weight the importance of all coarse sampling points Normalization yields the probability density function PDF along the ray, and the corresponding cumulative distribution function CDF is constructed based on the PDF; N c Indicates the number of coarse sampling points; Based on the constructed CDF, the inverse transform sampling method is used to perform non-uniform secondary sampling on each ray passing through the housing scene to obtain a fine sampling point set. N f This indicates the number of fine sampling points.

5. The Bayesian neural radiation field modeling method based on uncertainty perception and dynamic importance sampling according to claim 4, characterized in that, The importance weight α≥0 is a hyperparameter.

6. The Bayesian neural radiation field modeling method based on uncertainty perception and dynamic importance sampling according to claim 4, characterized in that, The non-uniform secondary sampling specifically includes: The spatial distribution density of sampling points is adjusted by CDF to make them denser in important areas; Based on the uncertainty index assessed along the entire ray, adjust the number of fine sampling points N. f The uncertainty index is the average uncertainty or the peak uncertainty.

7. The Bayesian neural radiation field modeling method based on uncertainty perception and dynamic importance sampling according to claim 1, characterized in that, The process of performing volume rendering based on the color and density of each sampling point to generate the final house viewing image specifically includes: performing volume rendering according to the following formula and fusing it into the final pixel color C; Where, N final c represents the final number of sampling points. l and σ l The color and density of sampling point l are respectively, δ l This represents the optical path length at sampling point l.

8. A Bayesian neural radiation field modeling system based on uncertainty perception and dynamic importance sampling, characterized in that, include: The network construction module is used to replace the fully connected layers in the multilayer perceptron with Bayesian linear layers, thereby constructing a Bayesian neural radiation field (BN-NeRF) model based on the replaced multilayer perceptron. The training module is used to acquire a dataset containing photos of properties from different viewpoints and their corresponding camera positions, and to train the BN-NeRF model using this dataset. The BN-NeRF model takes the position x and viewing direction d of a point in 3D space as input, and outputs the volume density σ and color c of the property scene; where x, d, c ∈ R 3 R 3 Represents the three-dimensional real number space; The coarse sampling module is used to perform preliminary coarse sampling on each ray of light passing through the housing scene to obtain a coarse sampling point set, and to use a trained BN-NeRF model to evaluate the uncertainty of each sampling point in the coarse sampling point set; The fine sampling module is used to perform secondary sampling on each ray of light passing through the housing scene based on the uncertainty assessment results corresponding to the initial coarse sampling, so as to obtain a fine sampling point set; The volume rendering module integrates the coarse and fine sampling point sets to generate the final sampling point set. It uses a trained BN-NeRF model to calculate the color and density of each sampling point in the final sampling point set, and performs volume rendering based on the color and density of each sampling point to generate the final house viewing image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images