An active ray sampling method and system for accelerating radiance field training

Abstract

The application provides an active ray sampling method and system for accelerating radiance field training. The method comprises the following steps: establishing a ray loss diffusion model based on rendering gradient, diffusing the new loss value of the ray sampled in the current round to the remaining rays, and calculating the confidence of the diffusion according to the difference of the ray color; establishing an update model of the ray state, updating the current state of the ray with the loss value and the confidence obtained by diffusion; establishing an online reinforcement learning ray sampling strategy based on UCB, solving the problem of low training efficiency under uniform ray sampling; in addition, the estimation deviation of the ray state is included in the confidence interval, solving the problem that the ray state is always outdated; without pre-training, the problem of overfitting to a specific scene is avoided. Therefore, the active ray sampling method and system for accelerating radiance field training provided by the application can adaptively select the most valuable ray sample for training, thereby greatly accelerating the reconstruction process of the radiance field.

Description

Technical Field

[0001] This invention belongs to the field of inverse rendering technology, specifically relating to an active ray sampling method and system for accelerating radiation field training. Background Technology

[0002] Inverse rendering aims to reconstruct the 3D information of a scene from a series of calibrated 2D images. This is an ill-posed problem, but it plays a crucial role in many fields, such as virtual reality, autonomous driving, and medical imaging. One of the mainstream paradigms of current inverse rendering technology is Neural Radiance Fields (NeRF), which represents a continuous 3D scene as a compact implicit field, such as a multilayer perceptron (MLP). Specifically, NeRF uses differentiable volume rendering to generate 2D images and then back-optimizes the 3D scene parameters through gradient descent of the loss value. Due to its ultra-high accuracy in reconstruction quality and photorealistic rendering effects, NeRF has received widespread attention and continuous improvement, making it more consistent with physical priors.

[0003] Despite their impressive visual effects, the enormous training time of neural radiation fields limits their practical applications. To accelerate training, some works decompose the end-to-end multilayer perceptron of neural radiation fields into coordinate-based latent features and a lightweight decoder, partially replacing them with explicit representations. These methods significantly accelerate radiation field reconstruction due to the easier optimization and lower inference time of explicit representations. On the other hand, some works have designed sampling strategies for points on the ray tracing during volume rendering, allowing the algorithm to select sampling points only near the object's surface, thus reducing the computational cost of radiation field training. Although these methods have achieved significant training speedups, a considerable gap remains before real-time reconstruction. This is because radiation field reconstruction is an ill-posed problem; gradient descent-based methods require a large number of iterations to converge the optimization process.

[0004] Existing methods typically select rays uniformly from the training set when training a radiation field. However, the importance of different rays often varies significantly. For example, rays in high-frequency regions are often more difficult to fit than those in low-frequency regions. Therefore, uniform, passive ray sampling leads to inefficient training. Conversely, actively selecting rays and concentrating a limited number of training iterations on the most valuable rays could significantly improve the speed of radiation field training. However, actively selecting the most valuable rays presents several challenges. First, the state of a ray is locally observable. Specifically, to fully and accurately observe the state of each ray, all rays in the training set must be rendered, which is extremely time-consuming. Therefore, in practice, only a small portion of the ray states can be observed. Second, the observed ray states are often outdated. To avoid the additional time consumption from extra rendering, existing algorithms often reuse the loss values ​​calculated by gradient descent for each ray as the observed state values. However, since the radiation field parameters have been updated by gradient descent, these reused loss values ​​often deviate from the true values, i.e., they are outdated. Third, the selection of light sources needs to balance "exploring potential better light sources" with "utilizing known better light sources." If always choosing known better light sources, the algorithm may get stuck in local optima because the state values ​​of the light sources are not accurate. Conversely, excessive exploration can lead to wasting too many training iterations on low-value light sources. Fourth, light sampling strategies are prone to overfitting to specific scenes. Data-driven sampling strategies tend to remember specific prior knowledge of the scene in the training set, resulting in poor performance in unfamiliar scenes. Fifth, the execution of the light sampling strategy needs extremely low complexity. This is because the training process involves many iterations, and in each iteration, the sampling strategy may be executed several times to sample a batch of light sources. Therefore, even a slight time overhead introduced by the sampling strategy can drastically increase the overall training time. Consequently, some computationally intensive decision-making algorithms or neural networks with a large number of parameters are not suitable for this light sampling strategy. Summary of the Invention

[0005] This invention provides an active ray sampling method and system for accelerating radiation field training, which adaptively samples the most valuable rays for training, thereby enabling training to converge with fewer iterations and accelerating radiation field reconstruction.

[0006] One aspect of the present invention is to provide an active ray sampling method for accelerating radiation field training, the steps of which include:

[0007] Establish a ray loss diffusion model based on rendering gradient;

[0008] Establish an update model for the state of light;

[0009] Design an online reinforcement learning ray sampling strategy based on the upper confidence bound (UCB);

[0010] Furthermore, the establishment of a ray loss diffusion model based on rendering gradients, which propagates the new loss value of the current sampled ray to the remaining rays, includes:

[0011] By using a ray loss propagation model based on rendering gradients, the loss value of local rays is propagated to other rays that observe the same object point;

[0012] The confidence level of diffusion is calculated based on the differences in light color to filter out meaningless diffusion.

[0013] Furthermore, the establishment of the ray state update model, which updates the current estimated loss value of the ray using the loss value obtained from diffusion, includes:

[0014] The loss value obtained from diffusion does not directly overwrite the original estimated loss value of the ray. Instead, it updates the current estimated loss value of the ray with the diffusion confidence as the weight. Furthermore, a confidence level is maintained for the estimated loss value of each ray, and the confidence level of the estimated loss value is updated using the diffusion confidence level in an exponentially weighted average manner.

[0015] For light rays that are not diffused, they directly inherit the estimated loss value and confidence level from the previous round.

[0016] Furthermore, the establishment of the online reinforcement learning ray sampling strategy based on the upper bound of the confidence interval includes:

[0017] An online reinforcement learning algorithm based on the upper bound of the confidence interval is employed to maintain not only the estimated loss value for each ray. Also estimate the upper bound of the confidence interval for the loss value. The upper bound of the confidence interval determines the probability of light sampling, i.e. The higher the value, the higher the sampling probability of the light.

[0018] Another aspect of the present invention is to provide an active light sampling system for accelerating radiation field training, comprising:

[0019] The probability sampling module for light rays is used to select a batch of light rays from the training set at the beginning of each iteration according to the current light ray sampling probability for this round of training;

[0020] The ray state update and radiation field training module is used to render the rays sampled by the ray probability sampling module and calculate the loss value. The loss value is used for gradient descent optimization of the radiation field and loss diffusion and update of the state of all rays.

[0021] The ray sampling probability update module is used to learn the optimal ray sampling strategy online and determine the new ray sampling probability based on the updated ray state for the next round of training.

[0022] Furthermore, the probability sampling module for the light rays is also used for:

[0023] At the start of each iteration t, this module updates the ray sampling probability given by the module based on the ray sampling probability in the previous iteration t-1. Sample a batch number of rays R from the training set Ω. t The state update transmitted to the light source and the radiation field training module are used for radiation field training;

[0024] Update the time interval between each ray and the last sampled ray.

[0025] Furthermore, the light state update and radiation field training module is also used for:

[0026] Receive the set of rays R sampled in this round from the ray probability sampling module. t Perform volume rendering on these rays and calculate the loss value.

[0027] During the rendering process, the coordinates of the object point observed by each ray are calculated based on the rendering gradient;

[0028] Loss value The diffusion is performed on other light rays that observe the same object point, and filtered according to the color difference between the original light ray and the diffused light ray to calculate the confidence level of the diffusion.

[0029] Update the estimated loss value of the ray with the loss value of diffusion. In addition, the confidence level of the loss value is also updated.

[0030] Calculate the loss value Gradient descent is used to optimize the radiation field parameters by considering the derivatives of the parameters.

[0031] Furthermore, the light sampling probability update module is also used for:

[0032] The sampling time interval is calculated based on the probability sampling module of the light. The estimated loss value calculated by the light state update and radiation field training module. Loss value confidence level Calculate the new ray sampling probability;

[0033] The new sampling probability is passed to the probability sampling module of the light beam for use in the next round of light beam sampling.

[0034] The active ray sampling method and system for accelerating radiation field training provided by this invention propagates the new loss value of the sampled ray to the remaining rays through a ray loss diffusion model based on rendering gradients. It then updates the current ray state using the diffused loss value through a ray state update model, thereby addressing the problem that ray states are always locally observable. By employing an online reinforcement learning ray sampling strategy based on UCB, the estimation bias of ray states is included within the confidence interval, solving the problem of outdated ray states. It requires no pre-training, avoiding overfitting to specific scenes; it has low complexity, avoiding excessive additional time overhead; and it balances exploration and utilization, avoiding getting trapped in local optima or causing low training efficiency. Therefore, the active ray sampling method and system for accelerating radiation field training provided by this invention can adaptively select the most valuable rays for training, thus significantly accelerating the radiation field reconstruction process.

[0035] This invention can be applied to the fields of 3D reconstruction and virtual viewpoint synthesis, with applications including immersive 3D volumetric videos. Utilizing high-quality 3D reconstruction and photorealistic rendering effects from radiation fields, these volumetric videos allow users to freely view them from different angles, significantly enhancing the user's sense of immersion and realism. The technology described in this invention can greatly accelerate the training process of radiation fields, reduce the time and hardware costs of generating immersive volumetric video content, enable real-time communication (RTC) based on radiation fields, and facilitate the implementation of many data-real fusion applications, such as free-viewpoint viewing and virtual reality live streaming. Attached Figure Description

[0036] Figure 1 A flowchart of the active light sampling method provided in an embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram of the structure of an active light sampling system provided in an embodiment of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] This invention addresses the problem of low training efficiency caused by uniform light sampling in existing radiation field training methods by providing an active light sampling method. Figure 1 The flowchart for active light sampling provided in this embodiment of the invention mainly includes the following steps:

[0040] Step 101: Establish a ray loss propagation model based on rendering gradients to propagate the new observation loss value of the current sampled ray to the remaining rays.

[0041] In this embodiment, to select the most valuable ray, the decision algorithm needs to evaluate the state of each ray in the training set. Specifically, in this embodiment, the loss value of a ray represents its state. However, to avoid introducing additional computational time, this embodiment does not additionally render any ray for precise loss value evaluation. Instead, it maintains an estimated loss value for each ray and reuses the loss values ​​of local rays calculated for gradient descent in each iteration to update the loss estimate of all rays.

[0042] This embodiment utilizes a ray loss diffusion model based on rendering gradients to propagate the loss values ​​of local rays to other rays, thereby updating the loss values ​​of more rays. Let R... t Let r be the set of rays sampled in the t-th round. Perform volume rendering on each ray r and calculate the loss value according to the following formula.

[0043]

[0044] in Let C(r) be the color obtained by volume rendering of ray r in the radiation field, and let C(r) be the true color of ray r.

[0045] This embodiment assumes that the light loss values ​​observed at the same object point should be similar. Therefore, in order to minimize the loss value... The light rays diffuse to the remaining rays that observe the same object point. In this embodiment, the object point coordinates observed by ray r are first estimated based on the rendering gradient using the following formula:

[0046]

[0047] Where c i Let N be the color of the i-th sampling point during the volume rendering process, and N be the total number of sampling points on the ray. This embodiment will determine the final rendered color. The sampling point with the greatest impact is considered as the object point observed by ray r, therefore, by taking... For c i The maximum gradient is then calculated. Subsequently, this point is mapped onto the imaging planes of the other cameras k∈K to find the other rays r that also observed this point. k .

[0048] Due to occlusion, r and r k What may still be observed are different object points. Furthermore, even if r and r kEven when the same object point is observed, due to the perspective-dependent nature of color (such as reflection), the two objects will still appear to be completely different colors. In the above situation, the loss of ray r is diffused to ray r. k It is meaningless. Therefore, the spread of loss will be filtered out. First, r and r k The color differences will be calculated:

[0049]

[0050] Where C(r) represents the true color of r, C(r) k ) represents r k The true color.

[0051] Then, the spread of loss will be filtered based on color differences:

[0052]

[0053] in τ is the confidence level of the diffusion. τ is the filtering threshold, used to control the range of loss diffusion. In this embodiment, τ is specifically set to 0.2.

[0054] In summary, the loss of ray r is diffused to more rays r. k (k∈K). Confidence level when diffusion has no practical significance. A value of 0 indicates that the diffusion was filtered out.

[0055] Step 102: Establish an update model for the light state, and update the current estimated loss value of the light with the loss value obtained from diffusion.

[0056] In this embodiment, the loss value obtained from diffusion does not directly overwrite the original loss value of the light rays, but is updated using the confidence level of diffusion as the weight. Specifically:

[0057]

[0058] in Estimate the loss value for the updated ray. The confidence level for estimating the loss value of a ray. μ is a hyperparameter controlling the update rate. For rays that are not diffused, they directly inherit the estimated loss value and confidence level from the previous round:

[0059]

[0060] In summary, based on the new loss values ​​of the sampling rays in this round, the estimated loss values ​​for each ray are... With confidence level All of them were updated.

[0061] Step 103: Design an online reinforcement learning ray sampling strategy based on UCB.

[0062] This embodiment employs a reinforcement learning algorithm to determine the sampling probability of each ray under different states, balancing exploration and utilization to maximize training efficiency. The characteristics of radiation field reconstruction impose several limitations on the design of this reinforcement learning algorithm. First, the reconstruction process involves numerous iterations, and in each iteration, the reinforcement learning algorithm needs to make decisions to sample rays. Therefore, even a slight increase in time overhead can lead to a sharp rise in the total training time. This limits the requirement for extremely low complexity in reinforcement learning algorithms. Second, most advanced reinforcement learning algorithms integrate data-driven neural networks, which may cause the algorithm to memorize prior knowledge of specific scenes in the training set, resulting in poor performance in unseen new scenes. Third, due to updates to radiation field parameters, the loss value used for diffusion is often outdated, leading to biases in the estimated ray loss values ​​and affecting the decision-making of the reinforcement learning algorithm.

[0063] Considering the aforementioned limitations, this embodiment proposes an online reinforcement learning algorithm based on UCB. First, this algorithm has extremely low complexity, thus avoiding excessive additional time consumption. Second, this algorithm does not require pre-training and can be co-trained from scratch with the radiation field, thus exhibiting robustness to unseen new scenes. Third, this algorithm not only utilizes the estimated loss value of each ray... The upper bound of the confidence interval for the loss value was also estimated. Compared to selecting rays solely based on their loss values, this algorithm determines ray sampling probabilities based on the upper bound of the confidence interval, thereby reducing the impact of bias in the ray loss values. (Upper bound of the confidence interval) The calculation method is as follows:

[0064]

[0065] in The time interval between the light ray r and the last sampled point is initialized to 0. It is also initialized to 0. Initialize to a small non-zero value, such as 1. -8 γ is a temperature coefficient, used to control... The effect on the upper bound of the confidence interval. Ω is the set of all rays in the training set. r ′ For each ray traversed in Ω, For light r ′ The time interval since the last sampling. Ultimately, the sampling probability of the light ray is the normalized value of the upper bound of the confidence interval:

[0066]

[0067] This UCB algorithm addresses the following issues: First, the loss of light rays decreases as optimization progresses; therefore, bias or outdated estimations of the loss value can lead to it being greater than the true value. Furthermore, the loss value from the previous sampling is often greater than the current loss value, making it reasonable to use the previous sampled value to estimate the upper confidence interval of the current loss value. Second, when the confidence level of the loss value... The smaller the value, the less accurate the estimated loss, and therefore the larger the confidence interval. Third, in the early stages of training, the confidence level is [not specified] for light rays that are neither sampled nor diffused. The initial value of 0 results in an infinitely large upper bound for the confidence interval. In this case, the algorithm will prioritize selecting completely unexplored rays. Simultaneously, due to diffusion, both direct sampling and diffusion will serve an exploration function, thus increasing the exploration efficiency by a factor of |K|. Fourth, since the actual loss value of a ray changes continuously with the optimization of the radiation field parameters, the longer a ray has not been sampled, the greater the deviation in its estimated loss value will be, thus increasing the upper bound of the confidence interval. Fifth, due to the significant scale differences among the three terms in formula (7) (estimated loss value, confidence interval based on confidence level, and confidence interval based on sampling time), unlike the addition-based merging in traditional UCB, this embodiment uses multiplication to merge the three.

[0068] In this embodiment, a ray loss diffusion model based on rendering gradients is used to propagate the new loss value of the ray sampled in the current round to the remaining rays. A ray state update model is then used to update the current state of the rays with the diffused loss value, thus addressing the problem that ray states are always locally observable. An online reinforcement learning ray sampling strategy based on UCB incorporates the estimation bias of ray states into the confidence interval, solving the problem of outdated ray states. No pre-training is required, avoiding overfitting to specific scenes; the complexity is low, avoiding excessive additional time overhead; and it balances exploration and utilization, avoiding getting trapped in local optima or causing low training efficiency. Therefore, the active ray sampling method and system for accelerating radiation field training provided by this invention can select the most valuable rays for training without introducing excessive additional computational overhead, thereby significantly accelerating the reconstruction process of the radiation field.

[0069] Figure 2 This is a schematic diagram of the structure of an active ray sampling system for accelerating radiation field training provided in an embodiment of the present invention. The system includes:

[0070] The probability sampling module 11 for light rays is used to select a batch of light rays from the training set according to the current light ray sampling probability at the beginning of each iteration for this round of training.

[0071] The ray state update and radiation field training module 12 is used to render the sampled rays and calculate the loss value. The loss value is used for both gradient descent optimization of the radiation field and loss diffusion and updating of the overall ray state.

[0072] The ray sampling probability update module 13 is used to learn the optimal ray sampling strategy online and decide on the new ray sampling probability based on the updated state of the ray for the next round of training.

[0073] Furthermore, the probability sampling module 11 for light rays is also used for:

[0074] At the start of each iteration t, module 11 uses the light sampling probability given by module 13 in the previous iteration t-1. Sample a batch number of rays R from the training set Ω. t This is passed to module 12 for radiation field training;

[0075] Update the time interval between each ray and the last sampled ray.

[0076] Furthermore, the state update and radiation field training module 12 for light is also used for:

[0077] Receive the set of rays R sampled in this round from module 11. t Perform volume rendering on these rays and calculate the loss value.

[0078] During the rendering process, the object point coordinates observed by each ray are calculated according to formula (2). Since most of the calculation steps can reuse the rendering formula, almost no additional time is introduced.

[0079] Loss value The diffusion is spread to other light rays that observe the same object point, and filtered according to formula (4) to calculate the confidence level of the diffusion;

[0080] Update the estimated loss value of the ray with the loss value of diffusion. In addition, the confidence level of the loss value is also updated.

[0081] Calculate the true loss value Gradient descent is used to optimize the radiation field parameters by considering the derivatives of the parameters.

[0082] Furthermore, the light sampling probability update module 13 is also used for:

[0083] The sampling time interval calculated by module 11 The estimated loss value calculated with module 12 Loss value confidence level The new light sampling probability is calculated according to formulas (7) and (8).

[0084] New sampling probability It is passed to module 11 for the next round of light sampling.

[0085] The active light sampling system for accelerated radiation field training in this embodiment can be used to perform... Figure 1 The technical solutions of the method embodiments shown are similar in principle and in effect, and will not be described again here.

[0086] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0087] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0088] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0089] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. For example:

[0090] One embodiment of the present invention provides a computer device including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the steps of the method of the present invention.

[0091] One embodiment of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a computer, implements the various steps of the method of the present invention.

[0092] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is merely an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An active light sampling method for accelerating radiation field training, characterized in that, Includes the following steps: Establish a ray loss diffusion model based on rendering gradients to diffuse the new loss value of the current sampled ray to the remaining rays; Establish an update model for the state of light, and update the current estimated loss value of the light using the loss value obtained from diffusion; An online reinforcement learning ray sampling strategy based on the upper bound of the confidence interval is established, and the ray sampling probability is determined according to the upper bound of the confidence interval of the loss value; The establishment of the ray state update model, which updates the current estimated loss value of the ray using the loss value obtained from diffusion, includes: the loss value obtained from diffusion does not directly overwrite the original estimated loss value of the ray, but updates the current estimated loss value of the ray with the diffusion confidence as the weight; furthermore, a confidence level is maintained for the estimated loss value of each ray, and the confidence level of the estimated loss value is updated using the diffusion confidence level in an exponentially weighted average manner; for rays that have not been diffused, they directly inherit the estimated loss value and confidence level from the previous round. The update of the current estimated loss value of the light rays using diffusion confidence as a weight includes: Calculate the loss value using the following formula. : in For light The color obtained by volume rendering in a radiation field. For light The true color; Calculate light and other light Color differences: in, express The true color, express The true color; The diffusion of loss is filtered based on color differences: in It is the diffusion confidence level; It is a filtering threshold used to control the range of loss propagation; Then update the current estimated loss value of the light: in Estimate the loss value for the updated ray. Confidence level for estimating the loss value of light rays It is a hyperparameter that controls the update rate; The establishment of the online reinforcement learning ray sampling strategy based on the upper bound of the confidence interval includes: employing an online reinforcement learning algorithm based on the upper bound of the confidence interval, which not only maintains the estimated loss value for each ray... It also estimates the upper bound of the confidence interval for the loss value. The upper bound of the confidence interval determines the probability of light sampling, i.e. The higher the value, the higher the sampling probability of the light. The upper bound of the confidence interval The calculation method is as follows: in For light The time interval since the last sampling. Confidence level for the loss value; For temperature coefficient, control Impact on the upper bound of the confidence interval; To train the set of all light rays, for Each ray of light traversed in the middle, For light The time interval since the last sampling; ultimately, the sampling probability of the light is the normalized upper bound of the confidence interval: .

2. The method according to claim 1, characterized in that, The establishment of a ray loss diffusion model based on rendering gradients, which diffuses the new loss value of the current sampled ray to the remaining rays, includes: By using a ray loss propagation model based on rendering gradients, the loss value of local rays is propagated to other rays that observe the same object point; The confidence level of diffusion is calculated based on the differences in light color to filter out meaningless diffusion.

3. An active light sampling system for accelerating radiation field training, characterized in that, The system for performing the method of claim 1 or 2 includes: The probability sampling module for light rays is used to select a batch of light rays from the training set at the beginning of each iteration according to the current light ray sampling probability for this round of training; The ray state update and radiation field training module is used to render the sampled rays and calculate the loss value. The loss value is used for gradient descent optimization of the radiation field and loss diffusion and update of the state of all rays. The ray sampling probability update module is used to learn the optimal ray sampling strategy online and determine the new ray sampling probability based on the updated state of the ray for the next round of training. The light state update and radiation field training module is also used for: Receive the set of rays sampled in this round from the ray probability sampling module. Perform volume rendering on these rays and calculate the loss value. ; During the rendering process, the coordinates of the object point observed by each ray are calculated based on the rendering gradient; Loss value The diffusion is performed on other light rays that observe the same object point, and filtered according to the color difference between the original light ray and the diffused light ray to calculate the confidence level of the diffusion. Update the estimated loss value of the ray with the loss value of diffusion. In addition, the confidence level of the loss value is also updated. ; Calculate the loss value Gradient descent is used to optimize the radiation field parameters by considering the derivatives of the parameters. The light sampling probability update module is also used for: The sampling time interval is calculated based on the probability sampling module of the light. The estimated loss value calculated by the light state update and radiation field training module. Confidence level of loss value Calculate the new light sampling probability; The new sampling probability is passed to the probability sampling module of the light beam for use in the next round of light beam sampling.

4. The system according to claim 3, characterized in that, The probability sampling module for the light rays is also used for: Each iteration Initially, the probability sampling module for light rays is based on the previous iteration. The ray sampling probability update module provides the ray sampling probability. From the training set Sample a batch of light The state update transmitted to the light source and the radiation field training module are used for radiation field training; Update the time interval between each ray and the last sampled ray. .

5. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of claim 1 or 2.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a computer, implements the method of claim 1 or 2.

Images