Optimization method and system for micro rendering of participating medium

Abstract

The invention relates to the technical field of graphic rendering and computer graphics, in particular to an optimization method and system for micro rendering of a participating medium, and the method comprises the following steps: introducing a path space integral formula to obtain a target integral value, performing material form re-parameterization conversion on the target integral value to obtain the differential of the material form path integral of any scene parameter; establishing a KD-Tree structure storage mechanism, and obtaining a leaf node output result by using the storage mechanism; determining a boundary according to a leaf node output result and obtaining a sampling boundary section, performing unbiased estimation on the boundary section and obtaining an unbiased estimation numerical result of the boundary section; constructing a gradient multiplexing model, and obtaining model parameters updated by the optimizer through the multiplexing model; and performing optimization adjustment on the initialized scene parameters by combining the differential of the substance form path integral, the unbiased estimation numerical result and the updated model parameters. According to the method, a solution is provided for micro rendering of the participating medium, and a clear direction is provided for further optimization.

Description

Technical Field

[0001] The present invention relates to the field of graphics rendering and computer graphics technology, and in particular to an optimization method and system for micro-rendering of participating media. Background Art

[0002] Physically based differentiable rendering techniques are seeing increasing application in computer vision and graphics. However, high computational complexity for participating media and the presence of redundant computations in the optimization process significantly hinder efficiency gains. Boundary integrals are a unique and critical component of differentiable rendering, and their integration domains often exhibit discontinuities and other issues.

[0003] To address the boundary integral problem, various techniques have been introduced, including reparameterization of the ordinary rendering integral and differentiable path integrals. Existing methods optimize the differentiable path integral for scenes containing only surfaces, thereby improving computational speed and the ability to handle complex scenes. However, Monte Carlo estimation of boundary integrals in scenes with participating media remains an area of ​​urgent research, ultimately resulting in low efficiency and other issues when handling complex retrograde rendering tasks.

[0004] Although progress has been made in differentiable rendering technology, how to more effectively estimate boundary integrals in complex scenes involving participating media remains an urgent problem to be solved. It is necessary to continue exploring and optimizing related technologies to improve rendering efficiency and accuracy in complex scenes. Summary of the Invention

[0005] In view of the defects of existing methods and the shortcomings of practical applications, in order to reduce the computational complexity of boundary integrals and improve overall efficiency; at the same time, gradient caching and reuse are realized, the number of rendering times required in the reverse rendering task is reduced, and the optimization process is quickly converged. In the first aspect, the present invention provides an optimization method for participating medium differentiable rendering, which includes the following steps: introducing a path space integral formula to obtain a target integral value, performing a material form reparameterization conversion on the target integral value, and obtaining the differential of the material form path integral of any scene parameter based on the reparameterization conversion result; establishing a KD-Tree structure storage mechanism, and using the KD-Tree structure storage mechanism to obtain the leaf node output result; determining the boundary according to the leaf node output result, obtaining a sampling boundary segment based on the boundary, performing an unbiased estimation on the sampling boundary segment and obtaining an unbiased estimation numerical result of the sampling boundary segment; constructing a gradient reuse model, updating the model parameters of the optimizer through the gradient reuse model, and obtaining the updated model parameters; optimizing and adjusting the initialized scene parameters in combination with the differential of the material form path integral, the unbiased estimation numerical result, and the updated model parameters. The present invention introduces the path space integral formula, establishes the KD-Tree structure storage mechanism, and constructs a gradient reuse model, which further improves the computational efficiency, sampling and estimation accuracy, and brings benefits to the optimization of model parameters and scene parameters.

[0006] Optionally, the introducing a path space integral formula to obtain a target integral value includes: introducing a path space integral formula to convert the light transmission path into an integral form and obtain the target integral value; The target integral value satisfies the following relationship: in, represents the color of the pixel, Represents the path space, represents the measurement contribution function, represents the optical transmission path, express The present invention obtains the target integral value through the path space integral formula, which can not only accurately simulate the light transmission process, but also handle complex geometric shapes and is easy to expand and optimize.

[0007] Optionally, performing a material form reparameterization conversion on the target integral value, and obtaining a differential of a material form path integral of any scene parameter based on the reparameterization conversion result includes: introducing a reference configuration to perform a material form reparameterization conversion on the target integral value; The reparameterization conversion result satisfies the following relationship: in, express The result of the reparameterization transformation is represents the material path space, represents the product of the ordinary measurement contribution and the determinant of the Jacobian resulting from the reparameterization, express The Lebesgue measure on , the path of light that represents material form; The differential of the material form path integral satisfies the following relationship: in, represents the differential of the result of the reparameterization transformation, represents the material path space, represents the product of the ordinary measurement contribution and the determinant of the Jacobian resulting from the reparameterization, express The Lebesgue measure on , representing the path of light in material form, represents a discontinuous space, represents the contribution of the boundary segment, represents the contribution of the source subpath, represents the contribution of the detector subpath, represents the internal item, The present invention performs a material reparameterization on the target integral value and, based on the reparameterization result, obtains the differential of the material path integral for any scene parameter. This method has broad application prospects and potential value in optics, computer graphics, and other related fields.

[0008] Optionally, establishing a KD-Tree structure storage mechanism and utilizing the KD-Tree structure storage mechanism to obtain a leaf node output result includes: dividing the sampling space and obtaining a plurality of divided intervals, wherein the divided intervals correspond to the contour boundaries of objects in the scene; processing the divided intervals to obtain a divided sampling space; generating leaf nodes based on the divided sampling space; setting a global list and storing the leaf nodes in the global list; and combining the leaf nodes and the global list to obtain a leaf node output result. The present invention divides the sampling space into a plurality of intervals, which can efficiently organize and manage multidimensional data, not only making data storage more orderly, but also improving data retrieval efficiency.

[0009] Optionally, obtaining a sampling boundary segment based on the boundary includes: selecting an arbitrary point and an arbitrary direction on the boundary; and obtaining a first sampling distance and a second sampling distance based on the arbitrary direction. The method of obtaining a sampling boundary segment based on a boundary of the present invention is flexible and easy to implement, applicable to various scenarios, supports complex boundaries, and facilitates subsequent processing.

[0010] Optionally, the sampling boundary segment satisfies the following relationship: in, represents the radiance of the boundary term, represents the contribution of the boundary segment, represents an endpoint of a boundary segment, represents the other endpoint of the boundary segment, represents the Jacobian matrix, Another way to express the contribution of the source subpath is, Another way to express the contribution of a detector subpath is represents a boundary segment, represents the source subpath, The model of the present invention can comprehensively consider the interaction between the boundary, light source and detector, as well as the transformation of geometric shapes, facilitates numerical calculation and optimization, and helps support the simulation of complex scenes.

[0011] Optionally, performing unbiased estimation on the sampling boundary segment and obtaining an unbiased estimation numerical result of the sampling boundary segment includes: introducing a Monte Carlo method, performing unbiased estimation on boundary terms in the sampling boundary segment based on the Monte Carlo method, and obtaining an unbiased estimation numerical result of the boundary terms; The unbiased estimation numerical results satisfy the following relationship: in, represents the contribution of the boundary segment, represents an endpoint of a boundary segment, represents the other endpoint of the boundary segment, represents the Jacobian matrix, represents the parameters of the sampling boundary segment, represents the contribution of the source subpath, represents the contribution of the detector subpath, Indicates the source subpath The probability density of the sample, Represents the detector subpath The probability density of the sample.

[0012] The present invention can perform unbiased estimation on the integral of a complex function, thereby avoiding the deviation of traditional numerical methods and further improving the accuracy of the estimation result.

[0013] Optionally, constructing a gradient reuse model, updating the optimizer's model parameters through the gradient reuse model, and obtaining updated model parameters includes: setting a predetermined threshold; obtaining the calculated gradient result in the previous iteration; obtaining the parameter change rate between adjacent iterations based on the gradient reuse model and the calculated gradient result; comparing the parameter change rate with the predetermined threshold, and obtaining updated model parameters based on the comparison result. The present invention constructs a gradient reuse model and updates the optimizer's model parameters through the model, which can improve computational efficiency, accelerate model training, and enhance model generalization capabilities, making the optimization method of the present invention have application prospects and potential value.

[0014] Optionally, the gradient reuse model satisfies the following relationship: in, It represents the rate of change of parameters between two iterations. Represents the parameters in the current iteration, represents the parameters in the previous iteration, Indicates a small positive constant introduced. The present invention uses the rate of change as a quantitative indicator to measure the degree of change in parameters between the current iteration and the previous iteration, which helps to intuitively understand the dynamic behavior of the model during training.

[0015] In a second aspect, the present invention also provides an optimization system for micro-rendering of participating media, which can efficiently execute an optimization method for micro-rendering of participating media provided by the present invention. The system includes an input device, a processor, an output device and a memory, wherein the input device, processor, output device and memory are interconnected. The system provided by the present invention has a compact structure, strong applicability, and greatly improves operating efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Flowchart of the optimization method for micro-rendering of participating media according to the present invention; Figure 2 Schematic diagram of the boundary light path in the optimization method for differentiable rendering of participating media of the present invention; Figure 3 A schematic diagram of boundary segment types in the optimization method for participating medium differentiable rendering of the present invention; Figure 4 A schematic diagram of an approximate gradient in the optimization method for differentiable rendering of participating media of the present invention; Figure 5Schematic diagram of the structure of the optimization system for participating medium micro-rendering of the present invention. DETAILED DESCRIPTION

[0017] Specific embodiments of the present invention will be described in detail below. It should be noted that the embodiments described herein are for illustrative purposes only and are not intended to limit the present invention. In the following description, numerous specific details are set forth to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that these specific details are not necessarily required to practice the present invention. In other instances, well-known circuits, software, or methods are not specifically described to avoid obscuring the present invention.

[0018] Throughout this specification, references to "one embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment of the present invention. Therefore, appearances of the phrases "in one embodiment," "in an embodiment," "an example," or "an example" in various places throughout this specification are not necessarily all referring to the same embodiment or example. Furthermore, the particular features, structures, or characteristics may be combined in any suitable combinations and / or subcombinations in one or more embodiments or examples. Furthermore, those of ordinary skill in the art will appreciate that the figures provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0019] See Figure 1 To reduce the computational complexity of differentiable rendering in scenes with participating media, improve the efficiency of the rendering process, reduce the number of iterations required in the inverse rendering task, and accelerate the convergence of the optimization process, this invention combines Monte Carlo boundary estimation optimization technology with gradient reuse methods to propose an efficient solution for differentiable rendering and point out the direction for subsequent optimization. The present invention provides an optimization method for differentiable rendering with participating media, which includes the following steps: S1. Introduce the path space integral formula to obtain the target integral value, perform a material reparameterization on this target integral value, and then, based on the reparameterization result, obtain the material path integral differential for any scenario parameter. The specific implementation steps and related content are as follows: Firstly, the path space integral formula is introduced to convert the light transmission path into integral form and obtain the target integral value.

[0020] In forward rendering, a path space integral formula was proposed, based on which the problem of light transmission along a path is transformed into an integral problem, which has promoted the development of computer graphics and rendering technology. The above path space integral formula can perform a weighted summation of all possible light paths to obtain the target integral value, where the weight of each path is determined by its transmission probability in the scene.

[0021] The above target integral value satisfies the following relationship: in, represents the color of the pixel, Represents the path space, represents the measurement contribution function, represents the optical transmission path, express The Lebesgue measure on .

[0022] For light transport involving only surfaces, the path space contains an infinite set of all possible light paths. The path space can be viewed as a high-dimensional geometric space where each point represents a specific light path. The above space has a complex topological structure because the light path can undergo multiple reflections, refractions, and scattering, forming a complex path network.

[0023] The path space in the embodiment satisfies the following relationship: in, Represents the path space, represents the union of all object surfaces, Indicates the number of segments in the path.

[0024] In volumetric light transport, the path space formula is no longer limited to the light path between surfaces, but is extended to the light transport in the entire three-dimensional space. It can then express the integral of all possible volumetric paths, where the weight of each path is determined by its transmission probability and radiation intensity in the volume, which can more accurately describe the light transmission process in the volume.

[0025] The Lebesgue measure is a generalization of the traditional concepts of length, area, and volume, allowing the measurement of more complex sets, including sets with fractal properties or infinite discontinuous points.

[0026] The above definition of Lebesgue measure needs to satisfy the following relationship: in, represents the total Lebesgue measure of a light path, Indicates the number of segments in the path, represents the Lebesgue measure, A node representing a ray.

[0027] in, represents the total Lebesgue measure of a light path, satisfying the following relationship: in, represents the Lebesgue measure of each node in this light path, represents the differential area, represents the differential volume, Represents the eigenvalue corresponding to any path node.

[0028] Go further into path space and measurement.

[0029] In this embodiment, the three-dimensional volume In the virtual scene represented by and remaining space When light propagates in the above environment, its path can be accurately described as a set of ordered points (or vertices), namely , in the above set, represents the starting point of the ray, and is the end point of the ray, and the middle point is represented by It can represent the various positions that the light passes through in the process of propagation.

[0030] In order to gain a deeper understanding of the relevant path, the concept of eigenvalue 1 is further introduced. Eigenvalue 1 is a binary number whose number of bits is equal to the number of vertices on the path. Correspondingly, in this binary number, each bit represents a vertex on the path and has a clear physical meaning: if a vertex is on the surface of an object If the vertex is located in the remaining space, the value of the corresponding bit is set to 1; In the object, the value of the corresponding bit is set to 0.

[0031] All possible ray paths can be classified by the eigenvalue 1. For a given number of path segments and a specific eigenvalue of 1, all paths that meet the relevant conditions can be found and grouped into a set.

[0032] Furthermore, the classified path set can be used to analyze the propagation behavior of light in the virtual scene. In an optional embodiment, the length distribution, direction distribution, and interaction between different paths and object surfaces of different categories of paths are calculated, thereby providing a deeper understanding of the propagation laws of light in the scene.

[0033] In summary, a more systematic classification and analysis of light transmission paths in three-dimensional virtual scenes through the concept of eigenvalue l not only helps to better understand the propagation behavior of light in the scene, but also provides strong support for subsequent rendering, lighting optimization and other applications.

[0034] In this embodiment, all paths that meet the conditions must satisfy the following relationship: in, Represents the set of all paths that meet the conditions. The symbol is defined as, Indicates the starting point of the light. represents the end point of the ray path, express One of them, Represents the surface of all objects, It represents the space remaining after removing the surface of the object. Represents the eigenvalue corresponding to any path node.

[0035] Then the path space in the target integral value can be transformed into: in, Represents the path space, Indicates the number of segments in the path, Indicates all paths that meet the conditions.

[0036] Contribution measurements are performed simultaneously.

[0037] The metric contribution of a lightpath is the cumulative product of all vertex contributions and segment contributions. For a given lightpath , whose metric contribution is the product of each vertex and each segment contribution: in, represents the measurement contribution function, represents the optical transmission path, Indicates the number of segments in the path, Indicates that the simplified expression is different according to n. Can be expressed as well as , represents the n-1 vertex in the light path, represents the n vertices in the ray path, represents the n+1 vertex in the light path, Represents a geometric item.

[0038] The light transmission path refers to the path from the light source to the To the camera point transmission path.

[0039] The contribution of each vertex above needs to satisfy the following relationship; in, Indicates different simplified expressions according to n, Can be expressed as well as , represents the n-1 vertex in the light path, express One of them, represents the n+1 vertex in the light path, represents geometric terms, represents the bidirectional scattering distribution function (BSDF), represents the scattering coefficient, Represents the phase function in volume rendering, represents the radiant energy of the light source, Represents pixel weight, which is equivalent to the importance function in rendering. Represents the surface of all objects, Indicates the space remaining after removing the surface of an object.

[0040] further, and They capture the radiance emitted by the light source and the weight of the camera respectively.

[0041] Therefore, each contribution needs to satisfy the following relationship; in, represents geometric terms, represents the visibility function, represents the transmittance between x and y, Represents light and The cosine value of the angle between the surface normal and the Represents light and The cosine value of the angle between the surface normal and the A point on the surface, Represents a point on another surface.

[0042] above The following relationship needs to be satisfied; in, Represents light and The cosine value of the angle between the surface normal and the represents the normal at the surface point x, Indicates the direction of the light connecting the two parts, Represents the surface of all objects, Indicates the space remaining after removing the surface of the object because .

[0043] Going further, represents the transmittance between x and y, and satisfies the following relationship; in, represents the transmittance between x and y, represents the length of the line segment connecting the two parts, represents the extinction coefficient, represents a measure of the length of the curve, Represents a point between x and y.

[0044] Then, a reference configuration is introduced to perform a material form reparameterization transformation on the target integral value.

[0045] Based on the relation Available The color of the pixel, differentiable rendering involves I for an arbitrary scene parameter Derivative, but when the scene geometry depends on the parameters The path space will also depend on , that is, the scene geometry changes with the parameter changes, the path space also depends accordingly , the above dependency increases the complexity of the derivation, because the path itself becomes a variable function that needs to be considered. In order to overcome the related difficulties, a strategy is adopted in the embodiment to introduce a mapping , which can change the path from depending on The scene geometry is mapped to an independent reference configuration, ensuring that calculations can be performed in a fixed, system-independent reference frame regardless of how the scene geometry changes.

[0046] Through mapping The path in the original path integral is reparameterized. Specifically, the original light path in the scene geometry Transformed into material paths in the reference configuration , you can get This transformation simplifies the form of the path integral and makes the derivation of the path integral more direct and feasible.

[0047] The above reparameterization conversion results satisfy the following relationship: in, express The result of the reparameterization transformation is represents the material path space, represents the product of the ordinary measurement contribution and the determinant of the Jacobian resulting from the reparameterization, express The Lebesgue measure on , the path of light that represents material form; In the embodiment, the material path space is defined as: in, represents the complete material path space in the scene, Indicates the number of segments in the path, Represents the various material light paths that may appear in the scene, and l represents the path characteristics.

[0048] The above-mentioned material path space needs to further satisfy the following relationship: in, Represents the various material light paths that may appear in the scene, represents the material path, Represents the surface in material space, Represents the volume of material space. Represents the eigenvalue corresponding to any path node.

[0049] In the framework of differentiable rendering, in order to study the scene parameters For images In this embodiment, the image I is firstly subjected to a reparameterization transformation. This transformation process transforms the image I and the scene parameters The relationship between them is expressed in a new form, which enables a more direct analysis of the parameters Impact on image I.

[0050] In the embodiment, mapping or transformation is introduced to convert the original The image I is converted into a new variable, which no longer directly depends on , but through a differentiable function and In association, the above transformation can be called a reparameterization transformation.

[0051] After the reparameterization is completed, the transformed result is further differentiated. The above process involves the derivation of the transformation function and its related physical quantities to reveal the scene parameters The differential relationship between the image I and the image I is not only helpful for understanding the parameters The local impact on the image I can also be used to guide parameter optimization and adjustment during the image rendering process.

[0052] The differential of the path integral of the above material form satisfies the following relationship: in, represents the differential of the result of the reparameterization transformation, represents the material path space, represents the product of the ordinary measurement contribution and the determinant of the Jacobian resulting from the reparameterization, express The Lebesgue measure on , representing the path of light in material form, represents a discontinuous space, represents the contribution of the boundary segment, represents the contribution of the source subpath, represents the contribution of the detector subpath, represents the internal item, Represents a boundary item.

[0053] In the context of differentiable rendering, we focus on computing the derivatives of the forward rendering result I with respect to the scene parameters θ , this derivative reveals how small changes in scene parameters such as object geometry, position, material, etc. affect the final rendered image.

[0054] in Indicates that I Differentiate the output, focusing on calculating the forward rendering result I relative to the scene parameters The derivative of The above derivative results reveal the final impact of slight changes in scene parameters such as object geometry, position, material, etc. on the rendered image.

[0055] In order to accurately calculate the above derivatives, the rendering process is decomposed into two main parts in this embodiment: source subpath , the source subpath represents the path from one end point of the boundary segment in the scene to the light source in the scene; and the detector subpath It represents the path from the other end point of the boundary segment to the camera.

[0056] In the differentiable rendering framework, the contributions of the above two sub-paths are clearly distinguished and modeled. The contribution of the source sub-path mainly reflects the influence of various rendering factors in the sub-path from the boundary segment to the light source on the rendering results. By differentiating the contributions of the two sub-paths, we can obtain their respective contributions to the derivatives. This not only reveals the direct impact of scene parameters on the rendering results, but also provides an effective method to optimize and adjust rendering parameters to obtain higher quality rendered images.

[0057] In the complex computational process of differentiable rendering, internal terms, i.e., contributions from paths that do not involve scene boundaries, can be easily solved using automatic differentiation techniques. This technique can efficiently compute derivatives, greatly simplifying the processing of internal terms.

[0058] However, the situation becomes complicated when we turn to boundary terms. Boundary terms involve sampling of a specific light path, which must contain a boundary segment. Since the determination of a specific light path requires precise mathematical modeling and complex sampling strategies, the calculation of boundary terms becomes extremely challenging. In order to accurately calculate the boundary terms, in this embodiment, it is first necessary to clarify the integration domain. The integration domain is composed of these specific light paths containing boundary segments. Once the boundary segment is determined, the contribution of the boundary segment, the contribution of the source subpath, and the contribution of the detector subpath can be further determined. The key terms can be expressed as 、 and , which together determine the precise impact of the boundary term on the final rendering result. For boundary light path conversion, please refer to Figure 2 ,in represents an endpoint of a boundary segment, represents the other endpoint of the boundary segment, represents the contribution of the boundary segment, represents the contribution of the source subpath, represents the contribution of the detector subpath, and Represent the direction of light travel, Indicates any selected point.

[0059] based on Figure 2 It can be seen that although the calculation of internal terms can be simplified through automatic differentiation technology, the calculation of boundary terms still requires precise sampling of specific optical paths and careful evaluation of the contribution of each related term. The above process not only tests mathematical modeling capabilities, but also places high demands on sampling strategies and numerical calculation methods.

[0060] Furthermore, in this embodiment, obtaining the differential of the material form path integral of any scene parameter is only an optional condition of the present invention. In one or some other embodiments, the method for obtaining the differential of the material form path integral can be adjusted according to the differentiable rendering requirements and actual optimization goals. By adjusting the differential acquisition method, the present invention can be applicable to a variety of different scenes and rendering requirements. Whether it is complex geometric shapes, changing material properties, or dynamic lighting conditions, the present invention can flexibly respond according to actual conditions, which helps to improve the quality of rendered images while reducing the waste of computing resources.

[0061] S2. Establish a KD-Tree structure storage mechanism and use it to obtain leaf node output results. The specific implementation steps and related content are as follows: Numerically computing boundary integrals is a challenging task in the prior art. Edge sampling methods were first proposed and subsequently extended to the rendering of participating media. However, existing methods require detecting the object's contour vertices one by one along the optical path, which significantly increases the computational cost when the scene contains complex geometries. To address this challenge, a multi-directional sampling strategy is proposed. This strategy first samples the boundary segments and then uses standard path sampling techniques to process the source subpath and detector subpath within the boundary segments separately.

[0062] In the framework of Monte Carlo estimation, the sampling step of the boundary segment is particularly critical. When considering the scenario of the presence of participating media, the boundary segment can be divided into four types according to the difference in the media on both sides. For details, please refer to Figure 3 , where a, b, c, and d represent the four types of media on both sides. and Represent two nodes in a light path respectively.

[0063] In order to sample the boundary segment, in this embodiment, instead of directly selecting the two endpoints of the boundary, a point x on the boundary (belonging to the boundary set M) and a direction vector ω (belonging to the unit sphere) are selected. ,Right now ),based on Figure 3 It can be seen that although existing methods avoid contour detection, how to effectively sample points and directions in scenes containing participating media remains a challenge. In this embodiment, an innovative method is proposed to address the challenges of effectively sampling points and directions, further improving the efficiency of inverse rendering.

[0064] Expected probability density function under ideal conditions Able to work with ∣ is proportional, but there are two major challenges in actual operation. First, and It is obtained by sampling from the boundary segment. Secondly, the sampling process of the boundary segment is extremely complex, and its contribution may be discontinuous.

[0065] In this embodiment, in order to more accurately describe the boundary segment, a KD-Tree structure storage mechanism is established, the sampling space is divided and multiple divided intervals are obtained, and the above divided intervals correspond to the contour boundaries of objects in the scene; the divided intervals are processed to obtain the divided sampling space; leaf nodes are generated based on the divided sampling space; a global list is set, the leaf nodes are stored in the global list, and the leaf node output results are obtained by combining the leaf nodes and the global list.

[0066] When constructing the KD-Tree structure storage mechanism, the five-dimensional sampling space The above sampling space can be represented as u1, u2, u3, and u4. In an optional embodiment, the space is first divided into multiple sub-intervals according to the first dimension (i.e., u1), and each sub-interval corresponds to the contour boundary of an object in the scene. Based on this, in each sub-interval From the sampling point To the corresponding boundary point The mapping maintains the corresponding continuity.

[0067] Next, for each such subinterval , further construct a Kd-tree to subdivide the remaining sampling space , a five-dimensional axis-aligned bounding box is calculated at each node of the Kd-tree, which covers the sampling space of the node. Then, a linear fit is performed on the function F within this five-dimensional axis-aligned bounding box. If the fitting error is small, that is, below the preset threshold, , then the fitting is considered close enough to the function F and no further division of the node is required. Otherwise, the node needs to be split into two equal-sized sub-nodes.

[0068] When selecting the segmentation dimension, the embodiment comprehensively considers all possible dimensions, where , that is, each coordinate axis in the five-dimensional space. For each dimension, the linear fitting error of the function F in the two child nodes after the split is calculated. Then, the dimension with the largest total fitting error is selected. As the split dimension, the node is split into two sub-nodes along the split dimension. The above process is performed recursively until the fitting error of all nodes is lower than the preset threshold. .

[0069] In this embodiment, the algorithm operation content of the KD-Tree structure storage mechanism is as follows: ALGORITHM 1: Build Kd-trees to approximate the function F in Eq. 1:BuildTree(r,aabb_5D) 2:Input: Current tree node r,the corresponding axis-aligned boundingbox aabb,,and error threshold 3: begin 4:Calculate the first-degree approximation of F within the aabbbounding box, using err to represent the fitting discrepancy 5: if err > then 6: for do 7: Split the box aabb in half along the axis ax 8: Compute first-order fits of F within the two split regions and let be the total fitting error 9: end for 10: ax←arg 11: Split aabb along ,into and 12: Make r an internal node with children and 13: BuildTree( , _5D) 14: BuildTree( , _5D 15:else 16. Make r a leaf node and add it to a (global) list 17: end if 18: end In the process of building the KD-Tree structure storage mechanism, a global list is maintained to record all generated leaf nodes. This strategy provides great convenience for creating discrete probability distributions. The above global list is not only crucial in the tree construction stage, but also plays a core role in the subsequent sampling and rendering processes.

[0070] Each leaf node in the KD-Tree represents a terminal region or unit. These regions are carefully divided during the construction process to ensure that the function F is fully approximated within these regions. The above approximate regions together form a fine division, providing a solid foundation for constructing both accurate and efficient probability distributions.

[0071] During rendering, this discrete distribution is first used to select a suitable leaf node. This step is based on pre-calculated probabilities, ensuring the randomness and uniformity of the selection process.

[0072] Next, we use the inverse method to extract samples from the selected leaf nodes. These samples include, but are not limited to, (u1, u2, u3, u4), representing specific locations within the region corresponding to the leaf node. The inverse method efficiently converts uniformly distributed random numbers into sample points that conform to a specific distribution.

[0073] Based on the above sample points, their boundary segments in the divided space can be further determined. These boundary segments not only help to understand the distribution of samples in the space, but also provide important reference information for subsequent rendering and image processing.

[0074] In summary, this embodiment, by constructing a KD-Tree structure storage mechanism and maintaining a global leaf node list, can effectively create a discrete probability distribution and perform efficient sampling in the divided sampling space. The present invention not only improves the accuracy and efficiency of rendering, but also provides new ideas and methods for research and application in the fields of image processing and computer graphics.

[0075] Furthermore, in this embodiment, the construction method of the KD-Tree structure storage mechanism is only an optional condition of the present invention. In one or some other embodiments, the setting method of the KD-Tree can be adjusted according to the actual rendering requirements and the model operation status, and the tree balance, depth and other parameters can be optimized, thereby reducing the computational complexity during rendering and improving the rendering speed, so that the present invention can better adapt to different rendering requirements and model characteristics.

[0076] S3. Determine the boundary based on the output of the leaf node, obtain the sampling boundary segment based on the boundary, perform an unbiased estimate of the sampling boundary segment, and obtain the unbiased estimate numerical result of the sampling boundary segment. The specific implementation steps and related content are as follows: Obtaining a sampling boundary segment based on the boundary includes: selecting any point and any direction on the boundary; and obtaining a first sampling distance and a second sampling distance according to the any direction.

[0077] In this embodiment, we first select an edge of a geometric object as the starting point; then, we randomly select a point on this edge and record it as (One-dimensional) is a one-dimensional sampling process; then, determine a direction (2D), because the 2D sampling direction has an angle and orientation in 2D space; in addition, in order to fully consider the participating media in the scene, two distance parameters are introduced and , which represent the light along Direction and its opposite direction Distance traveled.

[0078] By setting the above parameters, this embodiment is able to reconstruct the originally complex boundary integral, making it more intuitive and easier to handle. The above reconstruction process not only deepens the understanding of the boundary segment, but also greatly facilitates subsequent rendering and image processing.

[0079] Based on this, the differential of the above material form path integral is rewritten as a boundary integral, and the following relationship is satisfied: in, represents the edge sampling point, Indicates the sampling direction, Indicates the first distance of sampling, Indicates the second distance of sampling.

[0080] Then the above sampling boundary segments satisfy the following relationship: in, represents the radiance of the boundary term, represents the contribution of the boundary segment, represents an endpoint of a boundary segment, represents the other endpoint of the boundary segment, represents the Jacobian matrix, Another way to express the contribution of the source subpath is, Another way to express the contribution of a detector subpath is represents a boundary segment, represents the source subpath, represents the detector subpath.

[0081] 、 Represents the two endpoints of the boundary segment; Represents the source subpath, that is, the subpath from the boundary segment to the light source; represents the detector subpath, that is, the subpath from the boundary segment to the camera; in Represents the Jacobian matrix, which can be used to capture parameters arrive( ) changes; Indicates the above Another way to express the contribution of the source subpath is, Indicates the above Another way to express the contribution of detector subpaths; above The following relationship is satisfied: in, Another way to express the contribution of the source subpath is, represents an endpoint of a boundary segment, represents the other endpoint of the boundary segment, represents the contribution of the source subpath, represents the subpath from the light source to the boundary segment, express The Lebesgue measure on .

[0082] above The following relationship is satisfied: in, Another way to express the contribution of a detector subpath is represents an endpoint of a boundary segment, represents the other endpoint of the boundary segment, represents the contribution of the detector subpath, represents the subpath from the camera to the boundary segment, express The Lebesgue measure on .

[0083] in and They represent the subpath from the light source to the boundary segment, and the subpath from the camera to the boundary segment respectively.

[0084] In this embodiment, performing unbiased estimation on the sampling boundary segment and obtaining an unbiased estimation numerical result of the sampling boundary segment includes: introducing a Monte Carlo method, performing unbiased estimation on the boundary term in the sampling boundary segment based on the Monte Carlo method, and obtaining an unbiased estimation numerical result of the boundary term; By sampling 、 and , we further use the Monte Carlo method for unbiased estimation. The above unbiased estimation numerical results satisfy the following relationship: in, represents the contribution of the boundary segment, represents an endpoint of a boundary segment, represents the other endpoint of the boundary segment, represents the Jacobian matrix, represents the parameters of the sampling boundary segment, represents the contribution of the source subpath, represents the contribution of the detector subpath, Indicates the source subpath The probability density of the sample, Represents the detector subpath The probability density of the sample.

[0085] In practical applications, when faced with a boundary segment with two endpoints, and To define, in order to perform effective sampling, standard one-way or two-way sampling technology is used. Based on the above technology, subpaths are constructed from the two endpoints of the boundary segment. and The two subpaths extend across the scene until they meet or meet certain conditions, such as sampling a light source and camera or reaching maximum depth. During this process, special attention must be paid to accurately sampling boundary segments, which are important channels for light energy to propagate through the scene. Therefore, the sampling strategy for boundary segments directly affects the quality and efficiency of the final rendering results.

[0086] Furthermore, the method for obtaining an unbiased estimated numerical result of the sampling boundary segment in this embodiment is only an optional condition of the present invention. In one or some other embodiments, the calculation method of the unbiased estimated numerical result can be replaced according to the rendering target and actual situation. Since different calculation methods differ in computational complexity and resource consumption, choosing a more efficient calculation method can significantly reduce the time and computing resources required for rendering, thereby improving the overall performance of the system.

[0087] S4. Build a gradient reuse model, use it to update the optimizer's model parameters, and obtain the updated model parameters. The specific implementation steps and related content are as follows: Before running the gradient reuse model to update the optimizer parameters, it is first necessary to set a predetermined threshold and collect and obtain the gradient data generated by the previous round of iteration, that is, obtain the calculated gradient result in the previous iteration in the embodiment.

[0088] Then, the parameter change rate between adjacent iterations is obtained based on the gradient reuse model and the calculated gradient result, and the parameter change rate is compared with a predetermined threshold, and the updated model parameters are obtained according to the comparison result.

[0089] In existing physically-based differentiable rendering (PBDR) frameworks, the optimization process often suffers from inefficiencies, such as minimal gradient descent adjustments at each iteration, which in turn leads to extremely slow convergence of the loss function. Furthermore, this process involves significant computational redundancy. Therefore, in this embodiment, a gradient reuse strategy is designed to reduce unnecessary computational overhead. This strategy focuses on continuously reusing the gradient calculated in the previous iteration unless the parameter change rate between iterations exceeds a predetermined threshold, effectively reducing the frequency of gradient recalculation.

[0090] Furthermore, an iterative optimization diagram is drawn in the embodiment, for details, see Figure 4 Based on the straight line in Figure 1, we can see that the change in gradient descent in each iteration is very small; based on the straight line in Figure 2, we can see that the gradient reuse method uses the gradient calculated in the previous iteration.

[0091] The above gradient reuse model satisfies the following relationship: in, It represents the rate of change of parameters between two iterations. Represents the parameters in the current iteration, represents the parameters in the previous iteration, Indicates the introduction of a small positive constant.

[0092] refers to the parameter value in the previous iteration, In order to further quantify the magnitude of the parameter change between two iterations, the square of the Euclidean norm is used to calculate the difference between them, that is, the following relationship is satisfied: , the above square not only emphasizes the degree of change, but also makes larger parameter differences more significant in the calculation, thereby increasing the sensitivity of the optimization process to significantly changing parameters.

[0093] In order to obtain a standardized, dimensionless measure, a small positive constant is introduced , which can be used to prevent numerical instability and other related problems caused by zero denominators. Then, the overall expression is obtained by taking the square root of the ratio of the change magnitude to the previous parameter size. The above expression provides a proportional change measure relative to the original size of the parameter, which not only takes into account the absolute size of the change, but also eliminates the influence of dimension through normalization. Among them, This ensures that the denominator will not be zero even when the previous parameter value is close to zero, effectively avoiding potential numerical calculation problems. Finally, a standardized, dimensionless measure is provided to represent the change ratio relative to the original size of the parameter, which is of great significance for parameter monitoring and adjustment during the optimization process.

[0094] Furthermore, in this embodiment, the method of updating the optimizer model parameters through the gradient reuse model is only an optional condition of the present invention. In one or some other embodiments, the optimization method of the optimizer model parameters can be modified according to the optimizer model optimization requirements and the actual situation of the model, which can significantly reduce computational redundancy and improve the efficiency of the optimization process. At the same time, it can ensure that the present invention always adopts the optimization strategy that is most suitable for the current scenario, thereby achieving optimal performance.

[0095] S5. Optimize and adjust the initialized scenario parameters by combining the differential of the material form path integral, the unbiased estimation results, and the updated model parameters. The specific steps and implementation content are as follows: In this embodiment, the specific process of optimizing and adjusting the initialization scene parameters by combining the differential of the material form path integral, the unbiased estimation numerical results, and the updated model parameters is as follows: With the widespread application of physically based differentiable rendering in the fields of computer vision and computer graphics, the high computational complexity of scenes with participating media and the computational redundancy in the optimization process have become increasingly prominent. In order to meet practical challenges, this application proposes a new optimization strategy that combines Monte Carlo boundary estimation optimization and gradient reuse technology.

[0096] First, the Monte Carlo boundary estimation method was introduced, and combined with the preprocessing steps and KD-Tree structure, the fast and accurate calculation of boundary integrals was achieved, which significantly reduced the computational complexity and improved the rendering efficiency.

[0097] That is, the estimation algorithm of the boundary integral is improved in the embodiment, aiming to improve the efficiency of physically based differentiable rendering, and the surface primary sample space guidance technology is applied to the participating medium to achieve efficient importance sampling of the boundary light path. Based on this, the sampling efficiency is significantly improved and the computational complexity is reduced.

[0098] In the preprocessing stage, the KD-Tree structure storage mechanism is used to store relevant data and information, which greatly improves the search efficiency and further accelerates the drawing process.

[0099] Secondly, a gradient reuse model was constructed during the optimization phase. By caching and reusing gradient information, the number of renderings during the iteration process was effectively reduced. This not only accelerated the convergence of the optimization process but also further improved overall efficiency. In other words, in the subsequent optimization process, based on gradient reuse technology, combined with caching and reusing gradient information, the number of renderings required for each iteration was reduced, thereby accelerating the convergence of the optimization process.

[0100] The optimization method for participating medium differentiable rendering proposed in this paper performs well in various scenarios and significantly improves the overall efficiency of differentiable rendering. It not only provides an efficient and reliable solution for the application of differentiable rendering, but also points out the direction for further optimization in the future.

[0101] See Figure 5 In an optional embodiment, in order to efficiently execute the optimization method for participating medium micro-rendering provided by the present invention, the present invention also provides an optimization system for participating medium micro-rendering, the above-mentioned system includes a processor, an input device, an output device and a memory, and the processor, input device, output device and memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the optimization method for participating medium micro-rendering and the specific steps of related embodiments provided by the present invention. The optimization system for participating medium micro-rendering of the present invention is structurally complete and objectively stable.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and description of the present invention.

Claims

1. An optimization method for differentiable rendering of participating media, characterized in that: The steps include: A path space integral formula is introduced to obtain a target integral value, the target integral value is subjected to a material form reparameterization conversion, and the material form path integral differential of any scene parameter is obtained based on the reparameterization conversion result; Establishing a KD-Tree structure storage mechanism, and using the KD-Tree structure storage mechanism to obtain leaf node output results; Determine a boundary according to the output result of the leaf node, obtain a sampling boundary segment based on the boundary, perform an unbiased estimation on the sampling boundary segment and obtain an unbiased estimation numerical result of the sampling boundary segment; Constructing a gradient reuse model, updating the model parameters of the optimizer through the gradient reuse model, and obtaining the updated model parameters; The initialized scene parameters are optimized and adjusted in combination with the differential of the material form path integral, the unbiased estimated numerical result, and the updated model parameters.

2. The optimization method for differentiable rendering of participating media according to claim 1, characterized in that: The introducing of the path space integral formula to obtain the target integral value comprises: The path space integral formula is introduced to transform the light transmission path into integral form and obtain the target integral value; The target integral value satisfies the following relationship: in, represents the color of the pixel, represents the path space, represents the measurement contribution function, represents the optical transmission path, express The Lebesgue measure on .

3. The optimization method for differentiable rendering of participating media according to claim 1, characterized in that: The step of performing a material-form reparameterization conversion on the target integral value and obtaining a material-form path integral differential of any scene parameter based on the reparameterization conversion result includes: Introducing a reference configuration to perform a material-form reparameterization transformation on the target integral value; The reparameterization conversion result satisfies the following relationship: in, express The result of the reparameterization transformation is represents the material path space, represents the product of the ordinary measurement contribution and the Jacobian matrix determinant resulting from the reparameterization, express The Lebesgue measure on , The path of light that represents the physical form; The differential of the material form path integral satisfies the following relationship: in, represents the differential of the result of the reparameterization transformation, represents the material path space, represents the product of the ordinary measurement contribution and the Jacobian matrix determinant resulting from the reparameterization, express The Lebesgue measure on , represents the path of light in material form, represents a discontinuous space, represents the contribution of the boundary segment, represents the contribution of the source subpath, represents the contribution of the detector subpath, represents the internal item, Represents a boundary item.

4. The optimization method for differentiable rendering of participating media according to claim 1, characterized in that: The establishing of the KD-Tree structure storage mechanism and obtaining the leaf node output result by using the KD-Tree structure storage mechanism include: Dividing the sampling space and obtaining a plurality of divided intervals, wherein the divided intervals correspond to contour boundaries of objects in the scene; Processing the divided intervals to obtain divided sampling spaces; Generate leaf nodes based on the divided sampling space; Setting a global list, storing the leaf nodes in the global list; The leaf node output result is obtained by combining the leaf node and the global list.

5. The optimization method for differentiable rendering of participating media according to claim 1, characterized in that: The obtaining of a sampling boundary segment based on the boundary comprises: Select any point and any direction on the boundary; The first sampling distance and the second sampling distance are obtained according to any one direction.

6. The optimization method for differentiable rendering of participating media according to claim 5, characterized in that: The sampling boundary segment satisfies the following relationship: in, represents the radiance of the boundary term, represents the contribution of the boundary segment, represents an endpoint of a boundary segment, represents the other endpoint of the boundary segment, represents the Jacobian matrix, Another way to express the contribution of the source subpath is, Another way to express the contribution of a detector subpath is represents a boundary segment, represents the source subpath, represents a detector subpath.

7. The optimization method for differentiable rendering of participating media according to claim 6, characterized in that: The unbiased estimation of the sampling boundary segment and obtaining the unbiased estimation numerical result of the sampling boundary segment comprises: A Monte Carlo method is introduced, and an unbiased estimation is performed on the boundary items in the sampling boundary segment based on the Monte Carlo method, and an unbiased estimation numerical result of the boundary items is obtained; The unbiased estimation numerical result satisfies the following relationship: in, represents the contribution of the boundary segment, represents an endpoint of a boundary segment, represents the other endpoint of the boundary segment, represents the Jacobian matrix, represents the parameters of the sampling boundary segment, represents the contribution of the source subpath, represents the contribution of the detector subpath, Indicates the source subpath The probability density of the sample, Denotes the detector subpath The probability density of the sample.

8. The optimization method for differentiable rendering of participating media according to claim 7, characterized in that: The step of constructing a gradient reuse model, updating the model parameters of the optimizer through the gradient reuse model, and obtaining the updated model parameters includes: Setting a predetermined threshold; Get the calculated gradient results in the previous iteration; Obtaining a parameter change rate between adjacent iterations based on the gradient reuse model and the calculated gradient result; The parameter change rate is compared with the predetermined threshold, and updated model parameters are obtained according to the comparison result.

9. The optimization method for differentiable rendering of participating media according to claim 8, characterized in that: The gradient reuse model satisfies the following relationship: in, It represents the rate of change of parameters between two iterations. Represents the parameters in the current iteration, represents the parameters in the previous iteration, Represents a small positive constant introduced.

10. An optimization system for differentiable rendering of participating media, characterized in that: The system includes a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the optimization method for micro-rendering of participating media according to any one of claims 1 to 9.

Images