A three-dimensional radiation field rendering method, a rendering system, a reconstruction method and a reconstruction system

Abstract

The application provides a three-dimensional radiation field rendering method, a rendering system, a reconstruction method and a reconstruction system. The rendering method comprises the following steps: acquiring a sparse voxel radiation field and camera parameters of a target view angle; determining active voxels; determining anchor points on a virtual grid for each active voxel and generating spatial anchor point Morton coding; separating anisotropic scale information and subdivision mask information of the active voxels from each other by using a metadata hash table to obtain decoupled anchor point coding; determining an axial subdivision direction according to a local directional density gradient of the active voxels, performing anisotropic subdivision by axis bisection, and obtaining subdivided active voxels; and based on the decoupled anchor point coding and the subdivided active voxels, performing differentiable anisotropic sparse voxel rasterization to generate a two-dimensional rendering image of a three-dimensional scene at the target view angle. The application maintains the advantages of high-speed rendering of explicit sparse voxels, reduces redundant isotropic subdivision of thin structures and planar regions, and reduces memory occupation and model storage overhead.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a three-dimensional radiation field rendering method, rendering system, reconstruction method, and reconstruction system based on anisotropic sparse voxel rasterization, and also to a corresponding computer device and computer-readable storage medium. Background Technology

[0002] Novel perspective synthesis aims to generate images of a 3D scene from a target's perspective based on multi-view images of that scene. It is a crucial foundational technology in virtual reality, augmented reality, robot navigation, autonomous driving simulation, digital content creation, and 3D asset generation. Neural radiation fields represent continuous 3D scenes through multilayer perceptrons and can generate high-quality images. However, training and inference typically rely on extensive sampling and network queries, making it difficult to meet the deployment requirements of real-time interaction and resource-constrained devices.

[0003] To improve efficiency, explicit representation methods such as 3D Gaussian splashing, explicit voxel meshes, and sparse voxel rasterization have been widely studied. Among them, sparse voxel rasterization can combine voxel structures with differentiable rasterization, supporting end-to-end optimization based on image errors while maintaining high rendering speed. However, existing sparse voxel rasterization typically uses an isotropic octree voxel structure, where each subdivision simultaneously halves along three coordinate axes and generates eight child voxels. But in real 3D scenes, structures such as walls, tabletops, facades, railings, thin cables, and thin edges often have significant directionality and anisotropy. If isotropic voxels are used to represent these structures, in order to capture high-frequency changes in one direction, the system must also subdivide simultaneously in other low-varying directions, resulting in an exponentially increasing number of voxels and redundant parameters. This increases memory usage, model storage size, and training time, and may limit the effective reconstruction of thin structures.

[0004] Furthermore, traditional Morton coding typically serves the dual purpose of spatial location ordering and voxel-level scale representation. In anisotropic voxel scenes where each axis requires independent scale recording, directly stuffing axial hierarchy or scale information into a fixed-length code will compress the number of bits available for spatial coordinates, thereby reducing the highest achievable spatial resolution.

[0005] Therefore, there is a lack of a scheme in the art that can both preserve the spatial locality and sorting efficiency of Morton coding and express three-dimensional radiation field reconstruction at the anisotropic voxel scale. Currently, no descriptions or reports of techniques similar to this invention have been found, and no similar materials from domestic or international sources have been collected. Summary of the Invention

[0006] To address the aforementioned shortcomings in the existing technology, this invention provides a three-dimensional radiation field rendering method, rendering system, reconstruction method, and reconstruction system based on anisotropic sparse voxel rasterization, and also provides a corresponding computer device and computer-readable storage medium.

[0007] According to a first aspect of the present invention, a three-dimensional radiation field rendering method is provided, comprising: Acquire the sparse voxel radiation field representing the three-dimensional scene and the camera parameters corresponding to the target viewpoint; wherein, the sparse voxel radiation field includes multiple voxel units for recording density, color and spatial structure information; Identify the voxel units currently involved in encoding, subdivision, and / or rendering in the sparse voxel radiation field as active voxels; For each active voxel, an anchor point position is determined on a preset virtual mesh, and a spatial anchor point Morton code is generated based on the anchor point position. Construct a metadata hash table decoupled from the Morton encoding of the spatial anchor point, and store the anisotropic scale information and subdivision mask information of each active voxel in the metadata hash table to obtain the decoupled anchor point encoding; At least one axial subdivision direction is determined based on the local directional density gradient of each active voxel, and the active voxel is anisotropically subdivided along the axis according to the axial subdivision direction to obtain subdivided active voxels. Based on the decoupled anchor point encoding and the subdivided active voxels, differentiable anisotropic sparse voxel rasterization is performed to generate a two-dimensional rendered image of the three-dimensional scene from the target viewpoint.

[0008] According to a second aspect of the present invention, a three-dimensional radiation field rendering system is provided, comprising: The data acquisition module is used to acquire the sparse voxel radiation field representing the three-dimensional scene and the camera parameters corresponding to the target viewpoint; wherein, the sparse voxel radiation field includes multiple voxel units for recording density, color and spatial structure information. An active voxel determination module is used to determine the voxel units currently participating in encoding, subdivision, and / or rendering in the sparse voxel radiation field as active voxels. The spatial anchor point Morton encoding module is used to determine the anchor point position on a preset virtual grid for each active voxel and generate a spatial anchor point Morton code based on the anchor point position. A decoupled anchor coding module is used to construct a metadata hash table decoupled from the spatial anchor Morton coding. The anisotropic scale information and subdivision mask information of each active voxel are stored in the metadata hash table to obtain the decoupled anchor coding. An anisotropic subdivision module is used to determine at least one axial subdivision direction based on the local directional density gradient of each active voxel, and to perform anisotropic subdivision of the active voxel along the axis based on the axial subdivision direction to obtain subdivided active voxels. An anisotropic rasterization module performs differentiable anisotropic sparse voxel rasterization based on the decoupled anchor point encoding and the subdivided active voxels to generate a two-dimensional rendered image of the three-dimensional scene from the target viewpoint.

[0009] According to a third aspect of the present invention, a method for reconstructing a three-dimensional scene is provided, comprising: Acquire multi-view images and camera parameters of a 3D scene; Based on the multi-view images and the camera parameters, initialize the sparse voxel radiation field representing the three-dimensional scene; Based on the sparse voxel radiation field and the camera parameters corresponding to the target viewpoint, the three-dimensional radiation field rendering method described above in this invention is used to generate a two-dimensional rendering image of the three-dimensional scene under the target viewpoint. The loss function is determined based on the difference between the two-dimensional rendered image and the corresponding real image; The active voxel parameters in the sparse voxel radiation field are updated according to the loss function. Pruning, anisotropic subdivision, and parameter optimization are performed on the sparse voxel radiation field to obtain a three-dimensional radiation field reconstruction model of the three-dimensional scene.

[0010] According to a fourth aspect of the present invention, a three-dimensional scene reconstruction system is provided, comprising: The multi-view data acquisition module acquires multi-view images and camera parameters of the 3D scene. A voxel initialization module is used to initialize a sparse voxel radiation field representing the three-dimensional scene based on the multi-view images and the camera parameters. The radiation field rendering module is used to generate a two-dimensional rendering image of the three-dimensional scene from the target viewpoint based on the sparse voxel radiation field and the camera parameters corresponding to the target viewpoint, using the method described in any one of the above invention. The parameter optimization module is used to update the optimizable parameters of the sparse voxel radiation field based on the loss between the two-dimensional rendered image of the target view and the corresponding real image. The 3D reconstruction module is used to perform pruning, anisotropic subdivision, and parameter optimization on the sparse voxel radiation field to obtain a 3D radiation field reconstruction model of the 3D scene.

[0011] According to a fifth aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method of any one of the preceding claims of the present invention, or implements the system of any one of the preceding claims of the present invention.

[0012] According to a sixth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method of any one of the preceding claims of the present invention, or implements the system of any one of the preceding claims of the present invention.

[0013] By adopting the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art: This invention separates the spatial sorting key from the anisotropic metadata by decoupling anchor point encoding. While retaining the local aggregation characteristics of Morton encoding for spatially neighboring voxels and its efficient sorting ability, it supports independent axial scale expression, thus balancing spatial resolution and structural expression flexibility. This invention, through anisotropic subdivision bisected along an axis, can allocate voxel resolution according to local geometric complexity, thereby reducing redundant isotropic subdivision in planar and thin structures. This invention maintains the fast inference capability of explicit sparse voxel rendering by adapting to the differentiable rasterization and local coordinate mapping of non-cubic voxels. Attached Figure Description

[0014] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating the workflow of a three-dimensional radiation field rendering method provided in a preferred embodiment of the present invention.

[0015] Figure 2 This is a schematic diagram of the constituent modules of a three-dimensional radiation field rendering system provided in a preferred embodiment of the present invention.

[0016] Figure 3 This is a flowchart illustrating the workflow of a three-dimensional radiation field reconstruction method provided in a preferred embodiment of the present invention.

[0017] Figure 4 This is a schematic diagram of the constituent modules of a three-dimensional radiation field reconstruction system provided in a preferred embodiment of the present invention. Detailed Implementation

[0018] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention. Parts not described in detail in the following embodiments can be implemented using existing technology.

[0019] Existing sparse voxel rasterization methods typically employ isotropic octree voxel structures. When dealing with structures exhibiting significant directionality and anisotropy, such as walls, tabletops, facades, railings, thin cables, and thin edges, existing methods often require simultaneous subdivision in other directions to capture high-frequency variations in one direction. This leads to a rapid increase in the number of voxels, resulting in significant memory usage, high model storage overhead, and long training times. Furthermore, fixed encoding struggles to balance spatial resolution with anisotropic scale representation.

[0020] To address the aforementioned issues, one embodiment of the present invention provides a three-dimensional radiation field rendering method. This method is based on anisotropic sparse voxel rasterization, which can reduce redundant isotropic subdivision of thin structures and planar regions while maintaining the advantages of high-speed rendering of explicit sparse voxels, thereby reducing video memory usage and model storage overhead, so as to achieve efficient representation and high-fidelity rendering of directional structures.

[0021] Specifically, such as Figure 1 As shown, the three-dimensional radiation field rendering method provided in this embodiment may include: S1, obtain the sparse voxel radiation field representing the 3D scene and the camera parameters corresponding to the target viewpoint; wherein, the sparse voxel radiation field includes multiple voxel units used to record density, color and spatial structure information. S2, identify the voxel units currently involved in encoding, subdivision and / or rendering in the sparse voxel radiation field as active voxels; S3, for each active voxel, determine the anchor point position on the preset finest virtual mesh, and generate spatial anchor point Morton code based on the anchor point position; S4. Construct a metadata hash table decoupled from the spatial anchor Morton coding, and store the anisotropic scale information and subdivision mask information of each active voxel in the metadata hash table to obtain the decoupled anchor coding. S5. Determine at least one axial subdivision direction based on the local directional density gradient of each active voxel, and perform anisotropic subdivision of the active voxel along the axis based on the axial subdivision direction to obtain subdivided active voxels. S6, based on decoupled anchor point encoding and subdivided active voxels, performs differentiable anisotropic sparse voxel rasterization to generate a 2D rendered image of the 3D scene from the target's perspective.

[0022] In some preferred embodiments, S1 above, which involves obtaining the sparse voxel radiation field representing the three-dimensional scene and the camera parameters corresponding to the target viewpoint, may further include: This step provides a sparse voxel radiation field representing the 3D scene and the camera parameters corresponding to the target viewpoint. Specifically, the sparse voxel radiation field refers to the set of voxels used to represent the 3D scene. Unlike a dense voxel mesh that stores the entire regular 3D space, the sparse voxel radiation field only stores, indexes, and updates voxels corresponding to occupied areas, areas to be optimized, or areas that contribute to the rendering results of the target viewpoint, in order to reduce storage and computational overhead. Each voxel in the sparse voxel radiation field can be associated with density parameters, color parameters, spatial location parameters, scale parameters, and other structural parameters used for rendering or reconstruction. Color parameters can be represented using spherical harmonics to model viewpoint-dependent colors.

[0023] In some preferred embodiments, S2 above, which determines the voxel units currently participating in encoding, subdivision, and / or rendering in the sparse voxel radiation field as active voxels, may further include: This step provides active voxels. Specifically, active voxels refer to voxels in the sparse voxel radiation field currently participating in the processing. The processing may include one or more of the following: encoding, sorting, subdivision, pruning, rasterization, sampling, loss calculation, and parameter updating. For example, active voxels may be voxels with a density greater than a preset threshold, voxels located within the current view's visible range, voxels selected for subdivision or parameter updating in the current iteration, or voxels that contribute to the rendering result of the current target view. It should be understood that the number and distribution of active voxels may change dynamically during the rendering or optimization process.

[0024] In some preferred embodiments, S3 above, which involves determining the anchor point position on a preset finest virtual mesh for each active voxel and generating a spatial anchor point Morton code based on the anchor point position, may further include: S31, in the global coordinate system, obtain the first... The corner point with the smallest coordinate values ​​along each coordinate axis of the active voxel is taken as the first... The smallest corner point of an active voxel; S32, map the smallest corner point to the preset finest virtual mesh, as the first... The spatial anchor point of each active voxel, and the coordinates of the smallest corner point in the preset finest virtual mesh are used as the first... The spatial anchor point coordinates of each active voxel are then determined under the preset finest virtual mesh. Binary coordinates of an active voxel space anchor point ; S33, for spatial anchor points at The three-axis binary coordinates are interleaved using Morton bit interleaving to obtain the first... Morton encoding of spatial anchor points corresponding to each active voxel ; in, Indicates Morton bit interleaving operation; coordinates satisfy , , In the formula, This is the preset level of the finest virtual mesh.

[0025] This step performs decoupling anchor coding for each active voxel. Specifically: The finest virtual mesh refers to a reference mesh established within the pre-defined 3D scene enclosure to uniformly describe the spatial position and scale of voxels, and its mesh resolution corresponds to the highest pre-defined level. Each active voxel does not need to be actually subdivided to the highest level. Instead, the anchor point position and axial scale information of the voxel are recorded under the finest virtual mesh to uniformly represent voxels of different sizes and different length-width-height ratios.

[0026] The minimum corner point refers to the corner point where a voxel takes the smallest value along each coordinate axis in the global coordinate system. For axis-aligned voxels, the minimum corner point corresponds to the voxel's position in the coordinate system. axis, shaft and The corner point is determined by the minimum boundary position along the axis. By mapping the minimum corner point to the finest virtual mesh, the anchor point coordinates of the voxel can be determined, and the corresponding spatial anchor point Morton bond can be generated.

[0027] In one implementation example, for the first For each active voxel, determine the minimum corner coordinates of that voxel within a preset, finest virtual mesh. ,in, Subsequently, Morton key spatial anchors are generated through Morton bit interleaving operations. The Morton key for the spatial anchor only encodes the anchor location, not the scale of the voxels along each axis.

[0028] In some preferred embodiments, S4 above stores the anisotropic scale information and subdivision mask information of each active voxel in a metadata hash table to obtain decoupling anchor code, including: S41, the first The Morton code of the spatial anchor point corresponding to each active voxel is used as the spatial location index; for the first... Each active voxel establishes a metadata hash table entry associated with the Morton code of the spatial anchor point. The Morton encoding of the spatial anchor point is used to characterize the spatial location of the active voxel, and the metadata hash table entry is used to record the anisotropic scale information and subdivision mask information of the active voxel; wherein, Indicates the first Metadata corresponding to each activity voxel; , and They represent the first An active voxel in a preset virtual grid axis, shaft and The number of units occupied in the axial direction; Represents a three-bit binary direction mask A defined subdivision mask index that satisfies ; S42, the Morton code of the spatial anchor point and the metadata hash table entry associated with the Morton code of the spatial anchor point are used to jointly construct the decoupled anchor code. The metadata hash table entry is used to record the anisotropic scale information and subdivision mask information of the active voxel, and the Morton code of the spatial anchor point is used to characterize the spatial position of the active voxel; wherein: S421, Recover the first entry based on the metadata hash table entry. Individual active elements in axis, shaft and Physical scale along the axis , In the formula, This indicates the spatial scale corresponding to a single grid cell in a preset virtual grid. The bounding box scale of the 3D scene; used to recover the anisotropic scale of active voxels by querying the metadata hash table during gradient calculation or rasterization; S422, perform radix sorting based on the spatial anchor Morton code to obtain spatially ordered active voxels.

[0029] This step yields the decoupling anchor point code. Specifically: For the first Each activity voxel establishes a metadata hash table entry. .in, , and These represent the voxel in the preset finest virtual mesh. axis, shaft and The number of units occupied in the axial direction; Represents a three-dimensional directional mask Integer indices that satisfy .

[0030] During rendering, the first Individual active elements in axis, axis, The physical scale in the axial direction can be derived from Recovery, among which, , The scene bounding box scale. This represents the spatial scale corresponding to a single grid cell in the finest virtual grid.

[0031] By using the decoupled anchor point encoding described above, spatial sorting and voxel shape recording can be separated. The sorting stage can still perform radix sorting based solely on the spatial anchor point Morton key, preserving the local clustering characteristics of Morton encoding for spatially adjacent voxels and its efficient sorting capability; while the voxel's three-axis scale, subdivision state, and other structural information can be obtained through a metadata hash table. Therefore, this embodiment avoids the spatial bit compression problem caused by directly writing independent axial scales or levels into fixed-length Morton keys.

[0032] In some preferred embodiments, S5 above, determining at least one axial subdivision direction based on the local directional density gradient of each active voxel, and performing anisotropic subdivision of the active voxel along the axial subdivision direction to obtain subdivided active voxels, includes: In gradient calculation, the metadata hash table entries do not directly provide corner density; instead, they provide voxel-scale information for corner location, thereby reading corner density data from the grid point density parameter table. The grid point density parameter table stores the density parameters of the corners. For the... A single active voxel, through the spatial anchor point bond of that active voxel, yields the information about that active voxel in... axis, shaft and Dimensional description in the axial direction , and Based on this, the positions of the eight corner points in the virtual grid are determined and a corner point index mapping is established; then, the density of the eight corner points is read from the grid point density parameter table for the mean difference calculation of S51; where: S51, for the first For each active voxel, calculate the density field of that active voxel. axis, shaft and The gradient magnitude along the axial direction yields the local directional density gradient magnitude of the active voxel; where: S511, based on the density parameters of the eight corner points of the active voxel, respectively along... axis, shaft and The difference in the mean density at the four corner points of opposite surfaces on both sides of the axis approximates the local density gradient along the corresponding direction. S512, take the absolute value to obtain the first... Individual active elements in axis, shaft and The magnitude of the local directional density gradient along the axial direction; S52, when the magnitude of the local directional density gradient of any axis is greater than the preset threshold corresponding to that axis, the axis is determined as a subdivision direction; S53, if no axial gradient magnitude is greater than the corresponding preset threshold, then select the axial direction with the largest local directional density gradient magnitude from the axial directions that satisfy the divisible scale constraint as the subdivision direction. S54, based on the orientation mask Determine the number of daughter voxels ;in, , and These indicate whether or not along axis, shaft and The axis performs a two-part subdivision; S541, when At that time, the active voxel was subdivided into 8 isotropic daughter voxels; S541, when When there are fewer than three bits with a value of 1, subdivision is performed only along the axis with a value of 1 to obtain 2 or 4 anisotropic daughter voxels; S55, for each subdivision axis, halve the scale of the voxel to be subdivided along that axis, and determine its anchor point coordinate offset based on the position of the sub-voxel along that axis: in, , , These represent the voxels to be subdivided in Physical dimensions along three axes; , , express The position coefficients for the three axes are either 0 or 1 along the subdivision axis, where the position coefficient is 0 if the daughter voxel is located at the subdivision axis. , or The region axially closest to the smallest corner point of the voxel to be subdivided is the corresponding... , , If the daughter element is located at 0, , or The region that is halfway away from the smallest corner point of the voxel to be subdivided along the axis is then , , Pick In the non-subdivision axis, the value is fixed at 0. This step determines the axial subdivision direction based on the local directional density gradient of the active voxels and performs anisotropic voxel subdivision. Specifically: For any moving voxel, calculate its density field in... axis, shaft and Local gradient magnitude along an axis. When the gradient magnitude along a certain axis is greater than the corresponding preset threshold, the direction mask position corresponding to that axis is set to 1. If no axis has a gradient magnitude greater than the corresponding preset threshold, the axis with the largest gradient magnitude among the axes that satisfy the biscalar constraint is selected as the back-down subdivision direction.

[0033] The orientation mask can be represented as ,in, , and These indicate whether or not along axis, shaft and The axis undergoes bisection subdivision. Based on the direction mask. The number of daughter voxels can be determined as follows: For example, when At that time, only along The axis bisects and generates two slender daughter voxels; when At that time, along shaft and The axis is bisected and generates 4 daughter voxels; when At that time, 8 isotropic voxels are generated.

[0034] In one implementation, subdivision along a given axis is only permitted if the number of mesh cells of a voxel is divisible by 2, ensuring that the subdivision remains aligned with the finest global virtual mesh. Thus, the subdivision structure logically maintains a non-overlapping and continuous spatial coverage.

[0035] During subdivision, the scale of the voxel to be subdivided is halved along the activated axis, and the child voxels inherit other properties from the voxel to be subdivided or are initialized based on the properties of the voxel to be subdivided. For subdivision axes... axis, shaft and The offset of the axis and sub-voxel relative to the anchor point of the subdivided voxel can be expressed as: .in, Indicates the voxels to be subdivided in axis, axis, Physical dimensions along the axial direction. The values ​​are 0 or 1 along the subdivision axis and 0 along the non-subdivision axis. This method allows anisotropic voxels with different aspect ratios to maintain a uniform anchor point representation in the finest virtual mesh coordinate system.

[0036] Simultaneously, active voxels can be pruned based on voxel density, transparency, or contribution to the current rendering result. For example, when a voxel's density contribution is close to zero or its cumulative transparency is below a preset threshold, the voxel can be removed from the active set to maintain a sparse structure and save storage and computational overhead.

[0037] In some preferred embodiments, the above-mentioned S6, which performs differentiable anisotropic sparse voxel rasterization based on decoupled anchor point encoding and subdivision of active voxels, may further include: Differentiable anisotropic sparse voxel rasterization refers to the process of performing non-cubic axis-aligned bounding box construction, ray intersection, local coordinate mapping, corner attribute interpolation, and discretized volume rendering integration on subdivided active voxels; among which: S61, for the first A single active voxel, based on the metadata hash table entry, recovers the information of that active voxel in... axis, shaft and The physical scale along the axis constitutes the scale vector. According to the first The center coordinates of each active voxel and scale vector Create a non-cubic axis-aligned bounding box; where the center position coordinates are... The minimum corner coordinates and axial dimensions of the active voxel are determined; the boundary surfaces of the non-cubic axis aligned bounding box are parallel to the coordinate axes, and the side lengths are the same or different in the three coordinate axis directions. S62, find the intersection points between the ray corresponding to each pixel in the target view and the non-cubic axis aligned bounding box, and determine the sampling point within the intersection interval. ; S63, for the position located in the Sampling points within each active voxel , sampling points Mapped to local coordinates , is represented as: S64, based on local coordinates , for the Interpolate the corner density and color attributes of each active voxel to obtain the sampling points. Density and color; S65, sampling points according to the ray direction Perform discretized volume rendering integral to obtain the color of the corresponding pixel.

[0038] Finally, based on the sparse voxel radiation field after subdivision and rasterization, a two-dimensional rendered image of the three-dimensional scene from the target's perspective is output.

[0039] In some preferred embodiments, the above-described S6, which performs differentiable anisotropic sparse voxel rasterization, may further include: S66, based on the anisotropic scale of the active voxels along each coordinate axis, performs physical scale correction on the partial derivatives of the density field along each coordinate axis in the local coordinate system to obtain the density gradient in world space. And based on the density gradient in world space Perform normal estimation.

[0040] This step performs differentiable anisotropic sparse voxel rasterization. Specifically: Each active voxel can be located from the central position and scale vector Definition. The center position refers to the first... The geometric center coordinates of an active voxel in the global coordinate system. If the minimum corner coordinates and the scale of each axis of the active voxel are known, the center position can be obtained by adding half of the corresponding axial scale to the minimum corner coordinates. In one implementation example, It can be represented as: .

[0041] Construct a non-cubic axis-aligned bounding box and perform ray intersection with the bounding box. A non-cubic axis-aligned bounding box is a bounding box whose boundary surface is parallel to the coordinate axes and whose side lengths can be different along the three coordinate axes. It is used to describe the spatial extent of anisotropic voxels. The center position of the bounding box can be determined by the voxel center position, and the side lengths along the three axes can be determined by the corresponding axial scale. For pixels in the target viewpoint, generate the corresponding camera rays. The ray is then aligned with the bounding box corresponding to the non-cubic axis of the active voxel to find its intersection. If the ray intersects the bounding box, a sampling point is determined within the intersection interval. For those located at the . Sampling points inside individual pixels Map it to normalized local coordinates ,in, Using this local coordinate system, a unified trilinear interpolation logic can be applied within voxels of different aspect ratios to interpolate the voxel corner density and color attributes, thus obtaining the sampling point density. and color .

[0042] Discretized volumetric rendering integration is performed based on the depth order of voxels or sampling points along the ray. In one implementation example, pixel color... This can be represented as the cumulative result of the color contribution at each sampling point along the ray, where the opacity of each sampling point is determined by its density and sampling interval. This process can be executed in a CUDA parallel kernel function and supports backpropagation of density, color, scale, and structure parameters.

[0043] In an alternative implementation, for scenarios requiring normal estimation or mesh reconstruction, the density gradient can be physically scaled according to the anisotropic scale of the active voxels. For the partial derivative of density in local coordinates, the density gradient in world space can be expressed as... This scaling correction reduces the normal orientation bias caused by non-cubic elements, thereby improving the stability of normal estimation and geometric reconstruction.

[0044] Finally, based on the sparse voxel radiation field after subdivision and rasterization, a two-dimensional rendered image of the three-dimensional scene from the target's perspective is output.

[0045] The three-dimensional radiation field rendering method provided in this embodiment is based on anisotropic sparse voxel rasterization. By decoupling anchor point encoding, anisotropic voxel subdivision, and differentiable anisotropic sparse voxel rasterization, it can reduce redundant isotropic subdivision in thin structures and planar regions while retaining the advantages of high-speed rendering of explicit sparse voxels. This reduces video memory and storage overhead and improves the ability to express directional structures.

[0046] Based on the same inventive concept, another embodiment of the present invention provides a three-dimensional radiation field rendering system.

[0047] Specifically, such as Figure 2 As shown, the three-dimensional radiation field rendering system provided in this embodiment may include: The data acquisition module is used to acquire the sparse voxel radiation field representing the 3D scene and the camera parameters corresponding to the target viewpoint; wherein, the sparse voxel radiation field includes multiple voxel units for recording density, color and spatial structure information. The active voxel determination module is used to determine the voxel units currently involved in encoding, subdivision, and / or rendering in the sparse voxel radiation field as active voxels. The spatial anchor point Morton coding module is used to determine the anchor point position on a preset virtual mesh for each active voxel and generate a spatial anchor point Morton code based on the anchor point position. The decoupled anchor coding module is used to construct a metadata hash table decoupled from the spatial anchor Morton coding. The anisotropic scale information and subdivision mask information of each active voxel are stored in the metadata hash table to obtain the decoupled anchor coding. An anisotropic subdivision module is used to determine at least one axial subdivision direction based on the local directional density gradient of each active voxel, and to perform anisotropic subdivision of the active voxel along the axis based on the axial subdivision direction to obtain subdivided active voxels. An anisotropic rasterization module, based on decoupled anchor point encoding and subdivision of active voxels, performs differentiable anisotropic sparse voxel rasterization to generate a 2D rendered image of the 3D scene from the target's perspective.

[0048] It should be noted that the steps in the rendering method provided by the present invention can be implemented using corresponding modules, devices, units, etc. in the rendering system. Those skilled in the art can refer to the technical solution of the rendering method to realize the composition of the rendering system. That is, the embodiments in the rendering method can be understood as preferred examples of constructing the rendering system, and will not be elaborated here.

[0049] Based on the three-dimensional radiation field rendering method and system provided in the above embodiments of the present invention, another embodiment of the present invention provides a three-dimensional scene reconstruction method based on the above rendering method or system.

[0050] Specifically, such as Figure 3 As shown, the three-dimensional scene reconstruction method provided in this embodiment may include: M1 acquires multi-view images of a 3D scene and camera parameters.

[0051] In some preferred embodiments, the camera parameters in M1 include intrinsic and extrinsic camera parameters. Multi-view images can originate from real acquisition devices, simulation systems, or publicly available datasets.

[0052] M2 initializes the sparse voxel radiation field representing the 3D scene based on multi-view images and camera parameters.

[0053] In some preferred embodiments, in this M2, an initial sparse voxel set can be established within the scene bounding box, and density parameters, color parameters, and geometric correlation parameters can be set for each active voxel. The color parameter can be represented using spherical harmonics to model the viewpoint-related color. The initial level can be set according to the scene size, video memory budget, or target accuracy. Specifically, the scene bounding box is first determined based on camera parameters or sparse point cloud; then, a uniform voxel mesh is generated within the bounding box according to the preset initial level, and invalid voxels are removed based on camera visibility to obtain the initial active voxel set; finally, the corner density parameters, spherical harmonic color parameters, and geometric correlation parameters are initialized for each active voxel. The initial values ​​of the parameters can be preset constants, which are optimized and updated in subsequent training.

[0054] M3, based on the sparse voxel radiation field and the camera parameters corresponding to the target viewpoint, uses the three-dimensional radiation field rendering method or system provided in the foregoing embodiments of the present invention to generate a two-dimensional rendering image of the three-dimensional scene from the target viewpoint.

[0055] M4 determines the loss function based on the difference between the 2D rendered image and the corresponding real image.

[0056] In some preferred embodiments, the loss function in M4 may include one or more of the following: pixel color error term, structural similarity loss term, transmittance regularization term, distance regularization term, sparsity regularization term, total variation regularization term, and depth-normal consistency loss term.

[0057] M5 updates the active voxel parameters in the sparse voxel radiation field based on the loss function.

[0058] In some preferred embodiments, the updatable parameters in the M5 include voxel density, color spherical harmonic coefficients, scale-related metadata, sub-voxel attributes generated by subdivision, and other parameters used for rendering.

[0059] M6 performs pruning, anisotropic subdivision, and parameter optimization on the sparse voxel radiation field to obtain a 3D radiation field reconstruction model of the 3D scene.

[0060] In some preferred embodiments, in this M6, the subdivision mask corresponding to the active voxel can be determined according to the local directional density gradient of the active voxel in each coordinate axis direction, and anisotropic voxel subdivision by axis bisection can be performed on the selected active voxel; active voxels whose contribution to the current rendering result is lower than a preset threshold can be deleted according to the density, transparency, visibility or spatial scale of the active voxel, and an updated set of active voxels can be obtained; the optimizable parameters of the active voxel can be updated according to the gradient of the loss function relative to the density parameter, color parameter, spatial structure parameter and metadata parameter of the active voxel; after the active voxel is pruned or subdivided, the Morton key of the spatial anchor point and the metadata hash table corresponding to the active voxel are updated to obtain the three-dimensional radiation field reconstruction model of the three-dimensional scene.

[0061] In a specific training implementation, subdivision and pruning can be performed at preset iteration intervals, such as once every 1000 iterations. Parameters such as the finest virtual mesh level, initial voxel level, voxel subdivision ratio, transparency threshold, early stopping threshold, and oversampling scale can all be adjusted according to the complexity of the actual scene and hardware conditions.

[0062] The three-dimensional radiation field reconstruction method provided in this embodiment uses anisotropic sparse voxel rasterization as the basic rendering process, and combines multi-view supervision, loss optimization, pruning and anisotropic subdivision to obtain a three-dimensional scene reconstruction model with both good rendering quality and high efficiency.

[0063] Based on the same inventive concept, another embodiment of the present invention provides a three-dimensional radiation field rendering system.

[0064] Specifically, such as Figure 4 As shown, the three-dimensional radiation field rendering system provided in this embodiment may include: The multi-view data acquisition module acquires multi-view images and camera parameters of the 3D scene. The voxel initialization module is used to initialize the sparse voxel radiation field representing the 3D scene based on multi-view images and camera parameters. The radiation field rendering module is used to generate a two-dimensional rendered image of a three-dimensional scene from the target viewpoint based on the sparse voxel radiation field and the camera parameters corresponding to the target viewpoint, using the three-dimensional radiation field rendering method or system provided in the foregoing embodiments of the present invention. The parameter optimization module is used to update the optimizable parameters of the sparse voxel radiation field based on the loss between the 2D rendered image of the target view and the corresponding real image. The 3D reconstruction module is used to perform pruning, anisotropic subdivision, and parameter optimization on the sparse voxel radiation field to obtain a 3D radiation field reconstruction model of the 3D scene.

[0065] It should be noted that the steps in the reconstruction method provided by the present invention can be implemented using corresponding modules, devices, units, etc. in the reconstruction system. Those skilled in the art can refer to the technical solution of the reconstruction method to realize the composition of the reconstruction system. That is, the embodiments in the reconstruction method can be understood as preferred examples of constructing the reconstruction system, and will not be elaborated here.

[0066] Based on the same inventive concept, other embodiments of the present invention provide a computer device, which may include a processor, a memory, a graphics processor, and a communication interface. The memory stores a computer program, and when the processor or graphics processor executes the computer program, it can implement any of the rendering methods or reconstruction methods in the above embodiments of the present invention, or implement any of the rendering systems or reconstruction systems in the above embodiments of the present invention.

[0067] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs and functional modules that implement the above methods), computer instructions, etc., and the aforementioned computer programs and computer instructions can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0068] A processor is used to execute computer programs stored in memory to implement the various steps of the methods or various modules of the systems involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method and system embodiments.

[0069] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0070] Based on the same inventive concept, other embodiments of the present invention provide a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it can implement any of the rendering methods or reconstruction methods in the above embodiments of the present invention, or implement any of the rendering systems or reconstruction systems in the above embodiments of the present invention.

[0071] Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of computer programs from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a user device. Of course, the processor and storage medium can also exist as discrete components in a communication device.

[0072] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, combinations, or omissions made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0073] The three-dimensional radiation field rendering method, rendering system, reconstruction method, and reconstruction system provided in the above embodiments of the present invention, based on anisotropic sparse voxel rasterization, mainly adopt the following technical solutions: acquiring the sparse voxel radiation field representing the three-dimensional scene and the camera parameters corresponding to the target viewpoint; determining the voxel unit currently participating in encoding, subdivision, or rendering in the sparse voxel radiation field as the active voxel; for each active voxel, determining the anchor point position on a preset finest virtual mesh, and generating a spatial anchor point Morton code based on the anchor point position; constructing a metadata hash table decoupled from the spatial anchor point Morton code, storing the anisotropic scale information and subdivision mask information of the active voxel in the metadata hash table to obtain the decoupled anchor point code; determining at least one axial subdivision direction based on the local directional density gradient of the active voxel, and performing anisotropic subdivision of the active voxel along the axis based on the axial subdivision direction to obtain subdivided active voxels; and performing differentiable anisotropic sparse voxel rasterization based on the decoupled anchor point code and subdivided active voxels to generate a two-dimensional rendered image of the three-dimensional scene from the target viewpoint. Through the above technical solution, the present invention can reduce redundant isotropic subdivision of thin structures and planar regions while maintaining the advantages of high-speed rendering of explicit sparse voxels, thereby reducing memory usage and model storage overhead.

[0074] Any matters not covered in the above embodiments of the present invention are well-known in the art.

[0075] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention. The preferred features described above can be used in any combination without conflict.

Claims

1. A three-dimensional radiation field rendering method, characterized in that, include: Acquire the sparse voxel radiation field representing the three-dimensional scene and the camera parameters corresponding to the target viewpoint; wherein, the sparse voxel radiation field includes multiple voxel units for recording density, color and spatial structure information; Identify the voxel units currently involved in encoding, subdivision, and / or rendering in the sparse voxel radiation field as active voxels; For each active voxel, an anchor point position is determined on a preset virtual mesh, and a spatial anchor point Morton code is generated based on the anchor point position. Construct a metadata hash table decoupled from the Morton encoding of the spatial anchor point, and store the anisotropic scale information and subdivision mask information of each active voxel in the metadata hash table to obtain the decoupled anchor point encoding; At least one axial subdivision direction is determined based on the local directional density gradient of each active voxel, and the active voxel is anisotropically subdivided along the axis according to the axial subdivision direction to obtain subdivided active voxels. Based on the decoupled anchor point encoding and the subdivided active voxels, differentiable anisotropic sparse voxel rasterization is performed to generate a two-dimensional rendered image of the three-dimensional scene from the target viewpoint.

2. The three-dimensional radiation field rendering method according to claim 1, characterized in that, For each of the active voxels, determining the anchor point position on a preset virtual mesh and generating a spatial anchor point Morton code based on the anchor point position includes: In the global coordinate system, obtain the first... The corner point with the smallest coordinate values ​​along each coordinate axis of the active voxel is taken as the first... The smallest corner point of an active voxel; Mapping the minimum corner point to a preset virtual mesh is used as the first... The spatial anchor point of each active voxel, and the coordinates of the smallest corner point in the preset finest virtual mesh are used as the first... The spatial anchor point coordinates of each active voxel are then Under the preset virtual grid, determine the binary coordinates of the anchor point position. ; For the spatial anchor point at The three-axis binary coordinates are interleaved using Morton bit interleaving to obtain the first... Morton encoding of spatial anchor points corresponding to each active voxel ; in, Indicates Morton bit interleaving operation; the coordinates satisfy , , In the formula, The hierarchy of the preset virtual grid.

3. The three-dimensional radiation field rendering method according to claim 1, characterized in that, The step of storing the anisotropic scale information and subdivision mask information of each active voxel in the metadata hash table to obtain the decoupling anchor code includes: The first The Morton code of the spatial anchor point corresponding to each active voxel is used as the spatial location index; For the first Each active voxel establishes a metadata hash table entry associated with the Morton code of the spatial anchor point. ;in, Indicates the first Metadata corresponding to each activity voxel; , and Respectively represent the first An active voxel is located in the preset virtual grid. axis, shaft and The number of units occupied in the axial direction; Represents a three-bit binary direction mask A defined subdivision mask index that satisfies ; The spatial anchor Morton code and the metadata hash table entry associated with the spatial anchor Morton code together constitute a decoupled anchor code. The metadata hash table entry is used to record the anisotropic scale information and subdivision mask information of the active voxel, and the spatial anchor Morton code is used to characterize the spatial location of the active voxel; wherein: Radix sorting is performed based on the Morton code of the spatial anchor point to obtain spatially ordered active voxels; Recover the first entry based on the metadata hash table entry. Individual active elements in axis, shaft and Physical scale along the axis , , , , In the formula, This represents the spatial scale corresponding to a single grid cell in the preset virtual grid. The bounding box scale of the 3D scene; used to recover the anisotropic scale of active voxels by querying the metadata hash table during gradient calculation or rasterization.

4. The three-dimensional radiation field rendering method according to claim 1, characterized in that, The step of determining at least one axial subdivision direction based on the local directional density gradient of each active voxel, and performing anisotropic subdivision of the active voxel along the axis based on the axial subdivision direction to obtain subdivided active voxels includes: For the For each active voxel, calculate the density field of that active voxel. axis, shaft and The gradient magnitude along the axial direction is used to obtain the local directional density gradient magnitude of the active voxel; When the magnitude of the local directional density gradient in any axis is greater than the preset threshold corresponding to that axis, that axis is determined as a subdivision direction; If no axial gradient magnitude is greater than the corresponding preset threshold, then the axial direction with the largest local directional density gradient magnitude is selected from the axial directions that satisfy the divisible scale constraint as the subdivision direction. According to the direction mask Determine the number of daughter voxels ;in, , and These indicate whether or not along axis, shaft and The axis performs a two-part subdivision; when At that time, the active voxel was subdivided into 8 isotropic daughter voxels; when When there are fewer than three bits with a value of 1, subdivision is performed only along the axis with a value of 1 to obtain 2 or 4 anisotropic daughter voxels; For each subdivision axis, the scale of the voxel to be subdivided is halved along that axis, and the anchor point coordinate offset is determined based on the position of the sub-voxel along that axis: ； in, , , These represent the voxels to be subdivided in Physical dimensions along three axes; , , express The position coefficients for the three axes are either 0 or 1 along the subdivision axis, where the position coefficient is 0 if the daughter voxel is located at the subdivision axis. , or The region axially closest to the smallest corner point of the voxel to be subdivided is the corresponding... , , Set to 0; if the daughter element is located at , or The region that is halfway away from the smallest corner point of the voxel to be subdivided along the axis is then , , Pick On the non-subdivision axis, , , The value is fixed at 0.

5. The three-dimensional radiation field rendering method according to claim 1, characterized in that, The step of performing differentiable anisotropic sparse voxel rasterization based on the decoupled anchor point encoding and the subdivided active voxels includes: Regarding the first A single active voxel, based on the metadata hash table entry, recovers the information of that active voxel in... axis, shaft and The physical scale along the axis constitutes the scale vector. ; According to the The center coordinates of each active voxel and scale vector Construct a non-cubic axis-aligned bounding box; wherein the center position coordinates The boundary surface of the non-cubic axis aligned bounding box is parallel to the coordinate axes, and the side lengths are the same or different in the three coordinate axis directions, as determined by the minimum corner coordinates and the axial dimensions of the active voxel. For each pixel in the target view, the intersection point is calculated between the ray corresponding to the pixel and the non-cubic axis-aligned bounding box, and the sampling point is determined within the intersection interval. ; For those located at the Sampling points within each active voxel The sampling points Mapped to local coordinates , represented as: ； Based on the local coordinates , for the Interpolate the corner density and color attributes of each active voxel to obtain the sampling points. Density and color; Sampling points according to the direction of the ray Perform discretized volume rendering integral to obtain the color of the corresponding pixel.

6. The three-dimensional radiation field rendering method according to claim 5, characterized in that, The execution of differentiable anisotropic sparse voxel rasterization further includes: Based on the anisotropic scale of active voxels along each coordinate axis, the partial derivatives of the density field in local coordinates are physically scaled to obtain the density gradient in world space. And based on the density gradient in the world space Perform normal estimation.

7. A three-dimensional radiation field rendering system, characterized in that, include: The data acquisition module is used to acquire the sparse voxel radiation field representing the three-dimensional scene and the camera parameters corresponding to the target viewpoint; wherein, the sparse voxel radiation field includes multiple voxel units for recording density, color and spatial structure information. An active voxel determination module is used to determine the voxel units currently participating in encoding, subdivision, and / or rendering in the sparse voxel radiation field as active voxels. The spatial anchor point Morton encoding module is used to determine the anchor point position on a preset virtual grid for each active voxel and generate a spatial anchor point Morton code based on the anchor point position. A decoupled anchor coding module is used to construct a metadata hash table decoupled from the spatial anchor Morton coding. The anisotropic scale information and subdivision mask information of each active voxel are stored in the metadata hash table to obtain the decoupled anchor coding. An anisotropic subdivision module is used to determine at least one axial subdivision direction based on the local directional density gradient of each active voxel, and to perform anisotropic subdivision of the active voxel along the axis based on the axial subdivision direction to obtain subdivided active voxels. An anisotropic rasterization module performs differentiable anisotropic sparse voxel rasterization based on the decoupled anchor point encoding and the subdivided active voxels to generate a two-dimensional rendered image of the three-dimensional scene from the target viewpoint.

8. A method for reconstructing a three-dimensional scene, characterized in that, include: Acquire multi-view images and camera parameters of a 3D scene; Based on the multi-view images and the camera parameters, initialize the sparse voxel radiation field representing the three-dimensional scene; Based on the sparse voxel radiation field and the camera parameters corresponding to the target viewpoint, the three-dimensional radiation field rendering method according to any one of claims 1 to 6 is used to generate a two-dimensional rendering image of the three-dimensional scene from the target viewpoint. The loss function is determined based on the difference between the two-dimensional rendered image and the corresponding real image; The active voxel parameters in the sparse voxel radiation field are updated according to the loss function. Pruning, anisotropic subdivision, and parameter optimization are performed on the sparse voxel radiation field to obtain a three-dimensional radiation field reconstruction model of the three-dimensional scene.

9. The three-dimensional scene reconstruction method according to claim 8, characterized in that, The process of performing pruning, anisotropic subdivision, and parameter optimization on the sparse voxel radiation field to obtain a 3D radiation field reconstruction model of the 3D scene includes: Based on the local directional density gradient of the active voxel in each coordinate axis direction, determine the subdivision mask corresponding to the active voxel, and perform anisotropic voxel subdivision by axis bisection on the selected active voxel. Based on the density, transparency, visibility, or spatial scale of active voxels, remove active voxels that contribute less than a preset threshold to the current rendering result, and obtain an updated set of active voxels. The optimizable parameters of the active voxel are updated based on the gradient of the loss function with respect to the density parameter, color parameter, spatial structure parameter, and metadata parameter of the active voxel. After the active voxels are pruned or subdivided, the Morton key of the spatial anchor point and the metadata hash table corresponding to the active voxels are updated to obtain the three-dimensional radiation field reconstruction model of the three-dimensional scene.

10. A three-dimensional scene reconstruction system, characterized in that, include: The multi-view data acquisition module acquires multi-view images and camera parameters of the 3D scene. A voxel initialization module is used to initialize a sparse voxel radiation field representing the three-dimensional scene based on the multi-view images and the camera parameters. The radiation field rendering module is used to generate a two-dimensional rendering image of the three-dimensional scene from the target viewpoint by using the three-dimensional radiation field rendering method according to any one of claims 1 to 6, based on the sparse voxel radiation field and the camera parameters corresponding to the target viewpoint. The parameter optimization module is used to update the optimizable parameters of the sparse voxel radiation field based on the loss between the two-dimensional rendered image of the target view and the corresponding real image. The 3D reconstruction module is used to perform pruning, anisotropic subdivision, and parameter optimization on the sparse voxel radiation field to obtain a 3D radiation field reconstruction model of the 3D scene.

11. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1-6 or 8-9, or implements the system of any one of claims 7 or 10.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-6 or 8-9, or implements the system of any one of claims 7 or 10.

Images