A method for optimizing ray tracing of a three-dimensional model based on a deep neural network

Abstract

The application discloses a three-dimensional model light ray tracing optimization method based on a deep neural network, which comprises the following steps: acquiring multi-view images of a three-dimensional scene and constructing scene collection data; constructing a light tracing Neuralangelo model; performing multi-resolution hash coding based on the scene collection data and constructing a reconstruction tracing shared hash grid; performing SDF volume rendering based on the reconstruction tracing shared hash grid, determining an initial three-dimensional surface, and updating and generating a corrected three-dimensional surface; determining a light source incident direction, setting an offset sampling point, and generating light tracing geometric parameters; performing surface hit analysis based on the corrected three-dimensional surface, the reconstruction tracing shared hash grid and the light tracing geometric parameters, generating a low-sampling light shadow result; combining a radiation cache table, updating a dynamic indirect light result, and performing neural noise reduction reconstruction to generate a light shadow rendering result. The application improves hit precision, reduces invalid sampling, and improves dynamic light shadow rendering quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of ray tracing and global illumination rendering in computer graphics, and particularly relates to a three-dimensional model ray tracing optimization method based on a deep neural network. BACKGROUND

[0002] With the development of deep neural networks, neural implicit surface reconstruction and ray tracing technology, existing methods can recover three-dimensional models from multiple-view images, and combine camera poses, light source information and physical material parameters to complete light and shadow rendering. Some technologies use multi-resolution hash encoding and signed distance functions to represent three-dimensional surfaces, which have achieved certain results in improving the ability to recover geometric details.

[0003] However, existing three-dimensional reconstruction and ray tracing usually use independent processing procedures, and the three-dimensional space encoding results mainly serve surface reconstruction, which is difficult to provide effective data for ray hit search, reflection path judgment and shadow boundary detection. In complex curved surfaces, occluded interfaces and high reflection areas, a large range of ray traversal and a large number of repeated sampling are still required, resulting in large computational load and low rendering efficiency.

[0004] In addition, existing low sampling rendering methods mostly rely on fixed sampling strategies, conventional radiation caches and neural noise reduction processing, and are difficult to dynamically allocate sampling resources according to local surface changes, reflection direction changes and occlusion boundary changes. The correlation between cache data and three-dimensional space position, ray direction and material characteristics is insufficient, and dynamic indirect lighting update is prone to lag, and under low sampling conditions, reflection distortion, shadow edge blur and local noise are prone to occur, making it difficult to balance three-dimensional surface accuracy, light and shadow quality and rendering efficiency.

[0005] Therefore, how to provide a three-dimensional model ray tracing optimization method based on a deep neural network is a problem that those skilled in the art need to solve. SUMMARY

[0006] An object of the present application is to provide a three-dimensional model ray tracing optimization method based on a deep neural network. The present application constructs a ray tracing Neuralangelo model based on a Neuralangelo model, forms a reconstruction and tracing shared hash grid through the geometric channel and the traversal channel in the shared hash unit, generates ray tracing geometric parameters using the directed gradient unit, determines the candidate hit region, the local sampling weight, the reflection path update intensity and the shadow boundary update intensity by the hit output unit, adjusts the ray sampling direction and the ray sampling number. The light and shadow cache unit generates light and shadow rendering results in combination with the radiation cache table, the dynamic indirect lighting results and the denoising reconstruction results, which can improve the surface hit analysis accuracy, reduce invalid sampling, and improve the rendering quality of the reflection path and the shadow boundary.

[0007] According to an embodiment of the present application, a three-dimensional model light ray tracing optimization method based on a deep neural network comprises the following steps: Obtain multi-view images of a three-dimensional scene, perform distortion correction, frame time alignment and scene coordinate unification processing, and generate scene collection data; Construct a light tracing Neuralangelo model, including a shared hash unit, a directional gradient unit, a hit output unit and a light shadow cache unit, and set up a radiation cache table; In the shared hash unit, perform multi-resolution hash coding based on the scene collection data to construct a reconstruction tracing shared hash grid; Perform SDF volume rendering based on the reconstruction tracing shared hash grid to generate SDF values and surface features, determine an initial three-dimensional surface, and update to obtain a corrected three-dimensional surface; Call the directional gradient unit, determine the surface reflection direction based on the surface points of the corrected three-dimensional surface, set the offset sampling points, and calculate the light tracing geometric parameters; In the hit output unit, perform surface hit analysis on the tracing light rays based on the corrected three-dimensional surface, the reconstruction tracing shared hash grid and the light tracing geometric parameters to generate a low-sampling light shadow result; Based on the light shadow cache unit, combine the radiation cache table to calculate the radiation cache error, update the dynamic indirect lighting result, and perform neural noise reduction reconstruction to generate the light shadow rendering result of the three-dimensional model.

[0008] Optionally, the generation of the scene collection data comprises: Collect multi-view images corresponding to each shooting angle in the three-dimensional scene, and obtain camera poses; Obtain scale point coordinates of scale points in the three-dimensional scene, and use the actual spatial distance between the scale points as the scale correction basis in the scene coordinate unification processing; Obtain light source parameters, object poses and physical material parameters in the three-dimensional scene, the light source parameters including light source position, light source direction and illumination intensity; Obtain rendering precision parameters, including surface error limit value, light ray sampling number limit value and shadow boundary error limit value; Perform distortion correction on the multi-view images based on the camera poses to obtain corrected images, and perform frame time alignment on the corrected images, camera poses, light source parameters and object poses; Perform scale correction and coordinate conversion on the camera poses, scale point coordinates and object poses to form unified pose data and three-dimensional scale points; Based on the corrected images, unified pose data, three-dimensional scale points, light source parameters, object poses, physical material parameters and rendering precision parameters, combine to generate scene collection data.

[0009] Optionally, the construction of the light tracing Neuralangelo model comprises: The multi-resolution 3D hash grid, SDF volume rendering structure, and numerical gradient structure in the Neuralangelo model are invoked. Based on a multi-resolution 3D hash grid, the original hash grid structure used for encoding three-dimensional spatial location features is transformed into a hash grid structure with parallel geometric channels and traversing channels, and a shared hash unit is constructed. Based on the numerical gradient structure, the gradient calculation method is retained, and the camera line of sight, the incident direction of the light source, and the surface reflection direction are added to the setting of the offset sampling points to construct a directed gradient unit; After the SDF volume rendering structure, a new hit output unit is added to construct a surface hit analysis path that consists of correcting the 3D surface, reconstructing the shared hash grid for tracing, and ray tracing geometric parameters entering the hit output unit. After hitting the output unit, a new light and shadow cache unit is added to construct a light and shadow cache update path that uses the radiation cache index to locate the cache position in the radiation cache table and uses the local sampling weight to participate in the processing of low-sampled light and shadow results. Set the radiation cache table and cache location in the light and shadow cache unit; Based on shared hash units, directed gradient units, hit output units, and light and shadow buffer units, the SDF volume rendering structure is preserved, and the Neuralangelo ray tracing model is constructed by combining them according to the data connection relationship.

[0010] Optionally, the construction of the reconstruction trace shared hash grid includes: In the shared hash unit, the pixels in the corrected image are back-projected onto the three-dimensional scene coordinate system according to the unified pose data to determine the three-dimensional spatial position of the pixel and the tracing ray. The three-dimensional spatial location is input into the hash grid level in the shared hash unit for multi-resolution hash encoding to obtain the encoded location, and the same hash grid is divided into geometric channels and traversal channels. In the geometric channel, hash-coded features corresponding to the three-dimensional spatial position are extracted based on the encoded position; In the traversal channel, determine the hash grid through which the tracking ray passes, set the ray sampling points, and write the corresponding hash grid positions and sampling distances into the traversal channel; Based on the hash coding features in the geometric channel, the light sampling points in the traversing channel, the hash grid position and sampling distance, they are all associated with the coding position corresponding to the same hash grid level, forming a shared hash grid for reconstruction and tracing.

[0011] Optionally, the update to correct the three-dimensional surface includes: In the SDF volume rendering structure, the hash-encoded features of the geometric channels in the shared hash grid for reconstruction and tracing are read, input into the implicit surface network, and the SDF values ​​and surface features corresponding to the three-dimensional spatial points are generated to determine the surface points of the initial three-dimensional surface. Project the three-dimensional scale points onto the initial three-dimensional surface to obtain the mapping points. Calculate the actual spatial distance between adjacent three-dimensional scale points and the mapped spatial distance between adjacent mapping points, and generate the spatial distance deviation by subtracting them. The spatial distance deviation is compared with the surface error limit in the rendering accuracy parameters to construct the scale correction loss, which is then backpropagated to the geometric channel hash coding features corresponding to the coding position to be updated and updated accordingly. The updated hash-encoded features are input into the SDF volume rendering structure again to regenerate SDF values ​​and surface features, and to determine the corrected 3D surface. In the traversal channel, read the ray sampling points of the tracing ray in the corresponding hash grid, detect the change in the sign of the SDF value between adjacent ray sampling points, and construct the nearest surface distance range; Select light sampling points within the nearest surface distance range, calculate the SDF gradient direction, and generate the normal variation amplitude; Based on the direction of the tracking ray and the direction of the SDF gradient, the reflection direction corresponding to each ray sampling point is determined, and the amplitude of the reflection direction change is generated; Based on the light source parameters, the incident direction of the light source is determined, the SDF values ​​corresponding to adjacent light sampling points are read, the number of SDF value sign changes is counted, the distance abrupt change is calculated, and the occlusion boundary change amplitude is constructed.

[0012] Optionally, the calculation of the ray tracing geometric parameters includes: The directed gradient unit is invoked to determine the camera viewing direction corresponding to each surface point of the corrected 3D surface based on the unified pose data, and the initial surface normal and surface reflection direction are determined by combining the incident direction of the light source. Centered on the surface point of the 3D surface to be corrected, one positive offset sampling point and one negative offset sampling point are set along the camera line of sight, the light source incident direction and the surface reflection direction, respectively; Read the SDF value corresponding to the offset sampling point, and calculate the gradient value of the camera line of sight, the gradient value of the light source incident direction, and the gradient value of the surface reflection direction respectively; The surface normal is determined based on the initial surface normal and the gradient value of the camera's line of sight. Along the surface reflection direction, set reverse outer offset sampling points and forward outer offset sampling points, calculate the reverse interval gradient value and the forward interval gradient value, and generate stable values ​​in the reflection direction; One offset sampling point is added outside the positive offset sampling point and the negative offset sampling point in the incident direction of the light source. The number of intervals with different SDF values ​​in adjacent sampling intervals formed by the offset sampling points is counted, the sign change ratio is calculated, and it is correlated with the gradient value in the incident direction of the light source to generate a shadow boundary sensitivity value. By matching the same surface point, the surface normal, the stable value of the reflection direction, and the sensitive value of the shadow boundary are combined to form the ray tracing geometric parameters.

[0013] Optionally, the generation of low-sampled lighting results includes: In the hit output unit, obtain the positions of the hash grids that the tracking ray passes through in sequence, and determine the range of hash grid crossing distances; Compare the range of hash grid crossing distances corresponding to the same hash grid position with the range of nearest surface distances, retain the hash grid positions, determine the spatial occupancy range of the object, and generate candidate hit regions; Read the SDF values ​​corresponding to the ray sampling points in the candidate hit area sequentially along the direction of the tracing ray, determine the hit interval, and perform linear interpolation to obtain the coordinates of the hit point; Determine the hash grid position corresponding to the hit point coordinates, divide the tracing ray direction into tracing ray direction partitions, divide the physical material parameters into physical material parameter partitions, and generate material partition numbers; The radiation cache index is obtained by combining the hash grid position, the tracing ray direction partition, and the material partition number and performing a hash mapping. Calculate the difference between the termination distance and the starting distance in the nearest surface distance range corresponding to the hit point to obtain the distance width. Combine the normal change amplitude, reflection direction change amplitude and occlusion boundary change amplitude to calculate the local sampling weight, reflection path update intensity and shadow boundary update intensity. Based on the limit of the number of light samples, the base number of samples is calculated by combining the local sampling weight, the total number of additional samples is determined, and the additional number of reflection samples and shadow samples is allocated to generate the number of light samples in the hit area. The ray sampling direction corresponding to the base sampling quantity is distributed within the hemisphere of the surface normal orientation. The reflection direction of the hit point is determined, the ray sampling direction is adjusted, and ray tracing is performed in combination with the ray sampling quantity of the hit area to generate low-sampling lighting results.

[0014] Optionally, the lighting and shadow rendering results of the generated 3D model include: In the light and shadow cache unit, the radiation cache index is read, the corresponding cache position is located in the radiation cache table, and the cached radiation value is read. Based on the coordinates of the hit point, read the radiation values ​​returned from the sampling directions of each ray and calculate the actual sampled radiation value; Normalize the three color channels of the actual sampled radiation value and the buffered radiation value to 0 to 1 respectively, calculate the absolute difference between the actual sampled radiation value and the buffered radiation value in the same color channel, and determine it as the radiation buffer error. Based on the radiation cache error, the cache radiation value is updated to obtain the updated cache radiation value, which is written to the cache location. The cache radiation values ​​in the adjacent cache locations that are coplanar with the cache location are read, and the weighted sums are added to generate a dynamic indirect lighting result. The low-sampling lighting results, surface normals, local sampling weights, and physical material parameters are concatenated at the same pixel position and input into a neural noise reduction network to generate noise reduction and reconstruction results, which serve as low-sampling correction components. The dynamic indirect lighting results are determined as indirect lighting components, and material response is calculated to obtain the material response components. The lighting and shadow rendering results of the 3D model are generated based on the material response component and the low-sampling correction component.

[0015] The beneficial effects of this invention are: First, the present invention sets up geometric channels and traversal channels in the shared hash unit, and associates the SDF surface reconstruction information with the nearest surface distance range, normal variation range, reflection direction variation range and occlusion boundary variation range of the tracking ray to the same hash grid position. The hit output unit can reduce the surface search range of the tracking ray, reduce invalid hash grid traversal and repeated hit calculations, and improve the hit analysis efficiency and hit point positioning accuracy of the corrected three-dimensional surface.

[0016] Secondly, this invention obtains surface normals, stable reflection direction values, and shadow boundary sensitivity values ​​through directed gradient units, and combines this with local change data in the traversing channel to determine local sampling weights, reflection path update intensity, and shadow boundary update intensity, thereby adjusting the ray sampling direction and the number of ray samples. This processing can allocate more sampling resources to areas with significant changes in reflection direction and complex occlusion boundaries, reduce redundant sampling in flat areas, and balance low sampling efficiency, reflection path continuity, and shadow boundary clarity.

[0017] Furthermore, this invention establishes a radiation cache index based on hash grid positions, ray tracing direction partitions, and physical material parameter partitions. It updates the cached radiation values ​​according to radiation cache errors and combines adjacent cache positions to form dynamic indirect lighting results. Then, it utilizes local sampling weights to perform neural noise reduction and reconstruction on the low-sampled lighting results. This process improves the targeting of radiation cache positioning and updates, reduces cache lag, local noise, and lighting distortion caused by dynamic lighting changes, and enhances the stability and detail of 3D model lighting rendering results. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Fig. 1 This is an overall flowchart of a 3D model ray tracing optimization method based on deep neural networks proposed in this invention; Fig. 2 This is a flowchart illustrating the process of determining candidate hit regions and adjusting the ray sampling direction and the number of ray samples based on a shared hash grid for reconstruction tracking in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figs. 1-2 A ray tracing optimization method for 3D models based on deep neural networks includes the following steps: Acquire multi-view images of a 3D scene, camera pose, scale point coordinates, light source parameters, object pose, physical material parameters, and rendering accuracy parameters. Perform distortion correction, frame timing alignment, and scene coordinate unification processing on the multi-view images. Map the scale point coordinates to the 3D scene coordinate system to form scene acquisition data. A ray-tracing Neuralangelo model is constructed based on a multi-resolution 3D hash grid, SDF volume rendering structure, numerical gradient structure, and coarse-to-fine optimization structure of the Neuralangelo model. The ray-tracing Neuralangelo model includes a shared hash unit, a directed gradient unit, a hit output unit, and a light and shadow buffer unit, and a radiation cache table is set in the light and shadow buffer unit. In the shared hash unit, the three-dimensional spatial position and tracking ray are determined based on the corrected image and unified pose data in the scene acquisition data. The three-dimensional spatial position is multi-resolution hash encoded, and the same hash grid is divided into geometric channels and traversal channels. The geometric channels record the hash encoding features corresponding to the three-dimensional spatial points, and the traversal channels record the ray sampling points of the tracking rays in the corresponding hash grid, forming a shared hash grid for reconstruction and tracking. Based on the geometric channels in the shared hash grid for reconstruction and tracing, the SDF volume rendering structure is called to perform SDF volume rendering, generating SDF values ​​and surface features corresponding to 3D spatial points. The initial 3D surface is determined based on the SDF values. The actual spatial distances corresponding to the 3D scale points are compared with the mapped spatial distances in the initial 3D surface. The hash-encoded features in the geometric channels are updated based on the spatial distance deviations and rendering accuracy parameters obtained from the comparison, forming a corrected 3D surface. The nearest surface distance range, normal variation amplitude, reflection direction variation amplitude, and occlusion boundary variation amplitude are written to the traversal channel based on the SDF values. The system invokes a directed gradient unit to calculate the surface normal based on the SDF values ​​around the surface points of the corrected 3D surface. It determines the incident direction of the light source based on the light source parameters and the surface reflection direction based on the camera's line of sight, the incident direction of the light source, and the surface normal. It sets offset sampling points along the surface reflection direction and the incident direction of the light source, calculates the SDF change corresponding to the offset sampling points, generates a stable reflection direction value based on the SDF change in the surface reflection direction, and generates a shadow boundary sensitivity value based on the SDF change in the incident direction of the light source. It combines the surface normal, the stable reflection direction value, and the shadow boundary sensitivity value to form the ray tracing geometry parameters. In the hit output unit, surface hit analysis is performed on the tracking rays based on the corrected 3D surface, the reconstructed tracking shared hash grid, and the ray tracing geometric parameters. The range of the nearest surface distance, the magnitude of normal variation, the magnitude of reflection direction variation, and the magnitude of occlusion boundary variation in the traversing channel are read to determine the candidate hit region. When the tracking ray hits the corrected 3D surface, the coordinates of the hit point and the ray tracing geometric parameters are output to determine the radiation buffer index, local sampling weight, reflection path update intensity, and shadow boundary update intensity. The ray sampling direction and the number of ray samples are adjusted to generate low-sampled lighting and shadow results. The lighting and shadow caching unit locates the cache position in the radiation cache table based on the radiation cache index, reads the cached radiation value in the cache position, generates the actual sampled radiation value based on the hit point coordinates, ray tracing geometric parameters, and local sampling weights, generates the radiation cache error based on the actual sampled radiation value and the cached radiation value, and updates the cached radiation value and dynamic indirect lighting results based on the radiation cache error. It performs neural noise reduction and reconstruction on the low-sampled lighting and shadow results, surface normals, and physical material parameters to form a noise reduction and reconstruction result. It combines the dynamic indirect lighting result, physical material parameters, and noise reduction and reconstruction result to generate the lighting and shadow rendering result of the 3D model.

[0021] In this embodiment, the generated scene acquisition data includes: Set up no fewer than 6 shooting perspectives in the 3D scene, acquire multi-view images corresponding to each shooting perspective, and obtain the camera intrinsic parameters, camera extrinsic parameters and acquisition frame time corresponding to each multi-view image, and combine them to form the camera pose. Obtain the coordinates of the scale points in the 3D scene. The scale point coordinates include the 3D coordinates of the scale points in the coordinate system of the acquisition device and the actual spatial distance between the scale points. The actual spatial distance between the scale points is used as the basis for scale correction in the unified processing of scene coordinates. The system acquires the light source parameters, object pose, and physical material parameters in the 3D scene. The light source parameters include the light source position, light source direction, and light intensity. The object pose includes the object center coordinates and object pose angle. The physical material parameters include reflectivity, roughness, and metallicity. The light source parameters are used to determine the incident direction of the light source and the actual sampled radiation value. The object pose is used to determine the candidate hit area. The physical material parameters are used to determine the radiation cache index and generate the lighting and shadow rendering results. Obtain rendering accuracy parameters, including a 0.5mm surface error limit, a 64-ray-per-pixel sampling limit, and a 2-pixel shadow boundary error limit. The surface error limit is used to update the SDF surface features in the geometry channel, the ray sampling limit is used to limit the number of ray samples in the hit area, and the shadow boundary error limit is used to constrain the shadow boundary update intensity. Distortion correction is performed on multi-view images based on camera intrinsic parameters to obtain corrected images. The corrected images, camera pose, light source parameters and object pose are then aligned according to the acquisition frame time to form corresponding corrected images, camera pose, light source parameters and object pose at the same frame time. Using the 3D scene coordinate system as a unified coordinate system, the camera pose, scale point coordinates, and object pose are scaled and transformed based on the actual spatial distance between the scale points to form unified pose data and 3D scale points. The corrected image, unified pose data, 3D scale points, light source parameters, object pose, physical material parameters, and rendering accuracy parameters are combined to form scene acquisition data.

[0022] In this embodiment, constructing the ray-traced Neuralangelo model includes: The multi-resolution 3D hash grid, SDF volume rendering structure, numerical gradient structure, and coarse-to-fine optimization structure in the Neuralangelo model are used as the foundation for building the ray-traced Neuralangelo model. Based on the multi-resolution 3D hash grid, the hash grid hierarchy used for 3D spatial location encoding in the Neuralangelo model is retained. The original hash grid structure used for 3D spatial location feature encoding is transformed into a hash grid structure with parallel geometric and traversal channels. A shared hash unit is constructed, and the geometric channel is connected to the SDF volume rendering structure, while the traversal channel is connected to the SDF volume rendering structure and the hit output unit. Based on the numerical gradient structure, the gradient calculation method of calculating the SDF change by offset sampling points in the Neuralangelo model is retained. The camera line of sight, the incident direction of the light source and the surface reflection direction are added to the setting direction of the offset sampling points to construct a directed gradient unit, and the output of the shared hash unit is connected to the input of the directed gradient unit. After the SDF volume rendering structure, a new hit output unit is added. The corrected 3D surface, the reconstruction and tracing shared hash grid generated by the shared hash unit, and the ray tracing geometric parameters generated by the directed gradient unit are used as the processing objects of the hit output unit. A surface hit analysis path is constructed in which the corrected 3D surface, the reconstruction and tracing shared hash grid, and the ray tracing geometric parameters enter the hit output unit together. After the hit output unit, a new light and shadow cache unit is added. The hit point coordinates, radiation cache index, local sampling weight and low-sampled light and shadow results generated after the hit output unit are used as the processing objects of the light and shadow cache unit. A light and shadow cache update path is constructed, in which the radiation cache index locates the cache position in the radiation cache table and the local sampling weight participates in the processing of low-sampled light and shadow results. A radiation cache table is set in the light and shadow cache unit. The radiation cache table sets the cache position according to the hash grid position, the light direction partition and the material parameter partition. The light direction partition is set to 64 spherical direction intervals. The material parameter partition is divided into 16 intervals of reflectivity, 8 intervals of roughness and 2 intervals of metallicity. Based on the coarse-to-fine optimization structure, the hierarchical order of progressive updates from low-resolution hash grid level to high-resolution hash grid level in the Neuralangelo model is preserved; Based on shared hash units, directed gradient units, hit output units, and light and shadow buffer units, the SDF volume rendering structure is preserved and combined according to data connection relationships to construct a ray-traced Neuralangelo model. During the training of the Neuralangelo ray-traced model, training samples are constructed based on scene-acquired data. Training rays are emitted from the calibration image according to uniform pose data. Geometric channel features in the shared hash unit are read at the three-dimensional spatial position through which the training rays pass. The training rendered image, training SDF value, and training surface features are obtained through the SDF volume rendering structure. At the same time, three-dimensional scale points, light source parameters, physical material parameters, and rendering accuracy parameters are added to the training samples, so that the training samples simultaneously include image color supervision, spatial scale supervision, and lighting direction supervision. The training loss consists of image reconstruction loss, scale loss, directed gradient loss, and surface stabilization loss. Image reconstruction loss is obtained by comparing the color value of each pixel in the training rendered image with the color value of the corresponding pixel in the calibration image, calculating the color difference pixel by pixel and then averaging the values. Scale loss is obtained by comparing the actual spatial distance between 3D scale points with the mapped spatial distance between corresponding mapped points on the training SDF surface, calculating the distance difference group by group and then averaging the values. Directed gradient loss is obtained by comparing the SDF changes in the camera's line of sight, the light source's incident direction, and the surface reflection direction, averaging the difference in direction changes of the same surface point in adjacent training rounds. Surface stabilization loss is obtained by comparing the changes in SDF values ​​and surface normals of adjacent sampling points, statistically identifying sampling points with excessively large SDF jumps and abrupt changes in normal direction as sources of error. The total loss function is obtained by weighted summing of image reconstruction loss, scale loss, directed gradient loss, and surface stabilization loss. The Adam optimizer is used to update the parameters related to the shared hash unit, directed gradient unit, and SDF volume rendering structure. The initial learning rate is set to 0.001, and the batch size is set to 4096 training rays. The total loss function is calculated on the validation samples after each training epoch. When the decrease of the total loss function is less than 0.0001 for 10 consecutive training epochs, or when the training epochs reach 80, training is stopped and the model parameters corresponding to the minimum total loss function are saved. The radiation cache table is not used as a network parameter directly trained as part of the total loss function. It is initialized after the Neuralangelo ray tracing model is trained. Based on the trained shared hash units, directed gradient units, and SDF volume rendering structure, surface hit calculations are performed on the training rays to obtain the hit point coordinates and ray tracing geometric parameters corresponding to the training rays. The actual sampled radiation value is calculated by combining the physical material parameters, light source parameters, and local sampling weights. The actual sampled radiation value is written into the cache position of the radiation cache table according to the hash grid position corresponding to the hit point coordinates, the training ray direction partition, and the material parameter partition. When there are multiple actual sampled radiation values ​​in the same cache position, the multiple actual sampled radiation values ​​are averaged, and the averaged result is used as the initial cached radiation value for the corresponding cache position.

[0023] In this embodiment, constructing the reconstruction tracing shared hash grid includes: In the shared hash unit, the corrected image and unified pose data in the scene acquisition data are read. According to the camera position, camera orientation and camera intrinsic parameters in the unified pose data, the pixels in the corrected image are back-projected onto the three-dimensional scene coordinate system to determine the three-dimensional spatial position and tracking light corresponding to the pixel. The three-dimensional spatial location is input into the hash grid level in the shared hash unit for multi-resolution hash encoding. The hash grid level is set to 8 levels in order from low resolution to high resolution, and the encoded position of the three-dimensional spatial location in each hash grid level is obtained. Based on the encoding position, the same hash grid is divided into geometric channels and traversal channels. The geometric channels are connected to the SDF volume rendering structure, and the traversal channels are connected to the hit output units, so that the geometric channels and traversal channels record different types of data at the same encoding position. In the geometric channel, hash-coded features corresponding to the three-dimensional spatial position are extracted based on the encoded position, and the hash-coded features are written into the geometric channel as input features for the subsequent generation of SDF values ​​and surface features of the SDF volume rendering structure. In the traversal channel, determine the hash grid through which the tracing ray passes, and set 16 ray sampling points in the corresponding hash grid along the direction of the tracing ray. Write the hash grid position and sampling distance corresponding to each ray sampling point into the traversal channel. Based on the hash coding features in the geometric channel, the light sampling points, hash grid positions and sampling distances in the channel are collectively associated with the coding positions corresponding to the same hash grid level, forming a shared hash grid for reconstruction and tracing.

[0024] In this embodiment, the updated corrected three-dimensional surface includes: In the SDF volume rendering structure, the hash-encoded features of the geometric channels in the shared hash grid for reconstruction and tracing are read, and the hash-encoded features are input into the implicit surface network in the SDF volume rendering structure to generate the SDF value and surface features corresponding to the three-dimensional spatial points. The three-dimensional spatial points with an absolute SDF value of no more than 0.05 mm are determined as the surface points of the initial three-dimensional surface. Project the three-dimensional scale points onto the initial three-dimensional surface to obtain the mapping points of the three-dimensional scale points on the initial three-dimensional surface. Calculate the actual spatial distance between adjacent three-dimensional scale points and the mapped spatial distance between adjacent mapping points. The difference between the actual spatial distance and the mapped spatial distance is taken as the spatial distance deviation. The spatial distance deviation is compared with the surface error limit in the rendering accuracy parameters. When the spatial distance deviation is greater than the surface error limit, the grid cell in which the mapping point is located in each hash grid level is determined, and the 8 vertex encoding positions of the grid cell are selected as the encoding positions to be updated. The scale correction loss is constructed based on the difference between the actual spatial distance and the mapped spatial distance. The scale correction loss is backpropagated to the geometric channel hash encoding features corresponding to the encoding positions to be updated, and the geometric channel hash encoding features in the encoding positions to be updated are updated. The updated hash-encoded features are input into the SDF volume rendering structure again to regenerate the SDF values ​​and surface features corresponding to the three-dimensional spatial points, and the corrected three-dimensional surface is determined based on the regenerated SDF values. In the traversal channel, 16 ray sampling points of the tracing ray in the corresponding hash grid are read. Based on the regenerated SDF value, the change in the sign of the SDF value between adjacent ray sampling points is detected. The sampling interval where the SDF value sign changes occurs is written into the traversal channel along the direction of the tracing ray as the nearest surface distance range. Select light sampling points within the nearest surface distance range, calculate the SDF gradient direction based on the SDF value corresponding to the light sampling points, and write the maximum angle difference between each SDF gradient direction into the crossing channel as the normal change amplitude. Based on the direction of the tracing ray and the direction of the SDF gradient, the reflection direction corresponding to each ray sampling point is determined, and the maximum angle difference between each reflection direction is written into the crossing channel as the amplitude of the reflection direction change. Based on the light source parameters, the incident direction of the light source is determined. The SDF values ​​corresponding to adjacent light sampling points are read along the incident direction of the light source. The number of SDF value sign changes between adjacent light sampling points is counted. The starting distance difference and the ending distance difference of the nearest surface distance range are calculated. The larger value between the starting distance difference and the ending distance difference is taken as the distance change amount. The number of SDF value sign changes and the distance change amount are written into the crossing channel as the change amplitude of the occlusion boundary. The traversal channel is written based on the nearest surface distance range, normal variation range, reflection direction variation range, and occlusion boundary variation range.

[0025] In this embodiment, the calculation of ray tracing geometric parameters includes: The directed gradient unit is invoked to determine the camera viewing direction corresponding to each surface point on the corrected 3D surface based on the unified pose data, and the incident direction of the light source is determined based on the light source position and light source direction in the light source parameters; the SDF values ​​corresponding to the positive and negative adjacent positions of the surface points along the three coordinate axes of the 3D scene coordinate system are read, the initial SDF gradient direction is calculated using the central difference, the initial surface normal is determined after the initial SDF gradient direction is normalized, and the surface reflection direction is determined according to the incident direction of the light source and the initial surface normal; Centered on the surface point of the calibrated 3D surface, one positive offset sampling point and one negative offset sampling point are set along the camera line of sight, the light source incident direction and the surface reflection direction, respectively, forming 6 offset sampling points. The distances from the positive offset sampling point and the negative offset sampling point to the surface point are both set to the side length of the grid cell of the current hash grid level. Read the SDF values ​​corresponding to the 6 offset sampling points, calculate the SDF difference between the forward and reverse offset sampling points in the camera viewing direction, the light source incident direction, and the surface reflection direction, respectively, and divide each SDF difference by the distance between the corresponding forward and reverse offset sampling points to obtain the gradient values ​​in the camera viewing direction, the light source incident direction, and the surface reflection direction. Perform a dot product between the initial surface normal and the normalized camera viewing direction. If the product of the dot product and the camera viewing direction gradient value is not less than 0, the initial surface normal is determined as the surface normal. If the product is less than 0, the initial surface normal is reversed and determined as the surface normal. Along the surface reflection direction, a reverse outer offset sampling point is set on the side of the reverse offset sampling point away from the surface point, and a positive outer offset sampling point is set on the side of the positive offset sampling point away from the surface point. The distances from both the reverse outer offset sampling point and the positive outer offset sampling point to the surface point are twice the side length of the grid cell in the current hash grid level. The SDF difference between the reverse outer offset sampling point and the reverse offset sampling point is divided by the distance between them to obtain the reverse interval gradient value. The SDF difference between the positive offset sampling point and the positive outer offset sampling point is divided by the distance between them to obtain the positive interval gradient value. The gradient values ​​are then calculated according to the reverse interval gradient value... The gradient values ​​of the surface reflection direction and the gradient values ​​of the positive interval are arranged along the surface reflection direction. The absolute difference between the gradient value of the reverse interval and the gradient value of the surface reflection direction, and the absolute difference between the gradient value of the surface reflection direction and the gradient value of the positive interval are calculated respectively. When the sum of the absolute values ​​of the gradient values ​​of the reverse interval, the gradient value of the surface reflection direction, and the gradient value of the positive interval is 0, the stable value of the reflection direction is determined to be 1. When the sum of the absolute values ​​is greater than 0, the larger of the two absolute differences is selected, and the larger value is divided by the sum of the absolute values ​​of the three gradient values ​​to obtain the reflection direction change ratio. The reflection direction change ratio is then subtracted from 1 to obtain the stable value of the reflection direction. Add one offset sampling point outside the positive offset sampling point and one negative offset sampling point in the incident direction of the light source. Set the distance between the new offset sampling point and the adjacent offset sampling point to the side length of the grid cell in the current hash grid level. According to the arrangement order along the incident direction of the light source, count the number of intervals with different SDF values ​​in the three adjacent sampling intervals formed by the four offset sampling points, and divide the number of intervals by 3 to obtain the sign change ratio. Calculate the directional gradient value between the negative outer offset sampling point and the negative offset sampling point, and the directional gradient value between the positive offset sampling point and the positive outer offset sampling point, and match it with the gradient value in the incident direction of the light source. When the sum of the absolute values ​​of the three directional gradient values ​​is 0, the SDF mutation ratio is set to 0. When the sum of the absolute values ​​is greater than 0, divide the maximum value of the absolute difference between the adjacent directional gradient values ​​by the sum of the absolute values ​​of the three directional gradient values ​​to obtain the SDF mutation ratio. Calculate the average value of the sign change ratio and the SDF mutation ratio to obtain the shadow boundary sensitivity value. By matching the same surface point, the surface normal, the stable value of the reflection direction, and the sensitive value of the shadow boundary are combined to form the ray tracing geometric parameters.

[0026] In this embodiment, generating low-sampling lighting results includes: In the hit output unit, the positions of the hash grids that the tracking ray passes through in sequence are obtained. The entry distance of the tracking ray when it enters each hash grid position and the exit distance of the tracking ray when it leaves each hash grid position are calculated respectively. The distance interval between the entry distance and the exit distance is determined as the hash grid crossing distance range. The nearest surface distance range, normal change amplitude, reflection direction change amplitude and occlusion boundary change amplitude recorded in the crossing channel of each hash grid position are read. Compare the hash grid traversal distance range corresponding to the same hash grid position with the nearest surface distance range, and retain two hash grid positions with overlapping distance ranges; when the normal change range corresponding to the retained hash grid position is greater than 15°, retain the front and rear hash grid positions directly adjacent to the hash grid position along the tracing ray direction; based on the object pose, map the corrected 3D surface to the current 3D scene coordinate system, determine the object's spatial occupancy range, delete hash grid positions located outside the object's spatial occupancy range, and combine the remaining hash grid positions to form candidate hit regions; The SDF values ​​of the ray sampling points in the candidate hit area are read sequentially along the ray tracing direction. The sampling interval between the two ray sampling points where the sign of the adjacent SDF values ​​changes first is determined as the hit interval. Linear interpolation is performed based on the coordinates of the ray sampling points at both ends of the hit interval and the SDF values ​​to obtain the coordinates of the hit point with an SDF value of 0. The corresponding surface normal, reflection direction stability value and shadow boundary sensitivity value are read based on the hit point coordinates, and the hit point coordinates and ray tracing geometric parameters are output. The hash grid position corresponding to the hit point coordinates is determined. The tracing ray direction is divided into 8 tracing ray direction partitions according to the positive and negative relationship of the three coordinate components in the 3D scene coordinate system. The physical material parameters are normalized to 0 to 1 and divided into 4 physical material parameter partitions according to 0 to 0.25, 0.25 to 0.50, 0.50 to 0.75, and 0.75 to 1. The partition numbers corresponding to each physical material parameter are combined sequentially to form the material partition number. The hash grid position, tracing ray direction partition, and material partition number are combined and hash-mapped to obtain the radiation cache index. The distance width is obtained by calculating the difference between the termination distance and the starting distance within the nearest surface distance range corresponding to the hit point. Normalization is performed using the maximum values ​​of the distance width, normal variation amplitude, reflection direction variation amplitude, and occlusion boundary variation amplitude within the candidate hit area. The distance width weight is obtained by subtracting the normalized distance width result from 1. Normalized values ​​for normal variation, reflection direction variation, and occlusion boundary variation are also obtained. When the corresponding maximum value is 0, the distance width weight is set to 1; other normalized values ​​are set to 0. The weighted sum of the distance width weight, normal variation normalization value, reflection direction variation normalization value, and occlusion boundary variation normalization value is obtained to obtain the local sampling weight. The reflection path update intensity is obtained by multiplying the normalized reflection direction change value by 1 minus the reflection direction stability value. Within the candidate hit region, two adjacent ray sampling points with different SDF values ​​are selected. These two ray sampling points are projected onto the corrected image, and the pixel distance between the two projected positions is calculated. When the shadow boundary offset is less than the shadow boundary error limit, the shadow boundary update intensity is obtained by multiplying the ratio of the shadow boundary offset to the shadow boundary error limit, the normalized occlusion boundary change value, and the shadow boundary sensitivity value. When the shadow boundary offset is not less than the shadow boundary error limit, the shadow boundary update intensity is obtained by multiplying the normalized occlusion boundary change value and the shadow boundary sensitivity value. Read the ray sampling limit from the rendering precision parameters. Multiply the local sampling weight by the ray sampling limit and round up. The rounded result with a value not less than 1 is determined as the base sampling number. Calculate the difference between the ray sampling limit and the base sampling number to obtain the remaining sampling number. Calculate the average of the reflection path update intensity and the shadow boundary update intensity. Multiply the average with the remaining sampling number and round down to obtain the total increase in sampling number. When the sum of the reflection path update intensity and the shadow boundary update intensity is 0, both the increase in reflection sampling and the increase in shadow sampling are set to 0. When the sum of the reflection path update intensity and the shadow boundary update intensity is greater than 0, allocate the total increase in sampling to the increase in reflection sampling and the increase in shadow sampling according to the proportion of each to the sum. Add the base sampling number, the increase in reflection sampling, and the increase in shadow sampling to obtain the number of ray samples in the hit area that does not exceed the ray sampling limit. The ray sampling directions corresponding to the base sampling quantity are distributed within the hemisphere oriented by the surface normal. The reflection direction of the hit point is determined based on the ray tracing direction and the surface normal. The ray sampling directions corresponding to the increased reflection sampling quantity are distributed within a 15° range on both sides of the reflection direction of the hit point. The ray sampling directions corresponding to the increased shadow sampling quantity are evenly distributed within a 5° range on both sides of the incident light source direction. Ray tracing is performed according to the adjusted ray sampling direction and the number of ray samples in the hit area. The cosine value of the angle between each ray sampling direction and the surface normal is normalized and used as the cumulative weight of the corresponding radiation value. The radiation values ​​returned by each ray sampling direction are weighted and accumulated to generate low-sampling lighting and shadow results.

[0027] In this embodiment, the lighting and shadow rendering results of the generated 3D model include: Read the radiation cache index in the light and shadow cache unit, locate the corresponding cache position in the radiation cache table according to the hash grid position, tracing ray direction partition and material partition number in the radiation cache index, and read the cache radiation value in the cache position; The radiation values ​​returned from each ray sampling direction are read based on the coordinates of the hit point. The cosine of the angle between each ray sampling direction and the surface normal is calculated, and the cosine of the angle less than 0 is set to 0. Based on the reflectivity, roughness, and metallicity parameters in the physical material parameters, the material response value corresponding to each ray sampling direction is calculated. The material response value is multiplied by the corresponding cosine of the angle to obtain the radiation contribution coefficient of each ray sampling direction. The sum of all radiation contribution coefficients is calculated, and each radiation contribution coefficient is divided by the sum of all radiation contribution coefficients to obtain the radiation contribution weight of each ray sampling direction. The radiation values ​​returned from each ray sampling direction are weighted and summed based on the radiation contribution weight to obtain the actual sampled radiation value. When the sum of all radiation contribution coefficients is 0, the radiation values ​​returned from each ray sampling direction are averaged to obtain the actual sampled radiation value. Normalize the three color channels of the actual sampled radiation value and the buffered radiation value to 0 to 1 respectively, calculate the absolute difference between the actual sampled radiation value and the buffered radiation value in the same color channel, and determine the average of the three absolute differences as the radiation buffer error. When the radiation buffer error is no greater than 0.05, the buffered radiation value is multiplied by 0.9 and then added to the result of the actual sampled radiation value multiplied by 0.1 to obtain the updated buffered radiation value. When the radiation buffer error is greater than 0.05, the buffered radiation value is multiplied by 0.5 and then added to the result of the actual sampled radiation value multiplied by 0.5 to obtain the updated buffered radiation value. The number of ray samples in the hit area is then added to the product of the radiation buffer error and the limit of the number of ray samples, and rounded up. The result that does not exceed the limit of the number of ray samples is used as the number of ray samples in the hit area of ​​the next frame at the same buffer position. Write the updated cache radiation value to the cache location, and read the cache radiation values ​​of the 6 adjacent cache locations that are coplanar with the cache location. Multiply the updated cache radiation value by 0.5. Normalize and weight the cache radiation values ​​of the 6 adjacent cache locations according to the reciprocal of the distance between the centers of the corresponding hash grids, and then multiply by 0.5. Add the two calculation results to obtain the dynamic indirect lighting result. The low-sampling lighting results, surface normals, local sampling weights, and physical material parameters are stitched together at the same pixel position. The stitched result is then input into a neural noise reduction network. Low-sampling noise is removed through convolutional feature extraction and residual reconstruction to generate a noise reduction and reconstruction result. The dynamic indirect lighting result is determined as the indirect lighting component. The material response of the indirect lighting component is calculated based on the reflectivity, roughness and metallicity in the physical material parameters to obtain the material response component. The noise reduction and reconstruction result is determined as the low sampling correction component. Multiply the material response component by the local sampling weight, multiply the low-sampling correction component by 1 and subtract the result of the local sampling weight, add the two calculation results to generate the lighting and shadow rendering result of the 3D model.

[0028] Example 1: To verify the feasibility of this invention in practice, it was applied to a 3D device model reconstruction and lighting rendering scene containing a metal shell, frosted panel, curved connector, deep groove, and partial occlusion structure. This scene simultaneously contains highly reflective surfaces, rough surfaces, narrow shadow areas, and multiple layers of occlusion. Traditional methods, after completing the 3D surface reconstruction, need to search for ray hit locations step-by-step over a large spatial area and use a fixed number of rays for reflection and shadow sampling. Smooth areas suffer from excessive duplicate sampling, while curved reflection areas and occlusion edges are prone to undersampling, ultimately resulting in problems such as broken reflection textures, blurred shadow edges, delayed indirect lighting updates, and high rendering time.

[0029] This embodiment acquired 84 multi-view images with a resolution of 2560×1440 pixels. The shooting perspectives covered the front, sides, top, upper diagonally, and local recessed areas of the device model. Fourteen 3D ruler points were set in the scene, with the actual spatial distance between the ruler points ranging from 80mm to 650mm. Camera pose, light source position, light source direction, object pose, and physical material parameters were recorded simultaneously. The reflectivity of the physical material parameters ranged from 0.18 to 0.82, the roughness from 0.12 to 0.76, and the metallicity from 0 to 0.95. Rendering accuracy parameters included a 0.5mm surface error limit, a limit on the number of light samples per pixel, and a 2-pixel shadow boundary error limit. The test sequence consisted of 300 frames of continuously changing viewpoint images to verify the stability of ray hits, radiation buffer updates, and dynamic indirect lighting reconstruction.

[0030] After distortion correction, frame timing alignment, and scene coordinate unification processing of multi-view images, scene acquisition data is formed. A ray-tracing Neuralangelo model is constructed based on a multi-resolution 3D hash grid, SDF volume rendering structure, numerical gradient structure, and coarse-to-fine optimization structure. The multi-resolution 3D hash grid is transformed into a shared hash unit with parallel geometric and traversal channels. The geometric channels record the hash encoding features corresponding to the 3D spatial position, while the traversal channels record the ray sampling points, hash grid positions, and sampling distances of the tracking rays within the hash grid.

[0031] After the hash-coded features in the geometry channel are input into the SDF volume rendering structure, SDF values ​​and surface features corresponding to 3D spatial points are generated, and the initial 3D surface is determined accordingly. 3D scale points are mapped to the initial 3D surface, and the spatial distance deviation between the mapped spatial distance and the actual spatial distance is calculated. When the spatial distance deviation exceeds the surface error limit, the grid cell in each hash grid level where the mapped point is located is determined. The 8 vertex encoding positions of the grid cell are selected, and the hash-coded features in the geometry channel are updated using scale correction loss. The SDF volume rendering is then performed again to form a corrected 3D surface.

[0032] The traversal channel records the nearest surface distance range, normal variation amplitude, reflection direction variation amplitude, and occlusion boundary variation amplitude based on the corrected SDF values. The directed gradient unit sets forward and reverse offset sampling points based on the camera's viewing direction, the light source's incident direction, and the surface reflection direction. It then combines the SDF changes corresponding to these offset sampling points to generate surface normals, stable reflection direction values, and shadow boundary sensitivity values. A low stable reflection direction value indicates significant SDF changes near the reflection direction, requiring increased sampling density in that area. A high shadow boundary sensitivity value indicates concentrated occlusion changes near the light source's incident direction, necessitating enhanced light sampling on both sides of the shadow boundary.

[0033] The hit output unit eliminates hash grid positions that do not overlap with the effective hit interval of the tracing ray based on the nearest surface distance range. It then retains hash grid positions with significant surface transitions based on the normal variation amplitude, and combines this with the object pose to limit the object's spatial occupancy, forming candidate hit regions. The tracing ray only reads the SDF value corresponding to the ray sampling point within the candidate hit region. The hit interval is determined by the sign change of adjacent SDF values, and linear interpolation is used to obtain the hit point coordinates. Local sampling weights are calculated based on distance width, normal variation amplitude, reflection direction variation amplitude, and occlusion boundary variation amplitude. These weights are then combined with the reflection direction stability value and shadow boundary sensitivity value to determine the reflection path update intensity and shadow boundary update intensity, dynamically allocating the ray sampling direction and the number of ray samples.

[0034] The lighting and shading cache unit forms a radiation cache index based on the hash grid position corresponding to the hit point, the tracing ray direction partition, and the physical material parameter partition, and reads the cached radiation value from the radiation cache table. The actual sampled radiation value of the current frame is compared with the cached radiation value to form a radiation cache error. The cached radiation value is then written back to the cache position with different update magnitudes based on the radiation cache error. The updated cached radiation value is then weighted and smoothed with adjacent cache positions on the shared mesh surface to form the dynamic indirect lighting result. The low-sampled lighting and shading result, surface normals, local sampling weights, and physical material parameters are input into the neural denoising network to obtain the denoised reconstruction result, which is then combined with the dynamic indirect lighting result to form the lighting and shading rendering result of the 3D model.

[0035] Table 1. Comparison of overall performance of low-sampling ray tracing in 3D models

[0036] As shown in Table 1, the average surface hit error of the method in this invention is 0.61 mm, which is about 29.1% lower than the original Neuralangelo method with fixed sampling ray tracing, and about 17.6% lower than the traditional SDF volume rendering method with ordinary radiation buffer. The error reduction mainly comes from the fact that the geometry channel and the traversal channel share the same hash grid position. The tracing ray can directly utilize the nearest surface distance range and the normal change amplitude to narrow the hit search area. At the same time, the spatial distance deviation corresponding to the 3D scale point continuously participates in the geometry channel update, reducing the impact of the initial 3D surface scale error on the hit point coordinates.

[0037] The method of this invention has an average of 31.4 samples per pixel, which is about 32.0% less than the fixed-sampling ray tracing method and about 18.9% less than the traditional SDF volume rendering plus ordinary radiation buffer method. The reduction in the number of samples does not uniformly reduce the sampling of the entire region, but rather uses local sampling weights to distinguish between smooth surfaces, regions with varying reflections, and occlusion boundary regions. Regions with small surface changes maintain a lower sampling number, while regions with significant changes in reflection direction and sensitive shadow boundaries are assigned a higher sampling number. Therefore, local details are preserved while reducing the overall sampling amount.

[0038] The average shadow boundary error was reduced from 2.84 pixels in the fixed-sampling ray tracing method to 1.92 pixels, a reduction of approximately 32.4%; compared to the traditional SDF volume rendering plus ordinary radiation buffer method, it was reduced by approximately 16.9%. This result is related to the combined use of occlusion boundary variation amplitude and shadow boundary sensitivity value. The hit output unit can identify SDF sign changes and local abrupt changes near the light source incident direction, and increase the sampling density on both sides of the shadow boundary to reduce shadow contour breakage and edge diffusion under low sampling conditions.

[0039] The PSNR of the reflection area reached 33.4dB, which is 2.6dB higher than the fixed-sampling ray tracing method and 1.3dB higher than the traditional SDF volume rendering plus ordinary radiance buffer method. The amplitude of the reflection direction variation and the stability value of the reflection direction jointly control the sampling distribution near the reflection path. More targeted reflection sampling can be obtained for curved metal surfaces and areas with varying roughness, reducing specular abrupt changes and texture discontinuities.

[0040] The single-frame rendering time has been reduced to 70.8ms, which is about 23.5% shorter than the fixed-sampling ray tracing method and about 13.0% shorter than the traditional SDF volume rendering plus ordinary radiance buffer method. The time reduction is mainly due to the compression of the hash grid search range of the candidate hit region, the dynamic control of the number of samples by the local sampling weight, and the direct location of the buffer position by the radiance buffer index, which reduces invalid mesh traversal, repeated ray sampling, and repeated indirect lighting calculations.

[0041] The effective hit rate of the radiation cache reaches 87.6%, which is 13.3 percentage points higher than the fixed-sampling ray tracing method and 7.5 percentage points higher than the traditional SDF volume rendering plus ordinary radiation cache method. The radiation cache index is composed of hash mesh position, tracing ray direction partition, and physical material parameter partition, which enables the same spatial location to be located to a more suitable cache position under different ray directions and different material response conditions. The cached radiation value is updated according to the radiation cache error and combined with adjacent cache positions to complete dynamic indirect lighting smoothing, thereby reducing cache lag and local flicker during continuous viewpoint changes.

[0042] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A ray tracing optimization method for 3D models based on deep neural networks, characterized in that, Includes the following steps: Acquire multi-view images of a 3D scene, perform distortion correction, frame timing alignment, and scene coordinate unification processing to generate scene acquisition data; Construct a ray-traced Neuralangelo model, including a shared hash unit, a directed gradient unit, a hit output unit, and a ray buffer unit, and set up a radiation buffer table; In the shared hash unit, multi-resolution hash encoding is performed based on scene-collected data to construct a shared hash grid for reconstruction and tracing. SDF volume rendering is performed based on the shared hash grid of reconstruction and tracing to generate SDF values ​​and surface features, determine the initial 3D surface, and update to obtain the corrected 3D surface. The directed gradient unit is invoked to determine the surface reflection direction based on the surface points of the corrected 3D surface, set the offset sampling points, and calculate the ray tracing geometric parameters; In the hit output unit, surface hit analysis is performed on the tracing rays based on the corrected 3D surface, the reconstructed tracing shared hash grid and the ray tracing geometry parameters to generate low-sampled lighting results; Based on the light and shadow caching unit and combined with the radiation caching table, the radiation caching error is calculated, the dynamic indirect lighting results are updated, and neural noise reduction and reconstruction are performed to generate the light and shadow rendering results of the 3D model.

2. The ray tracing optimization method for a 3D model based on a deep neural network according to claim 1, characterized in that, The generated scene data includes: Acquire multi-view images corresponding to each shooting angle in a 3D scene and obtain the camera pose; Obtain the coordinates of the scale points in the 3D scene, and use the actual spatial distance between the scale points as the basis for scale correction in the unified processing of scene coordinates; Obtain the light source parameters, object pose, and physical material parameters in the 3D scene. The light source parameters include the light source position, light source direction, and light intensity. Obtain rendering accuracy parameters, including surface error limits, light sampling quantity limits, and shadow boundary error limits; Distortion correction is performed on multi-view images based on camera pose to obtain corrected images, and frame-time alignment is performed on the corrected images, camera pose, light source parameters and object pose. Scale correction and coordinate transformation are performed on camera pose, scale point coordinates and object pose to form unified pose data and three-dimensional scale points; Scene acquisition data is generated by combining corrected images, unified pose data, 3D scale points, light source parameters, object pose, physical material parameters, and rendering accuracy parameters.

3. The ray tracing optimization method for a 3D model based on a deep neural network according to claim 1, characterized in that, The construction of the ray-traced Neuralangelo model includes: The multi-resolution 3D hash grid, SDF volume rendering structure, and numerical gradient structure in the Neuralangelo model are invoked. Based on a multi-resolution 3D hash grid, the original hash grid structure used for encoding three-dimensional spatial location features is transformed into a hash grid structure with parallel geometric channels and traversing channels, and a shared hash unit is constructed. Based on the numerical gradient structure, the gradient calculation method is retained, and the camera line of sight, the incident direction of the light source, and the surface reflection direction are added to the setting of the offset sampling points to construct a directed gradient unit; After the SDF volume rendering structure, a new hit output unit is added to construct a surface hit analysis path that consists of correcting the 3D surface, reconstructing the shared hash grid for tracing, and ray tracing geometric parameters entering the hit output unit. After hitting the output unit, a new light and shadow cache unit is added to construct a light and shadow cache update path that uses the radiation cache index to locate the cache position in the radiation cache table and uses the local sampling weight to participate in the processing of low-sampled light and shadow results. Set the radiation cache table and cache location in the light and shadow cache unit; Based on shared hash units, directed gradient units, hit output units, and light and shadow buffer units, the SDF volume rendering structure is preserved, and the Neuralangelo ray tracing model is constructed by combining them according to the data connection relationship.

4. The ray tracing optimization method for a 3D model based on a deep neural network according to claim 1, characterized in that, The construction and reconstruction tracking shared hash grid includes: In the shared hash unit, the pixels in the corrected image are back-projected onto the three-dimensional scene coordinate system according to the unified pose data to determine the three-dimensional spatial position of the pixel and the tracing ray. The three-dimensional spatial location is input into the hash grid level in the shared hash unit for multi-resolution hash encoding to obtain the encoded location, and the same hash grid is divided into geometric channels and traversal channels. In the geometric channel, hash-coded features corresponding to the three-dimensional spatial position are extracted based on the encoded position; In the traversal channel, determine the hash grid through which the tracking ray passes, set the ray sampling points, and write the corresponding hash grid positions and sampling distances into the traversal channel; Based on the hash coding features in the geometric channel, the light sampling points in the traversing channel, the hash grid position and sampling distance, they are all associated with the coding position corresponding to the same hash grid level, forming a shared hash grid for reconstruction and tracing.

5. The ray tracing optimization method for a 3D model based on a deep neural network according to claim 1, characterized in that, The updated corrected three-dimensional surface includes: In the SDF volume rendering structure, the hash-encoded features of the geometric channels in the shared hash grid for reconstruction and tracing are read, input into the implicit surface network, and the SDF values ​​and surface features corresponding to the three-dimensional spatial points are generated to determine the surface points of the initial three-dimensional surface. Project the three-dimensional scale points onto the initial three-dimensional surface to obtain the mapping points. Calculate the actual spatial distance between adjacent three-dimensional scale points and the mapped spatial distance between adjacent mapping points, and generate the spatial distance deviation by subtracting them. The spatial distance deviation is compared with the surface error limit in the rendering accuracy parameters to construct the scale correction loss, which is then backpropagated to the geometric channel hash coding features corresponding to the coding position to be updated and updated accordingly. The updated hash-encoded features are input into the SDF volume rendering structure again to regenerate SDF values ​​and surface features, and to determine the corrected 3D surface. In the traversal channel, read the ray sampling points of the tracing ray in the corresponding hash grid, detect the change in the sign of the SDF value between adjacent ray sampling points, and construct the nearest surface distance range; Select light sampling points within the nearest surface distance range, calculate the SDF gradient direction, and generate the normal variation amplitude; Based on the direction of the tracking ray and the direction of the SDF gradient, the reflection direction corresponding to each ray sampling point is determined, and the amplitude of the reflection direction change is generated; Based on the light source parameters, the incident direction of the light source is determined, the SDF values ​​corresponding to adjacent light sampling points are read, the number of SDF value sign changes is counted, the distance abrupt change is calculated, and the occlusion boundary change amplitude is constructed.

6. The ray tracing optimization method for a 3D model based on a deep neural network according to claim 1, characterized in that, The calculation of the optical tracing geometric parameters includes: The directed gradient unit is invoked to determine the camera viewing direction corresponding to each surface point of the corrected 3D surface based on the unified pose data, and the initial surface normal and surface reflection direction are determined by combining the incident direction of the light source. Centered on the surface point of the 3D surface to be corrected, one positive offset sampling point and one negative offset sampling point are set along the camera line of sight, the light source incident direction and the surface reflection direction, respectively; Read the SDF value corresponding to the offset sampling point, and calculate the gradient value of the camera line of sight, the gradient value of the light source incident direction, and the gradient value of the surface reflection direction respectively; The surface normal is determined based on the initial surface normal and the gradient value of the camera's line of sight. Along the surface reflection direction, set reverse outer offset sampling points and forward outer offset sampling points, calculate the reverse interval gradient value and the forward interval gradient value, and generate stable values ​​in the reflection direction; One offset sampling point is added outside the positive offset sampling point and the negative offset sampling point in the incident direction of the light source. The number of intervals with different SDF values ​​in adjacent sampling intervals formed by the offset sampling points is counted, the sign change ratio is calculated, and it is correlated with the gradient value in the incident direction of the light source to generate a shadow boundary sensitivity value. By matching the same surface point, the surface normal, the stable value of the reflection direction, and the sensitive value of the shadow boundary are combined to form the ray tracing geometric parameters.

7. The ray tracing optimization method for a 3D model based on a deep neural network according to claim 1, characterized in that, The generated low-sample lighting and shadow results include: In the hit output unit, obtain the positions of the hash grids that the tracking ray passes through in sequence, and determine the range of hash grid crossing distances; Compare the range of hash grid crossing distances corresponding to the same hash grid position with the range of nearest surface distances, retain the hash grid positions, determine the spatial occupancy range of the object, and generate candidate hit regions; Read the SDF values ​​corresponding to the ray sampling points in the candidate hit area sequentially along the direction of the tracing ray, determine the hit interval, and perform linear interpolation to obtain the coordinates of the hit point; Determine the hash grid position corresponding to the hit point coordinates, divide the tracing ray direction into tracing ray direction partitions, divide the physical material parameters into physical material parameter partitions, and generate material partition numbers; The radiation cache index is obtained by combining the hash grid position, the tracing ray direction partition, and the material partition number and performing a hash mapping. Calculate the difference between the termination distance and the starting distance in the nearest surface distance range corresponding to the hit point to obtain the distance width. Combine the normal change amplitude, the reflection direction change amplitude and the occlusion boundary change amplitude to calculate the local sampling weight, reflection path update intensity and shadow boundary update intensity. Based on the limit of the number of light samples, the basic number of samples is calculated by combining the local sampling weight, the total number of additional samples is determined, and it is allocated as the number of additional reflection samples and the number of additional shadow samples to generate the number of light samples in the hit area. The ray sampling direction corresponding to the base sampling quantity is distributed within the hemisphere of the surface normal orientation. The reflection direction of the hit point is determined, the ray sampling direction is adjusted, and ray tracing is performed in combination with the ray sampling quantity of the hit area to generate low-sampling lighting results.

8. The ray tracing optimization method for a 3D model based on a deep neural network according to claim 1, characterized in that, The lighting and shadow rendering results of the generated 3D model include: In the light and shadow cache unit, the radiation cache index is read, the corresponding cache position is located in the radiation cache table, and the cached radiation value is read. Based on the coordinates of the hit point, read the radiation values ​​returned from the sampling directions of each ray and calculate the actual sampled radiation value; Normalize the three color channels of the actual sampled radiation value and the buffered radiation value to 0 to 1 respectively, calculate the absolute difference between the actual sampled radiation value and the buffered radiation value in the same color channel, and determine it as the radiation buffer error. Based on the radiation cache error, the cache radiation value is updated to obtain the updated cache radiation value, which is written to the cache location. The cache radiation values ​​in the adjacent cache locations that are coplanar with the cache location are read, and the weighted sums are added to generate a dynamic indirect lighting result. The low-sampling lighting results, surface normals, local sampling weights, and physical material parameters are concatenated at the same pixel position and input into a neural noise reduction network to generate noise reduction and reconstruction results, which serve as low-sampling correction components. The dynamic indirect lighting results are determined as indirect lighting components, and material response is calculated to obtain the material response components. The lighting and shadow rendering results of the 3D model are generated based on the material response component and the low-sampling correction component.

Images