A city-level high-precision three-dimensional scene construction method and system based on NeRF technology

Abstract

The application relates to a city-level high-precision three-dimensional scene construction method and system based on a NeRF technology, and belongs to the technical field of computer graphics. The method comprises the following steps: constructing a ray deflection field to simulate the refraction effect of a ray in a transparent / semi-transparent medium; predicting a view direction deflection amount through a direction deflection network; constructing a decoupled geometry field, a medium attribute field and a color field; generating a hierarchical sampling point sequence in a sampling interval containing a surface by positioning a surface boundary; applying the direction deflection amount output by the direction deflection network to the view direction of each sampling point; introducing a physically driven refraction consistency loss and a brightness anomaly constraint; and performing progressive training to sequentially optimize the parameters of the geometry field, the color field, the medium attribute field and the direction deflection network. The method effectively solves the problem of insufficient rendering precision caused by the fact that the traditional neural radiation field method does not fully consider the ray refraction and medium characteristics when processing such objects.

Description

Technical Field

[0001] This invention belongs to the field of computer graphics technology, specifically relating to a method and system for constructing high-precision 3D city-level scenes based on NeRF technology. Background Technology

[0002] Traditional NeRF assumes that light travels in a straight line and cannot handle the refraction effect of transparent / semi-transparent objects, resulting in rendering distortion. Existing improved solutions (such as Ref-NeRF and UniSDF) have optimized the rendering of reflective surfaces, but still have not solved the refraction problem and have insufficient support for the scattering properties of semi-transparent media.

[0003] NeRF couples geometry (density field) with appearance (color field) in its modeling, leading to optimization conflicts: transparent objects require independent control of physical properties such as refractive index and scattering coefficient, but the coupled representation is difficult to decouple precisely; NeRF training is time-consuming and unstable for complex media (such as dynamic refractive index and wavelength-dependent effects), and existing solutions lack loss functions for refractive physical constraints, resulting in violations of energy conservation. The core problems at this stage are:

[0004] 1. Transparent / semi-transparent objects cause light to bend due to refraction, which invalidates the traditional NeRF assumption of straight light rays and disrupts the geometric consistency of multiple views;

[0005] 2. The coupling of medium properties such as refractive index and scattering coefficient with geometric density leads to optimization conflicts (e.g., the density field cannot simultaneously fit geometric and optical properties).

[0006] 3. Refraction path calculation requires efficient differential support, and traditional NeRF sampling strategies are inefficient for bent light rays. Summary of the Invention

[0007] To address the aforementioned problems in the existing technology, this invention provides a method and system for constructing high-precision 3D city-level scenes based on NeRF technology. The objective of this invention can be achieved through the following technical solutions:

[0008] A method for constructing high-precision 3D city scenes based on NeRF technology includes:

[0009] S1: Construct a ray offset field to simulate the refraction effect of light in a transparent / semi-transparent medium. The directional offset field in the ray offset field is realized by a directional offset network. The directional offset network is used to predict the viewing angle directional offset.

[0010] S2: Construct a decoupled geometric field, medium property field, and color field; the geometric field models the geometric structure of the object surface based on the symbolic distance function; the medium property field independently models the optical absorption and scattering properties and refractive index distribution of the medium; the symbolic distance is converted into geometric density through a differentiable transformation function;

[0011] S3: To address the scale differences in city-level scenes, a hierarchical sampling point sequence is generated within the sampling interval containing the surface by locating the surface boundary; the directional offset output by the directional offset network is applied to the viewpoint direction of each sampling point;

[0012] S4: Introduce physically driven refractive consistency loss and brightness anomaly constraint. The refractive consistency loss is based on the refractive index output by the medium property field to calculate the theoretical refractive direction, and the constraint direction offset conforms to the law of refraction. The brightness anomaly constraint forces the color value output by volume rendering to not exceed 1.

[0013] S5: Perform progressive training, sequentially optimizing the parameters of the geometric field, color field, medium property field, and orientation offset network. Divide the urban scene into a geographic grid, train each sub-network independently, and share the orientation offset network parameters. Introduce a temporal optical flow field to model the spatiotemporal continuity of the dynamic transparent medium, and constrain the changes in the geometric field and medium property field of adjacent frames through optical flow consistency loss.

[0014] Specifically, the method for generating the hierarchical sampling point sequence includes:

[0015] S301: Determine the surface boundary interval by coarse sampling based on geometric field SDF;

[0016] S302: Resample within the boundary interval, and use the orientation offset network to predict the orientation offset for each sampling point to obtain the offset viewpoint orientation;

[0017] S303: Under the conditions of the original sampling point position and the offset viewing direction, obtain the geometric density of the geometric field; obtain the medium density and refractive index of the medium property field; obtain the color value of the color field;

[0018] S304: The effective volume density is calculated by using a blending function, with geometric density as the primary factor in the surface region and medium density as the primary factor in the interior of the medium. Based on the effective volume density and color value, the light color is synthesized using a volume rendering formula.

[0019] Specifically, the orientation offset field is used to correct the viewing direction at the sampling point, and the orientation offset network is used to predict the change in the direction of the light rays at the sampling point; the input data of the orientation offset network are the origin of the light ray, the direction, the spatial position, and the viewing direction; the output is the orientation offset, and the deformed light ray is redefined by the output offset.

[0020] Specifically, the concrete implementation of the refractive consistency loss and brightness anomaly constraint described in S4 includes:

[0021] S401: For each sampling point, the surface normal vector is calculated based on the gradient of the geometric field SDF. The theoretical refraction direction is calculated according to the law of refraction and the standard refractive index corresponding to the transparent / semi-transparent object. The actual refraction offset direction is obtained by normalizing the direction offset predicted by the direction offset network. The cosine loss of the angle between the theoretical refraction direction and the actual refraction offset direction is calculated.

[0022] S402: The brightness anomaly constraint is that the color value output by the volumetric rendering does not exceed 1;

[0023] S403: Add the refraction uniformity loss and brightness anomaly constraint to the total loss function in a weighted manner.

[0024] Furthermore, the total loss function also includes a path continuity regularization term, specifically a curl consistency constraint for directional offset, which ensures that the path remains a linear segment within the medium by penalizing the curvature change of the ray path.

[0025] Specifically, the optimization mechanism of the progressive training includes: a phased network parameter unlocking mechanism, adaptive learning rate scheduling, and gradient stabilization processing;

[0026] The phased network parameter unlocking mechanism fixes the output offset of the fixed-direction offset network to zero during the pre-training phase, optimizing only the basic parameters of the geometric field SDF, color field, and medium property field. During the medium property field optimization phase, the medium property field parameters are unlocked, and the geometric field, color field, and medium property field are jointly optimized, introducing a brightness anomaly constraint, with the weights increasing linearly from 0 to a predetermined value. During the offset field fine-tuning phase, the parameters of the directional offset network are unlocked, and all network parameters are jointly optimized, while introducing refraction consistency loss and path smoothing regularization terms.

[0027] The adaptive learning rate scheduling dynamically adjusts the learning rate, with the geometric field learning rate decaying exponentially and the offset network learning rate decaying linearly.

[0028] The gradient stabilization process includes truncating the magnitude of the offset field gradient and applying an Eikonal regularization term to the SDF field.

[0029] Specifically, the medium property field further extends the optical enhancement processing mechanism for translucent media, including:

[0030] For semi-transparent media, a scattering coefficient term is added to the output of the medium property field, and a diffusion approximation term is added to the volume rendering equation; a wavelength dependence term is added to the output of the direction offset network, and a dispersion uniformity loss is constructed; by defining the film thickness field in the surface neighborhood, the interference modulation factor is calculated, and the interference effect is incorporated into the color field; and a spatially varying refractive index field is established, introducing a dynamic refractive index into the refractive uniformity loss.

[0031] Specifically, the real-time rendering optimization mechanism also includes differential calculation of ray paths and adaptive sampling strategies, including: constructing the Jacobian matrix of deformable paths to support efficient automatic differentiation; optimizing sampling points based on curvature-sensitive step size and fixed step size of surface neighborhood; implementing ray bucketing for parallel computation; and establishing a dynamic update region detection mechanism to adjust the resolution of highly variable regions in real time.

[0032] Specifically, the method for locating the surface boundary includes:

[0033] Input the ray equation, and set the ray start point parameter, maximum number of iterations, surface distance threshold, and minimum step size distance;

[0034] Based on the light source parameters and light direction, the minimum step distance is used as the initial step size, and the step size is iterated along the light direction. In each iteration, the signed distance function value from the current point to the object surface is calculated. If it is less than the surface distance threshold, it is determined to be a surface boundary, and the point is recorded as a surface boundary point. If no boundary is found and the number of iterations reaches the maximum number of iterations, the iteration stops.

[0035] Specifically, the geometric field, color field, and medium property field are based on a neRF multilayer perceptron. The input to the geometric field is the position parameter, and the output is the SDF value. The input to the color field is the position parameter and the offset direction, and the output is the color value. The input to the medium property field is the position parameter, and the output is the medium density, refractive index, and scattering coefficient.

[0036] A city-level high-precision 3D scene construction system based on NeRF technology includes:

[0037] The refraction and deflection module is used to construct a light ray offset field to simulate the refraction effect of light in a transparent / semi-transparent medium. The direction offset field in the light ray offset field is realized by a direction offset network; the viewing angle direction offset is predicted by the direction offset network.

[0038] The decoupled physical property field module is used to construct decoupled geometric fields, medium property fields, and color fields. The geometric field models the geometric structure of the object surface based on the signed distance function, and the medium property field independently models the optical absorption and scattering properties and refractive index distribution of the medium. The signed distance is converted into geometric density through a differentiable transformation function.

[0039] The surface adaptive hierarchical sampling module is used to generate a hierarchical sampling point sequence within the sampling interval containing the surface by locating the surface boundary to address the scale differences in urban scenes; and to apply the directional offset output by the directional offset network to the viewpoint direction of each sampling point.

[0040] The physics decoupling constraint module is used to introduce physics-driven refractive consistency loss and brightness anomaly constraint. The refractive consistency loss is based on the refractive index output by the medium property field to calculate the theoretical refraction direction, and the constraint direction offset conforms to the law of refraction. The brightness anomaly constraint forces the color value output by volume rendering to not exceed 1.

[0041] The progressive spatiotemporal joint optimization module is used to perform progressive training, sequentially optimizing the parameters of the geometric field, color field, medium property field, and orientation offset network. The urban scene is divided into geographic grids, with each grid independently trained as a sub-network, while sharing the orientation offset network parameters. A temporal optical flow field is introduced to model the spatiotemporal continuity of the dynamic transparent medium, and the changes in the geometric field and medium property field of adjacent frames are constrained by optical flow consistency loss.

[0042] This method effectively solves many problems of traditional NeRF in rendering transparent / semi-transparent objects by constructing a ray offset field, a decoupled geometric field, a medium property field, and a color field, combined with a physics-driven loss function and a progressive training mechanism. The ray offset field simulates the effect of light refraction, enabling the rendering to more accurately represent the bending of light caused by refraction in transparent / semi-transparent objects, restoring geometric consistency from multiple viewpoints. The decoupled field structure avoids the coupling between medium properties and geometric density, resolves optimization conflicts, and allows independent control of physical properties such as refractive index and scattering coefficient, achieving precise decoupling.

[0043] Physically driven refractive consistency loss and brightness anomaly constraints ensure that the rendering results conform to physical laws, avoid violating energy conservation, and make the rendered colors more reasonable. The progressive training mechanism optimizes the parameters of each field sequentially, improving training efficiency and stability, especially for complex media.

[0044] The optical enhancement mechanism for medium property fields further improves the rendering quality of translucent media, taking into account factors such as scattering, dispersion, interference, and dynamic refractive index, resulting in more realistic rendering effects. Differential calculations of light paths and adaptive sampling strategies improve the efficiency of refraction path calculations, adapting to the sampling requirements of curved light rays.

[0045] Surface boundary localization methods accurately determine the position of object surfaces, providing a foundation for subsequent sampling and rendering. A multilayer perceptron based on neRF constructs each field, ensuring the model's versatility and scalability.

[0046] In summary, the high-precision rendering method for neural radiation fields of transparent and semi-transparent objects of the present invention has significant advantages in improving rendering accuracy, solving existing technical problems, and enhancing rendering efficiency and quality. It provides an effective solution for rendering transparent / semi-transparent objects in the field of computer graphics technology and has important theoretical and practical application value. Attached Figure Description

[0047] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0048] Figure 1 This is a flowchart illustrating a method for constructing a city-level high-precision 3D scene based on NeRF technology according to the present invention. Detailed Implementation

[0049] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0050] Please see Figure 1 A method for constructing high-precision 3D city scenes based on NeRF technology, comprising:

[0051] S1: Construct a ray offset field to simulate the refraction effect of light in a transparent / semi-transparent medium. The directional offset field in the ray offset field is realized by a directional offset network. The directional offset network is used to predict the viewing angle directional offset.

[0052] S2: Construct a decoupled geometric field, medium property field, and color field; the geometric field models the geometric structure of the object surface based on the symbolic distance function; the medium property field independently models the optical absorption and scattering properties and refractive index distribution of the medium; the symbolic distance is converted into geometric density through a differentiable transformation function;

[0053] S3: To address the scale differences in city-level scenes, a hierarchical sampling point sequence is generated within the sampling interval containing the surface by locating the surface boundary; the directional offset output by the directional offset network is applied to the viewpoint direction of each sampling point;

[0054] S4: Introduce physically driven refractive consistency loss and brightness anomaly constraint. The refractive consistency loss is based on the refractive index output by the medium property field to calculate the theoretical refractive direction, and the constraint direction offset conforms to the law of refraction. The brightness anomaly constraint forces the color value output by volume rendering to not exceed 1.

[0055] S5: Perform progressive training, sequentially optimizing the parameters of the geometric field, color field, medium property field, and orientation offset network. Divide the urban scene into a geographic grid, train each sub-network independently, and share the orientation offset network parameters. Introduce a temporal optical flow field to model the spatiotemporal continuity of the dynamic transparent medium, and constrain the changes in the geometric field and medium property field of adjacent frames through optical flow consistency loss.

[0056] Specifically, the method for generating the hierarchical sampling point sequence includes:

[0057] S301: Determine the surface boundary interval by coarse sampling based on geometric field SDF;

[0058] S302: Resample within the boundary interval, and use the orientation offset network to predict the orientation offset for each sampling point to obtain the offset viewpoint orientation;

[0059] S303: Under the conditions of the original sampling point position and the offset viewing direction, obtain the geometric density of the geometric field; obtain the medium density and refractive index of the medium property field; obtain the color value of the color field;

[0060] S304: The effective volume density is calculated by using a blending function, with geometric density as the primary factor in the surface region and medium density as the primary factor in the interior of the medium. Based on the effective volume density and color value, the light color is synthesized using a volume rendering formula.

[0061] In this embodiment, the ray equation is initialized as follows: the observation ray at pixel (800, 600) originates from the camera center, and its direction is set according to the normalized vector. The pseudocode for iteratively locating the surface boundary can be expressed as:

[0062] {

[0063] t_m=0, t_max=5.0 #Ray range

[0064] t_current=t_min

[0065] for i in range(max_iter):

[0066] p = o + t_current*d # Current sampling point

[0067] sdf=geometry_mlp(p) #Query SDF value

[0068] if abs(sdf) < δ: # Surface hit

[0069] surface_bound=[t_current-0.1,t_current+0.1]#boundary interval[t_min,t_max];

[0070] break

[0071] t_current += max(min_step, 0.5*sdf) # SDF-guided adaptive step size

[0072] }

[0073] Output the surface boundary interval, and uniformly generate sampling points within the interval, represented by a sample point set;

[0074] The refraction effect of light in the medium property field is compensated by the directional offset field;

[0075] The formula for fusing effective volume density in density calculation and volumetric rendering is as follows:

[0076] ,

[0077] ,

[0078] Where α is the surface weighting factor, dynamically controlled by the SDF value: α≈1 near the surface, geometric density dominates; α→0 deep into the medium, medium density dominates; σ eff For the mixed density, σ g For geometric density, σ m p is the density of the medium. i Here are the sampling points, and γ is the smoothing factor;

[0079] Light color is synthesized by accumulating transmittance and rendering equations:

[0080] ;

[0081] ;

[0082] Among them, T i Let be the cumulative transmittance at position i, and exp be the attenuation exponential function. This represents the mixing density at the j-th sampling point. In spatial analysis, c represents the distance between two consecutive sampling points; pixel For the final color of the pixel, c i Contribution value to the light source.

[0083] Specifically, the orientation offset field is used to correct the viewing direction at the sampling point, and the orientation offset network is used to predict the change in the direction of the light rays at the sampling point; the input data of the orientation offset network are the origin of the light ray, the direction, the spatial position, and the viewing direction; the output is the orientation offset, and the deformed light ray is redefined by the output offset.

[0084] Specifically, the concrete implementation of the refractive consistency loss and brightness anomaly constraint described in S4 includes:

[0085] S401: For each sampling point, the surface normal vector is calculated based on the gradient of the geometric field SDF. The theoretical refraction direction is calculated according to the law of refraction and the standard refractive index corresponding to the transparent / semi-transparent object. The actual refraction offset direction is obtained by normalizing the direction offset predicted by the direction offset network. The cosine loss of the angle between the theoretical refraction direction and the actual refraction offset direction is calculated.

[0086] S402: The brightness anomaly constraint is that the color value output by the volumetric rendering does not exceed 1;

[0087] S403: Add the refraction uniformity loss and brightness anomaly constraint to the total loss function in a weighted manner.

[0088] Furthermore, the total loss function also includes a path continuity regularization term, specifically a curl consistency constraint for directional offset, which ensures that the path remains a linear segment within the medium by penalizing the curvature change of the ray path.

[0089] In this embodiment, the direction offset is strictly constrained based on Snell's law. An accurate normal vector can be efficiently obtained using automatic differentiation techniques. This normal vector is a key parameter for calculating the theoretical refraction direction and is crucial for subsequent accurate calculation of refraction consistency loss. During light propagation, when light encounters the surface of a transparent or translucent object, refraction occurs according to the law of refraction. By accurately calculating the surface normal vector and combining it with the standard refractive index corresponding to the transparent / translucent object, the theoretically correct direction of light refraction can be obtained.

[0090] The direction offset predicted by the direction offset network is normalized to obtain the actual refraction offset direction. Comparing these two directions and calculating their cosine loss allows us to measure the accuracy of the direction offset network's prediction. A smaller cosine loss indicates that the direction offset predicted by the direction offset network is closer to the theoretical refraction direction, and the rendering result will better conform to physical laws.

[0091] The constraint on brightness anomalies plays a crucial role in ensuring the realism of the rendering results. In real-world physical scenes, the brightness of light is limited and will not exceed the normal range. By forcing the color values ​​in volumetric rendering output to not exceed 1, overly bright anomalies are avoided, resulting in rendered images that better match visual perception.

[0092] Adding a path continuity regularization term to the total loss function, which is essentially a curl consistency constraint on directional offset, effectively penalizes changes in the curvature of light paths. Inside transparent or semi-transparent objects, the propagation path of light should remain as linear as possible. If the curvature of the light path changes too much, it will cause unrealistic bending in the rendering result. This regularization constraint ensures a more reasonable propagation path for light within the medium, improving the quality and accuracy of the rendering.

[0093] Specifically, the optimization mechanism of the progressive training includes: a phased network parameter unlocking mechanism, adaptive learning rate scheduling, and gradient stabilization processing;

[0094] The phased network parameter unlocking mechanism fixes the output offset of the fixed-direction offset network to zero during the pre-training phase, optimizing only the basic parameters of the geometric field SDF, color field, and medium property field. During the medium property field optimization phase, the medium property field parameters are unlocked, and the geometric field, color field, and medium property field are jointly optimized, introducing a brightness anomaly constraint, with the weights increasing linearly from 0 to a predetermined value. During the offset field fine-tuning phase, the parameters of the directional offset network are unlocked, and all network parameters are jointly optimized, while introducing refraction consistency loss and path smoothing regularization terms.

[0095] The adaptive learning rate scheduling dynamically adjusts the learning rate, with the geometric field learning rate decaying exponentially and the offset network learning rate decaying linearly.

[0096] The gradient stabilization process includes truncating the magnitude of the offset field gradient and applying an Eikonal regularization term to the SDF field.

[0097] Specifically, the medium property field further extends the optical enhancement processing mechanism for translucent media, including:

[0098] For semi-transparent media, a scattering coefficient term is added to the output of the medium property field, and a diffusion approximation term is added to the volume rendering equation; a wavelength dependence term is added to the output of the direction offset network, and a dispersion uniformity loss is constructed; by defining the film thickness field in the surface neighborhood, the interference modulation factor is calculated, and the interference effect is incorporated into the color field; and a spatially varying refractive index field is established, introducing a dynamic refractive index into the refractive uniformity loss.

[0099] In this embodiment, the diffusion approximation term is incorporated into the volume rendering equation, and new parameters are added to the output of the medium property field: scattering coefficient, absorption coefficient, and scattering anisotropy parameter. The volume rendering equation is then corrected as follows:

[0100] ,

[0101] in, Indicates self-illumination. Indicates a single scattering. L represents the diffusion approximation term; diffuse The approximate solution for dipole diffusion is expressed as follows:

[0102] ,

[0103] Where, σ tr For the transmission coefficient, d r d v These represent the distances from the positive and negative light sources to the sampling points, respectively; k d The diffusion term weights are predicted from the medium property field.

[0104] The above diffusion term simulates the soft light transmission of multiple scattering, avoiding the noise of single scattering;

[0105] The direction offset network is extended to include an additional wavelength in the input, and the output becomes a spectral shift. ;

[0106] Dispersion uniformity loss manifests as follows:

[0107] ,

[0108] in, For Cauchy's dispersion formula, Output by the medium property field;

[0109] The above is used under specific conditions to decompose white light into a spectrum when it passes through a prism, compensating for the dispersion deficiency of traditional NeRF.

[0110] The film thickness at a point in the neighborhood of the surface is derived from the SDF gradient of the geometric field:

[0111] ,

[0112] The formulas for calculating the interference modulation factor and color field correction are as follows:

[0113] ,

[0114] ,

[0115] Where I1 and I2 are the intensity of light reflected from the interface. The phase shift predicted for the medium property field;

[0116] In this embodiment, the dynamic refractive index field is modeled by the position-dependent dynamic refractive index output from the medium property field, and the refractive consistency loss can be modified as follows:

[0117] ,

[0118] Among them, the theoretical refraction direction d refr for:

[0119] ,

[0120] n is the surface normal, d offset This is the output of the offset network.

[0121] The above refraction correction is used for physical refraction scenarios in non-uniform media (underwater bubbles, gradient glass).

[0122] Specifically, the real-time rendering optimization mechanism also includes differential calculation of ray paths and adaptive sampling strategies, including: constructing the Jacobian matrix of deformable paths to support efficient automatic differentiation; optimizing sampling points based on curvature-sensitive step size and fixed step size of surface neighborhood; implementing ray bucketing for parallel computation; and establishing a dynamic update region detection mechanism to adjust the resolution of highly variable regions in real time.

[0123] Specifically, the method for locating the surface boundary includes:

[0124] Input the ray equation, and set the ray start point parameter, maximum number of iterations, surface distance threshold, and minimum step size distance;

[0125] Based on the light source parameters and light direction, the minimum step distance is used as the initial step size, and the step size is iterated along the light direction. In each iteration, the signed distance function value from the current point to the object surface is calculated. If it is less than the surface distance threshold, it is determined to be a surface boundary, and the point is recorded as a surface boundary point. If no boundary is found and the number of iterations reaches the maximum number of iterations, the iteration stops.

[0126] Specifically, the geometric field, color field, and medium property field are based on a neRF multilayer perceptron. The input to the geometric field is the position parameter, and the output is the SDF value. The input to the color field is the position parameter and the offset direction, and the output is the color value. The input to the medium property field is the position parameter, and the output is the medium density, refractive index, and scattering coefficient.

[0127] A city-level high-precision 3D scene construction system based on NeRF technology includes:

[0128] The refraction and deflection module is used to construct a light ray offset field to simulate the refraction effect of light in a transparent / semi-transparent medium. The direction offset field in the light ray offset field is realized by a direction offset network; the viewing angle direction offset is predicted by the direction offset network.

[0129] The decoupled physical property field module is used to construct decoupled geometric fields, medium property fields, and color fields. The geometric field models the geometric structure of the object surface based on the signed distance function, and the medium property field independently models the optical absorption and scattering properties and refractive index distribution of the medium. The signed distance is converted into geometric density through a differentiable transformation function.

[0130] The surface adaptive hierarchical sampling module is used to generate a hierarchical sampling point sequence within the sampling interval containing the surface by locating the surface boundary to address the scale differences in urban scenes; and to apply the directional offset output by the directional offset network to the viewpoint direction of each sampling point.

[0131] The physics decoupling constraint module is used to introduce physics-driven refractive consistency loss and brightness anomaly constraint. The refractive consistency loss is based on the refractive index output by the medium property field to calculate the theoretical refraction direction, and the constraint direction offset conforms to the law of refraction. The brightness anomaly constraint forces the color value output by volume rendering to not exceed 1.

[0132] The progressive spatiotemporal joint optimization module is used to perform progressive training, sequentially optimizing the parameters of the geometric field, color field, medium property field, and orientation offset network. The urban scene is divided into geographic grids, with each grid independently trained as a sub-network, while sharing the orientation offset network parameters. A temporal optical flow field is introduced to model the spatiotemporal continuity of the dynamic transparent medium, and the changes in the geometric field and medium property field of adjacent frames are constrained by optical flow consistency loss.

[0133] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for constructing high-precision 3D city scenes based on NeRF technology, characterized in that, include: S1: Construct a ray offset field to simulate the refraction effect of light in a transparent / semi-transparent medium. The directional offset field in the ray offset field is realized by a directional offset network. The directional offset network is used to predict the viewing angle directional offset. S2: Construct a decoupled geometric field, medium property field, and color field; the geometric field models the geometric structure of the object surface based on the symbolic distance function; the medium property field independently models the optical absorption and scattering properties and refractive index distribution of the medium; the symbolic distance is converted into geometric density through a differentiable transformation function; S3: To address the scale differences in city-level scenes, a hierarchical sampling point sequence is generated within the sampling interval containing the surface by locating the surface boundary; the directional offset output by the directional offset network is applied to the viewpoint direction of each sampling point; S4: Introducing physically driven refractive consistency loss and brightness anomaly constraint. For each sampling point, the refractive consistency loss calculates the surface normal vector based on the gradient of the geometric field SDF, calculates the theoretical refractive direction according to the law of refraction and the standard refractive index corresponding to transparent / semi-transparent objects, normalizes the direction offset predicted by the direction offset network to obtain the actual refractive offset direction, and calculates the cosine loss of the angle between the theoretical refractive direction and the actual refractive offset direction. The brightness anomaly constraint forces the color value output by volume rendering to not exceed 1. S5: Perform progressive training, sequentially optimizing the parameters of the geometric field, color field, medium property field, and orientation offset network. Divide the urban scene into a geographic grid, train each sub-network independently, and share the orientation offset network parameters. Introduce a temporal optical flow field to model the spatiotemporal continuity of the dynamic transparent medium, and constrain the changes in the geometric field and medium property field of adjacent frames through optical flow consistency loss.

2. The method according to claim 1, characterized in that, The method for generating the hierarchical sampling point sequence includes: S301: Determine the surface boundary interval by coarse sampling based on geometric field SDF; S302: Resample within the boundary interval, and use the orientation offset network to predict the orientation offset for each sampling point to obtain the offset viewpoint orientation; S303: Under the conditions of the original sampling point position and the offset viewing direction, obtain the geometric density of the geometric field; obtain the medium density and refractive index of the medium property field; obtain the color value of the color field; S304: The effective volume density is calculated by using a blending function, with geometric density as the primary factor in the surface region and medium density as the primary factor in the interior of the medium. Based on the effective volume density and color value, the light color is synthesized using a volume rendering formula.

3. The method according to claim 1, characterized in that, The orientation offset field is used to correct the viewing direction at the sampling point, and the orientation offset network is used to predict the change in the direction of light rays at the sampling point; the input data of the orientation offset network are the origin of the light ray, the direction, the spatial position, and the viewing direction. The output is a directional offset, which redefines the deformed ray.

4. The method according to claim 1, characterized in that, The specific implementation of the refractive consistency loss and brightness anomaly constraint described in S4 includes: S401: For each sampling point, the surface normal vector is calculated based on the gradient of the geometric field SDF. The theoretical refraction direction is calculated according to the law of refraction and the standard refractive index corresponding to the transparent / semi-transparent object. The actual refraction offset direction is obtained by normalizing the direction offset predicted by the direction offset network. The cosine loss of the angle between the theoretical refraction direction and the actual refraction offset direction is calculated. S402: The brightness anomaly constraint is that the color value output by the volumetric rendering does not exceed 1; S403: Add the refraction uniformity loss and brightness anomaly constraint to the total loss function in a weighted form; The total loss function also includes a path continuity regularization term, specifically a curl consistency constraint for directional offset, which ensures that the path remains a linear segment within the medium by penalizing changes in the curvature of the ray path.

5. The method according to claim 1, characterized in that, The optimization mechanism for progressive training includes: a phased network parameter unlocking mechanism, adaptive learning rate scheduling, and gradient stabilization processing. The phased network parameter unlocking mechanism fixes the output offset of the fixed-direction offset network to zero during the pre-training phase, optimizing only the basic parameters of the geometric field SDF, color field, and medium property field. During the medium property field optimization phase, the medium property field parameters are unlocked, and the geometric field, color field, and medium property field are jointly optimized, introducing a brightness anomaly constraint, with the weights increasing linearly from 0 to a predetermined value. During the offset field fine-tuning phase, the parameters of the directional offset network are unlocked, and all network parameters are jointly optimized, while introducing refraction consistency loss and path smoothing regularization terms. The adaptive learning rate scheduling dynamically adjusts the learning rate, with the geometric field learning rate decaying exponentially and the offset network learning rate decaying linearly. The gradient stabilization process includes truncating the magnitude of the offset field gradient and applying an Eikonal regularization term to the SDF field.

6. The method according to claim 1, characterized in that, The aforementioned medium property field also extends the optical enhancement processing mechanism for translucent media, specifically including: For semi-transparent media, a scattering coefficient term is added to the output of the medium property field, and a diffusion approximation term is added to the volume rendering equation; a wavelength dependence term is added to the output of the direction offset network, and a dispersion uniformity loss is constructed; by defining the film thickness field in the surface neighborhood, the interference modulation factor is calculated, and the interference effect is incorporated into the color field; and a spatially varying refractive index field is established, introducing a dynamic refractive index into the refractive uniformity loss.

7. The method according to claim 1, characterized in that, The proposed method for constructing a city-level high-precision 3D scene based on NeRF technology also includes a real-time rendering optimization mechanism. This mechanism includes differential calculation of ray paths and an adaptive sampling strategy, specifically: constructing the Jacobian matrix of the deformable path to support efficient automatic differentiation; optimizing sampling points based on curvature-sensitive step size and fixed step size of surface neighborhood; implementing ray binning for parallel computation; and establishing a dynamically updated region detection mechanism to adjust the resolution of highly variable regions in real time.

8. The method according to claim 1, characterized in that, The method for locating the surface boundary includes: Input the ray equation, and set the ray start point parameter, maximum number of iterations, surface distance threshold, and minimum step size distance; Based on the light source parameters and light direction, the minimum step distance is used as the initial step size, and the step size is iterated along the light direction. In each iteration, the signed distance function value from the current point to the object surface is calculated. If it is less than the surface distance threshold, it is determined to be a surface boundary, and the point is recorded as a surface boundary point. If no boundary is found and the number of iterations reaches the maximum number of iterations, the iteration stops.

9. The method according to claim 1, characterized in that, The geometric field, color field, and medium property field are based on a NeRF multilayer perceptron. The input to the geometric field is the position parameter, and the output is the SDF value. The input to the color field is the position parameter and the offset direction, and the output is the color value. The input to the medium property field is the position parameter, and the output is the medium density, refractive index, and scattering coefficient.

10. A city-level high-precision 3D scene construction system based on NeRF technology, used to perform any one of the methods described in claims 1-9, characterized in that, include: The refraction and deflection module is used to construct a light ray offset field to simulate the refraction effect of light in a transparent / semi-transparent medium. The direction offset field in the light ray offset field is realized by a direction offset network; the viewing angle direction offset is predicted by the direction offset network. The decoupled physical property field module is used to construct decoupled geometric fields, medium property fields, and color fields. The geometric field models the geometric structure of the object surface based on the signed distance function, and the medium property field independently models the optical absorption and scattering properties and refractive index distribution of the medium. The signed distance is converted into geometric density through a differentiable transformation function. The surface adaptive hierarchical sampling module is used to generate a hierarchical sampling point sequence within the sampling interval containing the surface by locating the surface boundary to address the scale differences in urban scenes; and to apply the directional offset output by the directional offset network to the viewpoint direction of each sampling point. The physics decoupling constraint module is used to introduce physics-driven refractive consistency loss and brightness anomaly constraint. The refractive consistency loss is based on the refractive index output by the medium property field to calculate the theoretical refraction direction, and the constraint direction offset conforms to the law of refraction. The brightness anomaly constraint forces the color value output by volume rendering to not exceed 1. The progressive spatiotemporal joint optimization module is used to perform progressive training, sequentially optimizing the parameters of the geometric field, color field, medium property field, and orientation offset network. The urban scene is divided into geographic grids, with each grid independently trained as a sub-network, while sharing the orientation offset network parameters. A temporal optical flow field is introduced to model the spatiotemporal continuity of the dynamic transparent medium, and the changes in the geometric field and medium property field of adjacent frames are constrained by optical flow consistency loss.

Images