A new view synthesis method based on convolutional neural radiance fields

Abstract

The application discloses a new view synthesis method based on convolutional neural radiance field, which is characterized by using implicit neural representation to model the change of radiance in the scene, reconstructing the scene with complex optical phenomena, and eliminating the ambiguity existing in the process of optimizing geometric information, and specifically comprises the following steps: calculating the starting point and direction of the ray, importance sampling, extracting the characteristics of the ray, decoding of the attribute network, unique surface constraint and new view synthesis, etc. Compared with the prior art, the application has the advantages that the optimization of geometric information is changed from a regression task to a classification-like task, the difficulty of the optimization problem is reduced, the ability of the method to represent the scene is improved, the effect of new view synthesis is improved, the coupling degree is low, the method can be easily combined into most of the neural radiance field-based methods designed for specific scenes, the rendering result is further improved, the quality of new view synthesis is improved, and the application has certain practical value.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and computer graphics, specifically a novel view synthesis method based on convolutional neural radiation fields. Background Technology

[0002] Recently, novel view synthesis techniques employing neural implicit representations have rapidly advanced due to their surprisingly high-quality images generated under various camera poses. Based on the initial Neural Radiation Field (NeRF) method, various research directions have been explored to further improve rendering quality, rendering speed, and other aspects. Furthermore, because NeRF links radiosity and geometric information, many other applications utilize NeRF as a building block for tasks in the field of computer graphics, such as extracting geometric, semantic, and material information from scenes, and extending static settings to dynamic scenes.

[0003] Currently, there are many methods that improve upon NeRF, but because they are still based on the NeRF framework, traditional volumetric rendering methods remain in the rendering pipeline. However, this rendering method makes NeRF's optimization of geometric information multifaceted. (See attached image) Figure 1 As shown, the same integral color is obtained under completely different density distributions. If geometric ambiguity exists, the synthesized image from the new viewpoint may contain errors, which will seriously affect the reliability of the synthesized result. Light field-based methods can reproduce complex optical phenomena with higher quality, but they also lead to geometric ambiguity. (See attached image) Figure 2 As shown, due to point P on the surface T and point P in space F They all have the same color, and their geometric information is blurred when viewed from two different angles. Specifically, different points P... T and P F The observation points o1 and o2 of the two cameras receive the same radiance L(P) T o i )=L(P F o i ), i∈{1,2}. This means that there may be multiple positional schemes that can satisfy the same line-dependent radiance, which is difficult for neural networks to optimize.

[0004] Existing new view synthesis algorithms suffer from ambiguity and vagueness in the process of optimizing geometric information, and cannot better reproduce phenomena such as reflection and subsurface scattering, resulting in poor ability to characterize scenes and poor new view synthesis effects. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a novel view synthesis method based on convolutional neural radiation fields. It employs an implicit scene representation method, building upon a scene representation method based on the derivative of radiance. Using a convolutional structure, it extracts local features along rays using a fully neural rendering scheme, eliminating ambiguity and vagueness in the process of optimizing geometric information. It also uses a concise neural network expression to recreate complex optical phenomena, synthesizing new views with high quality. The method is simple, requiring no storage of information within the entire blank space interval from the camera origin to the object intersection. Modeling radiation lines as derivatives generates sparse solutions along individual rays, transforming the optimization of geometric information from a regression task to a classification-like task, thus reducing the difficulty of the optimization problem. This method draws on experience from the development of deep learning, improving the network structure of neural radiation fields and enhancing the quality of new view synthesis with a concise approach. The proposed structure has low coupling and can be easily integrated into most scene-specific neural radiation field-based methods, further improving rendering results and possessing practical value.

[0006] The specific technical solution for implementing this invention is: a novel view synthesis method based on convolutional neural radiation fields, characterized in that the method specifically includes the following steps:

[0007] 1) The origin and direction of the ray are calculated using the camera's intrinsic parameters (i.e., inherent properties such as the camera's focal length, focal position, and lens distortion) and extrinsic parameters (i.e., the observation position and angle of view). Each ray originates from the camera's location and passes through the corresponding pixel on the focal plane.

[0008] 2) Sampling is performed on the ray to obtain several discrete sampling points in order to increase the quality of the rendered image and reduce noise. The sampling is specifically divided into two stages: in the coarse sampling stage, the ray is divided into a fixed number of intervals, and sampling is performed once on each interval. All positions in each interval have an equal probability of being sampled.

[0009] In the fine sampling stage, based on the radiance coefficients obtained in the coarse stage, the softmax function is used to convert the radiance coefficients of each sampling point into probabilities, assigning higher probabilities to locations where objects are more likely to appear, making it easier to sample objects near those locations.

[0010] 3) A ray feature extractor is used to extract features of the corresponding rays from the sampling points obtained in step 2. Utilizing the discrete sampling points from step 2, and based on the scene representation theory of radiance derivatives, features of the corresponding rays are extracted. The features of the sampling points on the rays are then fused, combining the advantages of radiance derivative representation with those of light field representation. The ray feature extractor is based on a U-shaped convolutional neural network, used to parameterize the scene representation based on radiance derivatives. It treats the space of each ray as a complete scene representation and uses one-dimensional convolution along the ray direction to extract features of the intersection points between the ray and the object surface. Through layer-by-layer convolution, the receptive field gradually increases, finally outputting a code containing local features related to location and global features related to light.

[0011] 4) Based on the scene representation theory of radiance derivative, ray features are decoded through a radiance attribute network and a geometric attribute network, and unique surface constraints are applied. The features obtained in step 3 are in a high-dimensional space and need to be converted into quantities with physical meaning before subsequent rendering operations can be performed. The radiance attribute network is a two-layer MLP that converts ray features into radiance color at each sampling point, using sigmoid as the activation function to limit the output value to between 0 and 1, matching the range of normalized color values. The geometric attribute network mainly learns the position of the intersection point between a given ray and the object surface and the local incident luminance. To handle the occlusion relationship between points on the ray, the geometric attribute network uses a gated recurrent unit (GRU), which exhibits strong performance in sequence learning. In each recursive step, all points sampled before the current step are used as prior points. The GRU-based module outputs the original geometric attribute coefficients through a three-layer fully connected network, with the activation function of the last layer of the fully connected network being sigmoid.

[0012] Unique surface constraints are used to address ambiguity issues in fitting problems. Since the radiance reaches its maximum value only at the first intersection point and is negligible in the rest of the ray, this situation is well-suited for using softmax, which amplifies the differences between inputs and selects the largest value among a series of inputs in a differentiable manner.

[0013] 5) Using the Epipolar-based color expectation calculation module, combined with blank area regularization, the attributes obtained in step 4 are converted into pixel colors. The softmax constraint used in the unique surface constraint restricts the sum of the radiance coefficients of the sampled points on the ray to always be 1. However, if the ray does not pass through any object, the sum of the radiance coefficients should obviously be 1. Therefore, it is assumed that there is an epipolar point at infinity on the ray, where the radiance coefficient is sufficiently small but not 0. When all inputs are 0, i.e., the ray does not pass through any object, the epipolar point is selected after softmax calculation. Subsequently, the radiance color of each sampled point is weighted and summed according to the weights processed by the epipolar mechanism to obtain the final color of the corresponding ray (i.e., pixel).

[0014] This invention proposes a scene representation method based on the radiosity derivative. This scene representation method is based on the derivation of the rendering equation in computer graphics, and the specific derivation is as follows:

[0015] 1) In the scene settings of the new view composition, the position and intensity of the light source remain unchanged, and the radiance of a point p on the object's surface can be calculated using a simplified scattering equation:

[0016]

[0017] Among them, L out (p,ω o ) indicates that point p is at ω o The light emitted from the direction, L in (p,-ω i () represents the light reaching point p. s (p,ω i ω o ) is the two-way scattering distribution function (BSDFs), S 2 Indicates the direction of the incident ray.

[0018] 2) Given a ray originating at point o and directed by direction d, the intersection of the ray and the surface can be represented as p = o + td, where t is the distance from the intersection point to the origin of the line of sight. The direction of the emitted radiation is ω. o Opposite to the direction of the viewpoint d, i.e., d = -ω o Considering only the radiant flux along a single ray, the radiance can be rewritten as L(o, d, t). Based on the general function fitting capability of neural networks, a specially designed network F is used to fit the scattering equation:

[0019] L(o,d,t)=F(o,d,t;Θ),

[0020] Here, Θ is a network parameter that encodes fixed attributes for different scenarios.

[0021] 3) To obtain a single mapping space, the radiance ray needs to be modeled as the derivative of the radiance, i.e.

[0022]

[0023] Since radiance only changes significantly near the surface and remains zero in open space or inside an object, Dirac's delta function δ(o, d, t) is introduced here to approximate the derivative of radiance:

[0024]

[0025] Dirac's delta function has a value only at the intersection of the light ray and the object's surface.

[0026] 4) To achieve this radiance in practice, L(o, d, t) is discretized as...

[0027]

[0028] In this context, the indicator function II[o, d, i] is equivalent to Dirac's delta function in the case of discontinuity. Since the indicator function is non-zero only at sampling points near the surface, the range of summation can be extended from [t, D] to [1, D]. Therefore, the radiance does not change with distance t. Thus, the rendering equation can be divided into two parts: II[o, d, i] represents the intersection of the ray and the scene, and L(o, d, i) records the radiance at different locations in the scene.

[0029] 5) To fit the discretized L(o, d, t) with a neural function, a convolutional ray feature extractor V is first proposed. r =ρ F (o, d, t; θ) F ), where t = [t1, ..., t2] D [This is used for ray information embedding. Then, a geometric attribute network ρ is used.] G (V r ;θ G The intersection point is represented by α, and a unique surface constraint α is used to fit the unimodal indicator function II[o, d, i]. The radiometric attribute network ρ R (V r ;θ R Calculate the intersection radiance. Finally, the scattering equation for ray radiance can be approximately expressed as:

[0030] Compared with existing technologies, this invention eliminates ambiguity in the process of optimizing geometric information and can restore it in a concise manner.

[0031]

[0032] Complex optical phenomena are represented by a scene representation method based on radiance derivatives. Employing a convolutional structure, local features along rays are extracted, and new views are synthesized with high quality using a concise method. The method is simple, requiring no storage of information within the entire empty space from the camera origin to the object's intersection. Modeling radiation lines as derivatives generates sparse solutions along individual rays, transforming the optimization of geometric information from a regression task to a classification-like task, thus reducing the difficulty of the optimization problem and further improving rendering results, demonstrating practical value. Attached Figure Description

[0033] Figure 1 To solve the multi-solution graph of geometric information in the radiation field rendering method;

[0034] Figure 2 To solve the ambiguity of geometric information in light field rendering methods;

[0035] Figure 3 This is a schematic diagram of the process of the present invention;

[0036] Figure 4 This is a qualitative comparison diagram between the present invention and the prior art;

[0037] Figure 5 This is a mesh rendering extracted using information reconstructed using the present invention. Detailed Implementation

[0038] See Figure 3 This invention employs a method that uses implicit neural representation to depict radiance variations in a scene and models the radiance derivative. This eliminates ambiguity in optimizing geometric information and uses a concise representation to reconstruct complex optical phenomena. A fully neuralized process is used to fit the rendering equation to improve rendering quality. The new view synthesis method specifically includes the following steps:

[0039] Step 1: Calculation of ray origin and direction

[0040] The origin and direction of the rays are calculated using the camera's intrinsic and extrinsic parameters. Each ray originates from the camera's location and passes through the corresponding pixel on the focal plane. The intrinsic parameters are the camera's focal length, focal position, and inherent properties of lens distortion; the extrinsic parameters are the observation position and viewing angle.

[0041] Step 2: Importance Sampling

[0042] Several discrete sampling points are selected as importance sampling points. Coarse sampling and fine sampling are performed on the ray. In the coarse sampling stage, the ray is divided into a fixed number of intervals, and sampling is performed once in each interval. All positions in each interval have an equal probability of being sampled. In the fine sampling stage, the radiometric coefficients obtained in the coarse sampling stage are converted into probabilities using the softmax function.

[0043] Step 3: Extracting ray features

[0044] A ray feature extractor is used to extract features of the corresponding rays from the sampling points obtained in step 2. The ray feature extractor is based on a U-shaped convolutional neural network and is used to parameterize the scene representation based on the radiance derivative. Finally, it outputs a local feature encoding that includes location-related features and a global feature encoding that includes light-related features.

[0045] Step 4: Decoding the radiosity attribute network

[0046] Using a radiometric attribute network based on an MLP structure, the normalized color of each sampling point is decoded by utilizing the ray feature sequence.

[0047] Step 5: Decoding the geometric attribute network

[0048] By utilizing ray feature sequences and employing a geometric property network based on a GRU structure, the position and local incident luminosity of the intersection point between a given ray and the object surface are learned.

[0049] Step 6: Unique Surface Constraint

[0050] Taking advantage of the property that the radiance on a ray reaches its maximum value only at the first intersection point, a unique surface constraint is used to select the intersection point between the ray and the object surface.

[0051] Step 7: Color Expectation Calculation

[0052] By using an Epipolar-based color expectation calculation module and combining it with blank area regularization, the final color of the rays is calculated by decoding the radiance color and radiance coefficient, thus obtaining an observation image from a new perspective.

[0053] Step 3 describes the design of a ray feature extractor p based on a U-shaped convolutional neural network. F To extract ray features V r The scene representation based on radiance derivative is parameterized, and the number of sampling points along the light direction is compressed through convolutional structure to generate bottleneck features. Layer-by-layer convolution is then used to increase the receptive field and embed the entire light ray.

[0054] Step 4 involves designing a radiance attribute network ρ with a two-layer MLP structure. R To extract radiometric color Cr The high-dimensional ray features are decoded into radiance colors at each sampling point.

[0055] Step 5 involves designing a geometric attribute network ρ based on a GRU structure. G To extract the original radiance coefficient s r In each recursive step, all points sampled before the current step are used as prior points. If the current point is occluded by previous points, a gating mechanism is used to retain the information of previous points and reduce the influence of the current point on the GRU state. The geometric attribute network based on GRU outputs the original geometric attribute coefficients through a three-layer fully connected network. The activation function of the last layer of the fully connected network is sigmoid.

[0056] Step 6 introduces a unique surface constraint as α, and takes advantage of the fact that the radioactivity reaches its maximum value only at the first intersection point, using the softmax operation to select the first intersection point.

[0057] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. The embodiments of the present invention include, but are not limited to, the following embodiments.

[0058] Example 1

[0059] See Figure 3 Perform the following steps to synthesize a new view based on the convolutional neural radiation field:

[0060] Step 1: Calculate the starting point and direction of the ray based on the observation position and viewing angle.

[0061] Using the camera's intrinsic parameters (i.e., inherent properties such as focal length, focal position, and lens distortion) and extrinsic parameters (i.e., viewing position and angle of view), the origin and direction of the ray can be calculated. Each ray originates from the camera's location and passes through the corresponding pixel position on the focal plane.

[0062] Assuming the camera's intrinsic parameter matrix is ​​K, its extrinsic parameter matrix is ​​[R|t], and the pixel coordinates are (u, v), the starting point and direction of the ray can be calculated using the following steps:

[0063] 1-1: Convert the pixel coordinates (u, v) to three-dimensional coordinates (X, v) in the camera coordinate system using the following equation (a). c Y c Z c ):

[0064]

[0065] 1-2: Convert the three-dimensional coordinates (X, Y, Z) in the camera coordinate system. c Y c Z cThe following equation (b) is used to convert the coordinates to three-dimensional coordinates in the world coordinate system (X). w Y w Z w ):

[0066]

[0067] 1-3: Calculate the origin and direction of the ray. The origin of the ray is the position of the camera, and the direction is from the camera position to the (X) coordinate system. w Y w Z w )point.

[0068] Step 2: Importance Sampling

[0069] To improve the quality of the rendered image and reduce noise, a phased importance sampling method was employed. Specifically, ray sampling is divided into a coarse sampling phase and a fine sampling phase.

[0070] In the coarse sampling phase, the ray is divided into 128 equal intervals, and sampling is performed once with equal probability in each interval. The purpose of this is to cover the entire ray as much as possible and to ensure that each point has an equal probability of being sampled.

[0071] In the fine-sampling stage, based on the radiance coefficients obtained in the coarse-sampling stage, the softmax function is used to convert the radiance coefficient of each sampling point into a probability. In this stage, locations more likely to contain objects are assigned a higher probability, making their vicinity easier to sample. The goal of this stage is to minimize noise and improve the quality of the rendered image while ensuring that the sampling points cover the rays. A total of 64 points are sampled in the fine-sampling stage.

[0072] Step 3: Extracting ray features

[0073] Using the D sampling points obtained in step 2, features of the corresponding rays are extracted based on the scene representation theory of radiance derivative, and the features of the sampling points on the rays are fused, thus combining the advantages of light field representation with the scene representation method based on radiance derivative. The ray feature extractor is based on a U-shaped convolutional neural network ρ. F Used to obtain a given ray The compact representation of V r =ρ F (o, d, t; θ) F ).

[0074] The ray feature extractor is designed as an 8-layer network, with the first four layers being MLPs with a width of 256. Subsequent U-shaped convolutional layers learn local and global information about the rays in the latent space representation. Two convolutional downsampling layers within this network have a kernel size of 3 and a stride of 2, considering the positions and features of neighboring points when generating feature values. This feature embedding structure implicitly encodes the viewing angle direction along the ray sampling position. The convolutional structure compresses the number of sampling points along the ray direction, creating bottleneck features. Through this layer-by-layer convolution, the receptive field gradually increases, and the bottleneck features are embedded throughout the ray. Finally, two upsampling layers perform upsampling using linear interpolation and concatenate the interpolated feature vector with the relative vector from the downsampling stage. The output ray feature encoding V is... r It includes location-related local features and light-related global features.

[0075] Step 4: Decode the ray features

[0076] The feature V obtained in step 3 r In high-dimensional space, it needs to be converted into a quantity with physical meaning before subsequent rendering operations can be performed. Specifically, a radiometric attribute network ρ needs to be used. R Geometric attribute network ρ G Furthermore, a unique surface constraint α is added to enable decoding of ray features.

[0077] The radiometric attribute network ρ R It is a two-layer MLP that converts ray features into radiometric color C at each sampling point. r = [c1, c2, ..., c D ]=ρ R (V r ;θ R The network employs a hidden layer feature dimension of 128 and a sigmoid activation function, which restricts the output value to between 0 and 1 to match the range of normalized colors.

[0078] The geometric attribute network ρ G Used to calculate the original radiance coefficient s r = [s1, s2, ..., s D ]=ρ G (V r ;θ GThis network employs a GRU module with a hidden state dimension of 64, demonstrating strong performance in sequence learning for handling occlusion relationships between points on a light ray. Following the GRU module is a four-layer MLP to obtain one-dimensional raw geometric attribute coefficients. To progressively reduce the feature dimension, the hidden layer dimensions are 64, 64, and 32 respectively. The final layer of the MLP is designed to use a sigmoid activation function to represent the probability of an object existing at a point in space.

[0079] The unique surface constraint, α, is used to address ambiguity in the fitting problem. Since the radiance reaches its maximum value only at the first intersection point and is negligible in the rest of the ray, this situation is well-suited for softmax. To avoid output instability caused by all inputs being very small, the original geometric properties are first multiplied by a scaling factor θ. α Then the result is input into the softmax function. Based on the experimental results, the scaling factor θ... α The effect is best when set to 10, and the final radiance coefficient can be expressed by the following equation (c):

[0080] w r，e =α(s r，e ;θ α ) = softmax(θ a *s r，e (c).

[0081] Where s r，e =concat(s r s e ), w r，e =concat(w r w e ), where the variable with the subscript 'e' represents the properties of the epipolar point.

[0082] Step 5: Calculate color expectation based on Epipolar

[0083] Using an epipolar-based color expectation calculation module combined with blank area regularization, the attributes obtained in step 4 are converted into pixel colors. The softmax operation in the unique surface constraint limits the radiance coefficient w of the ray sampling points. r The sum is 1, but this constraint no longer applies if the ray does not pass through any object. To solve this problem, assume there is an epipolar point at infinity where the radiance coefficient s is located. e It is set to 1 / N, where N is the number of sampling points on the ray. When all inputs are 0, i.e., the ray does not pass through any object, the epipolar point is selected after softmax calculation, and w is then... rThe values ​​are all 0. Then, the radiance color of each sampling point is weighted and summed to obtain the final color of the corresponding ray (i.e., pixel) represented by the following equation (d):

[0084]

[0085] Blank region regularization applies L1 regularization to the geometric attribute coefficients of the sampling points on the ray, forcing the geometric attribute coefficients wr to become sparse, as expressed in equation (e) below:

[0086]

[0087] See Figure 4 The Blender Dataset is a commonly used dataset in novel view compositing tasks, where GT represents the ground truth and Ours represents the results of this invention. Compared with existing state-of-the-art methods, the algorithm of this invention significantly improves visual effects and achieves better compositing results for highlights, specular reflections, and objects with complex hierarchical relationships.

[0088] See Figure 5 The scene information represented by this invention can be converted into fine mesh data, where Figure a is the rendering result and Figure b is the extracted mesh.

[0089] The above examples are merely typical cases in this invention. Clearly, the technical solutions involved in this invention are not limited to the above embodiments, and many other variations and improvements are possible. Without departing from the innovative spirit of this invention, those skilled in the art can make various modifications or alterations to the above embodiments. Therefore, the scope of protection is not limited to the above embodiments, but should be the maximum scope conforming to the innovative features mentioned in the claims.

Claims

1. A novel view synthesis method based on convolutional neural radiation fields, characterized in that, The new view composition method specifically includes the following steps: Step 1: Calculation of ray origin and direction The origin and direction of the rays are calculated using the camera's intrinsic and extrinsic parameters. Each ray originates from the camera's location and passes through the corresponding pixel on the focal plane. The intrinsic parameters are the camera's focal length, the focal point, and the inherent properties of lens distortion. The extrinsic parameters are the observation position and the angle of view. Step 2: Importance Sampling Several discrete sampling points are selected as importance sampling points. Coarse sampling and fine sampling are performed on the ray. In the coarse sampling stage, the ray is divided into a fixed number of intervals, and sampling is performed once in each interval. All positions in each interval have an equal probability of being sampled. In the fine sampling stage, the radiometric coefficients obtained in the coarse sampling stage are converted into probabilities using the softmax function. Step 3: Extracting ray features A ray feature extractor is used to extract the features of the corresponding rays from the sampling points obtained in step 2. The ray feature extractor is based on a U-shaped convolutional neural network and is used to parameterize the scene representation based on the radiance derivative. Finally, it outputs a local feature encoding that includes location-related features and a global feature encoding that includes light-related features. Step 4: Decoding the radiosity attribute network Using a radiometric attribute network based on an MLP structure, the radiometric color of each sampling point is decoded by utilizing ray features; Step 5: Decoding the geometric attribute network By utilizing ray features and employing a geometric property network based on a GRU structure, the original radiance coefficients of each sampling point are decoded, and the position and local incident radiance of the intersection point between a given ray and the object surface are learned. Step 6: Unique Surface Constraint A unique surface constraint is applied to the original radiance coefficient. Taking advantage of the property that the radiance on the ray reaches its maximum value only at the first intersection point, the radiance coefficient of each sampling point is obtained through the softmax operation, and the intersection point of the ray and the object surface is selected. Step 7: Color Expectation Calculation Using the Epipolar-based color expectation calculation module, combined with blank area regularization, the final color of the ray is calculated based on the radiance color and the radiance coefficient obtained in step 6, resulting in an observation image from a new perspective.

2. The novel view synthesis method based on convolutional neural radiation fields according to claim 1, characterized in that, Step 3 describes the design of a ray feature extractor based on a U-shaped convolutional neural network. To extract ray features The scene representation based on radiance derivative is parameterized, and the number of sampling points along the light direction is compressed through convolutional structure to generate bottleneck features. Layer-by-layer convolution is then used to increase the receptive field and embed the entire light ray.

3. The novel view synthesis method based on convolutional neural radiation fields according to claim 1, characterized in that, Step 4 describes the design of a radiometric attribute network with a two-layer MLP structure. To extract radiometric color The high-dimensional ray features are decoded into radiance colors at each sampling point.

4. The novel view synthesis method based on convolutional neural radiation fields according to claim 1, characterized in that, Step 5 involves designing a geometric attribute network based on a GRU structure. To extract the original radiance coefficient In each recursive step, all points sampled before the current step are used as prior points. If the current point is occluded by previous points, a gating mechanism is used to retain the information of previous points and reduce the influence of the current point on the GRU state. The geometric attribute network based on GRU outputs the original geometric attribute coefficients through a three-layer fully connected network. The activation function of the last layer of the fully connected network is sigmoid.

5. The novel view synthesis method based on convolutional neural radiation fields according to claim 1, characterized in that, Step 6 introduces a unique surface constraint as... Taking advantage of the fact that the radiance reaches its maximum value only at the first intersection point, a softmax operation is used to select the first intersection point.

Images