Three-dimensional scene representation model construction method based on scene adaptive local voxel multi-plane image

Abstract

The invention belongs to the field of three-dimensional reconstruction, and particularly relates to a three-dimensional scene representation model construction method based on scene adaptive local voxel multi-plane images, which comprises the following steps: dividing a three-dimensional scene to be represented into voxel grids, and constructing a global multi-plane image MPI of each voxel; determining intersection points of different light rays and each voxel MPI under each known view angle, encoding the three-dimensional position coordinates of each intersection point corresponding to each light ray into a multi-dimensional position vector, and encoding the direction of each intersection point corresponding to each light ray into a multi-dimensional direction vector; a multi-dimensional position vector set and a multi-dimensional direction vector set corresponding to the light and a color label of a pixel corresponding to the light in the collected scene image form a training sample; constructing a three-dimensional scene representation network, wherein the three-dimensional scene representation network comprises a position multi-layer MLP network, a direction multi-layer MLP network and a color calculation unit of each voxel; and training by adopting the training sample to obtain a three-dimensional scene representation model. The detail features of the local area can be effectively captured.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional reconstruction, and more specifically, relates to a method for constructing a three-dimensional scene representation model based on scene-adaptive local voxel multi-plane images. Background Art

[0002] Existing 3D scene expression methods based on global multi-plane images decompose the 3D scene into multiple parallel forward semi-transparent planes MPI along the reference camera's viewing cone direction, that is, a certain number of multi-planes at a fixed depth perpendicular to the reference camera constructed under the reference camera. The points on the planes are observed by the reference camera and the target camera respectively. This perspective consistency allows MPI to express the entire scene at a certain perspective with fewer sampling rays. A typical example is NeX, which uses a global MPI with a fixed depth distribution to represent 3D scenes, and uses a large MLP network for global learning to encode the color and opacity information of MPI. However, this method is not rich enough for local texture rendering in 3D scenes, and the synthesized 3D scene images are not good when synthesizing 3D scene images at a certain perspective.

[0003] Recent studies have improved the local texture representation in the scene by using local planes. For example, Neurmips uses point clouds to construct local planes. Specifically, it uses the Sfm 3D reconstruction method and colmap software to obtain a 3D point cloud (the 3D point cloud coordinates in the scene), clusters the 3D point cloud into multiple local planes, and uses a large global MLP network to learn the texture and geometric features of the local planes. Finally, a small MLP network is used for distillation learning and fine-tuning to achieve the purpose of scene local learning. However, this method still cannot achieve good rendering results. Summary of the invention

[0004] In view of the defects of the prior art and the need for improvement, the present invention provides a method for constructing a three-dimensional scene representation model based on scene-adaptive local voxel multi-plane images, which aims to effectively capture the detailed features of the local area and improve the rendering effect of the three-dimensional scene.

[0005] To achieve the above object, according to one aspect of the present invention, a method for constructing a three-dimensional scene representation model based on scene-adaptive local voxel multi-plane images is provided, comprising:

[0006] The three-dimensional scene to be represented is divided into a voxel grid, and a global multi-plane image MPI of each voxel is constructed; the intersection points of different light rays with each voxel MPI under each known viewing angle are determined, and the three-dimensional position coordinates of each intersection point corresponding to each light ray are encoded as a multi-dimensional position vector, and the direction is encoded as a multi-dimensional direction vector, and the multi-dimensional position vector set corresponding to the light ray, the multi-dimensional direction vector set and the color label of the pixel corresponding to the light ray in the collected scene image constitute a training sample;

[0007] Constructing a multi-layer MLP network for each voxel, a multi-layer MLP network for direction, and a color calculation unit; the multi-layer MLP network for position includes n layers of self-attention units respectively connected to the output ends of the first n layers of fully connected layers, which are respectively used to divide the feature vector output by the previous fully connected layer into multiple non-overlapping windows, and perform self-attention learning on each window, and splice the learning results of each window and input them into the next fully connected layer; the input of the multi-layer MLP network for position is the multi-dimensional position vector of all intersections of each light on the corresponding voxel of the network, and the output is the color coefficient vector and opacity corresponding to all intersections; the input of the multi-layer MLP network for direction is the multi-dimensional direction vector of all intersections of each light on the corresponding voxel of the network, and the output is the basis function of all intersections; the color calculation unit is used to calculate the color of all intersections based on the basis function and color coefficient vector of all intersections of each light received on all voxels, combined with the basic color vector to be optimized, so as to synthesize the opacity of all intersections to obtain the pixel rendering color corresponding to the light;

[0008] The training samples are used to optimize the position multi-layer MLP network, orientation multi-layer MLP network and basic color vector of each voxel until the preset termination condition is reached to obtain a three-dimensional scene representation model.

[0009] Furthermore, a sine function with a frequency length of m is used to perform m-dimensional vector encoding to obtain the multi-dimensional position vector and the multi-dimensional direction vector.

[0010] Further, the encoding method of the multidimensional position vector is: using a sine function with a frequency length of 20 to encode the plane position coordinates of each intersection corresponding to each light ray into a 40-dimensional vector; using a sine function with a frequency length of 8 to encode the depth coordinates of each intersection corresponding to the light ray into a 16-dimensional vector; splicing the 40-dimensional vector and the 16-dimensional vector to obtain the multidimensional position vector of the corresponding intersection;

[0011] The encoding method of the multi-dimensional direction vector is: using a sine function with a frequency length of 4 to encode the direction of each intersection point corresponding to each light ray into an 8-dimensional vector, so as to obtain the multi-dimensional direction vector of the intersection point.

[0012] Furthermore, the value of n is 2.

[0013] Furthermore, the calculation of the color and the opacity is performed using a preset linear relationship between a basis function set, a color coefficient vector set and a basic color vector.

[0014] The present invention also provides a method for synthesizing a new perspective of a three-dimensional scene, which uses the three-dimensional scene representation model constructed by the three-dimensional scene representation model construction method as described above to perform color rendering of pixels corresponding to multiple light rays under a new perspective, thereby synthesizing a new perspective image.

[0015] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the three-dimensional scene representation model construction method as described above and / or the three-dimensional scene new perspective synthesis method as described above.

[0016] The present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the three-dimensional scene representation model construction method as described above and / or the three-dimensional scene new perspective synthesis method as described above.

[0017] The present invention also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the above-mentioned three-dimensional scene representation model construction method and / or the above-mentioned three-dimensional scene new perspective synthesis method.

[0018] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0019] (1) The present invention proposes a voxel-based multi-plane three-dimensional scene representation method, which constructs a local MPI at the voxel level (that is, constructs an MPI for each voxel), retains the simplicity and perspective consistency of the MPI structure, and at the same time finely models each local MPI through the independently encoded MLP (implicit neural radiation field) of each voxel, which can effectively capture the detailed features of each local area. Among them, in order to solve the block effect existing in the independent modeling of each voxel, a voxel plane attention mechanism is introduced in the MLP network of each voxel, and the content and geometric consistency of the scene are enhanced by coordinating the scene information independently encoded in adjacent voxel blocks, so as to achieve scene adaptation, improve scene expression performance, ensure modeling accuracy, and significantly improve the continuity of scene geometry and content rendering. The three-dimensional scene representation model obtained by this method has a more detailed representation of the three-dimensional scene, and can achieve better subsequent new perspective rendering effects in scenes with complex geometry and color textures. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1A flowchart of a method for constructing a three-dimensional scene representation model based on scene-adaptive local voxel multi-plane images provided by an embodiment of the present invention;

[0021] Figure 2 A voxel multi-plane construction diagram provided by an embodiment of the present invention;

[0022] Figure 3 A voxel fully connected network diagram with a voxel attention block provided for an embodiment of the present invention;

[0023] Figure 4 End-to-end learning of color and opacity maps provided for embodiments of the present invention; DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0025] Embodiment 1

[0026] A method for constructing a three-dimensional scene representation model based on scene-adaptive local voxel multi-plane images, such as Figure 1 As shown, including:

[0027] The three-dimensional scene to be represented is divided into a voxel grid, and a global multi-plane image MPI of each voxel is constructed; the intersection points of different light rays with each voxel MPI under each known viewing angle are determined, and the three-dimensional position coordinates of each intersection point corresponding to each light ray are encoded as a multi-dimensional position vector, and the direction is encoded as a multi-dimensional direction vector, and the multi-dimensional position vector set corresponding to the light ray, the multi-dimensional direction vector set and the color label of the pixel corresponding to the light ray in the collected scene image constitute a training sample;

[0028] A three-dimensional scene representation network is constructed, including a multi-layer MLP network for the position of each voxel, a multi-layer MLP network for the direction, and a color calculation unit; wherein the multi-layer MLP network for the position includes n layers of self-attention units respectively connected to the output ends of the first n layers of fully connected layers, which are respectively used to divide the feature vector output by the previous fully connected layer into a series of non-overlapping windows, and perform self-attention learning on each window, and splice the self-attention learning results of each window and input them into the next fully connected layer; the input of the multi-layer MLP network for the position is the multi-dimensional position vector of all intersections of each light ray on the corresponding voxel, and the output is the multi-dimensional position vector of all intersections of each light ray on the corresponding voxel The color coefficient vector set and opacity set of the intersection points are as follows: the input of the directional multi-layer MLP network is the multi-dimensional direction vector of all the intersection points of each ray on the corresponding voxel, and the output is the basis function set of all the intersection points of each ray on the corresponding voxel; the color calculation unit is used to calculate the color set and opacity set of all the intersection points of each ray on the corresponding voxel based on the basis function set and color coefficient vector set of all the intersection points of each ray on all the voxels received, combined with the basic color vector to be optimized, so as to perform opacity synthesis on all the intersection points of each ray with all the voxels, and obtain the pixel rendering color corresponding to the ray;

[0029] The training samples are used to optimize the position multi-layer MLP network, orientation multi-layer MLP network and basic color vector of each voxel until the preset termination condition is reached to obtain a three-dimensional scene representation model.

[0030] Considering that the existing multi-plane learning method using fixed depth and global learning cannot capture the local geometric features and texture information of the scene in detail, especially in some scenes with complex textures and fine structures, the global multi-plane learning method will cause texture aliasing or missing when rendering the scene. The existing three-dimensional scene expression method based on local planes constructs local planes through point clouds, which can effectively capture geometric shapes and textures in simple scenes. However, this method ignores the correlation between cameras, resulting in the plane structure (position and normal vector) being too flexible. When using a large global MLP network to learn the texture and geometric features of the local plane, it is difficult to accurately learn the local plane (color, opacity) used for local texture rendering, resulting in holes or repetitions in the texture of the new perspective image rendered based on the local plane. In addition, it relies on the global MLP network learning method to fit the local plane without independent modeling, making it still a challenge to accurately express local information in complex scenes.

[0031] This embodiment proposes a voxel-based multi-plane 3D scene representation method. In scenes with complex geometry and color textures, this method uses a reference camera to construct a local MPI at the voxel level (that is, to construct an MPI for each voxel), retaining the simplicity and perspective consistency of the MPI structure, while finely modeling each local MPI through independently encoded voxel MLPs (implicit neural radiance fields). In order to solve the block effect existing in the independent modeling of each voxel, a voxel plane attention mechanism is introduced in the MLP network of each voxel, which enhances the content and geometric consistency of the scene by coordinating the scene information independently encoded in adjacent voxel blocks.

[0032] The above method first constructs a local voxel multi-plane image MPI. Specifically, Figure 2 As shown, first, the coordinates of the multi-plane image Define an axis-aligned bounding box (AABB), denoted as bd min :(x min ,y min d min ), bd max :(x max ,y max , d max ), where bd min , bd max The entire scene is then subdivided into blocks with a resolution of M = M x ·M y ·M d Assume that each voxel V i，j，k Located at a specific position determined by its index in the 3D grid, where i, j, k correspond to the index (number) of the voxel in the direction. Inside each voxel, there is a global multi-plane image corresponding to its position, which is a local multi-plane image of the entire 3D scene.

[0033] For each specific voxel V i，j，k , where the multiplane coordinate set can be defined as P i，j,k , where one of the plane coordinates x: (x i ,y j , d k ), and is assigned to the corresponding three-dimensional voxels through voxel identification, as follows:

[0034] V(x)=[(x-bd min ) / ((bd max -bd min ) / M)]

[0035] Here V(x)∈V i，j,k, each voxel corresponds to two voxel MLP (multi-layer perceptron) networks containing two independent sets of parameters, fitting the color and opacity of multiple planes within the voxel.

[0036] The second involves voxel-based fully connected networks and attention mechanisms. Unlike the large global MLP network outlined in NeX, the voxel-based MLP uses a small fully connected network that is learned independently within each voxel, such as Figure 3 Specifically, consider a point pixel x in a plane within a voxel: (x i ,y j , d k ), preferably, according to the formula: γ L (u)=[sin(u), cos(u),..., sin(2 L-1 u,2 L-1 u), cos(2 L-1 x, 2 L-1 u)], where u represents the three-dimensional position coordinates or direction information of a point in a plane, and L represents the encoding dimension. Preferably, a sine function with a frequency length of 20 is used to convert a position (x i ,y j ) is encoded into 40 dimensions. Similarly, a sine function with a frequency length of 8 is used to encode the depth d k Encoded into 16 dimensions, the concatenated vector of the 40-dimensional and 16-dimensional vector of a point is used as the multidimensional position vector E. A sine function with a frequency length of 4 is used to convert the plane direction Converted to 8 dimensions, as the multidimensional direction vector dir of a point.

[0037] Each multidimensional position vector E is input into the position multilayer MLP network F of the voxel where the multidimensional position vector E corresponds to, and is converted into a D-dimensional point embedding T through the first fully connected layer in the position multilayer MLP network. i，j，k (Eigenvector of each local MPI), as in the following formula Add the bias vector T i，j，k =W i，j，k E i，j，k +b i，j，k .

[0038] In order to promote the collaborative learning of local multi-planar representations across different voxel blocks, a voxel attention block is designed. The above D-dimensional embedding is input into the voxel attention block, where the attention network adopts a lightweight self-attention module: MobileAttention V2 to effectively balance model complexity and performance. This embedding T i，j，k is divided into a series of non-overlapping small windows: T i，j，k →{(Ti，j，k )1,(T i，j，k )2,...,(T i，j，k ) W}, each window contains L points. Self-attention is then applied independently within each window. The self-attention outputs of these windows are combined using a multi-head self-attention mechanism, expressed as:

[0039] As a preferred implementation, there are 1 to 2 voxel attention layers, and after stacking learning of voxel fully connected layers, the last layer of the fully connected network outputs color coefficients and opacity.

[0040] In order to better learn the color variation of the local voxel global multi-plane image with the change of viewing angle, the plane p in the voxel global multi-plane MPI is transformed into l Color Parameterized as the viewing direction dir = (dir x ,dir y ) basis function. The pixel color is determined by the basic color vector b(x i ,y j , d k ), and the basis functions related to the light direction The linear combination is expressed as: p l represents the lth local plane in voxel (i, j, k), and N represents the order (dimension).

[0041] Each voxel involves two multi-layer MLP networks: the position multi-layer MLP network F and the direction multi-layer MLP network G. Among them, the position multi-layer MLP network F concatenates the multi-dimensional position vector corresponding to a point and its corresponding embedding vector and maps it to opacity and color coefficients It is expressed as:

[0042] At the same time, another multi-layer MLP network, the directional multi-layer MLP network G, is specifically used to understand the basis function based on the observation direction dir, expressed as:

[0043]

[0044] Here, Representing voxels The pth l The transparency value of a global multi-plane image, b(x i ,y j , d k ), and denote the pthl The base color vector set, color coefficient set and basis function set of a global multi-plane image. F and G are two independently trained multi-layer MLP networks for each voxel (i, j, k). It is worth noting that the scale of the F network (i.e., the position multi-layer MLP network F) is much smaller than that of the existing MPI-based global learning method NeX, and can be used as a preferred method. It can have only 64 hidden neurons, balancing complexity and accuracy. In this case, its network parameters are only 1 / 36 of the F network in nex, while the scale of the G network is very close to that of nex, maintaining the same as ne x The same number of layers as the G network.

[0045] like Figure 4 As shown, the color calculation unit is used to calculate the color set and opacity set corresponding to all intersections based on the basis function set and color coefficient vector set of all intersections of each received light ray and each voxel MPI, combined with the basic color vector to be optimized, so as to perform opacity synthesis on all the intersections to obtain the pixel rendering color corresponding to the light ray.

[0046] Specifically, in order to optimize the base color, the weights of the MLP and attention block, Voxel-MPI is rendered into the target view and the loss value for backpropagation is calculated. Specifically, this embodiment uses a variant of the Axis Aligned Bounding Box (AABB) algorithm to find the exact intersection of the line of sight with the local multiplane of the voxel. Then using bilinear interpolation and inference MLP, the color of the target pixel is calculated by alpha synthesis in the following formula:

[0047]

[0048] Among them, N L is the number of planes that the ray intersects with Voxel-MPI, and Respectively represent the pth l The color image and transparency in the sub-plane. Calculate the loss value between the rendered result and the real result. Optimize the weights and base colors of the two multi-layer Voxel MLP by minimizing the following loss function: in, and I t They are rendered images and real images. represents the gradient operator, TV(b) represents the total variational error of the base color image, and ω and γ are balancing weights.

[0049] Compared with existing methods, the results of this embodiment show that the voxel multi-planar method surpasses the existing multi-planar method and the Nerf method in multiple data sets, and compared with the benchmark Nex, the voxel multi-planar method produces better results while requiring less memory and training time.

[0050] Embodiment 2

[0051] A method for synthesizing a new perspective of a three-dimensional scene uses the three-dimensional scene representation model constructed by the three-dimensional scene representation model construction method as described above to perform color rendering of pixels corresponding to multiple light rays under a new perspective, thereby synthesizing a new perspective image.

[0052] The relevant technical solution is the same as that in Embodiment 1 and will not be described in detail here.

[0053] Embodiment 3

[0054] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, it implements the three-dimensional scene representation model construction method described in the first embodiment and / or the three-dimensional scene new perspective synthesis method described in the second embodiment.

[0055] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The memory may be used to store computer programs and / or modules, and the processor may perform various functions of the electronic device by running or executing computer programs and / or modules stored in the memory, and calling data stored in the memory.

[0056] The relevant technical solution is the same as the above embodiment and will not be described again here.

[0057] Embodiment 4

[0058] A computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the three-dimensional scene representation model construction method described in the first embodiment and / or the three-dimensional scene new perspective synthesis method described in the second embodiment.

[0059] Specifically, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, an internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0060] The relevant technical solution is the same as the above embodiment and will not be described again here.

[0061] Embodiment 5

[0062] A computer program product includes a computer program or instructions, which, when executed by a processor, implements the three-dimensional scene representation model construction method described in the first embodiment and / or the three-dimensional scene new perspective synthesis method described in the second embodiment.

[0063] The relevant technical solution is the same as the above embodiment and will not be described again here.

[0064] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a three-dimensional scene representation model based on scene-adaptive local voxel multi-plane images, characterized in that: include: The three-dimensional scene to be represented is divided into voxel grids, and a global multi-plane image MPI of each voxel is constructed; Determine the intersection points of different light rays with each voxel MPI under each known viewing angle, encode the three-dimensional position coordinates of each intersection point corresponding to each light ray into a multi-dimensional position vector, and encode the direction into a multi-dimensional direction vector. The multi-dimensional position vector set and the multi-dimensional direction vector set corresponding to the light ray and the color label of the pixel corresponding to the light ray in the acquired scene image constitute a training sample; Constructing a multi-layer MLP network for each voxel, a multi-layer MLP network for direction, and a color calculation unit; the multi-layer MLP network for position includes n layers of self-attention units respectively connected to the output ends of the first n layers of fully connected layers, which are respectively used to divide the feature vector output by the previous fully connected layer into multiple non-overlapping windows, and perform self-attention learning on each window, and splice the learning results of each window and input them into the next fully connected layer; the input of the multi-layer MLP network for position is the multi-dimensional position vector of all intersections of each light on the corresponding voxel of the network, and the output is the color coefficient vector and opacity corresponding to all intersections; the input of the multi-layer MLP network for direction is the multi-dimensional direction vector of all intersections of each light on the corresponding voxel of the network, and the output is the basis function of all intersections; the color calculation unit is used to calculate the color of all intersections based on the basis function and color coefficient vector of all intersections of each light received on all voxels, combined with the basic color vector to be optimized, so as to synthesize the opacity of all intersections to obtain the pixel rendering color corresponding to the light; The training samples are used to optimize the position multi-layer MLP network, orientation multi-layer MLP network and basic color vector of each voxel until the preset termination condition is reached to obtain a three-dimensional scene representation model.

2. The method for constructing a three-dimensional scene representation model according to claim 1, characterized in that: The m-dimensional vector is encoded by using a sine function with a frequency length of m to obtain the multi-dimensional position vector and the multi-dimensional direction vector.

3. The method for constructing a three-dimensional scene representation model according to claim 2, characterized in that: The encoding method of the multidimensional position vector is: using a sine function with a frequency length of 20 to encode the plane position coordinates of each intersection corresponding to each light ray into a 40-dimensional vector; using a sine function with a frequency length of 8 to encode the depth coordinates of each intersection corresponding to the light ray into a 16-dimensional vector; splicing the 40-dimensional vector and the 16-dimensional vector to obtain the multidimensional position vector of the corresponding intersection; The encoding method of the multi-dimensional direction vector is: using a sine function with a frequency length of 4 to encode the direction of each intersection point corresponding to each light ray into an 8-dimensional vector, so as to obtain the multi-dimensional direction vector of the intersection point.

4. The method for constructing a three-dimensional scene representation model according to claim 1, characterized in that: The value of n is 2.

5. The method for constructing a three-dimensional scene representation model according to claim 1, characterized in that: The calculation of the color and the opacity is performed using a preset linear relationship between a basis function set, a color coefficient vector set and a basic color vector.

6. A method for synthesizing a new perspective of a three-dimensional scene, characterized in that: The three-dimensional scene representation model constructed by the three-dimensional scene representation model construction method according to any one of claims 1 to 5 is used to perform color rendering of pixels corresponding to multiple light rays under a new perspective, thereby synthesizing a new perspective image.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method for constructing a three-dimensional scene representation model according to any one of claims 1 to 5 and / or the method for synthesizing a new perspective of a three-dimensional scene according to claim 6 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the three-dimensional scene representation model construction method as described in any one of claims 1 to 5 and / or the three-dimensional scene new perspective synthesis method as described in claim 6.

9. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the method for constructing a three-dimensional scene representation model as described in any one of claims 1 to 5 and / or the method for synthesizing a new perspective of a three-dimensional scene as described in claim 6 are implemented.

Images