Indoor three-dimensional object reconstruction method and device, computer equipment and storage medium
Through the improved NeRF model and monocular camera technology, the problems of insufficient research and excessive sampling times in monocular cameras in three-dimensional reconstruction are solved, and efficient three-dimensional object reconstruction is achieved, which expands the scope of application and reduces resource requirements.
Patent Information
- Application Number
- CN202310527082.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-10
- Publication Date
- 2024-11-12
AI Technical Summary
In the existing three-dimensional reconstruction technology, the SFM algorithm based on multiple images taken at different angles has two-dimensional local feature matching errors, the MVS algorithm lacks inference ability and is complex in operation, and the NeRF method mainly relies on binocular or RGB-D cameras and monocular cameras. There are few studies and the sampling scheme increases the three-dimensional reconstruction time.
A monocular camera is used to collect indoor scene images from multiple different perspectives. Through the improved NeRF model, the volume density and RGB color are output using the MLP network. The combined rendering algorithm generates three-dimensional reconstruction images, reduces the number of samples, and optimizes the network to improve reconstruction efficiency.
It realizes efficient three-dimensional object reconstruction under monocular camera conditions, expands the scope of application of NeRF, reduces the storage space and computing resource requirements, and avoids the complex process of traditional manual modeling.
Smart Images

Figure CN118941699A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of three-dimensional reconstruction technology, and in particular, relates to a method, device, computer equipment and storage medium for indoor three-dimensional object reconstruction. Background Art
[0002] 3D reconstruction refers to the establishment of a mathematical model suitable for computer representation and processing of a 3D object. It is the basis for processing, operating and analyzing its properties in a computer environment. It is also a key technology for establishing a virtual reality that expresses the objective world in a computer. After more than ten years of development, 3D reconstruction has made great achievements. Compared with 2D images, 3D images contain more information and can more intuitively reflect the real world. Therefore, how to use computers to restore the 3D information of the target and establish a 3D model of the real scene has become an indispensable part before starting in-depth research in many fields. For example, in medical imaging, 3D CT images are used to judge the disease, and in cultural relics protection, cultural relics are restored in 3D.
[0003] At present, researchers mainly use the following three methods to obtain the three-dimensional model of indoor objects:
[0004] 1. Traditional geometric modeling technology: Geometric modeling technology is the most mature. It can be used to obtain very accurate three-dimensional models, but it is time-consuming and labor-intensive, and there are still differences from the real scene.
[0005] Second, based on multiple images taken from different angles, three-dimensional modeling is performed using computer vision theory; representative methods of this technology include the SFM (Structure from motion) and MVS (Multi-viewstereo) algorithms. Among them, when using the SFM algorithm to reconstruct the shape of indoor three-dimensional objects, the matching of two-dimensional local features in actual situations often results in serious errors due to the small overlap between different views, missing object textures, or light reflections on objects; and the MVS algorithm, due to its lack of inference capabilities, cannot solve the problem of three-dimensional structure completion when there is less input information. At the same time, this method also has problems such as long cycle and complex operation. For many irregular natural or man-made objects, its modeling effect is still different from the real scene.
[0006] 3. Scan the target using a 3D scanning device, and then reconstruct the 3D model of the target. This method is a 3D reconstruction based on point cloud. Currently, NeRF (Neural Radiance Fields) is an emerging 3D reconstruction method. It uses a neural network model to represent the radiation field of a 3D scene, thereby realizing the reconstruction of a 3D scene. NeRF can use 2D posed images as supervision to represent complex 3D scenes. At present, most 3D modeling methods based on NeRF use binocular cameras or RGB-D cameras to capture images, and then use the known depth information after shooting to model indoor 3D objects. However, there are few studies on monocular cameras, which limits the scope of use of the NeRF method. In addition, the current NeRF method uses a coarse and fine dual network cascade sampling scheme to fit the spatial light field. In this sampling scheme, it is equivalent to performing 2 uniform samplings and one non-uniform sampling, which will increase the time consumed by 3D reconstruction to a certain extent. Summary of the invention
[0007] The present application provides a method, apparatus, computer device and storage medium for reconstructing indoor three-dimensional objects, aiming to solve at least one of the above-mentioned technical problems in the prior art to a certain extent.
[0008] In order to solve the above problems, this application provides the following technical solutions:
[0009] A method for indoor three-dimensional object reconstruction, comprising:
[0010] Use a monocular camera to collect images of indoor scenes from multiple different perspectives;
[0011] Input the indoor scene image into the trained neural radiation field model, the neural radiation field model outputs the volume density and RGB color of each indoor scene image through the MLP network, and uses the volume rendering algorithm to render the scene image at each viewing angle according to the volume density and RGB color;
[0012] The scene images from multiple different perspectives are fused to generate a 3D reconstructed image of the indoor scene.
[0013] The technical solution adopted in the embodiment of the present application also includes: using a monocular camera to collect indoor scene images from multiple different perspectives also includes: recording the position and direction of each indoor scene image.
[0014] The technical solution adopted by the embodiment of the present application also includes: after using the monocular camera to collect indoor scene images under multiple different perspectives, it also includes:
[0015] The indoor scene image is preprocessed to obtain a corrected indoor scene image.
[0016] The technical solution adopted by the embodiment of the present application also includes: the preprocessing of the indoor scene image is specifically:
[0017] Obtaining an intrinsic parameter matrix and an extrinsic parameter matrix of the monocular camera;
[0018] The indoor scene image is subjected to distortion removal processing according to the intrinsic parameter matrix and the extrinsic parameter matrix to obtain a corrected indoor scene image.
[0019] The technical solution adopted by the embodiment of the present application also includes: before inputting the indoor scene image into the trained neural radiation field model, it also includes:
[0020] The corrected indoor scene image is input into the improved NeRF for training. The improved NeRF outputs the volume density and RGB color of each pixel in the indoor scene image through the MLP network. Hierarchical pixel sampling of the coarse network and the fine network is performed according to the volume density and RGB color. Three-dimensional reconstruction of the indoor scene image is performed according to the hierarchical pixel sampling result. The improved NeRF is optimized according to the three-dimensional reconstruction result to obtain a trained neural radiation field model.
[0021] The technical solution adopted by the embodiment of the present application also includes: the training process of the improved NeRF is specifically as follows:
[0022] For each indoor scene image r, it is processed into a 5D vector input N c (x i c ,d), the 5D vector input includes the 3D position coordinates x and the viewing direction d of each indoor scene image in the Cartesian coordinate system; the 5D vector N is c (x i c ,d) Perform uniform sampling, the sampling interval of uniform sampling is Δt, obtain the distribution of effective sampling positions, fit the probability density function of effective sampling positions, and output the uniform sampling result N c (c i c ,d);
[0023] Based on the probability density function, the 5D vector N is quantized at the position with high probability through a delicate network. c (x i c ,d) Perform non-uniform sampling and output the non-uniform sampling results in, represents the volume density;
[0024] The uniform sampling result N of the rough network c (ci c ,d) Non-uniform sampling results with delicate network output Mix and get the final RGB color output result C f (r).
[0025] The technical solution adopted by the embodiment of the present application also includes: the optimization of the improved NeRF according to the three-dimensional reconstruction result is specifically:
[0026] The mean square error of the uniform sampling result of the coarse network and the non-uniform sampling result of the fine network is calculated at each pixel as the loss function to simultaneously optimize the coarse network and the fine network:
[0027]
[0028] in, is the RGB color sampling result of the rough network sampling the light r, C gt (r) is the true RGB color value of light r.
[0029] Another technical solution adopted by the embodiment of the present application is: a method and device for indoor three-dimensional object reconstruction, comprising:
[0030] Image acquisition module: used to acquire indoor scene images from multiple different perspectives, the image acquisition module is a monocular camera;
[0031] Image rendering module: used for inputting the indoor scene image into the trained neural radiation field model, the neural radiation field model outputs the volume density and RGB color of each indoor scene image through the MLP network, and uses the volume rendering algorithm to render the scene image at each viewing angle according to the volume density and RGB color;
[0032] Image fusion module: used to fuse scene images from multiple different perspectives to generate a three-dimensional reconstructed image of the indoor scene.
[0033] Another technical solution adopted by the embodiment of the present application is: a computer device, the computer device includes a processor and a memory coupled to the processor, wherein:
[0034] The memory stores program instructions for implementing the indoor three-dimensional object reconstruction method;
[0035] The processor is used to execute the program instructions stored in the memory to control a method for reconstructing indoor three-dimensional objects.
[0036] Another technical solution adopted by the embodiment of the present application is: a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute the indoor three-dimensional object reconstruction method.
[0037] Compared with the prior art, the beneficial effects of the embodiments of the present application are as follows: the indoor three-dimensional object reconstruction method, device, computer equipment and storage medium of the embodiments of the present application collect indoor scene images from multiple perspectives through a monocular camera, and implicitly represent the scene as a continuous 5D radiation field through an improved NeRF model, and use the MLP network to output the volume density of each indoor scene image and the RGB color related to the input image, and render pictures from different perspectives according to the volume density and RGB color, and fuse the pictures from multiple perspectives to generate a three-dimensional image of the indoor scene. The embodiments of the present application reduce the number of sampling times for fitting the spatial light field by improving the original NeRF, thereby reducing the three-dimensional reconstruction time. Secondly, the embodiments of the present application do not need to store a large amount of scene information, can avoid the complex process of manual modeling required in traditional methods, and greatly reduce storage space and computing resources. In addition, the embodiments of the present application use a monocular camera for image acquisition, which expands the scope of application of NeRF. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 is a flow chart of a method for reconstructing indoor three-dimensional objects according to an embodiment of the present application;
[0039] Figure 2 An introduction to the original NeRF (Neural Radiance Field) principle and a schematic diagram of the rendering process;
[0040] Figure 3 is a schematic diagram of the improved NeRF interpretation of model input data in the embodiment of the present application;
[0041] Figure 4 This is a schematic diagram of the improved NeRF network structure of an embodiment of the present application;
[0042] Figure 5 This is a schematic diagram of the structure of a device for indoor three-dimensional object reconstruction according to an embodiment of the present application;
[0043] Figure 6 A schematic diagram of the structure of a computer device according to an embodiment of the present application;
[0044] Figure 7 A schematic diagram of the structure of a storage medium according to an embodiment of the present application. DETAILED DESCRIPTION
[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0046] The terms "first", "second", "third" in this application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second", "third" can expressly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In the embodiments of this application, all directional indications (such as up, down, left, right, front, back...) are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or computer device that includes a series of steps or units is not limited to the steps or units listed, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or computer devices.
[0047] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0048] See also Figure 1 , is a flow chart of a method for reconstructing an indoor three-dimensional object according to an embodiment of the present application. The method for reconstructing an indoor three-dimensional object according to an embodiment of the present application comprises the following steps:
[0049] S100: Use a monocular camera to collect indoor scene images from multiple perspectives and record the position and direction of each indoor scene image;
[0050] S110: preprocessing the indoor scene image to obtain a corrected indoor scene image;
[0051] In this step, more accurate three-dimensional information can be obtained by preprocessing the indoor scene image. Preprocessing the indoor scene image specifically includes:
[0052] S111: Obtaining the intrinsic parameter matrix and extrinsic parameter matrix of the monocular camera;
[0053] Among them, the intrinsic parameter matrix is the parameters such as focal length and pixel size related to the camera's own characteristics; the extrinsic parameter matrix is the parameters such as the position and rotation direction of the camera in the world coordinate system.
[0054] S112: performing distortion removal processing on the indoor scene image according to the intrinsic parameter matrix and the extrinsic parameter matrix to obtain a corrected indoor scene image.
[0055] S120: input the corrected indoor scene image into the improved NeRF for training, and the NeRF outputs the volume density and RGB color of each pixel in the indoor scene image through the MLP network, performs hierarchical pixel sampling of the coarse network and the fine network according to the volume density and RGB color, performs three-dimensional reconstruction of the indoor scene according to the hierarchical pixel sampling results, and optimizes the NeRF according to the three-dimensional reconstruction results to obtain a trained neural radiation field model;
[0056] In this step, please also refer to Figure 2 , which is an introduction to the original NeRF (Neural Radiance Field) principle and a schematic diagram of the rendering process. Figure 2 As shown in the multi-level sampling parts of (c) and (d), since the rendering process of NeRF is very computationally intensive, each ray (the line connecting the focus and the pixel) needs to sample many points, but in fact, most of the areas on each ray are empty areas or occluded areas, which do not contribute to the final 3D reconstructed color. The original NeRF will repeatedly sample these empty areas or occluded areas, thereby reducing the efficiency of 3D reconstruction.
[0057] Based on this, the embodiment of the present application adopts the improved NeRF to represent the scene, and the input data of the improved NeRF is a function of a 5D vector, as shown in the following example: Figure 3 As shown, it is a schematic diagram of the improved NeRF interpretation of the model input data in the embodiment of the present application. The 5D vector includes the 3D position coordinates x = (x, y, z) of each indoor scene image in the Cartesian coordinate system and the viewing direction Where θ represents the angle between the straight line from point p to the origin o and the z-axis. Represents the angle between the straight line oq formed by the projection point q and point o from point p to the xoy plane and the x-axis. A fully connected MLP neural network is used to implicitly learn a static three-dimensional scene. During the model training process, the 3D position coordinates and viewing direction of the corrected indoor scene image are input into the improved NeRF model. The model outputs the volume density σ of each indoor scene image and the RGB color related to the input image. Then, the volume rendering algorithm is used to project the output volume density σ and RGB color into the image, and images from different perspectives are rendered. The images from multiple perspectives are fused to generate a three-dimensional image of the indoor scene image. At the same time, the network parameters are adjusted by optimizing the loss function so that the scene output by the model can be as close to the real scene as possible to obtain the optimized neural radiation field model.
[0058] Specifically, see Figure 4 , is a schematic diagram of the improved NeRF network structure of the embodiment of the present application. For each indoor scene image r, it is first processed into a 5D vector input N c (x i c ,d), d is the direction of all sampling positions; then the 5D vector N is c (x i c ,d) Perform uniform sampling, the sampling interval of uniform sampling is Δt, obtain the distribution of effective sampling positions, fit the probability density function of effective sampling positions, and output the uniform sampling result N c (c i c ,d); Based on the probability density function, the 5D vector N is centered at the position with high probability through a delicate network c (x i c ,d) Perform denser non-uniform sampling and output the non-uniform sampling results in, After the two samplings are completed, the uniform sampling result of the coarse network is mixed with the non-uniform sampling result output by the fine network to obtain the final RGB color output result c f (r), thereby reducing the number of sampling times of the original NeRF from three to two, speeding up the 3D reconstruction of NeRF. Specifically, the improved NeRF sampling derivation formula is as follows:
[0059]
[0060] In formula (1), sort is the sorting function, x k represents the kth sampling position, x ic represents the sampling position under uniform sampling, Represents non-uniform sampling position, each x k Corresponding to their respective sampling results, the two sampling results are then merged to calculate the final RGB color:
[0061]
[0062]
[0063] In the above formula, x k -x k-1 is the distance between the kth sampling position and the previous sampling position, and the value of x0 is 0. k For x k The volume density at the location, c k For x k The sampling results of the location.
[0064] Finally, the mean square error of the two sampling results of the coarse network and the fine network is calculated at each pixel as the loss function to simultaneously optimize the coarse network and the fine network:
[0065]
[0066] in, is the RGB color sampling result of the rough network sampling the light r, C gt (r) is the true RGB color value of light r.
[0067] S130: inputting the indoor scene image to be reconstructed into the trained neural radiation field model, and outputting a three-dimensional reconstructed image of the indoor scene through the neural radiation field model;
[0068] In this step, after obtaining the trained neural radiation field model, the 3D model can be reconstructed through the model. Specifically, for a new perspective, the volume density and RGB color of each pixel at that perspective can be predicted by the neural radiation field model, and the scene image at each perspective can be rendered based on the volume density and RGB color using the volume rendering algorithm; finally, the scene images at multiple perspectives are fused to obtain a complete 3D reconstructed image of the indoor scene.
[0069] Based on the above, the indoor three-dimensional object reconstruction method of the second embodiment of the present application collects indoor scene images from multiple different perspectives through a monocular camera, and implicitly represents the scene as a continuous 5D radiation field through an improved NeRF model, and uses the MLP network to output the volume density and RGB color of each indoor scene image, and renders pictures from different perspectives according to the volume density and RGB color, and fuses the pictures from multiple perspectives to generate a three-dimensional reconstructed image of the indoor scene. The embodiment of the present application reduces the number of sampling times for fitting the spatial light field by improving the original NeRF, thereby reducing the three-dimensional reconstruction time. Secondly, the embodiment of the present application does not need to store a large amount of scene information, can avoid the complex process of manual modeling required in traditional methods, and greatly reduces storage space and computing resources. In addition, the embodiment of the present application uses a monocular camera for image acquisition, which expands the scope of application of NeRF.
[0070] See also Figure 5 , is a schematic diagram of the structure of an indoor three-dimensional object reconstruction method device according to an embodiment of the present application. The indoor three-dimensional object reconstruction method device 40 according to an embodiment of the present application comprises:
[0071] Image acquisition module 41: used to acquire indoor scene images from multiple different viewing angles, and the image acquisition module is a monocular camera;
[0072] Image rendering module 42: used for inputting the indoor scene image into the trained neural radiation field model, the neural radiation field model outputs the volume density and RGB color of each indoor scene image through the MLP network, and uses the volume rendering algorithm to render the scene image at each viewing angle according to the volume density and RGB color;
[0073] Image fusion module 43: used to fuse scene images from multiple different perspectives to generate a three-dimensional reconstructed image of the indoor scene.
[0074] See also Figure 6 , is a schematic diagram of the computer device structure of an embodiment of the present application. The computer device 50 includes:
[0075] A memory 51 storing executable program instructions;
[0076] A processor 52 connected to the memory 51;
[0077] The processor 52 is used to call the executable program instructions stored in the memory 51 and perform the following steps: use a monocular camera to collect indoor scene images from multiple different perspectives; input the indoor scene images into a trained neural radiation field model, the neural radiation field model outputs the volume density and RGB color of each indoor scene image through the MLP network, and uses a volume rendering algorithm to render the scene picture from each perspective according to the volume density and RGB color; fuse the scene pictures from multiple different perspectives to generate a three-dimensional reconstructed image of the indoor scene.
[0078] The processor 52 may also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip having signal processing capabilities. The processor 52 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0079] See also Figure 7, is a schematic diagram of the structure of the storage medium of the embodiment of the present application. The storage medium of the embodiment of the present application stores a program instruction 61 that can implement the following steps: using a monocular camera to collect indoor scene images under multiple different viewing angles; inputting the indoor scene images into a trained neural radiation field model, the neural radiation field model outputs the volume density and RGB color of each indoor scene image through an MLP network, and uses a volume rendering algorithm to render the scene picture under each viewing angle according to the volume density and RGB color; fuses the scene pictures under multiple different viewing angles to generate a three-dimensional reconstructed image of the indoor scene. Among them, the program instruction 61 can be stored in the above-mentioned storage medium in the form of a software product, including several instructions for a computer device (which can be a personal computer, a server, or a network computer device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: various media that can store program instructions, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, or a terminal computer device such as a computer, a server, a mobile phone, and a tablet. Among them, the server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.
[0080] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0081] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the description and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present application.
Claims
1. A method for indoor three-dimensional object reconstruction, characterized in that: include: Use a monocular camera to collect images of indoor scenes from multiple different perspectives; Input the indoor scene image into the trained neural radiation field model, the neural radiation field model outputs the volume density and RGB color of each indoor scene image through the MLP network, and uses the volume rendering algorithm to render the scene image at each viewing angle according to the volume density and RGB color; The scene images from multiple different perspectives are fused to generate a 3D reconstructed image of the indoor scene.
2. The indoor three-dimensional object reconstruction method according to claim 1, characterized in that: The method of using a monocular camera to collect indoor scene images at multiple different viewing angles also includes: recording the position and direction of each indoor scene image.
3. The indoor three-dimensional object reconstruction method according to claim 2, characterized in that: After the monocular camera is used to collect indoor scene images at multiple different viewing angles, the method further includes: The indoor scene image is preprocessed to obtain a corrected indoor scene image.
4. The indoor three-dimensional object reconstruction method according to claim 3, characterized in that: The preprocessing of the indoor scene image is specifically as follows: Obtaining an intrinsic parameter matrix and an extrinsic parameter matrix of the monocular camera; The indoor scene image is subjected to distortion removal processing according to the intrinsic parameter matrix and the extrinsic parameter matrix to obtain a corrected indoor scene image.
5. The indoor three-dimensional object reconstruction method according to claim 3 or 4, characterized in that: Before inputting the indoor scene image into the trained neural radiation field model, the method further includes: The corrected indoor scene image is input into the improved NeRF for training. The improved NeRF outputs the volume density and RGB color of each pixel in the indoor scene image through the MLP network. Hierarchical pixel sampling of the coarse network and the fine network is performed according to the volume density and RGB color. Three-dimensional reconstruction of the indoor scene image is performed according to the hierarchical pixel sampling result. The improved NeRF is optimized according to the three-dimensional reconstruction result to obtain a trained neural radiation field model.
6. The indoor three-dimensional object reconstruction method according to claim 5, characterized in that: The training process of the improved NeRF is specifically as follows: For each indoor scene image r, it is processed into a 5D vector input N c (x i c ,d), the 5D vector input includes the 3D position coordinates x and the viewing direction d of each indoor scene image in the Cartesian coordinate system; the 5D vector N is c (x i c ,d) Perform uniform sampling, the sampling interval of uniform sampling is Δt, obtain the distribution of effective sampling positions, fit the probability density function of effective sampling positions, and output the uniform sampling result N c (c i c ,d); Based on the probability density function, the 5D vector N is quantized at the position with high probability through a delicate network. c (x i c ,d) Perform non-uniform sampling and output the non-uniform sampling results in, represents the volume density; The uniform sampling result N of the rough network c (c i c ,d) Non-uniform sampling results with delicate network output Mix and get the final RGB color output result C f (r).
7. The indoor three-dimensional object reconstruction method according to claim 5 or 6, characterized in that: The optimization of the improved NeRF according to the three-dimensional reconstruction result is specifically as follows: The mean square error of the uniform sampling result of the coarse network and the non-uniform sampling result of the fine network is calculated at each pixel as the loss function to simultaneously optimize the coarse network and the fine network: in, is the RGB color sampling result of the rough network sampling the light r, C gt (r) is the true RGB color value of light r.
8. A method and device for indoor three-dimensional object reconstruction, characterized in that: include: Image acquisition module: used to acquire indoor scene images from multiple different viewing angles, the image acquisition module is a monocular camera; Image rendering module: used for inputting the indoor scene image into the trained neural radiation field model, the neural radiation field model outputs the volume density and RGB color of each indoor scene image through the MLP network, and uses the volume rendering algorithm to render the scene image at each viewing angle according to the volume density and RGB color; Image fusion module: used to fuse scene images from multiple different perspectives to generate a three-dimensional reconstructed image of the indoor scene.
9. A computer device, characterized in that: The computer device includes a processor and a memory coupled to the processor, wherein: The memory stores program instructions for implementing the indoor three-dimensional object reconstruction method according to any one of claims 1 to 7; The processor is used to execute the program instructions stored in the memory to control a method for reconstructing indoor three-dimensional objects.
10. A storage medium, characterized in that: The device stores program instructions executable by a processor, wherein the program instructions are used to execute the indoor three-dimensional object reconstruction method according to any one of claims 1 to 7.
Citation Information
Cited By
Method and device for reconstructing scene grabbed by robot and electronic equipment
CN119445029A
Multi-view-angle-based three-dimensional model generation method and system
CN120599157A
Three-dimensional indoor scene generation method and related equipment
CN121746643A
A three-dimensional indoor scene generation method and related device
CN121746643B
