3D reconstruction of large models: construction methods, controllers, systems, media, and products.

Abstract

This application proposes a method, controller, system, medium, and product for constructing a large-scale 3D reconstruction model. The method includes: acquiring training images; rendering the training images based on mesh rasterization to obtain a photometric stereo image; inputting the photometric stereo image into a preset explicit network to obtain a three-dimensional representation of the photometric stereo image; rendering the three-dimensional representation to obtain various rendered images, and predicting the albedo of the three-dimensional representation to obtain an albedo map; calculating the loss based on various rendered images, the albedo map, and the photometric stereo image using a preset loss function to obtain a loss value; and optimizing the explicit network based on the loss value to obtain a constructed large-scale 3D reconstruction model. Therefore, this application calculates the loss using various rendered images, the albedo map, and the photometric stereo image, and then optimizes the explicit network based on the loss value to obtain a constructed large-scale 3D reconstruction model, thereby improving the accuracy and quality of the 3D reconstruction model construction.

Description

Technical Field

[0001] This application relates to the field of 3D modeling technology, and in particular to a method, controller, system, medium and product for constructing large 3D reconstruction models. Background Technology

[0002] In related technologies, the methods for constructing image-derived 3D models are mainly divided into three categories. The first category relies on the prior knowledge of 2D diffusion models. By generating images of multiple views and reconstructing based on these multiple views, the conversion from images to 3D models is achieved. This method makes full use of the advantages of 2D diffusion models in image generation, but the generated multiple views may have problems in terms of consistency, which may lead to uneven surfaces or other defects in the final reconstructed 3D model.

[0003] The second type of method is based on the technique of generating large 3D models. It relies on the training of geometric variational autoregressive models and latent space diffusion models to generate high-quality 3D models. However, its effectiveness depends to a large extent on the number of available 3D models and the accuracy of the labels. In addition, the second stage of this method consumes a lot of resources, and the generated 3D model may not be completely consistent with the given image input. It also faces optimization challenges.

[0004] The third type of method is forward 3D reconstruction of large models. It uses a pre-trained model to project images into 3D space and optimizes the entire framework through the rendering loss of 2D images to achieve a 3D reconstruction model that is highly consistent with the input image. It only requires 2D images as supervision information. However, this method often performs poorly in terms of local details and is sensitive to the complex appearance of the input image, thus affecting the accuracy and realism of the reconstruction model.

[0005] Therefore, how to construct large-scale 3D reconstruction models is currently a major problem facing the industry. Summary of the Invention

[0006] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a method, controller, system, medium, and product for constructing large 3D reconstruction models, aiming to improve the accuracy and quality of large 3D reconstruction model construction.

[0007] In a first aspect, embodiments of this application provide a method for constructing a large 3D reconstruction model, the method comprising:

[0008] Acquire training images, and render the training images based on grid rasterization to obtain a photometric stereo image;

[0009] The photometric stereo image is input into a preset explicit network to obtain a three-dimensional representation of the photometric stereo image;

[0010] The three-dimensional representation is rendered to obtain various rendered images, and the albedo of the three-dimensional representation is predicted to obtain an albedo map.

[0011] The loss value is obtained by calculating the loss based on the various rendered images, the albedo map, and the photometric stereo image using a preset loss function;

[0012] The explicit network is optimized based on the loss value to obtain the constructed 3D reconstruction model.

[0013] According to some embodiments of this application, the step of rendering the training image based on grid rasterization to obtain a photometric stereo image includes:

[0014] Collect multiple light maps and obtain lighting information based on the light maps;

[0015] Obtain the material properties of a preset network model, and rasterize the network model to obtain the rasterization information of the network model. The material properties include albedo, metallicity, and roughness.

[0016] The diffuse albedo and specular albedo are calculated based on the albedo and the metallicity.

[0017] Define the sampling range for the metallicity and the roughness;

[0018] Based on the illumination information, the rasterization information, the diffuse albedo, the specular albedo, and the sampling range, the training image is subjected to diffuse and specular reflection to obtain the photometric stereo image from multiple perspectives.

[0019] According to some embodiments of this application, the explicit network includes an image encoder, and the step of inputting the photometric stereo image into a preset explicit network to obtain a three-dimensional representation of the features of the photometric stereo image includes:

[0020] The photometric stereo image is input into an image encoder in a preset explicit network for encoding to obtain the first image features;

[0021] A three-dimensional representation of the photometric stereo image features is obtained based on the features of the first image.

[0022] According to some embodiments of this application, the explicit network further includes a three-plane decoder, and the step of obtaining a three-dimensional representation of the photometric stereo image features based on the first image features includes:

[0023] The first image features are input into the three-plane decoder in the explicit network for decoding to obtain a three-dimensional representation of the first image features.

[0024] According to some embodiments of this application, various rendered images include specular maps, diffuse maps, depth maps, and normal maps.

[0025] According to some embodiments of this application, the loss function is:

[0026]

[0027] in, For loss function, Let C be the mean squared error loss function, and C be the true photometric stereo image. For the photometric stereo image, λ LPIPS As the first coefficient, To learn the perceptual image patch similarity loss function, where 'a' is the true albedo map, For the albedo map, L * This provides a true specular light map and a true diffuse light map. Here, n represents the specular light map and the diffuse light map, and n is the true normal map. For the normal diagram, λ normal Here, m is the second coefficient, and m is the actual mask. For the prediction mask, λ reg The third coefficient, Let λ be the geometric constraint loss function. depth The fourth coefficient is d, where d is the true depth map. For the depth map, λ mask It is the fifth coefficient.

[0028] Secondly, embodiments of this application provide a controller, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for constructing a large 3D reconstruction model as described in the first aspect when running the computer program.

[0029] Thirdly, embodiments of this application provide a 3D reconstruction system for large models, including the controller described in the second aspect above.

[0030] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions for performing a method for constructing a large 3D reconstruction model as described in the first aspect above.

[0031] Fifthly, embodiments of this application provide a computer program product, including a computer program or computer instructions, characterized in that the computer program or computer instructions are stored in a computer-readable storage medium, a processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions, causing the computer device to perform a method for constructing a large 3D reconstruction model as described in the first aspect above.

[0032] According to the technical solution of the embodiments of this application, at least the following beneficial effects are achieved: The embodiments of this application propose a method, controller, system, medium, and product for constructing a large 3D reconstruction model. The method includes: acquiring training images; rendering the training images based on mesh rasterization to obtain a photometric stereo image; inputting the photometric stereo image into a preset explicit network to obtain a three-dimensional representation of the photometric stereo image; rendering the three-dimensional representation to obtain various rendered images, and predicting the albedo of the three-dimensional representation to obtain an albedo map; calculating the loss based on various rendered images, the albedo map, and the photometric stereo image using a preset loss function to obtain a loss value; and optimizing the explicit network based on the loss value to obtain a constructed large 3D reconstruction model. Therefore, this application calculates the loss using various rendered images, the albedo map, and the photometric stereo image, and optimizes the explicit network based on the loss value to obtain a constructed large 3D reconstruction model, thereby improving the accuracy and quality of the large 3D reconstruction model construction.

[0033] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0034] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0035] Figure 1 This is a flowchart of a method for constructing a large 3D reconstruction model according to an embodiment of this application;

[0036] Figure 2 This is a flowchart of a method for constructing a large 3D reconstruction model according to another embodiment of this application;

[0037] Figure 3 This is a flowchart of a method for constructing a large 3D reconstruction model according to another embodiment of this application;

[0038] Figure 4This is an overall flowchart of a method for constructing a large 3D reconstruction model according to one embodiment of this application;

[0039] Figure 5 This is an overall flowchart of a method for constructing a large 3D reconstruction model provided in another embodiment of this application;

[0040] Figure 6 This is a schematic diagram of a 3D object provided in one embodiment of this application;

[0041] Figure 7 This is a schematic diagram of a 3D object provided in another embodiment of this application;

[0042] Figure 8 This is a schematic diagram of a 3D object provided in another embodiment of this application;

[0043] Figure 9 This is a schematic diagram of a controller for performing a method for constructing a large 3D reconstruction model, provided in one embodiment of this application. Detailed Implementation

[0044] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0045] In the description of this application, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0046] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0047] In the description of this application, unless otherwise expressly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0048] In some cases, the methods for constructing image-derived 3D models can be mainly divided into three categories. The first category relies on the prior knowledge of 2D diffusion models. By generating images of multiple views and reconstructing based on these multiple views, the conversion from image to 3D model can be achieved. This method makes full use of the advantages of 2D diffusion models in image generation, but the generated multiple views may have problems in terms of consistency, which may lead to uneven surfaces or other defects in the final reconstructed 3D model.

[0049] The second type of method is based on the technique of generating large 3D models. It relies on the training of geometric variational autoregressive models and latent space diffusion models to generate high-quality 3D models. However, its effectiveness depends to a large extent on the number of available 3D models and the accuracy of the labels. In addition, the second stage of this method consumes a lot of resources, and the generated 3D model may not be completely consistent with the given image input. It also faces optimization challenges.

[0050] The third type of method is forward 3D reconstruction of large models. It uses a pre-trained model to project images into 3D space and optimizes the entire framework through the rendering loss of 2D images to achieve a 3D reconstruction model that is highly consistent with the input image. It only requires 2D images as supervision information. However, this method often performs poorly in terms of local details and is sensitive to the complex appearance of the input image, thus affecting the accuracy and realism of the reconstruction model.

[0051] Therefore, how to construct large-scale 3D reconstruction models is currently a major problem facing the industry.

[0052] Based on the above, this application proposes a method, controller, system, medium, and product for constructing large 3D reconstruction models, aiming to improve the accuracy and quality of large 3D reconstruction model construction.

[0053] The following description, in conjunction with the accompanying drawings, further elaborates on various embodiments of a method for constructing a large 3D reconstruction model according to this application.

[0054] like Figure 1 As shown, Figure 1 This is a flowchart of a method for constructing a large 3D reconstruction model according to an embodiment of this application; the method for constructing a large 3D reconstruction model may include, but is not limited to, steps S110, S120, S130, S140 and S150.

[0055] Step S110: Obtain the training image, and render the training image based on grid rasterization to obtain a photometric stereo image;

[0056] Step S120: Input the photometric stereo image into a preset explicit network to obtain a three-dimensional representation of the photometric stereo image;

[0057] Step S130: Render the three-dimensional representation to obtain multiple rendered images, and predict the albedo of the three-dimensional representation to obtain an albedo map.

[0058] Step S140: Calculate the loss value by using a preset loss function based on various rendered images, albedo maps, and photometric stereo images;

[0059] Step S150: Optimize the explicit network based on the loss value to obtain the constructed 3D reconstruction model.

[0060] In one embodiment, firstly, training images are acquired and rendered using mesh rasterization to obtain photometric stereo images. Secondly, the photometric stereo images are input into a preset explicit network for encoding and decoding to obtain a three-dimensional representation of the photometric stereo images. Thirdly, the three-dimensional representation is rendered to obtain various rendered images, and albedo prediction is performed on the three-dimensional representation to obtain an albedo map. Then, a loss function is used to calculate the loss of various rendered images, albedo maps, and photometric stereo images to obtain loss values. Finally, the explicit network is optimized based on the loss values ​​to obtain a constructed large-scale 3D reconstruction model. Therefore, this application calculates the loss using various rendered images, albedo maps, and photometric stereo images, and then optimizes the explicit network based on the loss values ​​to obtain a constructed large-scale 3D reconstruction model, thereby improving the accuracy and quality of the 3D reconstruction model construction.

[0061] It is understood that this application uses mesh rasterization and physically based real-time rendering technology for online rendering. During the rendering process, the materials, environment maps and camera poses will change randomly, thereby rendering multi-view photometric stereoscopic images.

[0062] Understandably, the explicit network in this application supports rasterization rendering, which is faster and consumes less video memory compared to traditional voxel rendering. Secondly, the explicit network not only supports the rendering of photometric information, but also the rendering of geometric information, such as depth maps and normal maps, which provide additional optimization constraints. In addition, the explicit network supports physically based rendering, which can more accurately simulate reflected light, thus achieving better reconstruction results for highly reflective objects.

[0063] Understandably, mesh rasterization can quickly obtain surface points, masks, normals, and depth information. Combining this information with real-time rendering methods that use segmentation and summation, it is possible to render photometric information such as color, diffuse light map, and specular light map.

[0064] Understandably, various rendered images include specular maps, diffuse maps, depth maps, and normal maps.

[0065] Understandably, the loss function is:

[0066]

[0067] in, For loss function, Let C be the mean squared error loss function, and C be the true photometric stereo image. For the photometric stereo image, λ LPIPS As the first coefficient, To learn the perceptual image patch similarity loss function, where 'a' is the true albedo map, For the albedo map, L * This provides a true specular light map and a true diffuse light map. Here, n represents the specular light map and the diffuse light map, and n is the true normal map. For the normal diagram, λ normal Here, m is the second coefficient, and m is the actual mask. For the prediction mask, λ reg The third coefficient, Let λ be the geometric constraint loss function. depth The fourth coefficient is d, where d is the true depth map. For the depth map, λ mask It is the fifth coefficient.

[0068] Understandably, optimizing the loss function through multiple objectives greatly enhances the model's ability to recover geometry, thereby improving the quality of large-scale 3D reconstruction models.

[0069] In addition, such as Figure 2 As shown, Figure 2 This is a flowchart of a method for constructing a large 3D reconstruction model according to another embodiment of this application; regarding the rendering of the training image based on mesh rasterization to obtain a photometric stereo image in the above step S110, it may include, but is not limited to, steps S210, S220, S230, S240 and S250.

[0070] Step S210: Collect multiple light maps and obtain lighting information based on the light maps;

[0071] Step S220: Obtain the material properties of the preset network model and rasterize the network model to obtain the rasterization information of the network model. The material properties include albedo, metallicity and roughness.

[0072] Step S230: Calculate the diffuse albedo and specular albedo based on the albedo and metallicity.

[0073] Step S240: Define the sampling range for metallicity and roughness;

[0074] Step S250: Perform diffuse and specular reflection on the training image based on illumination information, rasterization information, diffuse albedo, specular albedo, and sampling range to obtain photometric stereo images from multiple perspectives.

[0075] In one embodiment, firstly, multiple light maps are acquired to obtain lighting information; next, the material properties of a preset network model are obtained, and the network model is rasterized. After rasterization, the rasterization message of the network model is obtained, where the material properties include albedo, metallicity, and roughness; then, diffuse albedo and specular albedo are calculated based on albedo and metallicity; furthermore, the sampling range of metallicity and roughness is defined; finally, diffuse and specular reflections are applied to the training image based on the obtained lighting information, rasterization information, diffuse albedo, specular albedo, and sampling range to obtain a stereoscopic image with multiple viewing angles.

[0076] Understandably, the rasterization information of the network model obtained after rasterization is an indispensable key information in the subsequent rendering process, which together form the basis of the rendering pipeline, thereby enabling the generation of realistic 3D graphics.

[0077] It is understandable that diffuse reflectance is expressed by formula a. d = (1-m)a is calculated, where a d denoted as diffuse reflectance, m as metallicity, and a as reflectance.

[0078] It is understandable that the specular albedo is expressed by the formula a. s = ((1-m)*0.04+m*a) is calculated, where a s denoted as specular reflectance.

[0079] It is understood that the physically based real-time rendering in this application includes two parts: diffuse reflection and specular reflection. Diffuse reflection is expressed by formula C. d (x,ω0)=a d (a,m)L diff (x,n,M d ) is obtained through formula C. s (x,ω0)=a s (a,m,n,ω0)L spec (x,n,ω0,ρ,M s This yields photometric stereo images with multiple views after diffuse and specular reflection of the training images, where C d Let x be the diffuse color, x be the surface point, ω0 be the viewing direction, and L be the line of sight. diff Let M be diffuse reflection light, n be the normal vector, and M be the diffuse reflection light.d For diffuse ambient light, C s For specular reflection color, L spec Let ρ be the specular reflection light, ρ be the surface roughness, and M be the surface roughness. s It is used to reflect ambient light through a mirror.

[0080] It is understood that the embodiments of this application randomly select metallicity, roughness and illumination information from predefined options to perform diffuse reflection and specular reflection on the training image.

[0081] It is understood that the range of metallicity and roughness can be 0-1, increasing by 0.1 each time, for a total of 121 possible combinations. This application does not specifically limit the range of metallicity and roughness.

[0082] It is understood that the number of light maps collected mentioned above could be 600 or 650, depending on actual needs. This application embodiment does not specify the number of light maps collected.

[0083] In addition, such as Figure 3 As shown, Figure 3 This is a flowchart of a method for constructing a large 3D reconstruction model according to another embodiment of this application; regarding the above step S120, it may include, but is not limited to, steps S310 and S320.

[0084] Step S310: Input the photometric stereo image into the image encoder in the preset explicit network for encoding to obtain the first image features;

[0085] Step S320: Obtain a three-dimensional representation of the photometric stereo image features based on the features of the first image.

[0086] In one embodiment, the rendered photometric stereo image is input into an image encoder in a preset explicit network for encoding, thereby obtaining first image features, and then obtaining a three-dimensional representation of the photometric stereo image features.

[0087] In addition, step S320 mentioned above also includes the following steps:

[0088] The first image features are input into a three-plane decoder in an explicit network for decoding to obtain a three-dimensional representation of the first image features.

[0089] Understandably, explicit networks include an image encoder and a three-plane decoder.

[0090] Based on the above embodiments, a method for constructing a large 3D reconstruction model is presented. The following are overall embodiments of the method for constructing a large 3D reconstruction model according to this application.

[0091] like Figure 4 and 5 As shown, Figure 4 This is an overall flowchart of a method for constructing a large 3D reconstruction model according to one embodiment of this application. Figure 5 This is an overall flowchart of a method for constructing a large 3D reconstruction model provided in another embodiment of this application.

[0092] Step S401: Obtain training images;

[0093] Step S402: Collect multiple light maps and obtain lighting information based on the light maps;

[0094] Step S403: Obtain the material properties of the preset network model and rasterize the network model to obtain the rasterization information of the network model. The material properties include albedo, metallicity and roughness.

[0095] Step S404: Calculate the diffuse albedo and specular albedo based on the albedo and metallicity.

[0096] Step S405: Define the sampling range for metallicity and roughness;

[0097] Step S406: Perform diffuse and specular reflection on the training image based on illumination information, rasterization information, diffuse albedo, specular albedo, and sampling range to obtain photometric stereo images from multiple perspectives.

[0098] Step S407: Input the photometric stereo image into the image encoder in the preset explicit network for encoding to obtain the first image features;

[0099] Step S408: Input the first image features into the three-plane decoder in the explicit network for decoding to obtain a three-dimensional representation of the first image features;

[0100] Step S409: Render the 3D representation to obtain multiple rendered images, and predict the albedo of the 3D representation to obtain an albedo map.

[0101] Step S410: Calculate the loss value by using a preset loss function based on various rendered images, albedo maps, and photometric stereo images;

[0102] Step S411: Optimize the explicit network based on the loss value to obtain the constructed 3D reconstruction model.

[0103] It is worth noting that this application performs loss calculations using various rendered images, albedo maps, and photometric stereo images, thereby optimizing the explicit network based on the loss values ​​to obtain a well-constructed 3D reconstruction model, thus improving the accuracy and quality of the 3D reconstruction model construction.

[0104] It is understood that, since the embodiments of this application utilize multi-view photometric stereo images as input and supervision of the model, the robustness of the model to the appearance of the input image and its ability to recover high-quality local details are significantly improved.

[0105] To better understand the solutions of the embodiments of this application, the following describes the method for constructing large 3D reconstruction models provided by the embodiments of this application through several specific application examples.

[0106] Example 1

[0107] like Figure 6 As shown, Figure 6 This is a schematic diagram of a 3D object provided in one embodiment of this application.

[0108] In one embodiment, the reconstructed 3D object can be subjected to downstream applications such as relighting and material editing.

[0109] It is understood that the 3D reconstructed large model in the embodiments of this application can be rendered under different environment textures, so that the object presents different appearance effects.

[0110] Example 2

[0111] like Figure 7 As shown, Figure 7 This is a schematic diagram of a 3D object provided in another embodiment of this application.

[0112] In one embodiment, the 3D reconstruction model of this application can vary the material of the object.

[0113] Example 3

[0114] like Figure 8 As shown, Figure 8 This is a schematic diagram of a 3D object provided in another embodiment of this application.

[0115] In one embodiment, the 3D reconstruction model of this application can gradually increase the metallicity of the object, making the object appear metallic.

[0116] Based on the above embodiments, a method for constructing a large 3D reconstruction model is provided. The following presents various embodiments of the controller, a system for constructing a large 3D reconstruction model, a computer-readable storage medium, and a computer program product of this application.

[0117] like Figure 9 As shown, Figure 9This is a schematic diagram of a controller for executing a method for constructing a large 3D reconstruction model, according to an embodiment of this application. The controller 700 implemented in this application includes: a processor 710, a memory 720, and a computer program stored in the memory 720 and executable on the processor 710, wherein... Figure 9 The example uses a processor 710 and a memory 720.

[0118] The processor 710 and memory 720 can be connected via a bus or other means. Figure 9 Taking the example of a connection between China and Israel via a bus.

[0119] Memory 720, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory 720 may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 720 may optionally include remotely located memories 720 relative to processor 710, which can be connected to controller 700 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0120] Those skilled in the art will understand that Figure 9 The device structure shown does not constitute a limitation on the controller 700 and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0121] exist Figure 9 In the controller 700 shown, the processor 710 can be used to call the control program stored in the memory 720, thereby implementing the above-described method for constructing a large 3D reconstruction model. Specifically, the non-transitory software program and instructions required to implement the method for constructing a large 3D reconstruction model according to the above embodiment are stored in the memory 720. When executed by the processor 710, the method for constructing a large 3D reconstruction model according to the above embodiment is executed.

[0122] It is worth noting that, since the controller 700 of this application embodiment is capable of executing a method for constructing a large 3D reconstruction model according to any of the above embodiments, the specific implementation method and technical effects of the controller 700 of this application embodiment can be referred to the specific implementation method and technical effects of a method for constructing a large 3D reconstruction model according to any of the above embodiments.

[0123] Furthermore, one embodiment of this application also provides a 3D reconstruction large model construction system, which includes the controller described in the above embodiment.

[0124] It is worth noting that, since the 3D reconstruction large model construction system of this application embodiment includes the controller of the above embodiment, and the controller of the above embodiment can execute a 3D reconstruction large model construction method of any of the above embodiments, the specific implementation method and technical effect of the multi-split system containing domestic hot water of this application embodiment can refer to the specific implementation method and technical effect of a 3D reconstruction large model construction method of any of the above embodiments.

[0125] Furthermore, one embodiment of this application provides a computer-readable storage medium storing computer-executable instructions for performing the aforementioned method for constructing a large 3D reconstruction model. Exemplarily, the above-described method is executed... Figures 1 to 4 The methods and steps in the text.

[0126] It is worth noting that, since the computer-readable storage medium of this application embodiment is capable of executing a method for constructing a large 3D reconstruction model according to any of the above embodiments, the specific implementation and technical effects of the computer-readable storage medium of this application embodiment can be referred to the specific implementation and technical effects of a method for constructing a large 3D reconstruction model according to any of the above embodiments.

[0127] Furthermore, one embodiment of this application also provides a computer program product, including a computer program or computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium and executes the computer program or computer instructions, causing the computer device to perform the aforementioned method for constructing a large 3D reconstruction model. Exemplarily, the above-described method is executed... Figures 1 to 4 The methods and steps in the text.

[0128] It is worth noting that, since the computer program product of this application embodiment can execute the method for constructing a 3D reconstruction large model of any of the above embodiments, the specific implementation method and technical effect of the computer program product of this application embodiment can refer to the specific implementation method and technical effect of the method for constructing a 3D reconstruction large model of any of the above embodiments.

[0129] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0130] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0131] In the several embodiments provided in this application, it should be understood that the disclosed systems, instruments, and methods can be implemented in other ways. For example, the instrument embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between instruments or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0132] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.

[0133] The above provides a detailed description of the preferred embodiments of this application. However, this application is not limited to the above-described embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for constructing a large 3D reconstruction model, characterized in that, The method includes: Acquire training images, and render the training images based on grid rasterization to obtain a photometric stereo image; The photometric stereo image is input into a preset explicit network to obtain a three-dimensional representation of the photometric stereo image, wherein the explicit network includes an image encoder and a three-plane decoder. The three-dimensional representation is rendered to obtain multiple rendered images, and the albedo of the three-dimensional representation is predicted to obtain an albedo map. The multiple rendered images include a specular map, a diffuse map, a depth map, and a normal map. The loss value is obtained by calculating the loss based on the various rendered images, the albedo map, and the photometric stereo image using a preset loss function; The explicit network is optimized based on the loss value to obtain the constructed 3D reconstruction model. The loss function is: in, For loss function, Let C be the mean squared error loss function, and C be the true photometric stereo image. For the rendered photometric stereo image, λ LPIPS As the first coefficient, To learn the perceptual image patch similarity loss function, where 'a' is the true albedo map, The albedo map obtained from albedo prediction, L * This provides a true specular light map and a true diffuse light map. The specular and diffuse light maps in the rendered image are represented by n, where n is the true normal map. For the normal map in the rendered image, λ normal Here, m is the second coefficient, and m is the actual mask. For the prediction mask, λ reg The third coefficient, Let λ be the geometric constraint loss function. depth The fourth coefficient is d, where d is the true depth map. For the depth map in the rendered image, λ mask It is the fifth coefficient.

2. The method for constructing a large 3D reconstruction model according to claim 1, characterized in that, The step of rendering the training image based on grid rasterization to obtain a photometric stereo image includes: Collect multiple light maps and obtain lighting information based on the light maps; Obtain the material properties of a preset network model, and rasterize the network model to obtain the rasterization information of the network model. The material properties include albedo, metallicity, and roughness. The diffuse albedo and specular albedo are calculated based on the albedo and the metallicity. Define the sampling range for the metallicity and the roughness; Based on the illumination information, the rasterization information, the diffuse albedo, the specular albedo, and the sampling range, the training image is subjected to diffuse and specular reflection to obtain the photometric stereo image from multiple perspectives.

3. The method for constructing a large 3D reconstruction model according to claim 1, characterized in that, The explicit network includes an image encoder. The step of inputting the photometric stereo image into a preset explicit network to obtain a three-dimensional representation of the features of the photometric stereo image includes: The photometric stereo image is input into an image encoder in a preset explicit network for encoding to obtain the first image features; A three-dimensional representation of the photometric stereo image features is obtained based on the features of the first image.

4. The method for constructing a large 3D reconstruction model according to claim 3, characterized in that, The explicit network further includes a three-plane decoder, wherein obtaining the three-dimensional representation of the photometric stereo image features based on the first image features includes: The first image features are input into the three-plane decoder in the explicit network for decoding to obtain a three-dimensional representation of the first image features.

5. A controller, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, performs the method for constructing a large 3D reconstruction model as described in any one of claims 1 to 4.

6. A system for constructing large 3D reconstruction models, characterized in that, Includes the controller as described in claim 5.

7. A computer-readable storage medium, characterized in that: The device stores computer-executable instructions for performing the method for constructing a large 3D reconstruction model as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program or computer instructions, characterized in that, The computer program or the computer instructions are stored in a computer-readable storage medium, the processor of the computer device reads the computer program or the computer instructions from the computer-readable storage medium, and the processor executes the computer program or the computer instructions, causing the computer device to perform the method for constructing a large 3D reconstruction model as described in any one of claims 1 to 4.

Images