Method for updating texture map of 3D mesh and computing device using same

Abstract

The present invention relates to a method for updating a texture map of a 3D mesh and a computing device using same. The method for updating a texture map of a 3D mesh using a computing device according to an embodiment of the present invention comprises the steps of: generating a 3D mesh for a target environment and an initial texture map for the 3D mesh from each target image obtained by imaging the target environment with a plurality of camera poses; parameterizing the initial texture map; generating a rendered image for the target environment from the initial texture map and the 3D mesh by using a differentiable renderer; fine-tuning the differentiable renderer by performing supervised learning on the generation of the rendered image on the basis of the target image; and regenerating a texture map of the 3D mesh from the initial texture map by using the fine-tuned differentiable renderer.

Description

Method for updating texture map of 3D mesh and computing device using same

[0001] The present invention relates to a method for updating a texture map of a 3D mesh, which implements a high-quality texture map by refining the texture map when generating a texture map for performing texturing on a 3D mesh, and to a computing device using the same.

[0002] When generating a 3D mesh with a texture reflected from an input image, a rendered image can be generated by rendering the 3D texture mesh, so it can be used in a wide range of fields such as games and animations.

[0003] Recently, there has been significant progress in the technology of restoring 3D meshes from 2D images based on deep learning. However, most research is on the restoration of 3D meshes, and research on the texture reconstruction process for meshes is limited.

[0004] However, texture plays a crucial role in recognizing and understanding objects, and reconstructing the texture of 3D meshes remains a challenging task.

[0005] The present invention provides a method for updating a texture map of a three-dimensional mesh, which can generate a high-quality texture mesh by refining a texture map when generating a texture map for performing texturing on a three-dimensional mesh, and a computing device using the same.

[0006] The present invention aims to provide a method for updating a texture map of a 3D mesh, which can reduce noise included in a 3D texture mesh generated through optimization of the texture map, and a computing device using the same.

[0007] The present invention aims to provide a method for updating a texture map of a three-dimensional mesh, which can be applied to both cases where the target environment is an object or a space, and a computing device using the same.

[0008] According to one embodiment of the present invention, a method for updating a texture map of a three-dimensional mesh using a computing device may include the steps of: generating a three-dimensional mesh for a target environment and an initial texture map for the three-dimensional mesh from respective target images captured with a plurality of camera poses of the target environment; parameterizing the initial texture map; generating a rendering image for the target environment from the initial texture map and the three-dimensional mesh using a differentiable renderer; fine-tuning the differentiable renderer by supervised learning the generation of the rendering image based on the target image; and regenerating a texture map of the three-dimensional mesh from the initial texture map using the fine-tuned differentiable renderer.

[0009] A computing device for updating a texture map of a three-dimensional mesh according to one embodiment of the present invention includes a processor, and the processor may perform the following operations: generating a three-dimensional mesh for a target environment and an initial texture map for the three-dimensional mesh from respective target images captured with a plurality of camera poses of the target environment; parameterizing the initial texture map; generating a rendering image for the target environment from the initial texture map and the three-dimensional mesh using a differentiable renderer; fine-tuning the differentiable renderer by supervised learning the generation of the rendering image based on the target image; and regenerating a texture map of the three-dimensional mesh from the initial texture map using the fine-tuned differentiable renderer.

[0010] Additionally, the solutions to the aforementioned problems do not enumerate all features of the present invention. The various features of the present invention, along with their corresponding advantages and effects, can be understood in more detail by referring to the specific embodiments below.

[0011] According to a method for updating a texture map of a 3D mesh according to one embodiment of the present invention and a computing device using the same, it is possible to generate a high-quality texture mesh by refining an initial texture map and regenerating the texture map.

[0012] According to a method for updating a texture map of a 3D mesh according to an embodiment of the present invention and a computing device using the same, the texture map can be optimized, so that noise occurring during a texturing process can be removed, and color consistency within a 3D texture mesh can be maintained.

[0013] However, the effects that can be achieved by the method for updating a texture map of a 3D mesh according to embodiments of the present invention and the computing device using the same are not limited to those mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from the description below.

[0014] Figure 1 is a schematic diagram showing a device for generating a three-dimensional model of a target environment based on a multi-view image according to one embodiment of the present invention.

[0015] Figure 2 is an exemplary diagram showing the generation of a multi-view image according to a target environment according to one embodiment of the present invention.

[0016] Figure 3 is a block diagram showing a texture map generation unit according to one embodiment of the present invention.

[0017] Figure 4 is a block diagram showing a computing device according to one embodiment of the present invention.

[0018] Figure 5 is a flowchart showing a method for updating a texture map of a 3D mesh according to one embodiment of the present invention.

[0019] Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are given or used interchangeably only for the convenience of writing the specification, and do not have distinct meanings or roles in themselves. That is, the term "part" used in the present invention means a hardware component such as software, FPGA, or ASIC, and the "part" performs certain roles. However, the "part" is not limited to software or hardware. The "part" may be configured to be on an addressable storage medium, or may be configured to reproduce one or more processors. Thus, as an example, a 'part' may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or further separated into additional components and 'parts'.

[0020] In addition, when describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0021]

[0022] Figure 1 is a schematic diagram showing a device for generating a three-dimensional model of a target environment based on a multi-view image according to one embodiment of the present invention.

[0023] Referring to FIG. 1, a three-dimensional model generation device (1000) according to one embodiment of the present invention may include a texture map generation unit (100) and a mesh generation unit (200).

[0024] Referring to FIG. 1 below, a three-dimensional model generation device (1000) according to one embodiment of the present invention will be described.

[0025] A three-dimensional model generation device (1000) generates a three-dimensional model for a target environment, and can receive, as input data (Input), target images (I) taken from various angles and directions of the target environment, and camera pose information (P) such as the angle and direction of the camera that took each of the target images (I). Thereafter, the three-dimensional model generation device (1000) can generate, as output data (Output), a three-dimensional mesh (M) and a texture map (T) for the target environment based on the input data (Input), and can provide a three-dimensional model using a texture mesh that combines these.

[0026] Here, the target environment may be a specific object or space, etc. That is, depending on the user's selection, the 3D model generation device (1000) can generate 3D models for various target environments such as objects or spaces.

[0027] The target image (I) may be a photograph taken using a camera, etc., and in some embodiments, an image captured from a video taken by a camera may be used as the target image (I). In this case, the camera may directly generate and provide camera pose information (P) regarding the angle or direction when capturing the target image (I), and in some embodiments, it may also measure and generate camera pose information (P) using a separate device, etc.

[0028] Specifically, as illustrated in FIG. 2(a), when the target environment is an object (O1), target images (I) for the object (O1) can be captured from various angles and directions while moving the camera (1) with the object (O1) at the center. In some embodiments, it is also possible to capture the target images (I) for the object (O1) while moving along a certain path (L). On the other hand, when the target environment is a space (O2), since it is impossible to move the camera (1) around the space (O2), the inside of the space (O2) can be captured from various angles and directions, such as by rotating the camera (1) while moving inside the space (O2), thereby generating target images (I).

[0029] Here, the camera (1) may include an image sensor based on a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), and may generate target images (I) based on light input to the image sensor.

[0030] Meanwhile, the camera (1) may be configured independently, but depending on the embodiment, it may also be implemented by being combined with various terminal devices such as a smartphone, tablet PC, PDA (Personal Digital Assistant), laptop computer, wearable device, etc. That is, a user can create two-dimensional target images of the target environment (O1, O2) by using the camera (1) equipped in his / her smartphone, etc.

[0031] Thereafter, the user can request the creation of a 3D model for the target environment by transmitting input data (Input) including the captured target image (I) and camera pose information (P) for each target image (I) to the 3D model creation device (1000) via a wired or wireless network.

[0032] At this time, the user can connect to the network using his / her terminal device and communicate with the 3D model generation device (1000) through the network. That is, the user can transmit input data (Input) such as a target image (I) and camera pose information (P) captured by a camera (1) using the terminal device to the 3D model generation device (1000), and the 3D model generation device (1000) can generate a 3D model for the corresponding target environment based on the received input data (Input).

[0033] The communication method between the terminal device and the 3D model generation device (1000) is not limited, and may include not only a communication method utilizing a communication network that the network may include (for example, a mobile communication network, wired Internet, wireless Internet, broadcasting network, satellite network, etc.), but also short-range wireless communication between devices. For example, the network may include any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, etc. In addition, the network may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree, or a hierarchical network.

[0034] A 3D model generation device (1000) can generate a 3D mesh (M) representing the structure (geometry) of the target environment based on input data (Input) for the target environment, and a texture map (T) including respective texture images mapped to the 3D mesh (M). In addition, a texture mesh that combines the texture map (T) with the 3D mesh (M) can be generated to implement a 3D model. For example, it is possible to provide images or videos from any point in time within the 3D model at the user's request. That is, images or videos from various angles and directions of the target environment that were not included in the target image can be generated and provided.

[0035] Meanwhile, in FIG. 1, the 3D model generation device (1000) is depicted as being configured independently. However, depending on the embodiment, the 3D model generation device (100) may be implemented to be included within a user's terminal device. In this case, the 3D model generation device (1000) may be implemented as an application within the terminal device, etc.

[0036] As illustrated in Fig. 1, a 3D model generation device (1000) may include a texture map generation unit (100) and a mesh generation unit (200). Here, the texture map generation unit (100) and the mesh generation unit (200) may each be implemented using a pre-learned neural network model, etc.

[0037] The texture map generation unit (100) can generate a texture map (T) for the target environment based on the target image (I) and camera pose information (P) included in the input data (Input). Here, the texture map generation unit (100) can obtain a UV mapping relationship between each texture image and each plane included in the 3D mesh (M), and can use this to generate the texture map. In addition, according to an embodiment, the texture map generation unit (100) can generate various sprites appearing in the target environment from the target image (I), and build a texture atlas including the sprites. In this case, each sprite can be utilized as a texture image, and it is possible to determine on which plane each sprite should be located in the 3D mesh (M) using the UV mapping relationship. Here, the texture map generation unit (100) generates a texture map (T) using an actual image of the target environment, so that texturing (high fidelity texturing) that is very similar to reality can be implemented for the 3D mesh (M).

[0038] The mesh generation unit (200) can generate a three-dimensional mesh (M) by implementing the structure of the target environment shown in the target image (I) in three dimensions. For example, the mesh generation unit (200) may utilize the MVS (Multi View Stereo) algorithm or NeuS (Neural Surface Reconstruction), but is not limited thereto, and any model or algorithm can be utilized as long as it generates a three-dimensional mesh for the target environment from a plurality of target images (I) and camera pose information (P).

[0039] Meanwhile, in the case of a 3D model created using a 3D model creation device (1000), a problem may occur in which a seam appears at the boundary of the 3D mesh (M) due to an error in view selection, such as a mismatch between the plane of the 3D mesh (M) and the view direction of the mapped texture image, or an error in the rasterization process of the 3D mesh (M). In addition, a problem may occur in which color consistency is poor between adjacent planes within the 3D model due to a difference in brightness included in each texture image.

[0040] Here, in the case of the texture map generation unit (100) according to one embodiment of the present invention, since the generated texture map is not directly used, but rather refined and regenerated, it is possible to solve problems such as the appearance of seams at the boundary or poor color consistency due to differences in brightness between adjacent planes. Hereinafter, the texture map generation unit (100) according to one embodiment of the present invention will be described with reference to FIG. 3.

[0041]

[0042] Figure 3 is a block diagram showing a texture map generation unit (100) according to one embodiment of the present invention.

[0043] Referring to FIG. 3, a texture map generation unit (100) according to one embodiment of the present invention may include an initial texture map generation unit (110) and a texture map update unit (120).

[0044] The initial texture map generation unit (110) can generate an initial texture map for a 3D mesh of the target environment from each of the target images (I) captured with a plurality of camera poses of the target environment. The initial texture map generation unit (110) can generate the initial texture map by using the MVS texturing technique, but is not limited thereto, and any algorithm or model that can generate a texture map for a 3D mesh can be utilized. In some embodiments, a texture atlas containing respective sprites corresponding to the 3D mesh (M) may be constructed in the initial texture map. Here, it is possible to generate a 3D model by performing texturing on the 3D mesh using the initial texture map, but in order to remove noises included in the initial texture map, the texture map may be regenerated and updated.

[0045] To this end, the texture map update unit (120) can first parameterize the initial texture map. That is, since the pixel values ​​of each texture image included in the initial texture map are treated as constants, the texture images can be mapped directly onto the plane of the 3D mesh when creating a 3D model. On the other hand, the texture map update unit (120) updates the texture map to remove noise, etc. included in the initial texture map, and can parameterize the pixel values ​​of each texture image included in the initial texture map by declaring them as changeable variables rather than fixed constants. Through this, the pixel values ​​of each texture image can be changed and refined, so that the quality of the texture map can be improved.

[0046] Specifically, the texture map update unit (120) may further include a rendering unit (121), a fine tuning unit (122), and a regeneration unit (123).

[0047] The rendering unit (121) may include a differentiable renderer, and may generate a rendering image using the differentiable renderer. That is, it is possible to generate a rendering image corresponding to arbitrary camera pose information (P) from an initial texture map and a 3D mesh. In some embodiments, the differentiable renderer may generate a raster image by rasterizing a 3D mesh based on the camera pose information (P), and may perform texture sampling on the raster image according to a UV mapping relationship between the 3D mesh and the texture image, thereby generating a rendering image.

[0048] Meanwhile, the fine-tuning unit (122) fine-tunes the differentiable renderer to update the initial texture map, and can perform supervised learning of the rendering image generation of the differentiable renderer based on the target images (I). Specifically, the fine-tuning unit (122) can generate a difference image by comparing the rendering image generated in response to the camera pose information of the target image (I) with the target image (I), and can train the differentiable renderer so that the difference image is minimized. At this time, since the initial texture map is parameterized, the differentiable renderer can be trained by resetting the pixel values ​​of the generated rendering image.

[0049] Specifically, the loss function of the differentiable renderer can be set as follows, and the differentiable renderer can be fine-tuned based on the photometric loss.

[0050]

[0051] Here, L photo is the optical loss, is a rendered image, I i is a ground-truth image, and may include a structural similarity index (SSIM) and a setting coefficient α. That is, the fine-tuning unit (122) can train a differentiable renderer by further reflecting SSIM in the optical loss. However, α can be set to an arbitrary value as a setting coefficient, and in some embodiments, α can be set to 0 so as not to reflect SSIM.

[0052] The fine tuning unit (122) is a differentiable renderer that renders a corresponding image (I) according to the camera pose information (P) of the target image (I). ) to generate the rendered image ( ) can be compared with the target image (I). In this case, since the target image (I) corresponds to the ground-truth image for the rendered image, it is possible for the differentiable renderer to learn to generate a rendered image close to the ground-truth image.

[0053] The regeneration unit (123) can regenerate the texture map of the 3D mesh from the initial texture map using a fine-tuned differentiable renderer. That is, the differentiable renderer can generate a rendered image similar to an actual image through fine-tuning. Therefore, by regenerating each texture image included in the initial texture map using the fine-tuned differentiable renderer, it is possible to update the texture map to a higher quality. For example, by generating rendered images using the fine-tuned differentiable renderer, each sprite in the texture atlas can be updated, thereby regenerating the texture map for the 3D mesh. That is, the UV mapping relationship between each sprite and the 3D mesh can be maintained, and the texture map can be updated by regenerating each sprite. By using the updated texture map in this way, it is possible to reduce noise, etc. occurring in the 3D model.

[0054]

[0055] Figure 4 is a block diagram illustrating a computing environment (10) suitable for use in exemplary embodiments. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components other than those described below.

[0056] The illustrated computing environment (10) includes a computing device (12). In one embodiment, the computing device (12) may be a device (e.g., a 3D model generation device (1000)) that generates a 3D model of a target environment based on a multi-view image.

[0057] A computing device (12) includes at least one processor (14), a computer-readable storage medium (16), and a communication bus (18). The processor (14) may cause the computing device (12) to operate according to the exemplary embodiments mentioned above. For example, the processor (14) may execute one or more programs stored in the computer-readable storage medium (16). The one or more programs may include one or more computer-executable instructions, which, when executed by the processor (14), may be configured to cause the computing device (12) to perform operations according to the exemplary embodiments.

[0058] A computer-readable storage medium (16) is configured to store computer-executable instructions or program code, program data, and / or other suitable forms of information. A program (20) stored in the computer-readable storage medium (16) includes a set of instructions executable by the processor (14). In one embodiment, the computer-readable storage medium (16) may be a memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, any other form of storage medium that is accessible by the computing device (12) and capable of storing desired information, or a suitable combination thereof.

[0059] A communication bus (18) interconnects various other components of the computing device (12), including the processor (14) and computer-readable storage media (16).

[0060] The computing device (12) may also include one or more input / output interfaces (22) that provide interfaces for one or more input / output devices (24) and one or more network communication interfaces (26). The input / output interfaces (22) and the network communication interfaces (26) are connected to the communication bus (18). The input / output devices (24) may be connected to other components of the computing device (12) via the input / output interfaces (22). Exemplary input / output devices (24) may include input devices such as pointing devices (such as a mouse or a trackpad), a keyboard, a touch input device (such as a touchpad or a touchscreen), a voice or sound input device, various types of sensor devices and / or photographing devices, and / or output devices such as display devices, printers, speakers and / or network cards. The exemplary input / output devices (24) may be included within the computing device (12) as a component constituting the computing device (12), or may be connected to the computing device (12) as a separate device distinct from the computing device (12).

[0061]

[0062] FIG. 5 is a flowchart illustrating a method for updating a texture map of a three-dimensional mesh according to an embodiment of the present invention. Here, each step of FIG. 5 can be performed by a computing device according to an embodiment of the present invention.

[0063] Referring to FIG. 5, the computing device can generate a three-dimensional mesh for the target environment and an initial texture map for the three-dimensional mesh from each of the target images captured with a plurality of camera poses of the target environment (S110). Here, the initial texture map can be generated using various methods such as the MVS texturing technique, and in some embodiments, a texture atlas containing respective sprites corresponding to the three-dimensional mesh may be constructed within the initial texture map.

[0064] Thereafter, the computing device can parameterize the initial texture map (S120). That is, since the pixel values ​​of each texture image included in the initial texture map are treated as constants, the texture images can be mapped directly onto the plane of the 3D mesh when creating a 3D model. Here, since the initial texture map is to be updated and refined, the pixel values ​​of each texture image can be parameterized by declaring them as changeable variables rather than fixed constants.

[0065] In addition, the computing device can generate a rendered image of the target environment from the initial texture map and the 3D mesh using a differentiable renderer (S130). That is, the differentiable renderer can generate a raster image by rasterizing the 3D mesh based on camera pose information, and can perform texture sampling on the raster image according to the UV mapping relationship between the 3D mesh and the texture image to generate a rendered image. Accordingly, the computing device can generate a corresponding rendered image according to the camera pose information of the target image, in which case each target image corresponds to a ground-truth image for the corresponding rendered image.

[0066] Thereafter, the computing device can fine-tune the differentiable renderer by supervised learning to generate a rendered image based on the target image (S140). Specifically, the computing device can compare the rendered image generated in response to the camera pose information of the target image with the target image to generate a difference image, and can train the differentiable renderer so that the difference image is minimized. At this time, since the initial texture map is parameterized, the differentiable renderer can be trained by resetting the pixel values ​​of the generated rendered image.

[0067] In some embodiments, the loss function of the differentiable renderer can be set as follows.

[0068]

[0069] Here, L photo is the optical loss, is a rendered image, I i is a ground-truth image, and may include a structural similarity index (SSIM) and a setting coefficient α. That is, the computing device can train a differentiable renderer by further reflecting SSIM in the optical loss. However, α can be set to an arbitrary value as a setting coefficient, and in some embodiments, α can be set to 0 so that SSIM is not reflected.

[0070] Thereafter, the computing device can regenerate the texture map of the 3D mesh from the initial texture map using the fine-tuned differentiable renderer (S150). That is, since the fine-tuned differentiable renderer can generate a rendered image similar to an actual image, it is possible to update the texture images in the initial texture map to a higher quality texture map by regenerating them using the fine-tuned differentiable renderer. For example, the rendered images can be generated by the fine-tuned differentiable renderer to update each sprite in the texture atlas, thereby regenerating the texture map for the 3D mesh. That is, it is possible to update the texture map by regenerating each sprite while maintaining the UV mapping relationship between each sprite and the 3D mesh.

[0071]

[0072] The present invention described above can be implemented as computer-readable code on a medium recording a program. The computer-readable medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. Furthermore, the medium may be a variety of recording or storage means, including a single or multiple hardware components, and is not limited to media directly connected to a computer system, but may also be distributed across a network. Examples of the medium include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and ROM, RAM, flash memory, and other media configured to store program instructions. Furthermore, other examples of media include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc. Therefore, the above detailed description should not be construed as limiting in all respects, but rather as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

[0073]

[0074] The present invention is not limited to the above-described embodiments and the attached drawings. It will be apparent to those skilled in the art that components of the present invention can be substituted, modified, and altered without departing from the technical spirit of the present invention.

Claims

1. A method for updating a texture map of a 3D mesh using a computing device, A step of generating a 3D mesh for the target environment and an initial texture map for the 3D mesh from each target image captured with multiple camera poses of the target environment; A step of parameterizing the initial texture map; A step of generating a rendering image for the target environment from the initial texture map and a 3D mesh using a differentiable renderer; A step of fine-tuning the differentiable renderer by supervised learning the generation of the rendered image based on the target image; and A method for updating a texture map of a 3D mesh, comprising the step of regenerating a texture map of the 3D mesh from the initial texture map using the fine-tuned differentiable renderer.

2. In the first paragraph, the initial texture map A method for updating a texture map of a 3D mesh, the method including a texture atlas of each sprite corresponding to the 3D mesh.

3. In the second paragraph, the parameterizing step A method for updating a texture map of a 3D mesh, wherein the pixel values ​​of the sprites included in the initial texture map are set as parameters so that they can be changed.

4. In the first paragraph, the differentiable renderer A method for updating a texture map of a 3D mesh, wherein the rendering image is generated by texture sampling sprites included in the initial texture map based on a mapping relationship between the 3D mesh and the initial texture map on a raster image obtained by rasterizing the 3D mesh.

5. In the first paragraph, the fine tuning step A method for updating a texture map of a 3D mesh, comprising: generating a difference image by comparing a rendered image generated in response to a camera pose of the target image with the target image; and learning to reset pixel values ​​of the rendered image so that the difference image is minimized.

6. In the fifth paragraph, the fine tuning step A method for updating a texture map of a 3D mesh by further reflecting SSIM (Structural Similarity Index) in the above loss function and learning it.

7. In the second paragraph, the regenerating step A method for updating a texture map of a 3D mesh, wherein the texture map is regenerated by updating each sprite in the texture atlas with a rendered image generated by the fine-tuned differentiable renderer.

8. A computing device including a processor and updating a texture map of a three-dimensional mesh, The above processor, Generating a 3D mesh for the target environment and an initial texture map for the 3D mesh from each target image captured with multiple camera poses; Parameterizing the initial texture map; Generating a rendering image of the target environment from the initial texture map and a 3D mesh using a differentiable renderer; Fine-tuning the differentiable renderer by supervised learning the generation of the rendered image based on the target image; and A computing device, comprising: regenerating a texture map of the three-dimensional mesh from the initial texture map using the fine-tuned differentiable renderer.

9. In the 8th paragraph, the initial texture map A computing device comprising a texture atlas of each sprite corresponding to the above three-dimensional mesh.

10. In the 9th paragraph, the parameterization is A computing device that allows pixel values ​​of the sprites included in the initial texture map to be changed by setting them as parameters.

11. In the 8th paragraph, the differentiable renderer A computing device that generates the rendered image by texture sampling sprites included in the initial texture map based on the mapping relationship between the 3D mesh and the initial texture map on a raster image obtained by rasterizing the 3D mesh.

Images