Three-dimensional texture super-resolution method and system based on multi-view fusion

Abstract

The invention discloses a three-dimensional texture super-resolution method and system based on multi-view fusion, and belongs to the technical field of computer graphics, and the method comprises the steps: constructing a standardized view set containing an in-block view for maintaining the texture continuity in UV blocks and a cross-seam view for maintaining the texture continuity between UV blocks; rendering the three-dimensional model at each view angle to obtain a low-resolution image and a current quality image, generating an initial mask through a quality image competition mechanism, and performing regularization processing to obtain a dynamic update mask; inputting the low-resolution image and the dynamic updating mask into a pre-trained local super-division model to execute local super-division to obtain a high-resolution image; and reversely projecting the high-resolution image to the current texture image, carrying out iterative execution until texture image fusion of all visual angles is completed, and outputting a final three-dimensional model high-resolution texture image. According to the method, the high-quality, globally consistent and seamless high-resolution three-dimensional model texture can be generated, and the method is suitable for application scenes with strict requirements on the visual quality of the three-dimensional model.

Description

Technical Field

[0001] This invention belongs to the field of computer graphics technology, specifically relating to a three-dimensional texture super-resolution method and system based on multi-view fusion. Background Technology

[0002] With the rapid development of the digital content industry, high-quality 3D digital assets are increasingly widely used in film, games, virtual reality, and other fields. Texture mapping, as a key element determining the visual realism and detail of 3D models, directly affects the final rendering effect through its resolution and quality. However, in practical applications, due to storage limitations, historical legacy issues, or production costs, many 3D assets come with low-resolution texture maps, severely restricting their application value in modern high-definition rendering environments. Therefore, super-resolution of low-resolution textures has become an important technical step in the asset reuse and quality improvement process.

[0003] Currently, image super-resolution technology has made significant progress, especially in the field of deep learning. Mainstream techniques, such as methods based on generative adversarial networks (GANs) or diffusion models, have demonstrated powerful capabilities in processing two-dimensional (2D) natural images, generating clear and detailed high-resolution images. However, existing techniques suffer from some insurmountable technical limitations when applied to 3D model texture super-resolution.

[0004] The lack of 3D topology awareness leads to UV seam artifacts. Existing 2D super-resolution models treat texture maps as isolated planar images. They cannot understand the surface topology of the corresponding 3D model behind the texture map. When a continuous surface in 3D space is segmented into different UV tiles during UV unwrapping, UV seams are formed. When 2D models process the edges of these seams, due to the lack of contextual information from the opposite side, obvious discontinuities, misalignments, or truncated patterns are produced. When the texture is re-rendered onto the 3D model, these discontinuities manifest as obvious visual artifacts.

[0005] Ignoring UV distortion leads to errors in detail generation. To tile a 3D surface onto a 2D plane, the UV unwrapping process typically introduces varying degrees of stretching or compression. This means that in a texture map, pixels in different regions may correspond to areas of significantly different sizes on the 3D surface. Existing 2D super-resolution methods treat all pixels uniformly, ignoring this distortion. Applying super-resolution to a region that is severely stretched in UV space can result in overly sharp or unnatural texture details on the 3D model; conversely, processing compressed regions can lead to insufficient detail.

[0006] In summary, existing 2D methods are insufficient to meet the specific requirements of 3D textures. Therefore, overcoming the limitations of traditional 2D methods and developing a super-resolution technique that can perceive 3D geometric and topological information and generate globally consistent, seamless, and high-quality 3D model textures is a pressing technical challenge in this field. Summary of the Invention

[0007] In view of the above, the purpose of this invention is to provide a method and system for super-resolution of three-dimensional textures based on multi-view fusion. By sampling three-dimensional surfaces from different perspectives and fusing multi-view texture information, it is possible to generate high-quality, globally consistent and seamless high-resolution three-dimensional model textures, which are suitable for application scenarios with stringent requirements for the visual quality of three-dimensional models, such as film production, high-end game development, virtual reality and digital cultural relic protection.

[0008] To achieve the above-mentioned objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides a three-dimensional texture super-resolution method based on multi-view fusion, comprising the following steps: Based on the geometry and UV partitioning of the 3D model, the intra-block viewpoint and the cross-slit viewpoint used to maintain the texture continuity within UV tiles are determined, forming a normalized viewpoint set. The 3D model is rendered in each viewpoint to obtain a low-resolution image and the current quality map. An initial mask is generated through a quality map competition mechanism and then normalized to obtain a dynamically updated mask. The low-resolution image and dynamically updated mask are input into a pre-trained local super-resolution model, and local super-resolution is performed within the masked region to obtain a high-resolution image. The high-resolution image is back-projected onto the current texture map, and this process is repeated iteratively until all selected viewpoints have completed texture map fusion. Finally, the high-resolution texture map of the 3D model is output.

[0009] Preferably, the step of determining the intra-block viewpoint for maintaining texture continuity within UV tiles and the cross-slit viewpoint for maintaining texture continuity between UV tiles based on 3D model geometry and UV partitioning includes: The 3D model is divided into patches to identify continuous UV tiles in UV space; for each UV tile, the area-weighted average direction of the normals of all the patches within it is used as the viewing direction of the viewpoint within the tile, with the viewpoint facing the geometric center of the UV tile. Identify geometrically continuous but UV-discontinuous seam edges, and define seam regions containing the two side panels based on adjacent seam edges; for each seam region, use the area-weighted average direction of the normals of all the side panels within it as the viewing direction of the cross-seam perspective, with the observation point facing the geometric center of the seam region.

[0010] Preferably, the generation of the initial mask through the quality graph competition mechanism includes: The pixel region covered by the current quality map is back-projected to the pre-initialized and iteratively updated global quality map to obtain the historical best quality map of the corresponding region. The quality value is compared pixel by pixel with the current quality image and the historical best quality image. If the current quality value is higher than the historical best quality value, the pixel is marked as an area to be updated in order to generate an initial mask.

[0011] Preferably, the dynamic update mask obtained through normalization processing includes: Construct a quadtree that covers the initial mask, with the root node corresponding to the complete initial mask region. By recursively dividing the quadtree, the mask is dynamically updated by the corresponding regions of all terminal nodes.

[0012] Preferably, the recursive partitioning of the quadtree follows these rules: For the current node, if the pixel values ​​within the node are consistent, then the division of the node is stopped; if the pixel values ​​within the node are inconsistent, then it is determined whether the size of the node has reached the preset minimum pixel block threshold. If the threshold is reached, then all pixels within the coverage area of ​​the node are forcibly marked as areas to be updated; if the threshold is not reached, then the division continues.

[0013] Preferably, the step of inputting a low-resolution image and a dynamically updated mask into a pre-trained local super-resolution model, and performing local super-resolution within the masked region to obtain a high-resolution image, includes: Low-resolution images are encoded to obtain low-resolution implicit variables, which are then concatenated with the downsampled dynamically updated mask along the channel dimension and input into the local super-resolution model. High-resolution latent variables are predicted by a local super-resolution model, and high-resolution images within the masked region are obtained by decoding the high-resolution latent variables.

[0014] Preferably, the step of back-projecting the high-resolution image onto the current texture map includes: The high-resolution image is back-projected back to the corresponding texture pixel position on the surface of the 3D model to update the high-resolution texture map; at the same time, the historical best quality value of the corresponding region in the global quality map is updated synchronously using the quality map of the current view.

[0015] Secondly, embodiments of the present invention also provide a three-dimensional texture super-resolution system based on multi-view fusion, which is implemented using the above-mentioned three-dimensional texture super-resolution method based on multi-view fusion, including: a normalized view construction module, a dynamically updated mask generation module, a mask conditional local super-resolution module, and a multi-view texture fusion module. The standardized perspective construction module is used to determine, based on the geometry and UV partitioning of the 3D model, the intra-block perspective for maintaining the texture continuity within UV tiles and the cross-slit perspective for maintaining the texture continuity between UV tiles, thus forming a standardized perspective set. The dynamic update mask generation module is used to render the 3D model in each viewpoint, obtain a low-resolution image and the current quality map, generate an initial mask through a quality map competition mechanism, and obtain a dynamic update mask through normalization processing. The mask conditional local super-resolution module is used to input a low-resolution image and a dynamically updated mask into a pre-trained local super-resolution model, and perform local super-resolution within the masked region to obtain a high-resolution image. The multi-view texture fusion module is used to back-project the high-resolution image onto the current texture map, and iterates until all selected views have completed texture map fusion, and then outputs the final high-resolution texture map of the three-dimensional model.

[0016] Thirdly, embodiments of the present invention also provide an electronic device, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-described method for three-dimensional texture super-resolution based on multi-view fusion when executing the computer program.

[0017] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-described method for three-dimensional texture super-resolution based on multi-view fusion.

[0018] Compared with the prior art, the beneficial effects of the present invention include at least the following: (1) Global consistency and seamless effect. This invention renders by determining a normalized view that includes both intra-block and cross-seam view, and processes texture super-resolution from a three-dimensional view space rather than a two-dimensional texture space. This fundamentally solves the problem of seam artifacts caused by UV discontinuity, and ensures the global consistency and seamless connection of the final texture on the three-dimensional model.

[0019] (2) High-fidelity detail generation. This invention renders a local view with minimal distortion from a normalized perspective and uses a local super-resolution model to generate details that seamlessly connect with the surrounding textures. This avoids the problems caused by directly processing UV maps with stretching and compression deformation, and can generate more accurate texture details that are more consistent with the three-dimensional surface structure.

[0020] (3) Quality-driven stable super-resolution. This invention ensures that the texture quality is monotonically improved in each iteration through a global quality map and a dynamic mask generation mechanism, avoiding the use of low-quality results to cover existing high-quality areas, and ensuring that the content generated in the local super-resolution area is seamlessly connected with the surrounding texture, thereby ensuring the stability, consistency and high quality of the entire super-resolution process and the final result.

[0021] (4) Improve the stability of local super-resolution. This invention uses a quadtree to normalize the dynamic mask, which can filter out the trivial noise at the edge of the update region and form a continuous and large super-resolution target. This provides a more stable and clearer guide for the subsequent local super-resolution model, enabling it to focus more effectively on the generation of details in the core region, thereby further improving the effect and stability of super-resolution. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a three-dimensional texture super-resolution method based on multi-view fusion provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the framework of a three-dimensional texture super-resolution method based on multi-view fusion provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the local super-resolution model provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the three-dimensional model texture super-resolution result provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a three-dimensional texture super-resolution system based on multi-view fusion provided in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0025] The inventive concept of this invention is as follows: Addressing the technical problem of poor visual quality in existing 3D digital assets due to low texture resolution, this invention provides a 3D texture super-resolution method and system based on multi-view fusion. By constructing a normalized set of views including intra-block views and cross-slit views, the 2D texture super-resolution problem is transformed into a 3D multi-view fusion problem. A quality graph competition mechanism is used to dynamically filter regions to be optimized, ensuring that texture quality is monotonically improved in each iteration. Simultaneously, by combining quadtree normalized masks and local super-resolution models, high-fidelity details that seamlessly blend with surrounding textures are directly generated on the surface of the 3D model. Finally, through iterative back-projection fusion of multi-view information, a globally consistent and seamlessly connected high-quality 3D texture reconstruction is achieved.

[0026] like Figure 1 As shown in the figure, the embodiment provides a three-dimensional texture super-resolution method based on multi-view fusion, including the following steps: S1, based on the geometry and UV partitioning of the 3D model, determines the intra-block viewpoint and the cross-slit viewpoint used to maintain the texture continuity within UV tiles, thus forming a normalized viewpoint set.

[0027] In the embodiments, reference is made to Figure 2 In step (a), the aim is to establish the optimal observation system from three-dimensional space rather than two-dimensional texture space. Based on the geometric information of the three-dimensional model and the UV spatial coordinate information, a standardized set of views covering the interior of UV tiles and UV seams is constructed. This fundamentally solves the UV seam artifact problem caused by the lack of three-dimensional topological perception in traditional two-dimensional super-resolution methods, laying the perspective foundation for subsequent multi-view fusion. Specifically, it includes the following sub-steps.

[0028] S1.1, Initialization.

[0029] Specifically, the first step is to obtain a 3D model to be processed. The model contains geometric information (such as vertices and faces) and corresponding low-resolution texture maps. Based on this low-resolution texture map An initial target resolution texture map is obtained by upsampling using a standard image interpolation algorithm. At the same time, initialize a... Global quality maps of the same size All pixel values ​​are initialized to zero. This global quality map is used to record the best rendering quality achieved for each texture pixel in subsequent iterations to determine whether the pixel needs to be updated from the current viewpoint.

[0030] S1.2, Construction from a Standardized Perspective.

[0031] Specifically, in order to perform super-resolution of textures in 3D space and avoid UV seam issues, it is necessary to perform super-resolution of textures in 3D space. Determine a fixed, optimal set of rendering perspectives.

[0032] First, the 3D model is divided into patches. An adjacency graph based on UV vertex sharing is constructed, and a graph traversal algorithm is used to assign all patches to their respective UV tiles, thus identifying all contiguous tiles in UV space. For each identified UV tile, its normalized intra-tile viewpoint is calculated. The viewing direction of this viewpoint is determined by the area-weighted average normal of all patch normals within the tile, with the viewpoint located along this normal direction and pointing towards the geometric center of the tile. This ensures minimal perspective distortion in the rendering of each tile.

[0033] Next, all geometrically continuous but discontinuous edges in UV space on the 3D model are identified, i.e., UV seams. Adjacent seam edges are grouped to form seam bands. For each seam band, a seam region is defined, encompassing the adjacent faces on both sides. A cross-seam view is calculated for this region, using the same method as the intra-block view. The area-weighted average normal of all face normals in this region is calculated, with the observation point located in the direction of this normal and facing the geometric center of the region. The aim is to observe the surfaces on both sides of the seam simultaneously with minimal distortion.

[0034] Finally, the normalized view set is composed of the intra-block view and the cross-seam view.

[0035] S2 renders the 3D model from each viewpoint, obtaining a low-resolution image and the current quality map. An initial mask is generated through a quality map competition mechanism, and a dynamically updated mask is obtained after normalization processing.

[0036] In the embodiments, reference is made to Figure 2 In step (b), the aim is to dynamically select the regions that need texture updates from the current perspective through a quality-driven competition mechanism, and to introduce a quadtree data structure to normalize the fragmented regions at the mask edges, forming a regular and continuous final dynamically updated mask, providing stable guidance for subsequent super-resolution. Specifically, it includes the following sub-steps.

[0037] S2.1, Initial mask generation.

[0038] Specifically, in the first In the next iteration, based on the current processing perspective and texture map Rendering 3D models This yields a low-resolution image from that viewpoint. Simultaneously, calculate the quality map of this view. The quality value of each pixel is determined by the dot product (cosine similarity) of the normal to the surface point of the corresponding 3D model and the camera's viewing direction, as well as the distance from the surface point to the camera. Specifically, the quality value = dot product value / (distance from the surface point to the camera). 2 Then, an initial mask is generated through a competition mechanism based on the quality graph. First, the pixel area covered by the current view is back-projected onto the global quality map. Above, the corresponding historical best quality map is obtained by sampling. Then, compare pixel by pixel. and The value, if The quality of a certain pixel is higher than The quality of the corresponding pixel in the middle, then in The corresponding position is set to 1, indicating that the area needs to be updated; otherwise, it is set to 0.

[0039] S2.2, Mask normalization processing.

[0040] Specifically, pixel-by-pixel comparisons may lead to an initial mask The edge regions contain numerous fragmented and discontinuous pixels, which can interfere with the stability of subsequent local super-resolution models. To address this issue, a quadtree is used... Normalization is performed. First, a root node covering the entire mask area is constructed, and a recursive partitioning check is performed. For the current quadtree node (the pixel block of the mask area), the internal pixel values ​​are checked for consistency. If all pixel values ​​inside the node are 1 or all are 0, the partitioning of the node is stopped, and the current state of the area is retained, which means that the area either needs to be updated or does not need to be updated at all. If the internal pixels of the node contain both 1 and 0 (i.e., it is in the edge blending area), it is further determined whether the size of the node has reached the preset minimum pixel block threshold (e.g., 4×4 pixels). If the minimum threshold has been reached, all pixels within the area covered by the node are forcibly set to 1, and the partitioning is stopped. The fragmented unselected pixels (0) at the edges are merged into the update area, thus forming a regular rectangular update block. If the minimum threshold has not been reached, the node is divided into four child nodes, and the above check steps are repeated for each child node. Finally, the corresponding areas of all terminating nodes constitute the normalized dynamically updated mask. .

[0041] S3 takes a low-resolution image and a dynamically updated mask as input to a pre-trained local super-resolution model, performs local super-resolution within the masked region, and obtains a high-resolution image.

[0042] In the embodiments, reference is made to Figure 2Step (c) aims to use a mask-conditional local super-resolution model to generate high-frequency details that seamlessly blend with the surrounding environment within the region specified by dynamically updated masks. This ensures texture consistency during multi-view super-resolution and avoids texture discontinuities and boundary artifacts between viewpoints caused by independently processing different viewpoints, thus achieving coherent super-resolution of 3D surface textures. Specifically, it includes the following sub-steps.

[0043] S3.1, Local super-resolution model pre-training.

[0044] Specifically, the Stable Diffusion 2.1 (SD) model is used as the local super-resolution model to be specifically adapted for super-resolution tasks that perform single-step, masked conditions, and is pre-trained.

[0045] (1) Training data preparation Training is performed using high-quality image datasets containing diverse content (e.g., the LSDIR and FFHQ datasets). The training data is constructed as follows: For each high-resolution baseline image (HR) in the dataset, firstly, a corresponding low-resolution image is generated using a pre-defined image degradation algorithm (Real-ESRGAN degradation algorithm); secondly, a binary mask is randomly generated; finally, the region specified by the mask in the high-resolution baseline image is replaced with the corresponding region in the low-resolution image, thereby generating a locally low-resolution training input image (LR) with surrounding high-resolution elements. This LR image, together with the original HR image, constitutes a training data pair.

[0046] (2) Forward propagation and residual prediction like Figure 3 In each training iteration, a training data pair and its corresponding mask are selected. The LR image is then passed through a VAE encoder with frozen parameters to obtain its initial low-resolution implicit variables. Mask downsampling and The input is concatenated along the channel dimension to form a 5-channel tensor. This tensor is then fed into the U-Net backbone of the diffusion model. Since the default input channels for the diffusion model are 4, the input convolutional layers of the U-Net are modified to accept 5-channel input. To adapt the model to mask-conditional single-step super-resolution tasks, the U-Net backbone integrates two trainable low-rank adaptation (LoRA) modules: "Pixel and Semantic LoRA" and "Mask LoRA". Guided by these LoRA modules, U-Net processes the input 5-channel tensor and predicts a latent variable residual in a single step. .

[0047] (3) Loss calculation and backpropagation The predicted residual From the initial implicit variable Subtracting from the middle yields the final high-resolution implicit variable. The resulting image is then decoded into a final high-resolution (HR) image using a VAE decoder with frozen parameters. The generated HR image is compared to a baseline HR image from the training data pair, and a hybrid loss function is computed. This loss function includes mean squared error loss (MSE Loss) and perceptual loss (LPIPS Loss) computed within the masked region. Finally, based on the computed loss values, the trainable parameters of only the input convolutional layers of the U-Net and the two LoRA modules are updated via backpropagation, while the remaining parameters of the pedestal diffusion model and the VAE remain frozen throughout the training process.

[0048] By repeating the above training steps, a high-efficiency and high-quality local super-resolution model specifically designed for the method flow of this invention can be obtained.

[0049] S3.2, Local Super-Digital Reasoning.

[0050] Specifically, this step utilizes a pre-trained local super-resolution model to perform super-resolution processing on the rendered view. The inference process of this model is as follows: First, the rendered low-resolution view image is processed... Transform the frozen VAE encoder into its low-resolution implicit variables. Next, the dynamically updated mask generated in the previous step is... After preprocessing such as downsampling, and The input tensor is concatenated along the channel dimension to form a 5-channel input tensor. This 5-channel tensor is then fed into the U-Net network. The U-Net has already incorporated pre-trained parameters for both pixel-and-semantic LoRA and mask LoRA. The U-Net performs a single forward propagation on the input, predicting the latent variable residuals used for updates in a single step. Finally, according to the formula The high-resolution implicit variables are calculated and decoded through a frozen VAE decoder to output the final high-resolution image. The image contains rich high-frequency details within the masked area and blends seamlessly with the surrounding area.

[0051] S4 projects the high-resolution image back onto the current texture map, iterating until all selected viewpoints have completed texture map fusion, and then outputs the final high-resolution texture map of the 3D model.

[0052] In the embodiments, reference is made to Figure 2Step (d) aims to iteratively render local views from a selected perspective, generate normalized masks, perform local super-resolution, and back-project the super-resolution details back to the high-resolution texture map while updating the global quality map. This ensures a monotonic improvement in texture quality with each iteration, ultimately outputting a globally consistent, seamless, and highly detailed 3D model texture. Specifically, it includes the following sub-steps.

[0053] S4.1, Local texture map update.

[0054] Specifically, in the obtained high-resolution image The mask is dynamically updated. The specified area is updated with a high-resolution texture map via back projection. The updated texture map is obtained. At the same time, use the quality map of the current view. Update the global quality graph The values ​​in the corresponding areas ensure Always record the best quality in history.

[0055] S4.2, Iterative Fusion and Output.

[0056] Specifically, the iterative process of steps S2.1, S2.2, S3.2, and S4.1 is repeated until all selected perspectives in the normalized perspective set have participated in the iterative process. After the loop terminates, the final result is... This is the final high-resolution texture map output, such as Figure 4 As shown, the comparison between the original low-resolution texture (LR), ground truth (GT), and the final generated high-resolution texture (HR) is presented, and zoom-in comparisons of the three types of images are shown with different colored boxes. It can be seen that the method of the present invention effectively eliminates UV seam artifacts and generates clear and natural texture details on the surface of the 3D model.

[0057] In summary, the multi-view fusion-based 3D texture super-resolution method provided by this invention achieves high-quality super-resolution reconstruction of low-resolution 3D textures through 3D perception viewpoint selection, quality-driven dynamic mask generation, mask conditional local super-resolution, and multi-view iterative texture fusion, thereby generating globally consistent and detail-rich 3D model textures.

[0058] Based on the same inventive concept, such as Figure 5 As shown, this embodiment of the invention also provides a three-dimensional texture super-resolution system 500 based on multi-view fusion, including: a normalized view construction module 510, a dynamically updated mask generation module 520, a mask conditional local super-resolution module 530, and a multi-view texture fusion module 540.

[0059] The normalized perspective construction module 510 is used to determine the intra-block perspective and the cross-slit perspective to maintain the texture continuity within UV tiles and the texture continuity between UV tiles based on the geometry and UV partitioning of the 3D model, thus forming a normalized perspective set.

[0060] The dynamic update mask generation module 520 is used to render the 3D model in each viewpoint, obtain a low-resolution image and the current quality map, generate an initial mask through a quality map competition mechanism, and obtain a dynamic update mask through normalization processing.

[0061] The mask-conditional local super-resolution module 530 is used to input a low-resolution image and a dynamically updated mask into a pre-trained local super-resolution model, and perform local super-resolution within the masked region to obtain a high-resolution image.

[0062] The multi-view texture fusion module 540 is used to back-project a high-resolution image onto the current texture map, iterates until all selected views have completed texture map fusion, and outputs the final high-resolution texture map of the 3D model.

[0063] Based on the same inventive concept, embodiments of the present invention also provide an electronic device, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-described method for three-dimensional texture super-resolution based on multi-view fusion when executing the computer program.

[0064] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-described method for three-dimensional texture super-resolution based on multi-view fusion.

[0065] It should be noted that the three-dimensional texture super-resolution system, electronic device, and computer-readable storage medium based on multi-view fusion provided in the above embodiments all belong to the same inventive concept as the three-dimensional texture super-resolution method based on multi-view fusion. For details of their specific implementation process, please refer to the embodiments of the three-dimensional texture super-resolution method based on multi-view fusion, which will not be repeated here.

[0066] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A three-dimensional texture super-resolution method based on multi-view fusion, characterized in that, Includes the following steps: Based on the geometry and UV partitioning of the 3D model, the intra-block viewpoint and the cross-slit viewpoint used to maintain the texture continuity within UV tiles are determined, forming a normalized viewpoint set. The 3D model is rendered in each viewpoint to obtain a low-resolution image and the current quality map. An initial mask is generated through a quality map competition mechanism and then normalized to obtain a dynamically updated mask. The low-resolution image and dynamically updated mask are input into a pre-trained local super-resolution model, and local super-resolution is performed within the masked region to obtain a high-resolution image. The high-resolution image is back-projected onto the current texture map, and this process is repeated iteratively until all selected viewpoints have completed texture map fusion. Finally, the high-resolution texture map of the 3D model is output.

2. The 3D texture super-resolution method based on multi-view fusion according to claim 1, characterized in that, The method of determining the intra-block viewpoint for maintaining texture continuity within UV tiles and the cross-slit viewpoint for maintaining texture continuity between UV tiles based on 3D model geometry and UV partitioning includes: The 3D model is divided into patches to identify continuous UV tiles in UV space; for each UV tile, the area-weighted average direction of the normals of all the patches within it is used as the viewing direction of the viewpoint within the tile, with the viewpoint facing the geometric center of the UV tile. Identify geometrically continuous but UV-discontinuous seam edges, and define seam regions containing the two side panels based on adjacent seam edges; for each seam region, use the area-weighted average direction of the normals of all the side panels within it as the viewing direction of the cross-seam perspective, with the observation point facing the geometric center of the seam region.

3. The 3D texture super-resolution method based on multi-view fusion according to claim 1, characterized in that, The generation of the initial mask through the quality graph competition mechanism includes: The pixel region covered by the current quality map is back-projected to the pre-initialized and iteratively updated global quality map to obtain the historical best quality map of the corresponding region. The quality value is compared pixel by pixel with the current quality image and the historical best quality image. If the current quality value is higher than the historical best quality value, the pixel is marked as an area to be updated in order to generate an initial mask.

4. The 3D texture super-resolution method based on multi-view fusion according to claim 1 or 3, characterized in that, The dynamic update mask obtained through normalization includes: Construct a quadtree that covers the initial mask, with the root node corresponding to the complete initial mask region. By recursively dividing the quadtree, the mask is dynamically updated by the corresponding regions of all terminal nodes.

5. The 3D texture super-resolution method based on multi-view fusion according to claim 4, characterized in that, The recursive partitioning of a quadtree follows these rules: For the current node, if the pixel values ​​within the node are consistent, then the division of the node is stopped; if the pixel values ​​within the node are inconsistent, then it is determined whether the size of the node has reached the preset minimum pixel block threshold. If the threshold is reached, then all pixels within the coverage area of ​​the node are forcibly marked as areas to be updated; if the threshold is not reached, then the division continues.

6. The 3D texture super-resolution method based on multi-view fusion according to claim 1, characterized in that, The process of inputting a low-resolution image and a dynamically updated mask into a pre-trained local super-resolution model, and performing local super-resolution within the masked region to obtain a high-resolution image includes: Low-resolution images are encoded to obtain low-resolution implicit variables, which are then concatenated with the downsampled dynamically updated mask along the channel dimension and input into the local super-resolution model. High-resolution latent variables are predicted by a local super-resolution model, and high-resolution images within the masked region are obtained by decoding the high-resolution latent variables.

7. The three-dimensional texture super-resolution method based on multi-view fusion according to claim 1 or 3, characterized in that, The step of back-projecting the high-resolution image onto the current texture map includes: The high-resolution image is back-projected back to the corresponding texture pixel position on the surface of the 3D model to update the high-resolution texture map; at the same time, the historical best quality value of the corresponding region in the global quality map is updated synchronously using the quality map of the current view.

8. A three-dimensional texture super-resolution system based on multi-view fusion, implemented using the three-dimensional texture super-resolution method based on multi-view fusion as described in any one of claims 1 to 7, characterized in that, include: The module includes a standardized view construction module, a dynamically updated mask generation module, a mask conditional local super-resolution module, and a multi-view texture fusion module. The standardized perspective construction module is used to determine, based on the geometry and UV partitioning of the 3D model, the intra-block perspective for maintaining the texture continuity within UV tiles and the cross-slit perspective for maintaining the texture continuity between UV tiles, thus forming a standardized perspective set. The dynamic update mask generation module is used to render the 3D model in each viewpoint, obtain a low-resolution image and the current quality map, generate an initial mask through a quality map competition mechanism, and obtain a dynamic update mask through normalization processing. The mask conditional local super-resolution module is used to input a low-resolution image and a dynamically updated mask into a pre-trained local super-resolution model, and perform local super-resolution within the masked region to obtain a high-resolution image. The multi-view texture fusion module is used to back-project the high-resolution image onto the current texture map, and iterates until all selected views have completed texture map fusion, and then outputs the final high-resolution texture map of the three-dimensional model.

9. An electronic device comprising a memory and one or more processors, the memory for storing a computer program, characterized in that, The processor is used to implement the three-dimensional texture super-resolution method based on multi-view fusion as described in any one of claims 1 to 7 when executing a computer program.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a computer, it implements the three-dimensional texture super-resolution method based on multi-view fusion as described in any one of claims 1 to 7.

Images