Texture map generation method and device, electronic equipment and storage medium

Abstract

Embodiments of the invention provide a texture map generation method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining a three-dimensional model of a to-be-generated texture map and each image for creating the three-dimensional model; determining a visible surface of the three-dimensional model relative to each view angle and an invisible surface of the three-dimensional model relative to all view angles; according to the pixel values of the pixel points in each image and the weights of the pixel points, determining the texture value of the visual surface of each visual angle and mapping the texture value to a first partition of the texture map; wherein the weight is the weight of the texture contributed by the pixel points in the image to the three-dimensional model; the texture map is provided with first partitions which are in one-to-one correspondence with the visual angles and are fixed in position; determining the texture value of the invisible surface according to the texture value of the visible surface adjacent to the invisible surface in the three-dimensional model, and mapping the texture value to a second partition of the texture map; wherein the texture map is provided with a second partition which corresponds to the invisible surface and is fixed in position.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a texture map generation method, device, electronic device and storage medium. Background Art

[0002] Volumetric video is a technology that can capture information in three-dimensional space (such as depth information and color information) and generate a sequence of three-dimensional models. These three-dimensional models can be connected to form a new video format that can be viewed from any perspective.

[0003] In the scenario of real-time transmission of volumetric video, it is necessary to transmit each frame of the 3D model and the texture map of each frame of the 3D model in the volumetric video. However, the current technology for generating texture maps for 3D models in real-time transmitted volumetric video has the problems of low compression rate and poor map quality in the time domain, which increases the bandwidth occupied by data transmission and reduces the accuracy of the 3D model in the volumetric video. Summary of the invention

[0004] The embodiments of the present disclosure provide a texture map generation method, device, electronic device and storage medium, which can improve the compression rate of texture maps in the time domain and the image quality of texture maps in the scenario of real-time transmission of volumetric video, thereby reducing the bandwidth occupied by data transmission and improving the accuracy of three-dimensional models in volumetric video.

[0005] In a first aspect, an embodiment of the present disclosure provides a texture map generation method, comprising:

[0006] Acquire a three-dimensional model for which a texture map is to be generated and various images used to create the three-dimensional model;

[0007] Determining a visible surface of the three-dimensional model relative to each viewing angle and an invisible surface of the three-dimensional model relative to all the viewing angles;

[0008] According to the pixel values ​​and weights of the pixels in each of the images, the texture value of the visible surface of each of the viewing angles is determined and mapped to the first partition of the texture map; wherein the weight is the weight of the pixel in the image contributing to the texture of the three-dimensional model; the texture map has a first partition corresponding to each of the viewing angles and having a fixed position;

[0009] According to the texture value of the visible surface adjacent to the invisible surface in the three-dimensional model, the texture value of the invisible surface is determined and mapped to the second partition of the texture map; wherein the texture map has a second partition corresponding to the invisible surface and fixed in position.

[0010] In a second aspect, an embodiment of the present disclosure provides a texture map generating device, comprising:

[0011] An acquisition unit, used for acquiring a three-dimensional model for which a texture map is to be generated and each image used to create the three-dimensional model;

[0012] a determining unit, configured to determine a visible surface of the three-dimensional model relative to each viewing angle and an invisible surface of the three-dimensional model relative to all the viewing angles;

[0013] A first mapping unit, configured to determine the texture value of the visible surface of each of the viewing angles according to the pixel values ​​and weights of the pixels in each of the images, and map the texture value to a first partition of the texture map; wherein the weight is the weight of the pixel in the image contributing to the texture of the three-dimensional model; and the texture map has a first partition corresponding to each of the viewing angles and having a fixed position;

[0014] A second mapping unit is used to determine the texture value of the invisible surface according to the texture value of the visible surface adjacent to the invisible surface in the three-dimensional model and map it to a second partition of the texture map; wherein the texture map has a second partition corresponding to the invisible surface and with a fixed position.

[0015] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, enable the processor to implement the steps of the method described in the first aspect above.

[0016] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions implement the steps of the method described in the first aspect when executed by a processor.

[0017] In one or more embodiments of the present disclosure, firstly, a three-dimensional model for which a texture map is to be generated and various images used to create the three-dimensional model are obtained, then the visible surface of the three-dimensional model relative to each viewing angle and the invisible surface of the three-dimensional model relative to all viewing angles are determined, and then the texture value of the visible surface of each viewing angle is determined and mapped to the first partition of the texture map according to the pixel value and weight of the pixel points in each image, and the texture value of the invisible surface is determined and mapped to the second partition of the texture map according to the texture value of the visible surface adjacent to the invisible surface in the three-dimensional model. The weight is the weight of the pixel point in the image contributing to the texture of the three-dimensional model, and the texture map has a first partition corresponding to each viewing angle and fixed in position, and a second partition corresponding to the invisible surface and fixed in position. It can be seen that when this embodiment is applied to the scene of real-time transmission of volumetric video, on the one hand, since the texture value of the visible surface of each perspective can be determined according to the pixel value and the weight of the pixel in each image, and the texture value of the invisible surface can be determined according to the texture value of the visible surface adjacent to the invisible surface in the three-dimensional model, the accuracy of determining the texture value is improved based on the weight of the pixel and the adjacency relationship between the visible surface and the invisible surface, thereby improving the image quality of the texture map and the accuracy of the three-dimensional model in the volumetric video. On the other hand, since the texture map has a first partition corresponding to each perspective and a fixed position and a second partition corresponding to the invisible surface and a fixed position, when the textures of multiple three-dimensional models in multiple frames of volumetric video with adjacent timestamps are similar, the image contents of multiple texture maps corresponding to the multiple three-dimensional models will also be similar, thereby improving the compression rate of the texture map in the time domain and reducing the bandwidth occupied by data transmission. In addition, the process of generating texture maps in this embodiment is highly efficient and can meet the real-time requirements for generating texture maps in the scene of real-time transmission of volumetric video. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate one or more embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

[0019] Figure 1 A schematic diagram of a process for generating a texture map according to an embodiment of the present disclosure;

[0020] Figure 2a A schematic diagram of a partition of a texture map provided by an embodiment of the present disclosure;

[0021] Figure 2bA schematic diagram of a partition of a texture map provided by another embodiment of the present disclosure;

[0022] Figure 3 A schematic diagram of the structure of a texture map generating device provided in one embodiment of the present disclosure;

[0023] Figure 4 A schematic diagram of the structure of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of the present disclosure, the technical solutions in one or more embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in one or more embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on one or more embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the protection scope of the present disclosure.

[0025] An embodiment of the present disclosure provides a texture map generation method, which can improve the compression rate of texture maps in the time domain and the image quality of texture maps in the scenario of real-time transmission of volumetric video, thereby reducing the bandwidth occupied by data transmission and improving the accuracy of three-dimensional models in volumetric video. The method can be applied on a server and executed by the server.

[0026] To facilitate understanding of this embodiment, the volumetric video is first introduced here. The volumetric video includes multiple video frames, each of which has a 3D model processed by mapping, and the 3D models processed by mapping in each video frame constitute a 3D model sequence. The process of playing the volumetric video can be understood as the process of playing the 3D model sequence.

[0027] The three-dimensional model in the volumetric video is a model obtained by modeling a specific object, such as a person, and the specific object moves over time. First, multiple cameras are arranged at different positions around the specific object. During the movement of the specific object, the specific object is photographed at each moment by multiple cameras. Each camera obtains an image of the specific object from its position at each moment, which is equivalent to using multiple cameras to record the movement process of the specific object from different positions. At each moment, a three-dimensional model is established for the specific object based on the multiple images obtained by the multiple cameras at that moment. The three-dimensional model can be understood as the model of the specific object at that moment. The three-dimensional model has not been mapped and does not have texture information. After the three-dimensional model is mapped, the three-dimensional model with texture of the specific object at each moment constitutes a three-dimensional model sequence, and the volumetric video can be generated using the three-dimensional model sequence.

[0028] The scene of real-time transmission of volumetric video may include a live broadcast scene of volumetric video. For example, a user watches a live broadcast of volumetric video through a terminal device, and the terminal device receives multiple three-dimensional models and texture maps corresponding to each three-dimensional model sent by a server, generates each three-dimensional model with texture based on each three-dimensional model and the corresponding texture map, and then renders and displays the video screen of each frame of volumetric video, and each frame of volumetric video includes a three-dimensional model with texture. Among them, the user can watch the live broadcast of volumetric video through a VR (Virtual Reality) device, or watch the live broadcast of volumetric video through a mobile phone, tablet computer, car computer, wearable device, etc.

[0029] Figure 1 A schematic diagram of a process flow of a texture map generation method provided by an embodiment of the present disclosure is shown in FIG. Figure 1 As shown, the method includes:

[0030] Step S102, obtaining a three-dimensional model for which a texture map is to be generated and various images used to create the three-dimensional model;

[0031] Step S104, determining the visible surface of the three-dimensional model relative to each viewing angle and the invisible surface of the three-dimensional model relative to all viewing angles;

[0032] Step S106, determining the texture value of the visible surface of each viewing angle according to the pixel value and the weight of the pixel in each image and mapping it to the first partition of the texture map; wherein the weight is the weight of the pixel in the image contributing to the texture of the three-dimensional model; the texture map has a first partition corresponding to each viewing angle and having a fixed position;

[0033] Step S108, determining the texture value of the invisible surface according to the texture value of the visible surface adjacent to the invisible surface in the three-dimensional model and mapping it to the second partition of the texture map; wherein the texture map has a second partition corresponding to the invisible surface and having a fixed position.

[0034] It can be seen that when this embodiment is applied to the scene of real-time transmission of volumetric video, on the one hand, since the texture value of the visible surface of each viewing angle can be determined according to the pixel value and the weight of the pixel in each image, and the texture value of the invisible surface can be determined according to the texture value of the visible surface adjacent to the invisible surface in the three-dimensional model, the accuracy of determining the texture value is improved based on the weight of the pixel and the adjacency relationship between the visible surface and the invisible surface, thereby improving the image quality of the texture map and the accuracy of the three-dimensional model in the volumetric video. On the other hand, since the texture map has a first partition corresponding to each viewing angle and a fixed position and a second partition corresponding to the invisible surface and a fixed position, when the textures of multiple three-dimensional models in multiple frames of volumetric video with adjacent timestamps are similar, the image contents of multiple texture maps corresponding to the multiple three-dimensional models will also be similar, thereby improving the compression rate of the texture map in the time domain and reducing the bandwidth occupied by data transmission. In addition, the process of generating texture maps in this embodiment is highly efficient and can meet the real-time requirements for generating texture maps in the scene of real-time transmission of volumetric video.

[0035] In the above step S102, a three-dimensional model for which a texture map is to be generated is obtained. The three-dimensional model may be a three-dimensional model in each frame of volumetric video in the volumetric video scene. Each frame of volumetric video includes a three-dimensional model, and each three-dimensional model corresponds to a texture map, and the texture map is used to perform mapping processing on the three-dimensional model. In the live broadcast scene of the volumetric video, the user watches the live broadcast of the volumetric video through a terminal device, and the terminal device receives multiple three-dimensional models and texture maps corresponding to each three-dimensional model sent by the server, and generates each three-dimensional model with texture based on each three-dimensional model and the corresponding texture map, and then renders and displays the video screen of each frame of the volumetric video, and the video screen of each frame of the volumetric video includes a three-dimensional model with texture.

[0036] In the above step S102, each image used to create the three-dimensional model is obtained. Each three-dimensional model in each frame of volumetric video is created based on multiple images, which can be RGB (Red-Green-Blue) images taken from different perspectives at the same time for the object to be modeled. The three-dimensional model at that moment is established based on the RGB images taken from each perspective at the same time.

[0037] In the above step S104, the visible surface of the three-dimensional model relative to each viewing angle and the invisible surface of the three-dimensional model relative to all viewing angles are determined. The viewing angles here include the viewing angles of each camera used to shoot each image and at least one virtual viewing angle. The virtual viewing angle refers to a viewing angle that is not real but virtual. In one embodiment, determining the visible surface of the three-dimensional model relative to each viewing angle and the invisible surface of the three-dimensional model relative to all viewing angles includes:

[0038] Obtain a depth map corresponding to the three-dimensional model at each viewing angle and a depth value of each point on the three-dimensional model at each viewing angle; the depth map is obtained by rasterizing the three-dimensional model to each viewing angle; the depth value is obtained by projecting the three-dimensional model to each viewing angle;

[0039] According to the depth map corresponding to each viewing angle and the depth values ​​of each point on the 3D model at each viewing angle, the visible surface of the 3D model relative to each viewing angle and the invisible surface of the 3D model relative to all viewing angles are determined.

[0040] First, obtain the viewing parameters of each viewing angle. Each viewing angle is equivalent to a camera. The viewing parameters include the camera's intrinsic parameters and extrinsic parameters. The camera's intrinsic parameters include the camera's optical center and focal length. The camera's extrinsic parameters include the camera's position, rotation matrix, translation matrix, and other parameters. According to the viewing parameters of each viewing angle, rasterize the established 3D model to each viewing angle to obtain the depth map corresponding to the 3D model at each viewing angle.

[0041] Next, according to the viewing parameters of each viewing angle, the established 3D model is projected to each viewing angle to obtain the depth values ​​of all points on the 3D model at each viewing angle. The difference between rasterization and projection is that during rasterization, for each viewing angle, only the points on the 3D model that can be seen by that viewing angle are rasterized to that viewing angle to obtain a depth map. Therefore, the depth map only records the depth values ​​of the points on the 3D model that can be seen by that viewing angle. During projection, all points on the surface of the 3D model are projected to each viewing angle to obtain the depth value of each point at each viewing angle.

[0042] Finally, according to the depth map corresponding to each viewing angle and the depth values ​​of each point on the three-dimensional model at each viewing angle, the visible surface of the three-dimensional model relative to each viewing angle and the invisible surface of the three-dimensional model relative to all viewing angles are determined. The visible surface of the three-dimensional model relative to each viewing angle refers to the surface of the three-dimensional model that can be seen from each viewing angle. One viewing angle corresponds to one visible surface, and there is no overlapping area between the visible surfaces corresponding to different viewing angles. The invisible surface of the three-dimensional model relative to all viewing angles refers to the surface of the three-dimensional model that cannot be seen from any viewing angle.

[0043] It can be seen that through this embodiment, the visible surface of the three-dimensional model relative to each perspective and the invisible surface of the three-dimensional model relative to all perspectives can be accurately determined according to the depth map corresponding to each perspective and the depth values ​​of each point on the three-dimensional model at each perspective, thereby preparing for the subsequent generation of high-quality texture maps.

[0044] In one embodiment, determining a visible surface of the three-dimensional model relative to each viewing angle and an invisible surface of the three-dimensional model relative to all viewing angles according to a depth map corresponding to each viewing angle and depth values ​​of each point on the three-dimensional model at each viewing angle includes:

[0045] Determine, according to the depth map corresponding to each viewing angle and the depth values ​​of each point on the three-dimensional model at each viewing angle, the visible facets of the three-dimensional model relative to each viewing angle and the invisible facets of the three-dimensional model relative to all viewing angles; the surface of the three-dimensional model is composed of at least the facets;

[0046] The visible surface of the 3D model relative to each viewing angle and the invisible surface of the 3D model relative to all viewing angles are determined according to the visible surface of the 3D model relative to each viewing angle and the invisible surface of the 3D model relative to all viewing angles.

[0047] The surface of the three-dimensional model is composed of multiple facets, such as triangular facets, and each triangular facet is composed of three vertices and three edges between the three vertices. In this embodiment, firstly, the visible facets of the three-dimensional model relative to each perspective and the invisible facets of the three-dimensional model relative to all perspectives are determined according to the depth map corresponding to each perspective and the depth values ​​of each point on the three-dimensional model at each perspective. The visible facets of the three-dimensional model relative to each perspective refer to the facets of the three-dimensional model that can be seen for each perspective. One perspective corresponds to one or more visible facets, and the same visible facets may exist in the visible facets corresponding to each perspective. The invisible facets of the three-dimensional model relative to all perspectives refer to the facets of the three-dimensional model that cannot be seen by any perspective.

[0048] Next, the visible surface of the 3D model relative to each viewing angle and the invisible surface of the 3D model relative to all viewing angles are determined according to the visible surface of the 3D model relative to each viewing angle and the invisible surface of the 3D model relative to all viewing angles. For example, for each viewing angle, the visible surface of each viewing angle is used as the visible surface of the viewing angle, and the invisible surface of the 3D model relative to all viewing angles is used as the invisible surface of the 3D model relative to all viewing angles.

[0049] It can be seen that through this embodiment, the visible patches of the three-dimensional model relative to each perspective and the invisible patches of the three-dimensional model relative to all perspectives can be determined according to the depth map corresponding to each perspective and the depth values ​​of each point on the three-dimensional model at each perspective. Based on the visible patches and the invisible patches, the visible surface of the three-dimensional model relative to each perspective and the invisible surface of the three-dimensional model relative to all perspectives are determined, thereby achieving the effect of accurately determining the visible surface and the invisible surface.

[0050] In one embodiment, determining visible patches of the three-dimensional model relative to each viewing angle and invisible patches of the three-dimensional model relative to all viewing angles according to the depth map corresponding to each viewing angle and the depth values ​​of each point on the three-dimensional model at each viewing angle includes:

[0051] For each viewing angle, a patch composed of the first target point in the three-dimensional model is determined as a visible patch of the viewing angle, and a patch including the second target point in the three-dimensional model is determined as an invisible patch of the viewing angle; the difference between the depth value of the first target point at the viewing angle and the depth value of the first target point recorded in the depth map corresponding to the viewing angle meets the preset difference requirement; the difference between the depth value of the second target point at the viewing angle and the depth value of the second target point recorded in the depth map corresponding to the viewing angle does not meet the preset difference requirement;

[0052] The same facets among the invisible facets of each viewing angle are determined as the invisible facets of the three-dimensional model with respect to all viewing angles.

[0053] Specifically, for each viewing angle, the depth values ​​of each point recorded in the depth map corresponding to the viewing angle are compared with the depth values ​​of each point on the three-dimensional model at each viewing angle to obtain the first target point and the second target point corresponding to the viewing angle, and the difference between the depth value of the first target point at the viewing angle and the depth value of the first target point recorded in the depth map corresponding to the viewing angle meets the preset difference requirement, for example, the absolute value of the difference is less than or equal to 2, and the difference between the depth value of the second target point at the viewing angle and the depth value of the second target point recorded in the depth map corresponding to the viewing angle does not meet the preset difference requirement, for example, the absolute value of the difference is greater than 2.

[0054] It can be understood that, since the difference between the depth value of the first target point at the viewing angle and the depth value of the first target point recorded in the depth map corresponding to the viewing angle meets the preset difference requirement, the first target point is a point that can be seen by the viewing angle, and therefore the patch composed of the first target point is also a patch that can be seen by the viewing angle. On the contrary, since the difference between the depth value of the second target point at the viewing angle and the depth value of the second target point recorded in the depth map corresponding to the viewing angle does not meet the preset difference requirement, the second target point is a point that cannot be seen by the viewing angle, and therefore the patch including the second target point is also a patch that cannot be seen by the viewing angle, and the patch including the second target point can be a patch on which one or more points are the second target point, and also a patch that is completely composed of the second target point.

[0055] For each viewing angle, the patch composed of the first target points corresponding to the viewing angle in the three-dimensional model is determined as the visible patch of the viewing angle. The patch composed of the first target points corresponding to the viewing angle means that each point on the patch is the first target point corresponding to the viewing angle. For example, when the patch is a triangular patch, when the three vertices are all the first target points corresponding to the viewing angle, the patch is a visible patch of the viewing angle.

[0056] For each viewing angle, the facet in the three-dimensional model that includes the second target point corresponding to the viewing angle is determined as an invisible facet of the viewing angle. The facet that includes the second target point corresponding to the viewing angle means that the second target point corresponding to the viewing angle exists on the facet. For example, when the facet is a triangular facet, when any one or more of the three vertices is the second target point corresponding to the viewing angle, the facet is an invisible facet of the viewing angle.

[0057] Next, the invisible patches of each viewing angle are determined, and the same patches among the invisible patches of each viewing angle, that is, patches that cannot be seen from any viewing angle, are determined as the invisible patches of the three-dimensional model with respect to all viewing angles.

[0058] In a specific embodiment, after rasterization obtains a depth map corresponding to each perspective, the boundary of the object is determined in the depth map and the boundary is eroded to remove the boundary in the depth map, and based on the depth map after removing the boundary corresponding to each perspective, the visible surface of the three-dimensional model relative to each perspective and the invisible surface of the three-dimensional model relative to all perspectives are determined, thereby determining the visible surface of the three-dimensional model relative to each perspective and the invisible surface of the three-dimensional model relative to all perspectives.

[0059] It can be seen that through this embodiment, the visible patches and invisible patches of each perspective can be accurately identified by comparing the depth values ​​of each point recorded in the depth map corresponding to each perspective with the depth values ​​of each point on the three-dimensional model at each perspective, and then the invisible patches of all perspectives can be determined. The process of determining the visible patches and invisible patches is simple and the determination result is highly accurate.

[0060] In one embodiment, determining the visible surface of the 3D model relative to each viewing angle and the invisible surface of the 3D model relative to all viewing angles according to the visible surface of the 3D model relative to each viewing angle and the invisible surface of the 3D model relative to all viewing angles includes:

[0061] Determining a visible surface of the three-dimensional model relative to each viewing angle according to the adjacency relationship between the facets in the three-dimensional model and the visible facets of the three-dimensional model relative to each viewing angle; the visible surface includes a plurality of mutually adjacent visible facets; and the visible surface of each viewing angle includes mutually non-repeating visible facets;

[0062] The invisible patches of the three-dimensional model relative to all viewing angles are determined as the invisible surfaces of the three-dimensional model relative to all viewing angles.

[0063] The surface of the three-dimensional model is composed of multiple facets, such as triangular facets, and each triangular facet is composed of three vertices and three edges between the three vertices. First, the adjacency relationship between each facet is determined. Two facets with common edges have an adjacency relationship, which is called adjacent facets or adjacent facets. According to the adjacency relationship between the facets in the three-dimensional model and the visible facets of each perspective, the visible surface of the three-dimensional model relative to each perspective is determined. The visible surface of each perspective includes multiple visible facets that are adjacent to each other in the adjacency relationship, and the figure surrounded by the outermost edge of the visible surface of each perspective does not include visible facets of other perspectives.

[0064] For example, for a viewing angle, the visible surface of the viewing angle is determined based on the adjacency relationship between the facets in the three-dimensional model and the visible facets of the viewing angle. The visible surface of the viewing angle includes multiple visible facets that are adjacent to each other in the adjacency relationship. The outermost edge of the visible surface of the viewing angle surrounds a figure such as an ellipse. The interior of the figure is the visible surface of the viewing angle and does not include visible facets of other viewing angles. If, in the process of determining the visible surface of the viewing angle, the interior of the figure surrounded by the outermost edge of the visible surface initially determined by the viewing angle includes visible facets of other viewing angles, such as a ring, and the interior of the ring includes visible facets of other viewing angles, then the interior of the figure is also determined as the visible surface of the viewing angle.

[0065] By ensuring that the visible surface of each perspective includes multiple visible patches that are adjacent to each other in an adjacency relationship, and that the figure surrounded by the outermost edge of the visible surface of each perspective does not include visible patches of other perspectives, the continuity and local consistency of the visible surface of each perspective can be ensured, thereby improving the accuracy of the subsequent determination of the texture value of the visible surface of each perspective.

[0066] In addition, the visible surfaces of each viewing angle include non-repeating visible patches, that is, after the surface of the three-dimensional model is divided into visible surfaces of each viewing angle, one viewing angle corresponds to one visible surface, and the visible surfaces of each viewing angle do not overlap.

[0067] By ensuring that one viewing angle corresponds to one visible surface and the visible surfaces of different viewing angles do not overlap, the uniqueness of the visible surface of each viewing angle can be ensured, thereby improving the accuracy of the texture value of the visible surface of each viewing angle that is subsequently determined.

[0068] In addition, invisible patches of the three-dimensional model relative to all viewing angles are determined as invisible surfaces of the three-dimensional model relative to all viewing angles.

[0069] It can be seen that through this embodiment, the visible surface of each perspective can be determined based on the adjacency relationship between the patches in the three-dimensional model and the visible patches of each perspective, and it is ensured that the visible surface includes multiple visible patches that are adjacent to each other in the adjacency relationship, and the figure surrounded by the outermost edge of the visible surface of each perspective does not include visible patches of other perspectives, and the visible surfaces of each perspective include non-repeating visible patches, thereby ensuring the continuity, local consistency and uniqueness of the visible surface of each perspective, and improving the accuracy of the subsequent determination of the texture value of the visible surface of each perspective.

[0070] After the visible and invisible surfaces are determined, Figure 1 In the above step S106, the texture value of the visible surface of each viewing angle is determined according to the pixel value and the weight of the pixel in each image and mapped to the first partition of the texture map. In each image used to create a three-dimensional model, each pixel has a corresponding weight, which is the weight of the pixel in the image contributing texture to the three-dimensional model. A texture map is preset, and the preset texture map has a first partition corresponding to each viewing angle and fixed in position, and each first partition is used to record the texture value of the visible surface of the corresponding viewing angle.

[0071] Figure 2a A schematic diagram of a partition of a texture map provided by an embodiment of the present disclosure, such as Figure 2a As shown, in the texture map, there are multiple first partitions, each of which corresponds to a viewing angle, and the pixel values ​​of the pixels in each first partition are in a pending state. The texture value of the visible surface of the viewing angle corresponding to the first partition is recorded by determining the pixel value of the pixel in the first partition, and the pixel value is the texture value.

[0072] In one embodiment, according to the pixel values ​​and weights of the pixels in each image, the texture value of the visible surface of each viewing angle is determined and mapped to the first partition of the texture map, including:

[0073] Determine the texture value of a specific point on the three-dimensional model according to the pixel value and the weight of the pixel point in each image to form a texture value set; the specific point is a point on the three-dimensional model for which a corresponding pixel point exists in each image;

[0074] According to the texture value set, the texture value of the visible surface of each viewing angle is determined and mapped into the first partition of the texture map.

[0075] The surface of a 3D model is composed of multiple facets, such as triangular facets, each of which is composed of multiple vertices and multiple edges between the vertices. These vertices are points on the 3D model. As mentioned above, the images used to create the 3D model are images taken from different perspectives of the modeled object. Therefore, an image represents a camera perspective. For each camera perspective, it can view a part of the 3D model, and the viewed part will have corresponding pixels in the image. If it is a part that cannot be viewed by any camera perspective, there will be no pixels in any image, that is, there will be no pixels in all images.

[0076] In this embodiment, first, among all points on the surface of the three-dimensional model, points having corresponding pixel points in each image are determined. The specific determination process may be: through the previous process, the visible surface of the three-dimensional model relative to each camera perspective is determined, and then the points on the visible surface of each camera perspective are points having corresponding pixel points in the image corresponding to the camera perspective, and the points on the visible surface of each camera perspective are all specific points.

[0077] After the specific point is determined, the texture value of the specific point on the three-dimensional model is determined according to the pixel value and the weight of the pixel in each image to form a texture value set, which includes the texture value of each specific point. Then, according to the texture value set, the texture value of the visible surface of each viewing angle is determined and mapped to the first partition of the texture map.

[0078] It can be seen that through this embodiment, based on the various images used to create the three-dimensional model, specific points on the three-dimensional model that have corresponding pixel points in each image can be determined, and the texture value of the specific point can be determined according to the pixel value and weight of the pixel point in each image. Based on the texture value of the specific point, the texture value of the visible surface of each perspective can be accurately determined and mapped to the first partition of the texture map.

[0079] In one embodiment, the texture value of a specific point on the three-dimensional model is determined according to the pixel value and the weight of the pixel in each image to form a texture value set, including:

[0080] Obtain pixel values ​​of each first pixel point obtained by mapping the specific point to each image;

[0081] The pixel values ​​of each first pixel point are weighted and summed according to the weight of each first pixel point to obtain the texture value of the specific point and form a texture value set.

[0082] A specific point is a point located on the surface of a three-dimensional model, and there are multiple specific points. For each specific point, it may be seen by multiple camera perspectives, so there will be corresponding pixel points in multiple images. First, for each specific point, according to the perspective parameters of each camera perspective, determine the multiple first pixel points and the pixel values ​​of the first pixel points obtained by mapping the specific point to each image. The specific point will be mapped to one or more images, that is, it will be seen by one or more camera perspectives, so one or more first pixel points will be obtained, and then the pixel value of each first pixel point will be determined, and each mapped image includes a first pixel point. The perspective parameters of the camera perspective include the intrinsic parameters and extrinsic parameters of the camera.

[0083] Each pixel in each image has a corresponding weight, which is the weight of the texture contributed by the pixel to the three-dimensional model. Based on this, for each specific point, the pixel values ​​of each first pixel corresponding to the specific point are weighted and summed according to the weights of each first pixel corresponding to the specific point to obtain the texture value of the specific point, and the texture values ​​of all specific points form a texture value set. For example, for a specific point, it corresponds to 3 first pixels, with pixel values ​​of 200, 50, and 100, respectively, and weights of 0.3, 0.2, and 0.5, respectively. Then the texture value of the specific point is 200*0.3+50*0.2+100*0.5=120.

[0084] It can be seen that through this embodiment, the texture value of a specific point can be determined according to the pixel value of the pixel point in each image and the weight of the pixel point to form a texture value set. Since the weight is the weight of the texture contributed by the pixel point to the three-dimensional model, the accuracy of the determined texture value can be improved by determining the texture value based on the weight. In addition, since the texture value of each specific point is obtained based on the weighted mixture of the pixel value and the weight of the pixel point, the problem of the texture seams on the surface of the three-dimensional model after mapping caused by mapping different areas of the three-dimensional model through different images in the prior art is effectively solved.

[0085] As mentioned above, the perspective includes a camera perspective corresponding to an image one by one and at least one virtual perspective. The virtual perspective is a hypothetical perspective that does not really exist. The process of determining the virtual perspective will be described later. Figure 2a As shown, there are multiple first partitions in the texture map, each of which corresponds to a viewing angle, which can be a camera viewing angle or a virtual viewing angle. Based on this, after obtaining the texture value set, in one embodiment, the texture value of the visible surface of each viewing angle is determined according to the texture value set and mapped to the first partition of the texture map, including:

[0086] Obtain texture values ​​of all points on the visible surface of the camera's viewing angle from the texture value set, and map them to the first partition of the texture map corresponding to the camera's viewing angle;

[0087] Acquire texture values ​​of a first portion of points on the visible surface of the virtual perspective from the texture value set, and map them to a first partition of the texture map corresponding to the virtual perspective; the first portion of points are points located in the visible surface of the virtual perspective and having corresponding pixel points in each image;

[0088] In the first partition corresponding to the virtual perspective in the texture map, the texture values ​​of the second part of the points on the visible surface of the virtual perspective are completed; the second part of the points are points located in the visible surface of the virtual perspective and for which there are no corresponding pixels in each image.

[0089] The texture value set includes the texture value of each specific point on the surface of the three-dimensional model. The specific point is a point that can be seen by the camera perspective, that is, there is a corresponding pixel point in each image. Therefore, for each camera perspective, all points on the visible surface of the three-dimensional model relative to the camera perspective are specific points, and the texture values ​​of these points are all located in the texture value set. Considering that when determining the visible surface of each perspective, one perspective corresponds to one visible surface, and the visible surfaces of each perspective do not overlap each other, but this does not mean that each perspective can only see its own visible surface. Each perspective can still see part of the visible surface of the adjacent perspective. Therefore, part of the area on the surface seen by the virtual perspective will also be seen by the adjacent camera perspective. Therefore, for each virtual perspective, part of the points on the visible surface of the three-dimensional model relative to the virtual perspective are also specific points, and the texture values ​​of these points are also in the texture value set. Further, for the points in the visible surface of the virtual perspective that are not seen by the adjacent camera perspective, these points are not specific points, and the texture values ​​of these points are not located in the texture value set.

[0090] Based on this, in this embodiment, the texture values ​​of all points on the visible surface of the camera's view are first obtained from the texture value set, and mapped to the first partition of the texture map corresponding to the camera's view. As mentioned above, the texture values ​​of all points on the visible surface of the camera's view are located in the texture value set.

[0091] Next, for the first part of points that are located in the visible surface of the virtual perspective and have corresponding pixel points in each image (that is, seen by each camera perspective), the texture value of the first part of points is also located in the texture value set, and the texture value of the first part of points on the visible surface of the virtual perspective is obtained from the texture value set and mapped to the first partition of the texture map corresponding to the virtual perspective.

[0092] At this point, it can be understood that there are still some points in the visible surface of the virtual perspective. These points are located in the visible surface of the virtual perspective but there are no corresponding pixels in each image (that is, they are not seen by each camera perspective). These points are called second part points. In the first partition of the texture map corresponding to the virtual perspective, a texture value completion strategy is used to complete the texture values ​​of the second part points on the visible surface of the virtual perspective.

[0093] It can be seen that through this embodiment, it is possible to fully rely on the texture value set, obtain the texture values ​​of all points on the visible surface of the camera perspective and the texture values ​​of the first part of the points on the visible surface of the virtual perspective from the texture value set and map them into the texture map, and, in the texture map, complete the texture values ​​of the second part of the points on the visible surface of the virtual perspective, so as to comprehensively determine the texture value of each point on the visible surface of each perspective, effectively improve the accuracy and efficiency of determining the texture value, so that the process of determining the texture value meets the real-time requirements of the scene of real-time transmission of volumetric video for generating texture maps.

[0094] In one embodiment, the texture values ​​of all points on the visible surface of the camera perspective are obtained from the texture value set and mapped to the first partition of the texture map corresponding to the camera perspective, including:

[0095] Obtain pixel coordinates of a second pixel point obtained by mapping a point on the visible surface of the camera's viewing angle to a first partition of the texture map corresponding to the camera's viewing angle;

[0096] The texture value of a point on the visible surface of the camera's viewing angle is obtained from the texture value set as the pixel value of the second pixel point, and is written into the first partition corresponding to the camera's viewing angle.

[0097] In this embodiment, a point on the visible surface of the camera's view has a mapped second pixel in the first partition corresponding to the camera's view, and the pixel value of the second pixel is the texture value of the point on the visible surface of the camera's view, thereby achieving the effect of recording the texture of the visible surface of the camera's view in the first partition. Based on this, the pixel coordinates of each second pixel obtained by mapping each point on the visible surface of the camera's view to the texture map in the first partition corresponding to the camera's view are obtained, and the texture value of each point on the visible surface of the camera's view is obtained from the texture value set as the pixel value of the corresponding second pixel, and written into the first partition corresponding to the camera's view, thereby achieving the effect of recording the texture of the visible surface of the camera's view in the first partition.

[0098] It can be seen that through this embodiment, the points on the visible surface of the camera's perspective are mapped to the first partition of the texture map corresponding to the camera's perspective, and the pixel value of the mapped second pixel point is set to the texture value of the point on the visible surface of the camera's perspective obtained from the texture value set, thereby realizing texture mapping and achieving the effect of recording the texture of the visible surface of the camera's perspective in the first partition.

[0099] In one embodiment, obtaining the pixel coordinates of a second pixel point obtained by mapping a point on a visible surface of a camera perspective to a first partition of a texture map corresponding to the camera perspective includes:

[0100] Projecting the visible surface of the camera's viewing angle to the camera's viewing angle to obtain a first projection image;

[0101] Based on the bounding box principle, a first enclosed area is obtained by cutting out in the first projection image;

[0102] The first enclosed area is projected into a first partition of the texture map corresponding to the camera viewing angle, and the pixel coordinates of the second pixel point corresponding to the point on the visible surface of the camera viewing angle are determined according to the pixel point mapping relationship obtained by the projection.

[0103] In order to complete the mapping of points on the visible surface of the camera perspective to the first partition of the texture map corresponding to the camera perspective, first, for each camera perspective, the visible surface of the camera perspective is projected to the same camera perspective according to the perspective parameters of the camera perspective to obtain a first projection image, which includes projection points obtained by projecting each point on the visible surface of the camera perspective.

[0104] Next, based on the bounding box principle, the valid area is determined in the first projection image. The valid area is the area corresponding to the visible surface of the camera perspective. The first projection image also includes an invalid area. The invalid area is the area where the projection points corresponding to the points on the visible surface of the camera perspective do not exist, which can also be understood as the background area. The valid area is cropped to obtain a rectangular area, which is the first enclosed area.

[0105] Finally, the first enclosed area of ​​the rectangle is projected to the first partition of the texture map corresponding to the camera's view. The first partition is also a rectangle, so this is equivalent to the projection between two rectangles. After the projection is completed, each pixel point in the first enclosed area corresponds one-to-one with each pixel point in the first partition corresponding to the camera's view. This one-to-one correspondence is the pixel mapping relationship. The pixel mapping relationship records the coordinate values ​​of a pair of corresponding pixels. Each pixel point in the first enclosed area is the projection point obtained by projecting each point on the visible surface of the camera's view. Therefore, based on the pixel mapping relationship, it is possible to determine the pixel coordinates of the second pixel point obtained by mapping each point on the visible surface of the camera's view to the first partition of the texture map corresponding to the camera's view.

[0106] It can be seen that through this embodiment, the pixel coordinates of the second pixel point obtained by mapping each point on the visible surface of the camera's perspective to the first partition of the texture map corresponding to the camera's perspective can be determined by means of multiple projections. The determination process is simple and efficient, and can meet the real-time requirements for generating texture maps in the scene of real-time transmission of volumetric video.

[0107] After obtaining texture values ​​of all points on the visible surface of the camera perspective from the texture value set and mapping them to the first partition of the texture map corresponding to the camera perspective, in one embodiment, obtaining texture values ​​of a first part of points on the visible surface of the virtual perspective from the texture value set and mapping them to the first partition of the texture map corresponding to the virtual perspective includes:

[0108] Obtain pixel coordinates of a third pixel point obtained by mapping a first portion of points on a visible surface of a virtual viewing angle to a first subarea of ​​a texture map corresponding to the virtual viewing angle;

[0109] The texture values ​​of the first part of the points on the visible surface of the virtual viewing angle are obtained from the texture value set as the pixel value of the third pixel point, and are written into the first partition corresponding to the virtual viewing angle.

[0110] The first part of the points on the visible surface of the virtual perspective are located in the visible surface of the virtual perspective and have corresponding pixel points in each image, that is, the points seen by the camera perspective. Therefore, similar to the logic of the camera perspective, the first part of the points on the visible surface of the virtual perspective has a mapped third pixel point in the first partition corresponding to the virtual perspective, and the pixel value of the third pixel point is the texture value of the first part of the points on the visible surface of the virtual perspective, thereby achieving the effect of recording the texture of the first part of the points on the visible surface of the virtual perspective in the first partition. Based on this, the pixel coordinates of each third pixel point obtained by mapping the first part of the points on the visible surface of the virtual perspective to the texture map in the first partition corresponding to the virtual perspective are obtained, and the texture value of each first part of the points on the visible surface of the virtual perspective is obtained from the texture value set as the pixel value of the corresponding third pixel point, and written into the first partition corresponding to the virtual perspective, thereby achieving the effect of recording the texture of the first part of the points on the visible surface of the virtual perspective in the first partition.

[0111] It can be seen that through this embodiment, the first part of the points on the visible surface of the virtual perspective is mapped to the first partition of the texture map corresponding to the virtual perspective, and the pixel value of the mapped third pixel point is set to the texture value of the first part of the points on the visible surface of the virtual perspective obtained from the texture value set, thereby realizing texture mapping and achieving the effect of recording the texture of the first part of the points on the visible surface of the virtual perspective in the first partition.

[0112] In one embodiment, obtaining pixel coordinates of a third pixel point obtained by mapping a first portion of points on a visible surface of a virtual viewing angle to a first partition of a texture map corresponding to the virtual viewing angle includes:

[0113] Projecting the visible surface of the virtual perspective onto the virtual perspective to obtain a second projection image;

[0114] Based on the bounding box principle, a second enclosed area is obtained by cutting out the second projection image;

[0115] The second enclosed area is projected into the first partition of the texture map corresponding to the virtual viewing angle, and the pixel coordinates of the third pixel point corresponding to the first part of the points on the visible surface of the virtual viewing angle are determined according to the pixel point mapping relationship obtained by the projection.

[0116] In order to complete the mapping of the first part of the points on the visible surface of the virtual perspective to the first partition of the texture map corresponding to the virtual perspective, first, for each virtual perspective, according to the perspective parameters of the virtual perspective, the visible surface of the virtual perspective is projected to the same virtual perspective to obtain a second projection map, which includes projection points obtained by projecting each point on the visible surface of the virtual perspective, and naturally also includes projection points obtained by projecting the first part of the points on the visible surface of the virtual perspective. The perspective parameters of the virtual perspective include the intrinsic parameters and extrinsic parameters of the camera of the virtual perspective.

[0117] Next, based on the bounding box principle, the valid area is determined in the second projection map. The valid area is the area corresponding to the visible surface of the virtual perspective. The second projection map also includes an invalid area. The invalid area is the area of ​​the projection points corresponding to the points on the visible surface without the virtual perspective, which can also be understood as the background area. The valid area is cropped to obtain a rectangular area, which is the second enclosed area.

[0118] Finally, the second enclosed area of ​​the rectangle is projected to the first partition of the texture map corresponding to the virtual perspective. The first partition is also a rectangle, so this is equivalent to the projection between two rectangles. After the projection is completed, each pixel point in the second enclosed area corresponds to each pixel point in the first partition corresponding to the virtual perspective. The one-to-one correspondence is the pixel mapping relationship. The pixel mapping relationship records the coordinate values ​​of a pair of corresponding pixels. Each pixel point in the second enclosed area is the projection point obtained by projecting each point on the visible surface of the virtual perspective. Therefore, based on the pixel mapping relationship, the pixel coordinates of the third pixel point obtained by mapping each point on the visible surface of the virtual perspective to the first partition of the texture map corresponding to the virtual perspective can be determined, and naturally the pixel coordinates of the third pixel point obtained by mapping the first part of the points on the visible surface of the virtual perspective to the first partition of the texture map corresponding to the virtual perspective are obtained.

[0119] It can be seen that through this embodiment, the pixel coordinates of the third pixel point obtained by mapping the first part of the points on the visible surface of the virtual perspective to the first partition of the texture map corresponding to the virtual perspective can be determined by means of multiple projections. The determination process is simple and efficient, and can meet the real-time requirements for generating texture maps in the scene of real-time transmission of volumetric video.

[0120] Through the above process, the texture values ​​of all points on the visible surface of the camera perspective can be recorded in the first partition of the texture map corresponding to the camera perspective, thereby obtaining the map texture of the visible surface of the camera perspective, and the texture values ​​of the first part of the points on the visible surface of the virtual perspective can be recorded in the first partition of the texture map corresponding to the virtual perspective, thereby obtaining the map texture of the first part of the points on the visible surface of the virtual perspective. In one embodiment, in the first partition of the texture map corresponding to the virtual perspective, the texture values ​​of the second part of the points on the visible surface of the virtual perspective are completed, including:

[0121] Based on the mapped first partition of the texture map corresponding to the virtual perspective, an image including texture values ​​of a first portion of points on a visible surface of the virtual perspective is obtained and input into a pre-trained neural network model for processing;

[0122] Through the neural network model, a processed image of the first partition corresponding to the virtual viewing angle is output; the processed image includes texture values ​​of the second part of points.

[0123] The second part of points is a point on the surface of the three-dimensional model that is seen by the virtual perspective but not by the camera perspective, so there is no corresponding pixel point in each image. For each virtual perspective, the visible surface it sees includes both the first part of points and the second part of points. Therefore, in this embodiment, for each virtual perspective, firstly, based on the first partition of the texture map corresponding to the virtual perspective after mapping, an image including the texture values ​​of the first part of points on the visible surface of the virtual perspective is obtained. For example, an image of the first partition of the texture map corresponding to the virtual perspective after mapping is obtained. Through the previous process, the image includes the texture values ​​of the first part of points but does not include the texture values ​​of the second part of points. The image is an image including the texture values ​​of the first part of points on the visible surface of the virtual perspective.

[0124] Next, the acquired image is input into a pre-trained neural network model, which can supplement the texture values ​​of the second part of the points in the image. Through the neural network model, the processed image of the first partition corresponding to the virtual perspective is output, and the processed image includes the texture values ​​of the second part of the points. The processed image is mapped back to the first partition corresponding to the virtual perspective in the texture map, thereby obtaining the texture values ​​of each point in the visible surface of the virtual perspective. It can be understood that since the visible surface of the virtual perspective is pre-mapped to the corresponding first partition by secondary projection, when the texture values ​​of the second part of the points are supplemented, only the pixel values ​​of the pixel points corresponding to the second part of the points in the first partition will be supplemented, and the pixel values ​​of the pixel points other than the pixel points corresponding to the first part of the points and the second part of the points will not be supplemented. The neural network in this embodiment can be a neural network that has been pre-trained by deep learning, which is not limited here.

[0125] It can be seen that through this embodiment, the texture values ​​of the second part of the points on the visible surface of the virtual perspective can be completed in the first partition of the texture map corresponding to the virtual perspective based on the neural network model, and the completion efficiency is high, thereby improving the efficiency of generating texture maps.

[0126] Regarding the virtual perspective introduced above, in one embodiment, the virtual perspective is determined by the following steps:

[0127] Projecting invisible facets of a specific three-dimensional model relative to each camera perspective to adjacent perspectives of each camera perspective; the surface of the three-dimensional model is composed of at least facets;

[0128] According to the number of invisible patches to which each adjacent view is projected, a virtual view is selected from each adjacent view.

[0129] As a video format, volumetric video includes multiple video frames, each of which includes a three-dimensional model. The surface of the three-dimensional model is composed of at least patches. The specific three-dimensional model may be the three-dimensional model included in the first video frame in the volumetric video. The texture map generation method in this embodiment can generate a texture map for the three-dimensional model in any video frame in the volumetric video.

[0130] First, determine the invisible facets of a specific three-dimensional model relative to each camera perspective. The invisible facets refer to the facets that cannot be seen by any camera perspective, that is, the invisible facets of a specific three-dimensional model relative to all camera perspectives. The determination process refers to the previous introduction and will not be repeated here. Next, project the invisible facets of the specific three-dimensional model relative to each camera perspective to the adjacent perspectives of each camera perspective. When establishing each three-dimensional model, each camera perspective is set at a certain angle, such as setting a camera perspective every 36 degrees, so as to set 10 camera perspectives. The adjacent perspective of a camera perspective refers to the perspective between two adjacent camera perspectives, which can be the middle perspective between two adjacent camera perspectives. For example, when a camera perspective is set every 36 degrees, the adjacent perspective of a 0-degree camera perspective can be an 18-degree perspective or a 342-degree perspective. Here, the adjacent perspective of the camera perspective is determined in advance based on the camera perspective, such as taking the intermediate position perspective between every two camera perspectives as the adjacent perspective, and then projecting the invisible facets of the specific three-dimensional model relative to each camera perspective to each adjacent perspective.

[0131] Different adjacent perspectives are projected onto different numbers of invisible surfaces. It can be understood that the more invisible surfaces are projected onto, the more invisible surfaces the adjacent perspective can see. Therefore, in this embodiment, according to the number of invisible surfaces projected onto each adjacent perspective, multiple perspectives with a larger number of projected invisible surfaces are selected from each adjacent perspective as virtual perspectives. For example, each adjacent perspective is sorted in order from high to low according to the number of invisible surfaces projected onto each adjacent perspective, and the first four perspectives in the sorting are selected as virtual perspectives.

[0132] It can be seen that through this embodiment, a virtual perspective can be selected in each adjacent perspective according to the number of invisible faces of the specific three-dimensional model relative to each camera perspective to which each adjacent perspective is projected, so that the virtual perspective can see as many invisible faces of the three-dimensional model as possible relative to the camera perspective, so that as much texture of the three-dimensional model as possible can be obtained by rendering through the camera perspective and the virtual perspective.

[0133] As mentioned above, in each image used to build a three-dimensional model, each pixel has a corresponding weight for contributing texture to the three-dimensional model. In one embodiment, the weight of each pixel in each image is determined by the following steps:

[0134] Obtain a depth map corresponding to the 3D model at each camera perspective; the depth map is obtained by rasterizing the 3D model to each camera perspective;

[0135] For each camera viewing angle, the weight of each pixel in the image corresponding to the camera viewing angle is determined according to the depth map corresponding to the camera viewing angle.

[0136] In this embodiment, each frame of the volumetric video includes a three-dimensional model, and each three-dimensional model corresponds to multiple camera perspectives. In the process of generating the volumetric video, the position and parameters of each camera perspective remain unchanged. Each three-dimensional model corresponds to an image at each camera perspective, and the image can be an RGB image. Based on this, first, the three-dimensional model for the texture map to be generated is rasterized to each camera perspective to obtain a depth map corresponding to each camera perspective. Then, for each camera perspective, the weight of each pixel in the image corresponding to the camera perspective is determined according to the depth map corresponding to the camera perspective.

[0137] It can be seen that through this embodiment, for each camera viewing angle, the weight of each pixel in the image corresponding to the camera viewing angle can be determined according to the depth map of the three-dimensional model corresponding to the camera viewing angle. The process of determining the weight does not need to rely on too many images or data, and the determination process is efficient and fast.

[0138] In one embodiment, for each camera viewing angle, determining the weight of each pixel in the image corresponding to the camera viewing angle according to the depth map corresponding to the camera viewing angle includes:

[0139] Determine the object boundary and the unsigned distance field value of each pixel based on the object boundary in the depth map corresponding to the camera perspective;

[0140] Determine the weight of each pixel in the image corresponding to the camera's viewpoint based on the object boundary and the unsigned distance field value.

[0141] The following describes the process of determining weights using any camera perspective as an example. For any camera perspective, the object boundary is determined in the depth map of the three-dimensional model corresponding to the camera perspective. The object boundary is the boundary where the depth changes greatly, which can also be understood as the boundary of the three-dimensional model, and the boundary pixels that constitute the boundary are marked in the depth map. The boundary pixels can be determined in the following way: a 3*3 sliding window is used in the depth map. If the difference between the maximum depth value and the minimum depth value in the sliding window is greater than a certain threshold, the middle pixel in the sliding window is determined to be the boundary pixel.

[0142] Next, determine the unsigned distance field value of each pixel in the depth map of the three-dimensional model corresponding to the camera perspective relative to the boundary pixel. The unsigned distance field value can use the Manhattan unsigned distance field to represent, for each pixel in the depth map, the unsigned Manhattan distance between the boundary pixel closest to the Manhattan distance between the pixel and the pixel. For example, for pixel A in the depth map, its unsigned distance field value is 5, indicating that the unsigned Manhattan distance between the boundary pixel closest to the Manhattan distance between the pixel and the pixel is 5.

[0143] Next, the weight of each pixel in the image corresponding to the camera's view is determined based on the object boundary and the unsigned distance field value using the following formula:

[0144]

[0145] In this formula, T represents the weight, i represents the serial number of the camera view, represents the i-th camera view, (u, v) is the pixel coordinate in the depth map, α is the normalization coefficient, and f i represents the focal length of the camera of the i-th camera perspective. In one embodiment, for a 1024x1024 image, the focal length of an 8mm lens is about 1600, and the position of the modeled object is 2m in front of the camera on average. The normalization coefficient α can be 2 / f=1 / 800. i Indicates the depth map corresponding to the i-th camera view. U represents the operation of obtaining the unsigned distance field, B represents the operation of obtaining the boundary, and F represents the attenuation function. An example of an attenuation function is:

[0146] F(x)=0.5*cos(π*Min(x,u max ))+0.5 (2)

[0147] In this attenuation function, x is the result of the above-mentioned operation of obtaining the unsigned distance field, that is, the unsigned distance field value of each pixel. Min represents the value of the distance field between x and u. max Truncate and normalize, with an upper limit of u max , usually 40 (Manhattan distance).

[0148] The meaning of the above two formulas is that for any pixel point in the i-th camera perspective, determine whether the pixel point is a boundary pixel point. If not, determine the unsigned distance field value of the pixel point. If the pixel point is a boundary pixel point, its unsigned distance field value can be 0, and the unsigned distance field value is substituted into the attenuation function. The output result of the attenuation function is substituted back into formula (1), and the weight of the pixel point is calculated by formula (1).

[0149] It can be seen that through this embodiment, the object boundary and the unsigned distance field value of each pixel based on the object boundary can be determined in the depth map corresponding to the camera perspective, and the weight of each pixel in the image corresponding to the camera perspective is determined according to the object boundary and the unsigned distance field value. The determination process is simple and easy to implement, thereby improving the efficiency of determining the weight and generating the texture map.

[0150] Through the above process, the texture value of the visible surface of the 3D model relative to each camera view and each virtual view can be determined according to the pixel value and pixel weight of each pixel in each image and mapped to the first partition of the texture map. In the texture map, there is also a second partition. Figure 2a As shown, in the texture map, there are multiple first partitions, each of which corresponds to a camera perspective or a virtual perspective, and the pixel values ​​of the pixels in each first partition are in a pending state. The texture value of the visible surface of the perspective corresponding to the first partition is recorded by determining the pixel values ​​of the pixels in the first partition, and the pixel value is the texture value. Figure 2a As shown, in the texture map, there is also a second partition, and the pixel values ​​of the pixels in the second partition are all in a pending state. The texture value of the invisible surface of the three-dimensional model relative to all camera perspectives and all virtual perspectives is recorded by determining the pixel values ​​of the pixels in the second partition. The pixel value is the texture value.

[0151] In one embodiment, in the above step S108, determining the texture value of the invisible surface and mapping it to the second partition of the texture map according to the texture value of the visible surface adjacent to the invisible surface in the three-dimensional model includes:

[0152] Determine a correspondence between a point on the invisible surface and a point on the visible surface adjacent to the invisible surface on the three-dimensional model;

[0153] Based on the correspondence and the texture values ​​of the points on the adjacent visible surface, the texture values ​​of the points on the invisible surface are obtained and mapped into the second partition of the texture map.

[0154] The invisible surface refers to the invisible surface of the three-dimensional model relative to all camera perspectives and all virtual perspectives. On the three-dimensional model, the invisible surface has an adjacent visible surface. Therefore, the visible surface adjacent to the invisible surface on the three-dimensional model is determined, and the correspondence between the points on the visible surface and the points on the invisible surface is determined. For example, invisible surface 1 is adjacent to visible surface 2, and it is determined that the three points ABC in invisible surface 1 correspond to the three points DEF in visible surface 2. Then, based on the correspondence and the texture values ​​of the points on the adjacent visible surface, the texture values ​​of the points on the invisible surface are determined and mapped to the second partition of the texture map.

[0155] It can be seen that through this embodiment, a correspondence between points on the invisible surface and points on the visible surface adjacent to the invisible surface in the three-dimensional model can be established. Based on the correspondence and the texture values ​​of the points on the adjacent visible surface, the texture values ​​of the points on the invisible surface are determined and mapped to the second partition of the texture map, thereby achieving the effect of texture propagation between the visible surface and the invisible surface.

[0156] In one embodiment, determining the correspondence between a point on the invisible surface and a point on a visible surface adjacent to the invisible surface on the three-dimensional model includes:

[0157] Obtaining pixel coordinates of a fourth pixel point obtained by mapping the point on the adjacent visible surface to the first partition, and pixel coordinates of a fifth pixel point obtained by mapping the point on the invisible surface to the second partition;

[0158] Based on the pixel coordinates of the fourth pixel point and the pixel coordinates of the fifth pixel point, the correspondence between the fourth pixel point and the fifth pixel point is determined by centroid interpolation of the pixel coordinates as the correspondence between the adjacent points on the visible surface and the points on the invisible surface.

[0159] When determining the texture value of the visible surface, the visible surface has been mapped to the corresponding first partition, so the pixel coordinates of the fourth pixel point obtained by mapping the point on the visible surface adjacent to the invisible surface to the first partition can be obtained. It can be understood that the point on the adjacent visible surface may be the specific point mentioned above, may be the first part of points, or may be the second part of points.

[0160] Next, the pixel coordinates of the fifth pixel point obtained by mapping the point on the invisible surface to the second partition are obtained. Here, each invisible surface can be traversed on the three-dimensional model in a preset order, and each traversed invisible surface can be mapped to each pre-planned position in the second partition, such as mapping the first traversed invisible surface to the pre-planned first position in the second partition, and mapping the second traversed invisible surface to the pre-planned second position in the second partition, thereby obtaining the pixel coordinates of the fifth pixel point obtained by mapping the point on the invisible surface to the second partition. In one embodiment, the shape of each pre-planned position in the second partition can be a triangle, that is, each traversed invisible surface is mapped to each triangle in the second partition to obtain each triangular facet, and by setting the pixel value of each triangular facet, the effect of recording the texture value of each corresponding invisible surface is achieved, and the triangular facet corresponds to the invisible surface one by one.

[0161] Next, based on the pixel coordinates of the fourth pixel point and the pixel coordinates of the fifth pixel point, the correspondence between the fourth pixel point and the fifth pixel point is determined by determining the centroid interpolation of the pixel coordinates, and the correspondence is used as the correspondence between the point on the adjacent visible surface and the point on the invisible surface. It can be understood that for an adjacent visible surface, there are multiple corresponding fourth pixel points, and for an invisible surface, there are also multiple corresponding fifth pixel points. The number of fourth pixel points may not be equal to the number of fifth pixel points. When obtaining the correspondence here, it is sufficient to obtain the fourth pixel point corresponding to each fifth pixel point, and the fourth pixel point that does not have a corresponding fifth pixel point can be left unprocessed.

[0162] Based on the pixel coordinates of the fourth pixel point and the pixel coordinates of the fifth pixel point, the corresponding relationship between the fourth pixel point and the fifth pixel point is determined by determining the barycentric interpolation of the pixel coordinates. For example, in a preset barycentric coordinate system, the barycentric coordinates of each fourth pixel point and each fifth pixel point are calculated, and the fourth pixel point and the fifth pixel point whose barycentric coordinates match each other are used as the corresponding fourth pixel point and the fifth pixel point. The fourth pixel point and the fifth pixel point whose barycentric coordinates match each other can be the fourth pixel point and the fifth pixel point whose barycentric coordinates have the same value or the fourth pixel point and the fifth pixel point whose barycentric coordinates have opposite values.

[0163] It can be seen that through this embodiment, the correspondence between the fourth pixel point and the fifth pixel point can be determined by determining the centroid interpolation of the pixel coordinates, and then the correspondence between the adjacent points on the visible surface and the points on the invisible surface can be obtained. The method of determining the centroid interpolation of the pixel coordinates is simple, efficient and easy to implement, which effectively improves the efficiency of determining the texture of the invisible surface.

[0164] In one embodiment, based on the above correspondence and the texture values ​​of the points on the adjacent visible surface, determining the texture values ​​of the points on the invisible surface and mapping them to the second partition of the texture map includes:

[0165] Obtain pixel coordinates of a fifth pixel point obtained by mapping the point on the invisible surface to the second partition;

[0166] Based on the above correspondence, the texture value of the point on the adjacent visible surface is used as the texture value of the corresponding point on the invisible surface, and is written into the second partition as the pixel value of the fifth pixel point.

[0167] The process of obtaining the pixel coordinates of the fifth pixel point obtained by mapping the point on the invisible surface to the second partition can refer to the previous description and will not be repeated here. According to the correspondence between the point on the invisible surface and the point on the adjacent visible surface determined previously, the texture value of the point on the adjacent visible surface is used as the texture value of the corresponding point on the invisible surface, and is written into the second partition as the pixel value of the fifth pixel point, that is, the pixel value of the fifth pixel point is the texture value of the point on the invisible surface.

[0168] In a specific embodiment, the pixel coordinates of a fourth pixel point obtained by mapping the point on the adjacent visible surface to the first partition and the pixel coordinates of a fifth pixel point obtained by mapping the point on the invisible surface to the second partition are obtained, and based on the pixel coordinates of the fourth pixel point and the pixel coordinates of the fifth pixel point, the correspondence between the fourth pixel point and the fifth pixel point is determined by determining the centroid interpolation of the pixel coordinates as the correspondence between the adjacent point on the visible surface and the point on the invisible surface, and based on the correspondence, the texture value of the point on the adjacent visible surface is used as the texture value of the corresponding point on the invisible surface, and is written into the second partition as the pixel value of the fifth pixel point.

[0169] It can be seen that through this embodiment, the texture value of the point on the adjacent visible surface can be used as the texture value of the corresponding point on the invisible surface, and written into the second partition as the pixel value of the fifth pixel point, thereby achieving the effect of recording the texture value of the invisible surface in the second partition.

[0170] Through the above process, the texture values ​​of the visible surfaces of each camera view and each virtual view can be recorded in the first partition of the texture map, and the texture values ​​of the invisible surfaces that are invisible to all camera view and all virtual view can be recorded in the second partition of the texture map. As can be seen from the previous introduction, the positions of the first partition and the second partition in the texture map are fixed, the number of the first partition is equal to the sum of the number of camera view and the number of virtual view, and the number of the second partition is one.

[0171] For multi-frame volumetric video images with adjacent timestamps, the texture of each three-dimensional model therein is also relatively similar. Since the positions of the first partition and the second partition are fixed, the image contents of multiple texture maps corresponding to the multiple three-dimensional models will also be similar, thereby improving the compression rate of the texture map in the time domain and reducing the bandwidth occupied by data transmission.

[0172] In one embodiment, after recording the texture values ​​of the visible surfaces of each camera view and each virtual view in the first partition of the texture map, and recording the texture values ​​of the invisible surfaces that are invisible to all camera view and all virtual view in the second partition of the texture map, the above method flow further includes:

[0173] Boundary extension and background filling are performed on the texture maps with texture values ​​mapped in the first partition and the second partition, wherein the boundary extension is used to extend the boundary of the object in the first partition, and the background filling is used to fill the background in the first partition and the second partition.

[0174] After the texture values ​​are mapped in each first partition and second partition of the texture map, the texture map is presented in the form of an RGB image, and an object will be presented in the first partition. The image content of the first partition can be simply understood as the image content obtained by photographing a three-dimensional model with a texture from different perspectives. The object in the first partition is the model object of the three-dimensional model. For example, if the three-dimensional model is a model of a person, the object in the first partition is the person, and the boundary of the object will be sharp. The second partition includes triangular facets arranged in a preset order, each of which is an RGB facet and corresponds one-to-one to the invisible surfaces of all the camera perspectives and all the virtual perspectives mentioned above.

[0175] By extending the boundary of the object in the first partition, the effect of reducing the boundary sharpness can be achieved. Also, by background filling, the effect of blurring the background in the first partition and the second partition can be achieved. The background filling method can be exemplified by smoothed push-pull background filling. By reducing boundary sharpness and blurring the background, the bit rate of the texture map can be reduced, and the bandwidth occupied by the server to transmit the texture map to the terminal device can be reduced.

[0176] Figure 2b A schematic diagram of a partition of a texture map provided by another embodiment of the present disclosure is shown in FIG. Figure 2b As shown, taking a 3D model of a person as an example, the image content in the first partition of the texture map is the texture image of the person, and the second partition of the texture map includes a plurality of triangular facets. After boundary extension and background filling, the boundary sharpness of the person in the first partition of the texture map is low, and the background in the first partition and the second partition is also blurred. Figure 2b The slash in the middle indicates the background blur state.

[0177] It can be seen that in this embodiment, by extending the boundaries and filling the background of the texture maps with texture values ​​mapped in the first partition and the second partition, the bit rate of the texture maps can be reduced, and the bandwidth occupied by the server to transmit the texture maps to the terminal device can be reduced.

[0178] In summary, this embodiment provides various embodiments of a method for generating a texture map. Various embodiments of the method can be implemented in parallel through CUDA (Compute Unified Device Architecture), supporting real-time generation and real-time transmission of texture maps. Various embodiments of the method have at least the following technical effects:

[0179] 1. Based on the weight of pixels and the adjacency between visible and invisible surfaces, the accuracy of determining texture values ​​is improved, thereby improving the clarity of texture mapping and the accuracy of 3D models in volumetric videos;

[0180] 2. Since the texture value of each specific point is obtained by mixing the pixel value of the pixel point and the weight, the problem of texture seams on the surface of the mapped 3D model caused by mapping different areas of the 3D model using different images in the prior art is effectively solved;

[0181] 3. For multi-frame volumetric video images with adjacent timestamps, the texture of each 3D model is also relatively similar. Since the positions of the first partition and the second partition in the texture map are fixed, the image contents of the multiple texture maps corresponding to the multiple 3D models are also similar, thereby improving the compression rate of the texture map in the time domain and reducing the bandwidth occupied by data transmission;

[0182] 4. For model surfaces that are not visible from the camera perspective, texture completion is performed through the virtual perspective. For model surfaces that are not visible from both the virtual perspective and the camera perspective, texture completion is performed by transferring texture values, solving the problem of texture missing.

[0183] 5. The process of generating texture maps is highly efficient and can meet the real-time requirements for generating texture maps in the scene of real-time transmission of volumetric video.

[0184] An embodiment of the present disclosure further provides a texture map generation device, which is used to implement the above-mentioned texture map generation method. Figure 3 A schematic diagram of the structure of a texture map generating device provided by an embodiment of the present disclosure is shown in FIG. Figure 3 As shown, the device comprises:

[0185] An acquisition unit 31 is used to acquire a three-dimensional model for which a texture map is to be generated and various images used to create the three-dimensional model;

[0186] A determination unit 32, configured to determine a visible surface of the three-dimensional model relative to each viewing angle and an invisible surface of the three-dimensional model relative to all viewing angles;

[0187] A first mapping unit 33 is used to determine the texture value of the visible surface of each of the viewing angles according to the pixel values ​​and weights of the pixels in each of the images and map the texture value to a first partition of the texture map; wherein the weight is the weight of the pixel in the image contributing to the texture of the three-dimensional model; the texture map has a first partition corresponding to each of the viewing angles and having a fixed position;

[0188] The second mapping unit 34 is used to determine the texture value of the invisible surface according to the texture value of the visible surface adjacent to the invisible surface in the three-dimensional model and map it to the second partition of the texture map; wherein the texture map has a second partition corresponding to the invisible surface and with a fixed position.

[0189] Optionally, the determining unit 32 is specifically configured to:

[0190] Acquire a depth map corresponding to the three-dimensional model at each viewing angle and a depth value of each point on the three-dimensional model at each viewing angle; the depth map is obtained by rasterizing the three-dimensional model to each viewing angle; the depth value is obtained by projecting the three-dimensional model to each viewing angle;

[0191] According to the depth map corresponding to each viewing angle and the depth values ​​of each point on the three-dimensional model at each viewing angle, the visible surface of the three-dimensional model relative to each viewing angle and the invisible surface of the three-dimensional model relative to all the viewing angles are determined.

[0192] Optionally, the determining unit 32 is further specifically configured to:

[0193] Determine, according to the depth map corresponding to each viewing angle and the depth values ​​of each point on the three-dimensional model at each viewing angle, a visible patch of the three-dimensional model relative to each viewing angle and an invisible patch of the three-dimensional model relative to all the viewing angles; the surface of the three-dimensional model is at least composed of patches;

[0194] The visible surface of the three-dimensional model relative to each of the viewing angles and the invisible surface of the three-dimensional model relative to all of the viewing angles are determined according to the visible surface of the three-dimensional model relative to each of the viewing angles and the invisible surface of the three-dimensional model relative to all of the viewing angles.

[0195] Optionally, the determining unit 32 is further specifically configured to:

[0196] For each of the viewing angles, a patch composed of the first target point in the three-dimensional model is determined as a visible patch of the viewing angle, and a patch including the second target point in the three-dimensional model is determined as an invisible patch of the viewing angle; the difference between the depth value of the first target point at the viewing angle and the depth value of the first target point recorded in the depth map corresponding to the viewing angle meets a preset difference requirement; the difference between the depth value of the second target point at the viewing angle and the depth value of the second target point recorded in the depth map corresponding to the viewing angle does not meet the preset difference requirement;

[0197] The same facets among the invisible facets of each of the viewing angles are determined as the invisible facets of the three-dimensional model relative to all the viewing angles.

[0198] Optionally, the determining unit 32 is further specifically configured to:

[0199] Determining a visible surface of the three-dimensional model relative to each of the viewing angles according to the adjacency relationship between the facets in the three-dimensional model and the visible facets of the three-dimensional model relative to each of the viewing angles; the visible surface includes a plurality of the visible facets adjacent to each other; and the visible surface of each of the viewing angles includes non-repeating visible facets;

[0200] The invisible patches of the three-dimensional model relative to all the viewing angles are determined as the invisible surfaces of the three-dimensional model relative to all the viewing angles.

[0201] Optionally, the first mapping unit 33 is specifically configured to:

[0202] Determine the texture value of a specific point on the three-dimensional model according to the pixel value and the weight of the pixel point in each of the images to form a texture value set; the specific point is a point on the three-dimensional model for which a corresponding pixel point exists in each of the images;

[0203] According to the texture value set, the texture value of the visible surface of each of the viewing angles is determined and mapped into the first partition of the texture map.

[0204] Optionally, the first mapping unit 33 is further specifically configured to:

[0205] Obtaining pixel values ​​of each first pixel point obtained by mapping the specific point to each of the images;

[0206] The pixel values ​​of each of the first pixel points are weightedly summed according to the weights of each of the first pixel points to obtain the texture value of the specific point and form a texture value set.

[0207] Optionally, the viewing angle includes a camera viewing angle and at least one virtual viewing angle corresponding to the image one by one; the first mapping unit 33 is further specifically configured to:

[0208] Acquire texture values ​​of all points on the visible surface of the camera perspective from the texture value set, and map them to a first partition of the texture map corresponding to the camera perspective;

[0209] Acquire texture values ​​of a first portion of points on the visible surface of the virtual perspective from the texture value set, and map them to a first partition of the texture map corresponding to the virtual perspective; the first portion of points are points located in the visible surface of the virtual perspective and having corresponding pixel points in each of the images;

[0210] In the first partition of the texture map corresponding to the virtual perspective, the texture values ​​of the second part of the points on the visible surface of the virtual perspective are completed; the second part of the points are points located in the visible surface of the virtual perspective and for which there are no corresponding pixel points in each of the images.

[0211] Optionally, the first mapping unit 33 is further specifically configured to:

[0212] Obtain pixel coordinates of a second pixel point obtained by mapping a point on the visible surface of the camera viewing angle to a first partition of the texture map corresponding to the camera viewing angle;

[0213] The texture value of a point on the visible surface of the camera viewing angle is obtained from the texture value set as the pixel value of the second pixel point, and is written into the first partition corresponding to the camera viewing angle.

[0214] Optionally, the first mapping unit 33 is further specifically configured to:

[0215] Projecting the visible surface of the camera viewing angle to the camera viewing angle to obtain a first projection image;

[0216] Based on the bounding box principle, a first enclosed area is obtained by cutting out the first projection image;

[0217] The first enclosed area is projected into a first partition of the texture map corresponding to the camera viewing angle, and the pixel coordinates of the second pixel point corresponding to the point on the visible surface of the camera viewing angle are determined according to a pixel point mapping relationship obtained by the projection.

[0218] Optionally, the first mapping unit 33 is further specifically configured to:

[0219] Obtain pixel coordinates of a third pixel point obtained by mapping a first portion of points on the visible surface of the virtual viewing angle to a first subarea of ​​the texture map corresponding to the virtual viewing angle;

[0220] The texture values ​​of the first part of the points on the visible surface of the virtual viewing angle are obtained from the texture value set as the pixel value of the third pixel point, and are written into the first partition corresponding to the virtual viewing angle.

[0221] Optionally, the first mapping unit 33 is further specifically configured to:

[0222] Projecting the visible surface of the virtual perspective onto the virtual perspective to obtain a second projection image;

[0223] Based on the bounding box principle, a second enclosed area is obtained by cutting out the second projection image;

[0224] The second enclosed area is projected into the first partition of the texture map corresponding to the virtual perspective, and the pixel coordinates of the third pixel point corresponding to the first part of the points on the visible surface of the virtual perspective are determined according to the pixel point mapping relationship obtained by the projection.

[0225] Optionally, the first mapping unit 33 is further specifically configured to:

[0226] Based on the mapped first partition of the texture map corresponding to the virtual perspective, an image including texture values ​​of a first portion of points on a visible surface of the virtual perspective is obtained and input into a pre-trained neural network model for processing;

[0227] Through the neural network model, a processed image of the first partition corresponding to the virtual viewing angle is output; the processed image includes texture values ​​of the second part points.

[0228] Optionally, a virtual viewing angle determination unit is further included, configured to:

[0229] Projecting invisible facets of a specific three-dimensional model relative to each camera perspective to adjacent perspectives of each camera perspective; the surface of the three-dimensional model is at least composed of facets;

[0230] The virtual perspective is selected from each of the adjacent perspectives according to the number of invisible patches to which each of the adjacent perspectives is projected.

[0231] Optionally, the perspective includes a camera perspective corresponding to the image one by one and at least one virtual perspective; and further includes a weight determination unit, which is used to:

[0232] Acquire a depth map corresponding to the three-dimensional model at each camera perspective; the depth map is obtained by rasterizing the three-dimensional model to each camera perspective;

[0233] For each of the camera viewing angles, a weight of each pixel in the image corresponding to the camera viewing angle is determined according to the depth map corresponding to the camera viewing angle.

[0234] Optionally, the weight determination unit is specifically configured to:

[0235] Determine an object boundary and an unsigned distance field value of each pixel based on the object boundary in the depth map corresponding to the camera perspective;

[0236] A weight of each pixel in the image corresponding to the camera viewing angle is determined according to the object boundary and the unsigned distance field value.

[0237] Optionally, the second mapping unit 34 is specifically configured to:

[0238] Determine a correspondence between points on the invisible surface and points on a visible surface adjacent to the invisible surface on the three-dimensional model;

[0239] Based on the correspondence and the texture values ​​of the points on the adjacent visible surface, the texture values ​​of the points on the invisible surface are determined and mapped into the second partition of the texture map.

[0240] Optionally, the second mapping unit 34 is further specifically configured to:

[0241] Obtaining pixel coordinates of a fourth pixel point obtained by mapping the point on the adjacent visible surface to the first partition, and pixel coordinates of a fifth pixel point obtained by mapping the point on the invisible surface to the second partition;

[0242] Based on the pixel coordinates of the fourth pixel point and the pixel coordinates of the fifth pixel point, the correspondence between the fourth pixel point and the fifth pixel point is determined by centroid interpolation of the pixel coordinates as the correspondence between the point on the adjacent visible surface and the point on the invisible surface.

[0243] Optionally, the second mapping unit 34 is further specifically configured to:

[0244] Obtain pixel coordinates of a fifth pixel point obtained by mapping the point on the invisible surface to the second partition;

[0245] Based on the corresponding relationship, the texture value of the point on the adjacent visible surface is used as the texture value of the corresponding point on the invisible surface, and is written into the second partition as the pixel value of the fifth pixel point.

[0246] Optionally, an extended filling unit is further included, for:

[0247] The texture maps having texture values ​​mapped thereto in the first partition and the second partition are subjected to boundary extension and background filling; the boundary extension is used to extend the boundary of the object in the first partition; and the background filling is used to fill the background in the first partition and the second partition.

[0248] The texture map generation device in the embodiment of the present disclosure can implement each process of the above-mentioned texture map generation method embodiment and achieve the same effect and function, which will not be repeated here.

[0249] An embodiment of the present disclosure further provides an electronic device, Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, such as Figure 4 As shown, the electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors 401 and memory 402, and one or more applications or data may be stored in the memory 402. Among them, the memory 402 may be a temporary storage or a permanent storage. The application stored in the memory 402 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the electronic device. Furthermore, the processor 401 may be configured to communicate with the memory 402 to execute a series of computer executable instructions in the memory 402 on the electronic device. The electronic device may also include one or more power supplies 403, one or more wired or wireless network interfaces 404, one or more input or output interfaces 405, one or more keyboards 406, etc.

[0250] In a specific embodiment, the electronic device includes a processor; and a memory configured to store computer executable instructions, wherein when the computer executable instructions are executed, the processor implements the following process:

[0251] Acquire a three-dimensional model for which a texture map is to be generated and various images used to create the three-dimensional model;

[0252] Determining a visible surface of the three-dimensional model relative to each viewing angle and an invisible surface of the three-dimensional model relative to all the viewing angles;

[0253] According to the pixel values ​​and weights of the pixels in each of the images, the texture value of the visible surface of each of the viewing angles is determined and mapped to the first partition of the texture map; wherein the weight is the weight of the pixel in the image contributing to the texture of the three-dimensional model; the texture map has a first partition corresponding to each of the viewing angles and having a fixed position;

[0254] According to the texture value of the visible surface adjacent to the invisible surface in the three-dimensional model, the texture value of the invisible surface is determined and mapped to the second partition of the texture map; wherein the texture map has a second partition corresponding to the invisible surface and fixed in position.

[0255] The electronic device in the embodiment of the present disclosure can implement each process of the above-mentioned texture map generation method embodiment and achieve the same effect and function, which will not be repeated here.

[0256] Another embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the following process is implemented:

[0257] Acquire a three-dimensional model for which a texture map is to be generated and various images used to create the three-dimensional model;

[0258] Determining a visible surface of the three-dimensional model relative to each viewing angle and an invisible surface of the three-dimensional model relative to all the viewing angles;

[0259] According to the pixel values ​​and weights of the pixels in each of the images, the texture value of the visible surface of each of the viewing angles is determined and mapped to the first partition of the texture map; wherein the weight is the weight of the pixel in the image contributing to the texture of the three-dimensional model; the texture map has a first partition corresponding to each of the viewing angles and having a fixed position;

[0260] According to the texture value of the visible surface adjacent to the invisible surface in the three-dimensional model, the texture value of the invisible surface is determined and mapped to the second partition of the texture map; wherein the texture map has a second partition corresponding to the invisible surface and fixed in position.

[0261] The storage medium in the embodiment of the present disclosure can implement each process of the above-mentioned texture map generation method embodiment and achieve the same effect and function, which will not be repeated here.

[0262] In various embodiments of the present disclosure, the computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0263] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0264] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0265] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0266] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing the embodiments of the present disclosure, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0267] Those skilled in the art will appreciate that one or more embodiments of the present disclosure may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0268] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0269] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0270] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0271] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0272] One or more embodiments of the present disclosure may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present disclosure may also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0273] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0274] The above description is only an embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of the claims of the present disclosure.

Claims

1. A texture map generation method, characterized in that: include: Acquire a three-dimensional model for which a texture map is to be generated and various images used to create the three-dimensional model; Determining a visible surface of the three-dimensional model relative to each viewing angle and an invisible surface of the three-dimensional model relative to all the viewing angles; According to the pixel values ​​and weights of the pixels in each of the images, the texture value of the visible surface of each of the viewing angles is determined and mapped to the first partition of the texture map; wherein the weight is the weight of the pixel in the image contributing to the texture of the three-dimensional model; the texture map has a first partition corresponding to each of the viewing angles and having a fixed position; According to the texture value of the visible surface adjacent to the invisible surface in the three-dimensional model, the texture value of the invisible surface is determined and mapped to the second partition of the texture map; wherein the texture map has a second partition corresponding to the invisible surface and fixed in position.

2. The method according to claim 1, characterized in that The determining of the visible surface of the three-dimensional model relative to each viewing angle and the invisible surface of the three-dimensional model relative to all the viewing angles comprises: Acquire a depth map corresponding to the three-dimensional model at each viewing angle and a depth value of each point on the three-dimensional model at each viewing angle; the depth map is obtained by rasterizing the three-dimensional model to each viewing angle; the depth value is obtained by projecting the three-dimensional model to each viewing angle; According to the depth map corresponding to each viewing angle and the depth values ​​of each point on the three-dimensional model at each viewing angle, the visible surface of the three-dimensional model relative to each viewing angle and the invisible surface of the three-dimensional model relative to all the viewing angles are determined.

3. The method according to claim 2, characterized in that The determining, according to the depth map corresponding to each of the viewing angles and the depth values ​​of each point on the three-dimensional model at each of the viewing angles, a visible surface of the three-dimensional model relative to each of the viewing angles and an invisible surface of the three-dimensional model relative to all of the viewing angles, comprises: Determine, according to the depth map corresponding to each viewing angle and the depth values ​​of each point on the three-dimensional model at each viewing angle, a visible patch of the three-dimensional model relative to each viewing angle and an invisible patch of the three-dimensional model relative to all the viewing angles; the surface of the three-dimensional model is at least composed of patches; The visible surface of the three-dimensional model relative to each of the viewing angles and the invisible surface of the three-dimensional model relative to all of the viewing angles are determined according to the visible surface of the three-dimensional model relative to each of the viewing angles and the invisible surface of the three-dimensional model relative to all of the viewing angles.

4. The method according to claim 3, characterized in that The determining, according to the depth map corresponding to each of the viewing angles and the depth values ​​of each point on the three-dimensional model at each of the viewing angles, visible patches of the three-dimensional model relative to each of the viewing angles and invisible patches of the three-dimensional model relative to all of the viewing angles comprises: For each of the viewing angles, a patch composed of the first target point in the three-dimensional model is determined as a visible patch of the viewing angle, and a patch including the second target point in the three-dimensional model is determined as an invisible patch of the viewing angle; the difference between the depth value of the first target point at the viewing angle and the depth value of the first target point recorded in the depth map corresponding to the viewing angle meets a preset difference requirement; the difference between the depth value of the second target point at the viewing angle and the depth value of the second target point recorded in the depth map corresponding to the viewing angle does not meet the preset difference requirement; The same facets among the invisible facets of each of the viewing angles are determined as the invisible facets of the three-dimensional model relative to all the viewing angles.

5. The method according to claim 3, characterized in that: Determining the visible surface of the three-dimensional model relative to each of the viewing angles and the invisible surface of the three-dimensional model relative to all of the viewing angles according to the visible surface of the three-dimensional model relative to each of the viewing angles and the invisible surface of the three-dimensional model relative to all of the viewing angles comprises: Determining a visible surface of the three-dimensional model relative to each of the viewing angles according to the adjacency relationship between the facets in the three-dimensional model and the visible facets of the three-dimensional model relative to each of the viewing angles; the visible surface includes a plurality of the visible facets adjacent to each other; and the visible surface of each of the viewing angles includes non-repeating visible facets; The invisible patches of the three-dimensional model relative to all the viewing angles are determined as the invisible surfaces of the three-dimensional model relative to all the viewing angles.

6. The method according to claim 1, characterized in that The step of determining the texture value of the visible surface of each viewing angle according to the pixel values ​​and the weights of the pixels in each of the images and mapping the texture value to the first partition of the texture map comprises: Determine the texture value of a specific point on the three-dimensional model according to the pixel value and the weight of the pixel point in each of the images to form a texture value set; the specific point is a point on the three-dimensional model for which a corresponding pixel point exists in each of the images; According to the texture value set, the texture value of the visible surface of each of the viewing angles is determined and mapped into the first partition of the texture map.

7. The method according to claim 6, characterized in that The step of determining the texture value of a specific point on the three-dimensional model according to the pixel value and the weight of each pixel in the image to form a texture value set includes: Obtaining pixel values ​​of each first pixel point obtained by mapping the specific point to each of the images; The pixel values ​​of each of the first pixel points are weightedly summed according to the weights of each of the first pixel points to obtain the texture value of the specific point and form a texture value set.

8. The method according to claim 6, characterized in that The viewing angles include camera viewing angles and at least one virtual viewing angle corresponding to the image one by one; and determining the texture value of the visible surface of each viewing angle according to the texture value set and mapping it to the first partition of the texture map, including: Acquire texture values ​​of all points on the visible surface of the camera perspective from the texture value set, and map them to a first partition of the texture map corresponding to the camera perspective; Acquire texture values ​​of a first portion of points on the visible surface of the virtual perspective from the texture value set, and map them to a first partition of the texture map corresponding to the virtual perspective; the first portion of points are points located in the visible surface of the virtual perspective and having corresponding pixel points in each of the images; In the first partition of the texture map corresponding to the virtual perspective, the texture values ​​of the second part of the points on the visible surface of the virtual perspective are completed; the second part of the points are points located in the visible surface of the virtual perspective and for which there are no corresponding pixel points in each of the images.

9. The method according to claim 8, characterized in that The acquiring the texture values ​​of all points on the visible surface of the camera perspective from the texture value set and mapping them to a first partition of the texture map corresponding to the camera perspective comprises: Obtain pixel coordinates of a second pixel point obtained by mapping a point on the visible surface of the camera viewing angle to a first partition of the texture map corresponding to the camera viewing angle; The texture value of a point on the visible surface of the camera viewing angle is obtained from the texture value set as the pixel value of the second pixel point, and is written into the first partition corresponding to the camera viewing angle.

10. The method according to claim 9, characterized in that The step of obtaining pixel coordinates of a second pixel point obtained by mapping a point on the visible surface of the camera viewing angle to a first partition of the texture map corresponding to the camera viewing angle comprises: Projecting the visible surface of the camera viewing angle to the camera viewing angle to obtain a first projection image; Based on the bounding box principle, a first enclosed area is obtained by cutting out the first projection image; The first enclosed area is projected into a first partition of the texture map corresponding to the camera viewing angle, and the pixel coordinates of the second pixel point corresponding to the point on the visible surface of the camera viewing angle are determined according to a pixel point mapping relationship obtained by the projection.

11. The method according to claim 8, characterized in that The step of acquiring texture values ​​of a first portion of points on the visible surface of the virtual perspective from the texture value set and mapping the texture values ​​to a first partition of the texture map corresponding to the virtual perspective comprises: Obtain pixel coordinates of a third pixel point obtained by mapping a first portion of points on the visible surface of the virtual viewing angle to a first subarea of ​​the texture map corresponding to the virtual viewing angle; The texture values ​​of the first part of the points on the visible surface of the virtual viewing angle are obtained from the texture value set as the pixel value of the third pixel point, and are written into the first partition corresponding to the virtual viewing angle.

12. The method according to claim 11, characterized in that The step of acquiring pixel coordinates of a third pixel point obtained by mapping a first portion of points on the visible surface of the virtual viewing angle to a first partition of the texture map corresponding to the virtual viewing angle comprises: Projecting the visible surface of the virtual perspective onto the virtual perspective to obtain a second projection image; Based on the bounding box principle, a second enclosed area is obtained by cutting out the second projection image; The second enclosed area is projected into the first partition of the texture map corresponding to the virtual perspective, and the pixel coordinates of the third pixel point corresponding to the first part of the points on the visible surface of the virtual perspective are determined according to the pixel point mapping relationship obtained by the projection.

13. The method according to claim 8, characterized in that The step of completing the texture values ​​of the second part of the points on the visible surface of the virtual perspective in the first partition corresponding to the virtual perspective in the texture map comprises: Based on the mapped first partition of the texture map corresponding to the virtual perspective, an image including texture values ​​of a first portion of points on a visible surface of the virtual perspective is obtained and input into a pre-trained neural network model for processing; Through the neural network model, a processed image of the first partition corresponding to the virtual viewing angle is output; the processed image includes texture values ​​of the second part points.

14. The method according to claim 8, characterized in that The virtual viewing angle is determined by the following steps: Projecting the invisible facets of the specific three-dimensional model relative to each of the camera perspectives to adjacent perspectives of each of the camera perspectives; The surface of the three-dimensional model is at least composed of facets; The virtual perspective is selected from each of the adjacent perspectives according to the number of invisible patches to which each of the adjacent perspectives is projected.

15. The method according to claim 1, characterized in that The perspective includes a camera perspective and at least one virtual perspective corresponding to the image one by one; the weight of each pixel in each of the images is determined by the following steps: Acquire a depth map corresponding to the three-dimensional model at each camera perspective; the depth map is obtained by rasterizing the three-dimensional model to each camera perspective; For each of the camera viewing angles, a weight of each pixel in the image corresponding to the camera viewing angle is determined according to the depth map corresponding to the camera viewing angle.

16. The method according to claim 15, characterized in that The step of determining the weight of each pixel in the image corresponding to the camera viewing angle according to the depth map corresponding to the camera viewing angle includes: Determine an object boundary and an unsigned distance field value of each pixel based on the object boundary in the depth map corresponding to the camera perspective; A weight of each pixel in the image corresponding to the camera viewing angle is determined according to the object boundary and the unsigned distance field value.

17. The method according to claim 1, characterized in that The step of determining the texture value of the invisible surface according to the texture value of the visible surface adjacent to the invisible surface in the three-dimensional model and mapping the texture value to the second partition of the texture map comprises: Determine a correspondence between points on the invisible surface and points on a visible surface adjacent to the invisible surface on the three-dimensional model; Based on the correspondence and the texture values ​​of the points on the adjacent visible surface, the texture values ​​of the points on the invisible surface are determined and mapped into the second partition of the texture map.

18. The method according to claim 17, characterized in that The determining of the correspondence between the points on the invisible surface and the points on the visible surface adjacent to the invisible surface on the three-dimensional model comprises: Obtaining pixel coordinates of a fourth pixel point obtained by mapping the point on the adjacent visible surface to the first partition, and pixel coordinates of a fifth pixel point obtained by mapping the point on the invisible surface to the second partition; Based on the pixel coordinates of the fourth pixel point and the pixel coordinates of the fifth pixel point, the correspondence between the fourth pixel point and the fifth pixel point is determined by centroid interpolation of the pixel coordinates as the correspondence between the point on the adjacent visible surface and the point on the invisible surface.

19. The method according to claim 17, characterized in that The step of determining the texture value of the point on the invisible surface based on the corresponding relationship and the texture value of the point on the adjacent visible surface and mapping the texture value to the second partition of the texture map comprises: Obtain pixel coordinates of a fifth pixel point obtained by mapping the point on the invisible surface to the second partition; Based on the corresponding relationship, the texture value of the point on the adjacent visible surface is used as the texture value of the corresponding point on the invisible surface, and is written into the second partition as the pixel value of the fifth pixel point.

20. The method according to claim 1, characterized in that The method further comprises: The texture maps having texture values ​​mapped thereto in the first partition and the second partition are subjected to boundary extension and background filling; the boundary extension is used to extend the boundary of the object in the first partition; and the background filling is used to fill the background in the first partition and the second partition.

21. A texture map generating device, characterized in that: include: An acquisition unit, used for acquiring a three-dimensional model for which a texture map is to be generated and each image used to create the three-dimensional model; a determining unit, configured to determine a visible surface of the three-dimensional model relative to each viewing angle and an invisible surface of the three-dimensional model relative to all the viewing angles; A first mapping unit, configured to determine the texture value of the visible surface of each of the viewing angles according to the pixel values ​​and weights of the pixels in each of the images, and map the texture value to a first partition of the texture map; wherein the weight is the weight of the pixel in the image contributing to the texture of the three-dimensional model; and the texture map has a first partition corresponding to each of the viewing angles and having a fixed position; A second mapping unit is used to determine the texture value of the invisible surface according to the texture value of the visible surface adjacent to the invisible surface in the three-dimensional model and map it to a second partition of the texture map; wherein the texture map has a second partition corresponding to the invisible surface and with a fixed position.

22. An electronic device, characterized in that: include: processor; as well as, A memory configured to store computer executable instructions, which, when executed, cause the processor to implement the steps of the method according to any one of claims 1 to 20.

23. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions, when executed by a processor, implement the steps of the method described in any one of claims 1 to 20.

Images