A texture mapping method, device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure disclose a texture mapping method and device, electronic equipment and storage medium, wherein the method comprises: obtaining a three-dimensional mesh model, a three-dimensional point cloud model, each sample image and a camera pose corresponding to each sample image; wherein the each sample image and the camera pose corresponding to each sample image are used to construct the three-dimensional mesh model and the three-dimensional point cloud model; determining a target region in the three-dimensional point cloud model according to position information of a to-be-mapped face in the three-dimensional mesh model; determining a first texture image from the each sample image according to a camera pose corresponding to a point cloud in the target region; and performing texture mapping on the to-be-mapped face according to the first texture image. The optimal image can be selected for texture mapping, and the texture mapping effect is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer, and particularly, to a texture mapping method and device, electronic equipment and storage medium. BACKGROUND

[0002] Through a texture mapping (also known as texture mapping) method, a corresponding region in a suitable image (view) can be selected as a texture for each face of a three-dimensional mesh model for mapping. The selection of the image greatly affects the texture mapping effect, and how to select a relatively optimal image becomes a technical problem to be solved. SUMMARY

[0003] Embodiments of the present disclosure provide a texture mapping method and device, electronic equipment and storage medium, which can select a relatively optimal image for texture mapping and improve the texture mapping effect.

[0004] In a first aspect, embodiments of the present disclosure provide a texture mapping method, comprising:

[0005] obtaining a three-dimensional mesh model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image; wherein the each sample image and the camera pose corresponding to each sample image are used to construct the three-dimensional mesh model and the three-dimensional point cloud model;

[0006] determining a target region in the three-dimensional point cloud model according to position information of a face to be mapped in the three-dimensional mesh model;

[0007] determining a first texture image from the each sample image according to a camera pose corresponding to a point cloud in the target region;

[0008] performing texture mapping on the face to be mapped according to the first texture image.

[0009] In a second aspect, embodiments of the present disclosure further provide a texture mapping device, comprising:

[0010] a data acquisition module configured to obtain a three-dimensional mesh model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image; wherein the each sample image and the camera pose corresponding to each sample image are used to construct the three-dimensional mesh model and the three-dimensional point cloud model;

[0011] a point cloud region determination module configured to determine a target region in the three-dimensional point cloud model according to position information of a face to be mapped in the three-dimensional mesh model;

[0012] an image determination module configured to determine a first texture image from the each sample image according to a camera pose corresponding to a point cloud in the target region;

[0013] a texture mapping module, configured to perform texture mapping on the to-be-mapped patch according to the first texture image.

[0014] In a third aspect, the present disclosure provides an electronic device, which comprises:

[0015] one or more processors; a storage device configured to store one or more programs,

[0016] when the one or more programs are executed by the one or more processors, the one or more processors implement the texture mapping method according to any one of the embodiments of the present disclosure.

[0017] In a fourth aspect, the present disclosure provides a storage medium comprising computer executable instructions for performing the texture mapping method according to any one of the embodiments of the present disclosure when executed by a computer processor.

[0018] In the technical solution of the present disclosure, a three-dimensional mesh model, a three-dimensional point cloud model, sample images and camera poses corresponding to the sample images are obtained, wherein the sample images and the camera poses corresponding to the sample images are used to construct the three-dimensional mesh model and the three-dimensional point cloud model; a target region in the three-dimensional point cloud model corresponding to a to-be-mapped patch in the three-dimensional mesh model is determined according to position information of the to-be-mapped patch; a first texture image is determined from the sample images according to camera poses corresponding to point clouds in the target region; and texture mapping is performed on the to-be-mapped patch according to the first texture image.

[0019] In the construction process of the three-dimensional point cloud model, it can be determined whether each point is visible under each camera pose, and this information can be referred to as visibility information of the point. By combining the visibility information of the point clouds in the region corresponding to the patch in the three-dimensional point cloud model in the patch mapping process, a texture image under an optimal camera pose can be selected from the sample images. Compared with the existing method of determining a texture image under a corresponding camera pose based on normal information of a patch, the technical solution of the present disclosure can determine a texture image under a corresponding camera pose based on the visibility information of the point cloud, which can make the selection of the texture image more accurate. Furthermore, texture mapping is performed according to the texture image under the optimal camera pose, which can improve the texture mapping effect.

[0020] In the construction process of the three-dimensional point cloud model, it can be determined whether each point is visible under each camera pose, and this information can be referred to as visibility information of the point. By combining the visibility information of the point clouds in the region corresponding to the patch in the three-dimensional point cloud model in the patch mapping process, a texture image under an optimal camera pose can be selected from the sample images. Compared with the existing method of determining a texture image under a corresponding camera pose based on normal information of a patch, the technical solution of the present disclosure can determine a texture image under a corresponding camera pose based on the visibility information of the point cloud, which can make the selection of the texture image more accurate. Furthermore, texture mapping is performed according to the texture image under the optimal camera pose, which can improve the texture mapping effect. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings. The same or similar components have the same or similar reference labels. It should be understood that the drawings are not necessarily to scale, with emphasis instead being placed upon illustrating the principles of the embodiments of the present disclosure.

[0022] Figure 1 A flowchart of a texture mapping method provided by an embodiment of the present disclosure;

[0023] Figure 2 A block diagram of a texture mapping method provided by an embodiment of the present disclosure;

[0024] Figure 3 A flowchart of a texture mapping method provided by an embodiment of the present disclosure;

[0025] Figure 4 A flowchart of a texture mapping method provided by an embodiment of the present disclosure;

[0026] Figure 5 A schematic diagram of pixel value comparison in a texture mapping method provided by an embodiment of the present disclosure;

[0027] Figure 6 A flowchart of a texture mapping method provided by an embodiment of the present disclosure;

[0028] Figure 7 A schematic diagram of wall mapping provided by an embodiment of the present disclosure;

[0029] Figure 8 A schematic diagram of a texture hole provided by an embodiment of the present disclosure;

[0030] Figure 9 A schematic diagram of hole gap filling in a texture mapping method provided by an embodiment of the present disclosure;

[0031] Figure 10 A schematic diagram of Gaussian blur in a texture mapping method provided by an embodiment of the present disclosure;

[0032] Figure 11 A structural schematic diagram of a texture mapping device provided by an embodiment of the present disclosure;

[0033] Figure 12 A structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein, but rather, these embodiments are provided so as to more completely and thoroughly understand the present disclosure. It is understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0035] It should be understood that each of the steps recited in the method embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0036] The term "comprising" and variations thereof as used herein are open-ended, that is, "comprising but not limited to." The term "based on" is "based, at least in part, on." The term "one embodiment" means "at least one embodiment." The term "another embodiment" means "at least one additional embodiment." The term "some embodiments" means "at least some embodiments." Related definitions are given below in the description of the various terms.

[0037] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0038] It should be noted that the modification of "one" or "multiple" mentioned in the present disclosure is illustrative and not limiting, and those skilled in the art should understand that, unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0039] It can be understood that the data involved in the technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the relevant laws, regulations and provisions.

[0040] Figure 1 A flowchart of a texture mapping method provided by an embodiment of the present disclosure is shown. The embodiment of the present disclosure is applicable to the case of texture mapping. The method can be performed by a texture mapping device, which can be implemented in the form of software and / or hardware, and can be configured in an electronic device, such as a computer.

[0041] As shown in Figure 1 The texture mapping method provided by the embodiment can include:

[0042] S110, obtaining a three-dimensional mesh model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image.

[0043] In the embodiments of the present disclosure, each sample image and the camera pose corresponding to each sample image are used to construct a three-dimensional mesh model and a three-dimensional point cloud model. The three-dimensional mesh model can be considered as a mesh model that needs to be texture mapped. The three-dimensional point cloud model can include a dense point cloud model. Since the three-dimensional mesh model and the three-dimensional point cloud model are constructed based on the same sample images, the three-dimensional mesh model and the three-dimensional point cloud model have a corresponding relationship in size and shape.

[0044] The camera pose corresponding to each sample image can be considered as the camera extrinsic parameters (i.e., the rotation parameters and translation parameters of the camera) corresponding to each sample image. The camera extrinsic parameters corresponding to each sample image can be determined using an existing pose solving method, such as the Structure from Motion (SfM). The camera intrinsic parameters can include the focal length, principal point coordinates, and distortion parameters, and can be considered as the same and known for each sample image. The pixel point coordinates in each sample image can be projected into the world coordinate system based on the camera pose and the camera intrinsic parameters, and the sparse point cloud data corresponding to each sample image can be obtained.

[0045] After obtaining the sparse point cloud data, the three-dimensional mesh model and the three-dimensional point cloud model can be constructed, and the steps of constructing the two models can include, for example:

[0046] On the one hand, the world coordinates of the sparse point cloud data can be used to construct a Signed Distance Field (SDF) model. The SDF model is a way to represent the geometric information of a three-dimensional space. For a point in the space, a signed distance value is defined, and the positive / negative sign in the signed distance value can represent that the point is inside / outside the three-dimensional model, and the distance value can represent the distance from the point to the nearest edge point of the three-dimensional model. The surface of the three-dimensional model can be determined based on the part of the signed distance value output by the constructed SDF model being 0, so that the three-dimensional model in the space can be determined. Furthermore, the three-dimensional model can be converted into a three-dimensional mesh model based on an existing mesh model conversion method.

[0047] On the other hand, a Multiple View Stereo (MVS) algorithm can be used to estimate a depth image based on the camera pose, the camera intrinsic parameters, the sparse point cloud, and the sample images. Then, the depth image can be fused and other reconstruction steps are performed to obtain a dense point cloud model.

[0048] In addition, other methods of constructing a three-dimensional mesh model and a three-dimensional point cloud model can also be applied, which are not limited here.

[0049] For example, Figure 2This is a block diagram illustrating a texture mapping method provided in an embodiment of this disclosure. See also... Figure 2 For object A that needs to be reconstructed in three dimensions Figure 2 Using a cuboid as an example, sample images from different perspectives can be acquired through an image acquisition device. Each sample image and its corresponding camera pose can be used to construct a 3D mesh model and a 3D point cloud model of object A. The 3D mesh model and the 3D point cloud model have a corresponding relationship in size and shape.

[0050] S120. Based on the position information of the facet to be mapped in the 3D mesh model, determine the corresponding target region in the 3D point cloud model.

[0051] In this embodiment of the disclosure, texture mapping needs to be performed on each facet (usually a triangular facet) in the 3D mesh model to make the reconstructed 3D model realistic. The facet to be mapped can be considered as the facet currently requiring texture mapping; the facets in the 3D mesh model can be selected sequentially as the facets to be mapped, or other methods can be used to determine the facets to be mapped, without specific limitations here.

[0052] The positional information of the facet to be mapped can include its relative position within the 3D mesh model. Since the 3D mesh model and the 3D point cloud model have a correspondence, the target region corresponding to the positional information of the facet to be mapped can be determined from the 3D point cloud model.

[0053] The process of determining the target region may include, for example, statistically analyzing parameters such as the side length or circumcircle radius of each facet in the 3D mesh model, and determining a radius value based on these parameters (e.g., using the average circumcircle radius as the radius value). Based on the center position of the facet to be mapped, the center point of the target region in the 3D point cloud model is determined. A spherical region can be defined in the 3D point cloud model with the center point as the center and the radius value as the radius; this spherical region can then be considered the target region.

[0054] S130. Determine the first texture image from each sample image based on the camera pose corresponding to the point cloud within the target area.

[0055] In the construction of the three-dimensional point cloud model, whether each point is visible under the camera pose corresponding to each sample image can be determined, and the information of whether the point is visible can be referred to as visibility information of the point. In the embodiments of the present disclosure, after the target region in the three-dimensional point cloud model is determined, the visibility information of each point in the target region can be viewed. Moreover, the camera pose corresponding to the sample image in which the visible point ratio in the target region reaches a threshold condition can be taken as the camera pose corresponding to the point cloud in the target region. Therefore, the sample image corresponding to the camera pose can be taken as the first texture image.

[0056] In traditional texture mapping, whether each point is visible under the camera pose corresponding to each sample image is determined based on the normal information of the patch to be mapped, so as to select the texture image. Since texture mapping is the last joint of the three-dimensional reconstruction link, it is often affected by the errors introduced in the previous links (for example, pose errors, three-dimensional mesh model construction errors, etc.) and the occlusion relationship between the foreground object and the background object, resulting in low accuracy of the texture image determined based on the normal information of the patch, and a large number of texture mapping errors of the foreground object to the background object, which seriously affects the authenticity of the three-dimensional model.

[0057] In the embodiments of the present disclosure, based on the visibility information of each point in the target region corresponding to the patch in the three-dimensional point cloud model, a more accurate first texture image that can completely capture the patch to be mapped can be selected, and sample images that do not completely capture the patch to be mapped can be screened out, which can to some extent avoid the problem of texture mapping error and is conducive to improving the texture mapping effect.

[0058] S140, performing texture mapping on the patch to be mapped according to the first texture image.

[0059] In the embodiments of the present disclosure, based on the existing texture mapping method, the texture of the region corresponding to the patch to be mapped in the first texture image can be used for texture mapping on the patch to be mapped.

[0060] In the embodiments of the present disclosure, a three-dimensional mesh model, a three-dimensional point cloud model, sample images, and camera poses corresponding to the sample images are obtained. The sample images and the camera poses corresponding to the sample images are used to construct the three-dimensional mesh model and the three-dimensional point cloud model. The position information of the patch to be mapped in the three-dimensional mesh model is used to determine the target region in the three-dimensional point cloud model. The camera pose corresponding to the point cloud in the target region is used to determine the first texture image from the sample images. The patch to be mapped is texture mapped according to the first texture image.

[0061] In the construction process of the three-dimensional point cloud model, whether each point is visible under each camera pose can be determined, and the information of whether visible can be referred to as visibility information of the point. By combining the visibility information of the point cloud in the region corresponding to the patch in the three-dimensional point cloud model in the patch mapping process, a texture image under an optimal camera pose can be selected from the sample images. Compared with the existing method of determining the texture image under the corresponding camera pose based on the normal information of the patch, in the technical solution of the embodiment of the present disclosure, the texture image under the corresponding camera pose can be determined based on the visibility information of the point cloud, so that the selection of the texture image is more accurate. Furthermore, the texture mapping effect can be improved by performing texture mapping according to the selected texture image under the optimal camera pose.

[0062] The embodiment of the present disclosure can be combined with the optional schemes in the texture mapping method provided in the above embodiments. The texture mapping method provided in the embodiment describes the selection process of the optimal texture image in detail. By selecting the optimal texture image by combining the strong and weak texture information provided by the dense point cloud, the problem of mapping error can be further reduced.

[0063] Figure 3 A flowchart of a texture mapping method provided in an embodiment of the present disclosure is shown in FIG. 1. As shown in FIG. 1, the texture mapping method provided in the embodiment can include the following steps. Figure 3

[0064] In S310, a three-dimensional grid model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image are acquired.

[0065] In the embodiment, each sample image and the camera pose corresponding to each sample image are used to construct the three-dimensional grid model and the three-dimensional point cloud model.

[0066] In S320, a target region in the three-dimensional point cloud model is determined according to position information of a to-be-mapped patch in the three-dimensional grid model.

[0067] In S331, a first texture image is determined from each sample image according to a camera pose corresponding to the point cloud in the target region.

[0068] In S332, an index item for image selection is set according to a number of point clouds in the target region.

[0069] The number of point clouds in the target region can be used as a standard for measuring the texture strength of the to-be-mapped patch. When the number of point clouds is large, it can be considered that the texture corresponding to the to-be-mapped patch is rich, that is, the patch belongs to a strong texture patch. Conversely, when the number of point clouds is small, it can be considered that the texture corresponding to the to-be-mapped patch is simple, that is, the to-be-mapped patch belongs to a weak texture patch.

[0070] ​The texture mapping process can be modeled based on a mathematical model of a Markov random field, and an objective function can be set for the mathematical model of the Markov random field. The objective function can include a data item and a smoothing item, where the data item can be used to evaluate the quality of the texture image, and the smoothing item can be used for continuity after texture mapping. In the embodiments of the present disclosure, the indicator item of image selection can include the data item in the objective function of the Markov random field.

[0071] The different indicator items of image selection can be set according to the number of point clouds in the target region. That is, different texture image selection strategies can be set for strong texture patches and weak texture patches to obtain texture images with better effects. For example, for a strong texture patch, an indicator item can be set according to the texture clarity of the region in the sample image projected by the to-be-mapped patch, so as to select an image with clear texture for mapping; for a weak texture patch, an indicator item can be set according to the area of the region in the sample image projected by the to-be-mapped patch, so as to select an image covering the patch most for mapping. Thus, the selected texture image can be more accurate, and the problem of mapping error can be further reduced.

[0072] In some optional implementations, setting the indicator item of image selection according to the number of point clouds in the target region can include: in a case where the number of point clouds in the target region is greater than or equal to a preset threshold, setting a texture gradient integral of a projection region as the indicator item; wherein the projection region includes a region after the to-be-mapped patch is projected to the first texture image; in a case where the number of point clouds in the target region is less than the preset threshold, setting an area of the projection region as the indicator item.

[0073] The preset threshold can be set according to an experimental value or an empirical value, for example, can be set to 20.

[0074] When the number of points in the point cloud in the target region is greater than or equal to 20, the to-be-mapped patch can be considered as a strong texture patch. At this time, the texture gradient integral of the region after the to-be-mapped patch is projected to each first texture image can be set as the indicator item. The greater the texture gradient integral, on the one hand, can represent the higher the texture clarity, and on the other hand, can represent the larger the area of the projection region. The first texture image with a larger value of the texture gradient integral can be selected as the second texture image of the to-be-mapped patch.

[0075] When the number of points in the point cloud in the target region is less than 20, the to-be-mapped patch can be considered as a weak texture patch. At this time, the area of the region after the to-be-mapped patch is projected to each first texture image can be set as the indicator item. The larger the area, the more the area of the first texture image covering the to-be-mapped patch, and the clearer the texture details. The first texture image with a larger value of the area can be selected as the second texture image of the to-be-mapped patch.

[0076] In the optional implementation, the texture gradient integral is set as the index item for the strong texture region, and the area is set as the index item for the weak texture region, so that the selection of the optimal texture image can be realized.

[0077] S340, determining the second texture image from the first texture image based on the set index item.

[0078] In the embodiment, the second texture image with better image quality can be selected from the first texture images according to the numerical value of the index item.

[0079] S350, performing texture mapping on the to-be-mapped patch according to the second texture image.

[0080] The technical solution of the embodiment of the present disclosure describes the selection process of the optimal texture image in detail. The selection of the optimal texture image is performed by combining the strong and weak texture information provided by the dense point cloud, which can further reduce the problem of texture mapping error. The texture mapping method provided by the embodiment of the present disclosure belongs to the same disclosure concept as the texture mapping method provided by the above embodiment, and the technical details not described in detail in the embodiment can be referred to the above embodiment, and the same technical features have the same beneficial effects in the embodiment and the above embodiment.

[0081] The texture mapping method provided by the embodiment of the present disclosure can be combined with the optional schemes in the texture mapping method provided in the above embodiment. The texture mapping method provided by the embodiment describes the selection process of the optimal texture image in detail. The first texture image with inconsistent pixel colors is screened out by combining the texture atlas rendered by the neural radiance field model, which can further reduce the problem of texture mapping error.

[0082] Figure 4 A flowchart of a texture mapping method provided by an embodiment of the present disclosure is shown in FIG. 1. Figure 4 As shown in FIG. 1, the texture mapping method provided by the embodiment can include the following steps.

[0083] S410, obtaining a texture atlas, a three-dimensional mesh model, a three-dimensional point cloud model, sample images, and camera poses corresponding to the sample images.

[0084] In the embodiment, the sample images and the camera poses corresponding to the sample images are used to construct the three-dimensional mesh model and the three-dimensional point cloud model.

[0085] The texture atlas is rendered based on the sample images by a neural radiance field model.

[0086] The neural radiance field (NeRF) model is a computer vision technology that uses deep learning techniques to extract the collective shape and texture information of an object from multiple perspective images, and then uses this information to generate a continuous three-dimensional radiance field, so that a highly realistic three-dimensional model / scene can be presented at any angle and distance. The rendering process of the texture atlas can include: the color (i.e., pixel value) of the three-dimensional model at each perspective can be rendered by the NeRF model; the color can be filled into the corresponding pixel position of the two-dimensional image; and the two-dimensional image with filled pixels can be referred to as a texture atlas.

[0087] S420, determining a corresponding target region in the three-dimensional point cloud model according to the position information of the to-be-mapped patch in the three-dimensional mesh model.

[0088] S430, determining a first texture image from each sample image according to the camera pose corresponding to the point cloud in the target region.

[0089] S440, determining a third texture image from the first texture image according to the pixel value of each pixel in the projection region and the pixel value of each pixel in the corresponding region in the texture atlas.

[0090] The projection region includes the region after the to-be-mapped patch is projected onto the first texture image.

[0091] In this embodiment, the pixel values of each pixel in the region after the to-be-mapped patch is projected onto the first texture image can be compared one by one with the pixel values of each pixel in the corresponding region of the to-be-mapped patch in the texture atlas. The first texture image with consistent comparison results can be selected as the third texture image, and the first texture image with inconsistent color can be screened out. The consistent comparison results can include that the number of pixel points with the same pixel value accounts for a preset proportion. By selecting the third texture image with high consistency of comparison results, the accuracy of the selected texture image can be improved, which is beneficial to improving the texture mapping effect.

[0092] Exemplarily, Figure 5 A schematic diagram of pixel value comparison in a texture mapping method provided by the embodiment of the present disclosure is shown in FIG. 4. Referring to FIG. 4, Figure 5 The to-be-mapped patch can be a patch of a sofa backrest. The pixel values of each pixel in the region after the to-be-mapped patch is projected onto the first texture image can be compared one by one with the pixel values of each pixel in the corresponding region of the to-be-mapped patch in the texture atlas. The first texture image with inconsistent pixel values can be screened out, for example, the first texture image in which the number of pixel points with the same pixel value does not account for a preset proportion, so that the problem of texture mapping error can be avoided to some extent.

[0093] S450, performing texture mapping on the to-be-mapped patch according to the third texture image.

[0094] The technical solution of the embodiments of the present disclosure describes the selection process of the better texture image in detail. By combining the texture map set rendered by the neural radiance field model, the first texture image with inconsistent pixel colors can be screened out, which can further reduce the texture mapping error problem. The texture mapping method provided by the embodiments of the present disclosure belongs to the same disclosure concept as the texture mapping method provided by the above embodiments. The technical details not described in detail in the present embodiment can be referred to the above embodiments, and the same technical features have the same beneficial effects in the present embodiment and the above embodiments.

[0095] The embodiments of the present disclosure can be combined with the various optional solutions in the texture mapping method provided in the above embodiments. The texture mapping method provided by the present embodiment describes the texture mapping process when the three-dimensional mesh model contains a target plane in detail. By mapping the patches in the target plane in the three-dimensional mesh model as a whole, the texture mapping error problem can be reduced, and the overall texture can be more compact.

[0096] Figure 6 A flowchart of a texture mapping method provided by an embodiment of the present disclosure is shown in FIG. 6. As shown in FIG. 6, the texture mapping method provided by the present embodiment can include the following steps. Figure 6

[0097] S610, obtaining a three-dimensional mesh model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image.

[0098] In the present embodiment, each sample image and the camera pose corresponding to each sample image are used to construct the three-dimensional mesh model and the three-dimensional point cloud model.

[0099] S620, determining a target plane in the three-dimensional mesh model according to normal information of each patch in the three-dimensional mesh model.

[0100] In the present embodiment, the target plane in the three-dimensional mesh model can be determined by using the normal information of each patch in the three-dimensional mesh model. The target plane can include at least two patches that satisfy the normal information condition. For example, in the case where the normal information of adjacent patches in the three-dimensional mesh model is consistent, the adjacent patches can be considered to constitute a target plane.

[0101] S630, merging the patches in each target plane to obtain a merged patch.

[0102] In the present embodiment, the three-dimensional mesh model can include at least one target plane, and each target plane exists independently. For each target plane region, the patches in the target plane can be merged as a whole, and the patches can be merged into one patch for texture mapping processing, which is beneficial to improve the texture mapping effect. ​

[0103] S640, determine a target region in the three-dimensional point cloud model according to the position information of the merged face patch in the three-dimensional mesh model.

[0104] In this embodiment, the merged face patch can be taken as a to-be-mapped face patch, and the target region corresponding to the merged face patch can be determined in the three-dimensional point cloud model.

[0105] S650, determine a first texture image from each sample image according to the camera pose corresponding to the point cloud in the target region.

[0106] S660, perform texture mapping on the merged face patch according to the first texture image.

[0107] Exemplarily, Figure 7 A schematic diagram of wall mapping provided by the embodiment of the present disclosure. Referring to Figure 7 , the target plane in the three-dimensional mesh model can be a wall. Figure 7 (a) The effect of separately mapping each face patch in the wall may have a mapping misplacement problem, which seriously affects the authenticity of the wall. Figure 7 (b) The effect of mapping the face patches in the wall as a whole can effectively avoid the mapping misplacement problem, making the overall texture more compact.

[0108] The technical solution of the embodiment of the present disclosure describes in detail the texture mapping process under the condition that the three-dimensional mesh model contains a target plane. By mapping the face patches in the target plane in the three-dimensional mesh model as a whole, the mapping misplacement problem can be reduced, and the overall texture can be more compact. The texture mapping method provided by the embodiment of the present disclosure belongs to the same disclosure concept as the texture mapping method provided by the above embodiment, and the technical details not described in detail in this embodiment can be referred to the above embodiment, and the same technical features have the same beneficial effects in this embodiment and the above embodiment.

[0109] The texture mapping method provided by the embodiment of the present disclosure can be combined with the various optional solutions in the texture mapping method provided in the above embodiment. The texture mapping method provided by the embodiment of the present disclosure describes in detail the texture hole repair under the condition that the three-dimensional model contains texture holes after texture mapping. By removing small texture regions in the texture holes, more texture holes can be repaired. By performing reverse Gaussian blur from the outside to the inside before texture hole repair, it is beneficial to avoid the occurrence of gaps after hole repair.

[0110] Figure 8 A schematic diagram of a texture hole provided by the embodiment of the present disclosure. As Figure 8As shown, when the sample image cannot completely cover the surface of the object to be reconstructed in 3D, the 3D mesh model after texture mapping will have areas without mapped texture, which can be called texture holes. Existing texture hole-filling algorithms can be used to fill these unmapped texture areas with appropriate colors. However, existing texture hole-filling algorithms have high requirements on the topological structure of the texture hole areas; for example, textured patches cannot exist inside the texture holes. Figure 8 The face within the Chinese frame is a face with texture within a texture hole, which makes it impossible to fill many texture holes.

[0111] The texture mapping method provided in this embodiment can identify the target surface that has been texture mapped inside the texture hole when there are texture holes in the textured 3D mesh model after texture mapping is completed; after removing the texture of the target surface, the texture hole is filled.

[0112] This involves using existing image processing methods, such as Breadth-First Search (BFS), to scan for textured patches within texture holes. Furthermore, the texture within these patches can be removed, thereby adjusting the topological structure of the texture holes. This allows for texture hole repair based on existing texture hole repair algorithms, enabling the repair of even more texture holes.

[0113] The principle behind texture patching can be as follows: Based on an existing image unwrapping method (e.g., UV unwrapping), the texture holes are unwrapped onto a 2D image, and the intersection of the unwrapped shape with the pixels of the 2D image is calculated to obtain the edges of the texture holes (usually circular). Since this is pixel-level processing, there is typically an error of 1 to 2 pixels. This results in points that were originally inside the texture hole being reclassified as outside, and vice versa. In such cases, gaps usually remain after texture patching.

[0114] For example, Figure 9 This is a schematic diagram illustrating the patching of holes and gaps in a texture mapping method provided in an embodiment of this disclosure. See also... Figure 9 The texture patching algorithm is affected by pixel-level errors, resulting in poor edge continuity after texture patching and gaps at the edges of the patched texture.

[0115] In some optional implementations, after removing the texture of the target patch, before the hole filling processing, the method further includes: performing Gaussian blur processing on the texture hole from outside to inside. In these optional implementations, before the hole filling, the texture hole is processed by a preset number of times of reverse Gaussian blur from outside to inside. The preset number of times can be set according to empirical values or experimental values. Through multiple reverse Gaussian blurs, pixel values can be set for points that were originally inside the texture hole but are divided outside the texture hole, and the edge gap problem after texture hole filling can be largely solved.

[0116] In addition, after the texture hole filling, the texture hole edge can be processed by forward Gaussian blur from inside to outside, and appropriate pixel values can be set for points that were originally outside the texture hole but are divided inside the texture hole. By performing reverse Gaussian blur before hole filling and forward Gaussian blur after hole filling, the final presented texture hole filling result can have no visible texture gap, and the texture hole filling effect is good.

[0117] An exemplary Figure 10 schematic diagram of Gaussian blur in a texture mapping method provided by an embodiment of the present disclosure. Referring to Figure 10 , the annular region part is the effect after a preset number of times of reverse Gaussian blur processing before texture hole filling, and the central blur region is the effect of hole filling processing based on existing Poisson editing. By performing multiple reverse Gaussian blurs before hole filling, the edge gap problem after texture hole filling can be largely solved.

[0118] The technical solutions of the embodiments of the present disclosure are described in detail for texture hole filling in the case that the three-dimensional model after texture mapping contains texture holes. By removing small texture regions inside the texture hole, more texture holes can be repaired. By performing reverse Gaussian blur from outside to inside before texture hole filling, it is beneficial to avoid the occurrence of a gap after hole filling. The texture mapping method provided by the embodiments of the present disclosure belongs to the same disclosure concept as the texture mapping method provided by the above embodiments, and the technical details not described in detail in the present embodiment can be referred to the above embodiments, and the same technical features have the same beneficial effects in the present embodiment and the above embodiments.

[0119] Figure 11 A structural schematic diagram of a texture mapping device provided by an embodiment of the present disclosure. The texture mapping device provided by the present embodiment is suitable for the case of texture mapping.

[0120] As Figure 11 shown, the texture mapping device provided by the present embodiment can include:

[0121] The data acquisition module 1101 is configured to acquire a three-dimensional mesh model, a three-dimensional point cloud model, sample images, and camera poses corresponding to the sample images; the sample images and the camera poses corresponding to the sample images are used to construct the three-dimensional mesh model and the three-dimensional point cloud model.

[0122] The point cloud region determination module 1102 is configured to determine a target region in the three-dimensional point cloud model according to position information of a to-be-mapped face in the three-dimensional mesh model.

[0123] The image determination module 1103 is configured to determine a first texture image from the sample images according to camera poses corresponding to point clouds in the target region.

[0124] The texture mapping module 1104 is configured to perform texture mapping on the to-be-mapped face according to the first texture image.

[0125] In some optional implementations, the image determination module can be further configured to:

[0126] After the target region in the three-dimensional point cloud model is determined, an index item for image selection is set according to a quantity of point clouds in the target region.

[0127] A second texture image is determined from the first texture image based on the set index item.

[0128] Correspondingly, the texture mapping module can be configured to perform texture mapping on the to-be-mapped face according to the second texture image.

[0129] In some optional implementations, the image determination module can be further configured to:

[0130] In a case where the quantity of point clouds in the target region is greater than or equal to a preset threshold, a texture gradient integral of a projection region is taken as the index item; the projection region includes a region after the to-be-mapped face is projected to the first texture image.

[0131] In a case where the quantity of point clouds in the target region is less than the preset threshold, an area of the projection region is taken as the index item.

[0132] In some optional implementations, the image determination module can be further configured to:

[0133] A texture atlas is acquired; the texture atlas is obtained based on the sample images by a neural radiance field model.

[0134] A third texture image is determined from the first texture image according to pixel values of pixels in a projection region and pixel values of pixels in a corresponding region in the texture atlas; the projection region includes a region after the to-be-mapped face is projected to the first texture image.

[0135] Correspondingly, the texture mapping module can be configured to perform texture mapping on the to-be-mapped patch according to the third texture image.

[0136] In some optional implementation manners, the texture mapping apparatus can further include:

[0137] a patch merging module configured to determine target planes in the three-dimensional mesh model according to normal information of the patches in the three-dimensional mesh model;

[0138] merge the patches in each target plane to obtain merged patches;

[0139] Correspondingly, the point cloud region determination module can be configured to:

[0140] determine a corresponding target region in the three-dimensional point cloud model according to position information of the merged patches in the three-dimensional mesh model.

[0141] In some optional implementation manners, the texture mapping apparatus can further include:

[0142] a hole filling module configured to, in a case where the three-dimensional mesh model after texture mapping has a texture hole, determine target patches that have been texture mapped inside the texture hole;

[0143] fill the texture hole after removing the texture of the target patches.

[0144] In some optional implementation manners, the hole filling module can be further configured to, before filling the texture hole, perform:

[0145] Gaussian blur processing on the texture hole from outside to inside.

[0146] The texture mapping apparatus provided in the embodiments of the present disclosure can perform the texture mapping method provided in any of the embodiments of the present disclosure, and has the corresponding functional modules and beneficial effects of performing the method.

[0147] It should be noted that each unit and module included in the apparatus is only logically divided according to the function, and is not limited to the above division, as long as the corresponding function can be implemented; in addition, the specific name of each functional unit is only for convenient distinction, and does not limit the protection scope of the embodiments of the present disclosure.

[0148] Reference will be made to the following Figure 12 which shows an electronic device (for example, a mobile phone) suitable for implementing the embodiments of the present disclosure. Figure 12The diagram below shows the structure of the terminal device or server 1200. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 12 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0149] like Figure 12 As shown, the electronic device 1200 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage device 1208 into a random access memory (RAM) 1203. The RAM 1203 also stores various programs and data required for the operation of the electronic device 1200. The processing unit 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0150] Typically, the following devices can be connected to I / O interface 1205: input devices 1206 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1207 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1208 including, for example, magnetic tape, hard disk, etc.; and communication devices 1209. Communication device 1209 allows electronic device 1200 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 12 An electronic device 1200 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0151] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1209, or installed from the storage device 1208, or installed from the ROM 1202. When the computer program is executed by the processing device 1201, the above-mentioned functions defined in the texture mapping method of the embodiment of the present disclosure are executed.

[0152] The electronic device provided by the embodiment of the present disclosure belongs to the same disclosure concept as the texture mapping method provided by the above-mentioned embodiment, and the technical details not described in detail in the present embodiment can be referred to the above-mentioned embodiment, and the present embodiment has the same beneficial effects as the above-mentioned embodiment.

[0153] The embodiment of the present disclosure provides a computer storage medium, which stores a computer program, and the program is executed by a processor to implement the texture mapping method provided by the above-mentioned embodiment.

[0154] It should be noted that the computer-readable medium described above can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave in a propagated data signal, in which the computer-readable program code is contained. Such a propagated data signal can take any of a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. The computer-readable signal medium can also be any computer-readable medium that is not a storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including, but not limited to, wire, cable, RF (radio frequency), or the like, or any suitable combination of the foregoing.

[0155] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (Hyper Text Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0156] The computer-readable medium described above can be included in the electronic device described above; or can exist separately from the electronic device described above, and can be accessed via the electronic device described above.

[0157] The computer-readable medium described above carries one or more programs, when the one or more programs are executed by the electronic device described above, cause the electronic device to:

[0158] The three-dimensional mesh model, the three-dimensional point cloud model, the sample images, and the camera poses corresponding to the sample images are acquired; the sample images and the camera poses corresponding to the sample images are used to construct the three-dimensional mesh model and the three-dimensional point cloud model; a target region in the three-dimensional point cloud model is determined according to position information of a to-be-mapped face in the three-dimensional mesh model; a first texture image is determined from the sample images according to the camera poses corresponding to the point cloud in the target region; and the to-be-mapped face is texture-mapped according to the first texture image.

[0159] Computer program code for carrying out operations of the present disclosure can be written in any of one or more programming languages, including object oriented programming languages such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0160] The flow and block diagrams in the drawings show architectural, functional, and operational representations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may be executed in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or combinations of hardware and software.

[0161] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the names of the units and modules do not constitute a limitation on the units and modules themselves.

[0162] The functionality described herein above can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, an example type of hardware logic components that can be used include Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Parts (ASSPs), System on Chips (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0163] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0164] According to one or more embodiments of the present disclosure, a texture mapping method is provided, which comprises:

[0165] obtaining a three-dimensional mesh model, a three-dimensional point cloud model, sample images, and camera poses corresponding to the sample images; wherein the sample images and the camera poses corresponding to the sample images are used to construct the three-dimensional mesh model and the three-dimensional point cloud model;

[0166] determining a target region in the three-dimensional point cloud model according to position information of a to-be-mapped patch in the three-dimensional mesh model;

[0167] determining a first texture image from the sample images according to camera poses corresponding to point clouds in the target region;

[0168] performing texture mapping on the to-be-mapped patch according to the first texture image.

[0169] According to one or more embodiments of the present disclosure, a texture mapping method is provided, which further comprises:

[0170] In some optional implementations, after the corresponding target region in the three-dimensional point cloud model is determined, the method further includes:

[0171] According to the number of point clouds in the target region, an index item for image selection is set;

[0172] Accordingly, the texture mapping of the to-be-mapped face sheet according to the first texture image includes:

[0173] Based on the set index item, a second texture image is determined from the first texture image;

[0174] According to the second texture image, the to-be-mapped face sheet is texture-mapped.

[0175] According to one or more embodiments of the present disclosure, a texture mapping method is provided, which further includes:

[0176] In some optional implementations, the setting of the index item for image selection according to the number of point clouds in the target region includes:

[0177] In a case where the number of point clouds in the target region is greater than or equal to a preset threshold, a texture gradient integral of a projection region is taken as the index item; the projection region includes a region after the to-be-mapped face sheet is projected to the first texture image;

[0178] In a case where the number of point clouds in the target region is less than the preset threshold, an area of the projection region is taken as the index item.

[0179] According to one or more embodiments of the present disclosure, a texture mapping method is provided, which further includes:

[0180] In some optional implementations, a texture atlas is obtained; the texture atlas is obtained based on the sample images by a neural radiance field model;

[0181] Accordingly, the texture mapping of the to-be-mapped face sheet according to the first texture image includes:

[0182] According to pixel values of each pixel in a projection region and pixel values of each pixel in a corresponding region in the texture atlas, a third texture image is determined from the first texture image; the projection region includes a region after the to-be-mapped face sheet is projected to the first texture image;

[0183] According to the third texture image, the to-be-mapped face sheet is texture-mapped.

[0184] According to one or more embodiments of the present disclosure, a texture mapping method is provided, which further includes:

[0185] In some optional implementations, a target plane in the three-dimensional mesh model is determined according to normal information of each face in the three-dimensional mesh model.

[0186] The face in each target plane is merged to obtain a merged face.

[0187] Correspondingly, a target region in the three-dimensional point cloud model is determined according to position information of a to-be-mapped face in the three-dimensional mesh model, including:

[0188] The target region in the three-dimensional point cloud model is determined according to position information of the merged face in the three-dimensional mesh model.

[0189] According to one or more embodiments of the present disclosure, a texture mapping method is provided, further including:

[0190] In some optional implementations, in a case where the three-dimensional mesh model after texture mapping has a texture hole, a target face in the texture hole that has been subjected to texture mapping is determined.

[0191] After the texture of the target face is removed, the texture hole is subjected to a hole filling process.

[0192] According to one or more embodiments of the present disclosure, a texture mapping method is provided, further including:

[0193] In some optional implementations, after the texture of the target face is removed, before the hole filling process is performed, the method further includes:

[0194] The texture hole is subjected to an outward-to-inward Gaussian blur process.

[0195] According to one or more embodiments of the present disclosure, a texture mapping device is provided, including:

[0196] A data acquisition module is configured to acquire a three-dimensional mesh model, a three-dimensional point cloud model, sample images, and camera poses corresponding to the sample images, wherein the sample images and the camera poses corresponding to the sample images are used to construct the three-dimensional mesh model and the three-dimensional point cloud model.

[0197] A point cloud region determination module is configured to determine a target region in the three-dimensional point cloud model according to position information of a to-be-mapped face in the three-dimensional mesh model.

[0198] An image determination module is configured to determine a first texture image from the sample images according to camera poses corresponding to point clouds in the target region.

[0199] The texture mapping module is configured to map the first texture image to the to-be-mapped face.

[0200] The above description is merely that of the preferred embodiments of the present disclosure and the description of the technical principles of the application. It should be understood by those skilled in the art that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by the combinations of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features with similar functions disclosed in the present disclosure (but not limited to) can be used.

[0201] In addition, although each operation is described in a particular order, this should not be understood as requiring the operations to be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing can be advantageous. Similarly, although several implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be combined in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented separately or in any suitable subcombination.

[0202] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method of texture mapping, characterized by, The method comprises: acquiring a three-dimensional mesh model, a three-dimensional point cloud model, sample images, and camera poses corresponding to the sample images; wherein the sample images and the camera poses corresponding to the sample images are used to construct the three-dimensional mesh model and the three-dimensional point cloud model; determining a target region in the three-dimensional point cloud model according to position information of a face to be mapped in the three-dimensional mesh model; determining a first texture image from the sample images according to camera poses corresponding to point clouds in the target region; performing texture mapping on the face to be mapped according to the first texture image.

2. The method of claim 1, wherein, After the target region in the three-dimensional point cloud model is determined, the method further comprises: setting an index item for image selection according to the number of point clouds in the target region; correspondingly, the texture mapping on the face to be mapped according to the first texture image comprises: determining a second texture image from the first texture image based on the set index item; performing texture mapping on the face to be mapped according to the second texture image.

3. The method of claim 2, wherein, The setting of the index item for image selection according to the number of point clouds in the target region comprises: in a case where the number of point clouds in the target region is greater than or equal to a preset threshold, setting a texture gradient integral of a projection region as the index item; wherein the projection region comprises a region after the face to be mapped is projected to the first texture image; in a case where the number of point clouds in the target region is less than the preset threshold, setting an area of the projection region as the index item.

4. The method of claim 1, wherein, The method further comprises: acquiring a texture atlas; wherein the texture atlas is obtained by rendering the sample images based on a neural radiance field model; correspondingly, the texture mapping on the face to be mapped according to the first texture image comprises: determining a third texture image from the first texture image according to pixel values of pixels in a projection region and pixel values of pixels in a corresponding region in the texture atlas; wherein the projection region comprises a region after the face to be mapped is projected to the first texture image; performing texture mapping on the face to be mapped according to the third texture image.

5. The method of claim 1, wherein, The method further comprises: determining target planes in the three-dimensional mesh model according to normal information of each face in the three-dimensional mesh model; merging the faces in each target plane to obtain merged faces; correspondingly, the determining of the target region in the three-dimensional point cloud model according to the position information of the face to be mapped in the three-dimensional mesh model comprises: determining the target region in the three-dimensional point cloud model according to position information of the merged faces in the three-dimensional mesh model.

6. The method of claim 1, wherein, The method further comprises: in a case where a three-dimensional mesh model after texture mapping has texture holes, determining target faces that have been texture mapped inside the texture holes; after removing the texture of the target faces, performing hole filling processing on the texture holes.

7. The method of claim 6, wherein, After the texture of the target faces is removed, before the hole filling processing is performed, the method further comprises: performing Gaussian blur processing on the texture holes from outside to inside.

8. A texture mapping apparatus characterized by comprising: The method comprises: The data acquisition module is configured to acquire a three-dimensional mesh model, a three-dimensional point cloud model, sample images, and camera poses corresponding to the sample images; wherein the sample images and the camera poses corresponding to the sample images are used to construct the three-dimensional mesh model and the three-dimensional point cloud model. The point cloud region determination module is configured to determine a target region in the three-dimensional point cloud model according to position information of a to-be-mapped face in the three-dimensional mesh model. The image determination module is configured to determine a first texture image from the sample images according to camera poses corresponding to point clouds in the target region. The texture mapping module is configured to perform texture mapping on the to-be-mapped face according to the first texture image.

9. An electronic device, comprising: The electronic device includes: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the texture mapping method of any one of claims 1-7.

10. A storage medium containing computer-executable instructions for performing the texture mapping method of any one of claims 1-7 when executed by a computer processor.

Images