Texture mapping method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a texture mapping method and device, electronic equipment and a storage medium. The method comprises the steps that a three-dimensional grid model, a three-dimensional point cloud model, sample images and camera poses corresponding to the sample images are acquired; wherein each sample image and the camera pose corresponding to each sample image are used for constructing the three-dimensional grid model and the three-dimensional point cloud model; determining a corresponding target area in the three-dimensional point cloud model according to position information of a to-be-mapped patch in the three-dimensional grid model; determining a first texture image from the sample images according to the camera pose corresponding to the point cloud in the target area; and performing texture mapping on the to-be-mapped patch according to the first texture image. A better image can be selected for texture mapping, and the texture mapping effect is improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and in particular to a texture mapping method, device, electronic device, and storage medium. Background Art

[0002] Through the texture mapping method, we can select the corresponding area in the appropriate image (view) as the texture for each face of the 3D mesh model and map it. The choice of image greatly affects the texture mapping effect, and how to select a better image has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present disclosure provide a texture mapping method, device, electronic device and storage medium, which can select a better image for texture mapping and improve the texture mapping effect.

[0004] In a first aspect, an embodiment of the present disclosure provides a texture mapping method, comprising:

[0005] Acquire a three-dimensional mesh model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image; wherein each sample image and the camera pose corresponding to each sample image are used to construct the three-dimensional mesh model and the three-dimensional point cloud model;

[0006] Determine a corresponding target area in the three-dimensional point cloud model according to position information of the to-be-mapped face in the three-dimensional grid model;

[0007] Determining a first texture image from each of the sample images according to a camera pose corresponding to a point cloud in the target area;

[0008] According to the first texture image, texture mapping is performed on the surface to be mapped.

[0009] In a second aspect, the present disclosure also provides a texture mapping device, including:

[0010] A data acquisition module, used to acquire a three-dimensional mesh model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image; wherein each sample image and the camera pose corresponding to each sample image are used to construct the three-dimensional mesh model and the three-dimensional point cloud model;

[0011] A point cloud region determination module, used to determine the corresponding target region in the three-dimensional point cloud model according to the position information of the to-be-mapped face in the three-dimensional grid model;

[0012] An image determination module, configured to determine a first texture image from each of the sample images according to a camera pose corresponding to a point cloud in the target area;

[0013] The texture mapping module is used to perform texture mapping on the surface to be mapped according to the first texture image.

[0014] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0015] one or more processors;

[0016] a storage device for storing one or more programs,

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the texture mapping method as described in any of the embodiments of the present disclosure.

[0018] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the texture mapping method as described in any one of the embodiments of the present disclosure.

[0019] The technical solution of the embodiment of the present disclosure is to obtain a three-dimensional mesh model, a three-dimensional point cloud model, each sample image and a camera pose corresponding to each sample image; wherein each sample image and the camera pose corresponding to each sample image are used to construct the three-dimensional mesh model and the three-dimensional point cloud model; according to the position information of the facet to be mapped in the three-dimensional mesh model, the corresponding target area in the three-dimensional point cloud model is determined; according to the camera pose corresponding to the point cloud in the target area, a first texture image is determined from each sample image; and according to the first texture image, texture mapping is performed on the facet to be mapped.

[0020] During the construction of the three-dimensional point cloud model, it is possible to determine whether each point is visible under each camera pose, and the information on whether it is visible can be called the visibility information of the point. By combining the visibility information of the point cloud in the area corresponding to the patch in the three-dimensional point cloud model during the patch mapping process, a texture image under a better camera pose can be selected from the sample image. Compared with the existing method of determining the texture image under the corresponding camera pose based on the normal information of the patch, in the technical solution of the embodiment of the present disclosure, the texture image under the corresponding camera pose can be determined based on the visibility information of the point cloud, which can make the texture image selection more accurate. Furthermore, texture mapping can be performed based on the texture image under the selected better camera pose, which can improve the texture mapping effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0022] Figure 1 A flowchart of a texture mapping method provided by an embodiment of the present disclosure;

[0023] Figure 2 A schematic diagram of a block diagram of a texture mapping method provided by an embodiment of the present disclosure;

[0024] Figure 3 A flowchart of a texture mapping method provided by an embodiment of the present disclosure;

[0025] Figure 4 A flowchart of a texture mapping method provided by an embodiment of the present disclosure;

[0026] Figure 5 A schematic diagram of pixel value comparison in a texture mapping method provided by an embodiment of the present disclosure;

[0027] Figure 6 A flowchart of a texture mapping method provided by an embodiment of the present disclosure;

[0028] Figure 7 A schematic diagram of a wall map provided by an embodiment of the present disclosure;

[0029] Figure 8 A schematic diagram of a texture hole provided by an embodiment of the present disclosure;

[0030] Fig. 9 A schematic diagram of filling holes and gaps in a texture mapping method provided in an embodiment of the present disclosure;

[0031] Fig.10 A schematic diagram of Gaussian blur in a texture mapping method provided by an embodiment of the present disclosure;

[0032] Fig.11 A structural schematic diagram of a texture mapping device provided by an embodiment of the present disclosure;

[0033] Fig.12 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0035] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0036] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0037] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0038] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0039] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0040] Figure 1 A flow chart of a texture mapping method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to the case of texture mapping. The method can be performed by a texture mapping device, which can be implemented in the form of software and / or hardware, and the device can be configured in an electronic device, such as a computer.

[0041] like Figure 1 As shown, the texture mapping method provided in this embodiment may include:

[0042] S110, obtaining a three-dimensional mesh model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image.

[0043] In the disclosed embodiment, each sample image and the camera pose corresponding to each sample image are used to construct a 3D mesh model and a 3D point cloud model. The 3D mesh model can be considered as a mesh model that needs texture mapping. The 3D point cloud model may include a dense point cloud model. Since the 3D mesh model and the 3D point cloud model are constructed based on the same sample image, the 3D mesh model and the 3D point cloud model have a corresponding relationship in size and shape.

[0044] The camera pose corresponding to each sample image can be considered as the camera extrinsic parameters corresponding to each sample image (i.e., the rotation parameters and translation parameters of the camera). Existing pose solving methods, such as the Structure from Motion (SfM) method, can be used to determine the camera pose corresponding to each sample image. The camera intrinsic parameters may include the focal length of the camera, the coordinates of the principal point, and the distortion parameters, etc. The camera intrinsic parameters of each sample image can be considered to be the same and known. Based on the camera pose and the camera intrinsic parameters, the pixel coordinates in each sample image can be projected into the world coordinate system to obtain the sparse point cloud data corresponding to each sample image.

[0045] After obtaining the sparse point cloud data, a three-dimensional mesh model and a three-dimensional point cloud model may be constructed, and the steps of constructing the two models may include, for example:

[0046] On the one hand, the world coordinates of the sparse point cloud data can be used to construct a signed distance field model (SDF). The signed distance field model is a way to represent the geometric information of a three-dimensional space. For a point in space, a signed distance value is defined, and the positive / negative sign in the signed distance value can represent the point inside / outside the three-dimensional model, and the distance value can represent the distance from the point to the nearest edge point of the three-dimensional model. The surface of the three-dimensional model can be determined based on the part of the signed distance value of 0 output by the constructed signed distance field model, thereby determining the three-dimensional model in the space. In addition, the three-dimensional model can be converted into a three-dimensional mesh model based on the existing mesh model conversion method.

[0047] On the other hand, the depth image can be estimated through the Multiple View Stereo (MVS) algorithm based on the camera pose, camera intrinsic parameters, sparse point cloud and sample images, and then the depth image can be fused and reconstructed to obtain a dense point cloud model.

[0048] In addition, other methods of constructing a three-dimensional mesh model and a three-dimensional point cloud model may also be applied here, and are not specifically limited here.

[0049] For example, Figure 2A block diagram of a texture mapping method provided by an embodiment of the present disclosure. Figure 2 , for the object A that needs to be reconstructed in three dimensions ( Figure 2 In the example of a rectangular parallelepiped, sample images of different viewing angles can be collected by an image acquisition device. Each sample image and the camera pose corresponding to each sample image can be used to construct a 3D mesh model and a 3D point cloud model of object A. The 3D mesh model and the 3D point cloud model have a corresponding relationship in size and shape.

[0050] S120 . Determine a corresponding target area in the three-dimensional point cloud model according to the position information of the to-be-mapped face in the three-dimensional mesh model.

[0051] In the disclosed embodiment, it is necessary to perform texture mapping on each facet (usually a triangular facet) in the three-dimensional mesh model so that the reconstructed three-dimensional model has authenticity. The facet to be mapped can be considered as the facet that currently needs to be texture mapped; the facets in the three-dimensional mesh model can be selected in sequence as the facets to be mapped, or the facets to be mapped can be determined based on other methods, which are not specifically limited here.

[0052] The position information of the surface to be mapped may include the relative position information of the surface to be mapped in the 3D mesh model. Since the 3D mesh model and the 3D point cloud model have a corresponding relationship, the target area corresponding to the position information can be determined from the 3D point cloud model based on the position information of the surface to be mapped.

[0053] The process of determining the target area may include, for example, counting parameters such as the side length or circumscribed circle radius of each facet in the three-dimensional mesh model, and determining a radius value based on the statistical parameters (for example, taking the mean of the circumscribed circle radius as the radius value, etc.). According to the center position of the facet to be mapped, the center point of the target area in the three-dimensional point cloud model is determined. The corresponding spherical area in the three-dimensional point cloud model can be determined with the center point as the center and the radius value as the radius, and the spherical area can be used as the target area.

[0054] S130 , determining a first texture image from each sample image according to the camera pose corresponding to the point cloud in the target area.

[0055] During the construction of the three-dimensional point cloud model, it can be determined whether each point is visible under the camera pose corresponding to each sample image, and the information on whether it is visible can be called the visibility information of the point. In the embodiment of the present disclosure, after determining the target area in the three-dimensional point cloud model, the visibility information of each point in the target area can be viewed. In addition, the camera pose corresponding to the sample image in which each point in the target area is visible can be used as the camera pose corresponding to the point cloud in the target area; or, the camera pose corresponding to the sample image in which the proportion of visible points in the target area reaches a threshold condition can be used as the camera pose corresponding to the point cloud in the target area. Thus, the sample image corresponding to the camera pose can be used as the first texture image.

[0056] In traditional texture mapping, the texture image selection is usually performed based on the normal information of the face to be mapped to determine whether it is visible under the camera pose corresponding to each sample image. Since texture mapping is the last link in the 3D reconstruction chain, it is often affected by the errors introduced in the previous links (such as pose error, 3D mesh model construction error, etc.) and the occlusion relationship between foreground objects and background objects. As a result, the accuracy of the texture image determined based on the face normal information is low, and a large number of foreground object textures will be mapped to the background objects. The mapping error problem seriously affects the authenticity of the 3D model.

[0057] In the embodiment of the present disclosure, based on the visibility information of each point in the target area corresponding to the facet in the three-dimensional point cloud model, a more accurate first texture image that can completely capture the facet to be mapped can be selected, and sample images that do not completely capture the facet to be mapped can be screened out. This can avoid the problem of mapping errors to a certain extent and is conducive to improving the texture mapping effect.

[0058] S140: Perform texture mapping on the surface to be mapped according to the first texture image.

[0059] Herein, based on an existing texture mapping method, texture mapping may be performed on the surface to be mapped according to the texture of an area corresponding to the surface to be mapped in the first texture image.

[0060] In the disclosed embodiment, a three-dimensional mesh model, a three-dimensional point cloud model, each sample image and a camera pose corresponding to each sample image are obtained; wherein each sample image and the camera pose corresponding to each sample image are used to construct a three-dimensional mesh model and a three-dimensional point cloud model; based on the position information of the facet to be mapped in the three-dimensional mesh model, the corresponding target area in the three-dimensional point cloud model is determined; based on the camera pose corresponding to the point cloud in the target area, a first texture image is determined from each sample image; and based on the first texture image, texture mapping is performed on the facet to be mapped.

[0061] During the construction of the three-dimensional point cloud model, it is possible to determine whether each point is visible under each camera pose, and the information on whether it is visible can be called the visibility information of the point. By combining the visibility information of the point cloud in the area corresponding to the patch in the three-dimensional point cloud model during the patch mapping process, a texture image under a better camera pose can be selected from the sample image. Compared with the existing method of determining the texture image under the corresponding camera pose based on the normal information of the patch, in the technical solution of the embodiment of the present disclosure, the texture image under the corresponding camera pose can be determined based on the visibility information of the point cloud, which can make the texture image selection more accurate. Furthermore, texture mapping can be performed based on the texture image under the selected better camera pose, which can improve the texture mapping effect.

[0062] The various optional solutions in the texture mapping method provided in the embodiment of the present disclosure and the above embodiment can be combined. The texture mapping method provided in this embodiment describes in detail the process of selecting a better texture image. By combining the strong and weak texture information provided by the dense point cloud to select a better texture image, the problem of mapping errors can be further reduced.

[0063] Figure 3 A flow chart of a texture mapping method provided by an embodiment of the present disclosure. Figure 3 As shown, the texture mapping method provided in this embodiment may include:

[0064] S310, obtaining a three-dimensional mesh model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image.

[0065] In this embodiment, each sample image and the camera pose corresponding to each sample image are used to construct a three-dimensional mesh model and a three-dimensional point cloud model.

[0066] S320: Determine a corresponding target area in the three-dimensional point cloud model according to the position information of the to-be-mapped face in the three-dimensional mesh model.

[0067] S331 . Determine a first texture image from each sample image according to the camera pose corresponding to the point cloud in the target area.

[0068] S332. According to the number of point clouds in the target area, set the index items for image selection.

[0069] Among them, the number of point clouds in the target area can be used as a standard to measure the strength of the texture of the patch to be mapped. When the number of point clouds is large, it can be considered that the texture corresponding to the patch to be mapped is richer, and the patch can be considered to be a patch with strong texture; conversely, when the number of point clouds is small, it can be considered that the texture corresponding to the patch to be mapped is simpler, and the patch to be mapped can be considered to be a patch with weak texture.

[0070] The texture mapping process can be modeled based on a mathematical model of a Markov random field, and an objective function can be set for the mathematical model of the Markov random field. The objective function may include a data item and a smoothing item, wherein the data item can be used to evaluate the quality of the texture image, and the smoothing item can be used for the continuity after the texture mapping. In the disclosed embodiment, the index item for image selection may include the data item in the objective function of the Markov random field.

[0071] Among them, different image selection index items can be set according to the number of point clouds in the target area. That is, different texture image selection strategies can be set for strong texture patches and weak texture patches to obtain texture images with better effects. For example, for strong texture patches, index items can be set according to the texture clarity of the area where the patch to be mapped is projected into the sample image, so as to select an image with clear texture for mapping; for weak texture patches, index items can be set according to the area where the patch to be mapped is projected into the sample image, so as to select an image with the most covered patches for mapping. In this way, the selected texture image can be made more accurate, and the problem of mapping errors can be further reduced.

[0072] In some optional implementations, image selection index items are set according to the number of point clouds in the target area, which may include: when the number of point clouds in the target area is greater than or equal to a preset threshold, the texture gradient integral of the projection area is used as the index item; wherein the projection area includes the area after the surface to be mapped is projected onto the first texture image; when the number of point clouds in the target area is less than a preset threshold, the area of ​​the projection area is used as the index item.

[0073] The preset threshold value may be set according to an experimental value or an empirical value, for example, may be set to 20.

[0074] When the number of points in the point cloud in the target area is greater than or equal to 20, the patch to be mapped can be considered to be a strong texture patch. At this time, the texture gradient integral of the area after the patch to be mapped is projected onto each first texture image can be used as an indicator item. Among them, the larger the texture gradient integral, on the one hand, it can represent a higher texture clarity, and on the other hand, it can represent a larger area of ​​the projection area. Among them, the first texture image with a larger value of the texture gradient integral can be selected as the second texture image of the patch to be mapped.

[0075] When the number of points in the point cloud in the target area is less than 20, the patch to be mapped can be considered to be a weak texture patch. At this time, the area of ​​the area after the patch to be mapped is projected onto each first texture image can be used as an indicator item. Among them, the larger the area, the more the first texture image covers the area of ​​the patch to be mapped, and the clearer the texture details. Among them, the first texture image with a larger area value can be selected as the second texture image of the patch to be mapped.

[0076] In these optional implementations, by setting the texture gradient integral as an index item for a strong texture region and setting the area as an index item for a weak texture region, selection of a better texture image can be achieved.

[0077] S340: Determine a second texture image from the first texture image based on the set index item.

[0078] In this embodiment, a second texture image with better image quality may be selected from each first texture image according to the value of the indicator item.

[0079] S350: Perform texture mapping on the surface to be mapped according to the second texture image.

[0080] The technical solution of the embodiment of the present disclosure describes in detail the process of selecting a better texture image. By combining the strong and weak texture information provided by the dense point cloud to select a better texture image, the problem of mapping errors can be further reduced. The texture mapping method provided in the embodiment of the present disclosure and the texture mapping method provided in the above embodiment belong to the same public concept. The technical details not described in detail in this embodiment can be referred to the above embodiment, and the same technical features have the same beneficial effects in this embodiment and the above embodiment.

[0081] The various optional schemes in the texture mapping method provided in the embodiment of the present disclosure can be combined with the above-mentioned embodiments. The texture mapping method provided in this embodiment describes in detail the process of selecting a better texture image. By combining the texture atlas rendered by the neural radiation field model to filter out the first texture image with inconsistent pixel colors, the problem of mapping errors can be further reduced.

[0082] Figure 4 A flow chart of a texture mapping method provided by an embodiment of the present disclosure. Figure 4 As shown, the texture mapping method provided in this embodiment may include:

[0083] S410, obtaining a texture atlas, a three-dimensional mesh model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image.

[0084] In this embodiment, each sample image and the camera pose corresponding to each sample image are used to construct a three-dimensional mesh model and a three-dimensional point cloud model.

[0085] Among them, the texture atlas is rendered based on each sample image through a neural radiation field model.

[0086] Among them, the Neural Radiance Fields (NeRF) model is a computer vision technology that uses deep learning technology to extract the collective shape and texture information of objects from images of multiple perspectives, and then uses this information to generate a continuous three-dimensional radiation field, so that highly realistic three-dimensional models / scenes can be presented at any angle and distance. Among them, the rendering process of the texture atlas can include: the color (i.e., pixel value) of the three-dimensional model at each perspective can be rendered through the NeRF model; the color can be filled into the corresponding pixel position in the two-dimensional image; the two-dimensional image with the filled pixels can be called a texture atlas.

[0087] S420: Determine a corresponding target area in the three-dimensional point cloud model according to the position information of the to-be-mapped face in the three-dimensional mesh model.

[0088] S430 , determining a first texture image from each sample image according to the camera pose corresponding to the point cloud in the target area.

[0089] S440: Determine a third texture image from the first texture image according to the pixel value of each pixel in the projection area and the pixel value of each pixel in the corresponding area in the texture atlas.

[0090] The projection area includes the area after the surface to be mapped is projected onto the first texture image.

[0091] In this embodiment, the pixel values ​​of each pixel in the area after the surface to be mapped is projected onto the first texture image can be compared one by one with the pixel values ​​of each pixel in the area corresponding to the surface to be mapped in the texture atlas. The first texture image with consistent comparison results can be used as the third texture image, and the first texture image with inconsistent color can be screened out. Among them, the consistency of the comparison result can include that the proportion of the number of pixels with the same pixel value reaches a preset proportion. By screening out the third texture image with high consistency of the comparison result, the accuracy of the selected texture image can be improved, which is conducive to improving the texture mapping effect.

[0092] For example, Figure 5 A schematic diagram of pixel value comparison in a texture mapping method provided by an embodiment of the present disclosure. Figure 5 The surface to be mapped may be a surface of a sofa backrest. Each pixel value in the projection area after the surface to be mapped is projected onto the first texture image may be compared one by one with the pixel value of each pixel in the corresponding area of ​​the surface to be mapped in the texture atlas, and the first texture image with inconsistent pixel values ​​may be screened out. For example, the first texture image in which the number of pixel points with the same pixel value does not account for a preset proportion may be screened out, thereby avoiding the problem of mapping errors to a certain extent.

[0093] S450: Perform texture mapping on the surface to be mapped according to the third texture image.

[0094] The technical solution of the disclosed embodiment describes in detail the process of selecting a better texture image. By combining the texture atlas rendered by the neural radiation field model to screen out the first texture image with inconsistent pixel colors, the problem of mapping errors can be further reduced. The texture mapping method provided in the disclosed embodiment and the texture mapping method provided in the above embodiment belong to the same public concept. The technical details not described in detail in this embodiment can be found in the above embodiment, and the same technical features have the same beneficial effects in this embodiment and in the above embodiment.

[0095] The various optional solutions in the texture mapping method provided in the embodiment of the present disclosure and the above embodiment can be combined. The texture mapping method provided in this embodiment describes in detail the texture mapping process when the three-dimensional mesh model includes a target plane. By mapping the patches in the target plane in the three-dimensional mesh model as a whole, the problem of misalignment of the mapping can be reduced, making the overall texture more compact.

[0096] Figure 6 A flow chart of a texture mapping method provided by an embodiment of the present disclosure. Figure 6 As shown, the texture mapping method provided in this embodiment may include:

[0097] S610, obtaining a three-dimensional mesh model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image.

[0098] In this embodiment, each sample image and the camera pose corresponding to each sample image are used to construct a three-dimensional mesh model and a three-dimensional point cloud model.

[0099] S620: Determine a target plane in the three-dimensional mesh model according to normal information of each facet in the three-dimensional mesh model.

[0100] In this embodiment, the target plane in the three-dimensional mesh model can be determined by using the normal information of each facet in the three-dimensional mesh model. The target plane can include at least two facets that meet the normal information condition. For example, when the normal information of adjacent facets in the three-dimensional mesh model is consistent, it can be considered that the adjacent facets can constitute the target plane.

[0101] S630: Merge the face patches in each target plane to obtain a merged face patch.

[0102] In this embodiment, the three-dimensional grid model may include at least one target plane, wherein each target plane exists independently. For each target plane area, the facets in the target plane may be taken as a whole, and the facets may be merged into one facet for mapping, which is beneficial to improving the texture mapping effect.

[0103] S640: Determine a corresponding target area in the three-dimensional point cloud model according to the position information of the merged facets in the three-dimensional mesh model.

[0104] In this embodiment, the merged facet may be used as a facet to be mapped, and a target area corresponding to the merged facet may be determined in the three-dimensional point cloud model.

[0105] S650 , determining a first texture image from each sample image according to the camera pose corresponding to the point cloud in the target area.

[0106] S660: Perform texture mapping on the merged face patch according to the first texture image.

[0107] For example, Figure 7 This is a schematic diagram of a wall map provided by an embodiment of the present disclosure. Figure 7 , the target plane in the three-dimensional mesh model can be a wall. Figure 7 (a) is the effect of mapping each face of the wall separately. There may be a problem of misalignment of the mapping, which seriously affects the authenticity of the wall. Figure 7 (b) The effect of mapping the facets on the wall as a whole can effectively avoid the problem of mapping misalignment and make the overall texture more compact.

[0108] The technical solution of the embodiment of the present disclosure describes in detail the texture mapping process when the three-dimensional mesh model contains a target plane. By mapping the patches in the target plane in the three-dimensional mesh model as a whole, the problem of mapping misalignment can be reduced, making the overall texture more compact. The texture mapping method provided in the embodiment of the present disclosure and the texture mapping method provided in the above embodiment belong to the same public concept. The technical details not described in detail in this embodiment can be referred to the above embodiment, and the same technical features have the same beneficial effects in this embodiment and in the above embodiment.

[0109] The various optional schemes in the texture mapping method provided in the embodiment of the present disclosure and the above-mentioned embodiment can be combined. The texture mapping method provided in this embodiment provides a detailed description of texture hole filling when the three-dimensional model after texture mapping contains texture holes. By removing small texture areas in texture holes, it is possible to support the repair of more texture holes. By performing reverse Gaussian blurring from the outside to the inside before texture hole filling, it is helpful to avoid the occurrence of gaps after hole filling.

[0110] Figure 8 Schematic diagram of texture holes provided by the embodiment of the present disclosure. Figure 8As shown in the figure, when the sample image cannot completely cover the surface of the object to be reconstructed in three dimensions, there will be areas in the three-dimensional mesh model after texture mapping where the texture is not mapped. These areas are called texture holes. Among them, the unmapped texture areas can be filled with reasonable colors based on the existing texture hole filling algorithm. However, the existing texture hole filling algorithm has high requirements on the regional topological structure of the texture hole. For example, there cannot be any patches with texture inside the texture hole ( Figure 8 The patches in the box are patches with texture in texture holes), which results in many texture holes being unable to be filled.

[0111] The texture mapping method provided in this embodiment can determine the target surface patch that has been texture mapped inside the texture hole when there are texture holes in the three-dimensional mesh model after texture mapping; after removing the texture of the target surface patch, the texture hole is filled.

[0112] Among them, the target patch in the texture hole that has been texture mapped can be scanned based on the existing image processing method, such as the Breadth First Search (BFS) algorithm. In addition, the texture in the target patch can be removed, so as to achieve the effect of adjusting the topological structure of the texture hole. Then, the texture hole can be filled based on the existing texture hole filling algorithm, so that more texture holes can be filled on the basis of the existing texture hole filling algorithm.

[0113] Since the operating principle of texture hole filling may include: based on the existing image expansion method (such as UV expansion), the texture hole is expanded onto the two-dimensional image, and the expanded shape is intersected with the pixels of the two-dimensional image to obtain the edge of the texture hole (usually the edge is circular). Since this is a pixel-level processing, there is usually an error of 1 to 2 pixels, which leads to the situation that the points originally inside the texture hole are divided outside the texture hole, and the points originally outside the texture hole are divided into the texture hole. In this case, after the texture hole is filled, there will usually be a hole filling gap.

[0114] For example, Fig. 9 A schematic diagram of filling holes and gaps in a texture mapping method provided by an embodiment of the present disclosure. Fig. 9 ,The texture hole filling algorithm is affected by the pixel level error, which leads to poor edge continuity after ,texture hole filling and the problem of gap at the edge of the texture hole filling.

[0115] In some optional implementations, after removing the texture of the target patch and before performing the hole filling process, the process may further include: performing Gaussian blur processing from the outside to the inside on the texture holes. In these optional implementations, before filling the holes, a preset number of reverse Gaussian blurs from the outside to the inside may be performed to process the texture holes. The preset number of times may be set based on an empirical value or an experimental value. By performing multiple reverse Gaussian blurs, pixel values ​​may be set for points that were originally inside the texture holes but are classified outside the texture holes, which can largely solve the problem of edge gaps after texture hole filling.

[0116] In addition, after the texture hole is filled, the edge of the texture hole can be processed by using the forward Gaussian blur from the inside to the outside, so that the appropriate pixel value can be set for the point that was originally outside the texture hole but is divided into the texture hole. By performing the reverse Gaussian blur before the hole is filled and the forward Gaussian blur after the hole is filled, the final texture hole filling result can be presented without any visible texture gaps, and the texture hole filling effect is good.

[0117] For example, Fig.10 A schematic diagram of Gaussian blur in a texture mapping method provided by an embodiment of the present disclosure. Fig.10 The annular area is the effect of the preset number of reverse Gaussian blurs before the texture is patched, and the central blurred area is the effect of the patching based on the existing Poisson editing. By performing multiple reverse Gaussian blurs before patching, the edge gap problem after the texture is patched can be solved to a large extent.

[0118] The technical solution of the embodiment of the present disclosure describes in detail the texture hole filling when the three-dimensional model after the texture mapping is completed contains texture holes. By removing small texture areas in the texture holes, it is possible to support the repair of more texture holes. By performing a reverse Gaussian blur from the outside to the inside before the texture hole filling, it is helpful to avoid the existence of gaps after the hole filling. The texture mapping method provided in the embodiment of the present disclosure and the texture mapping method provided in the above embodiment belong to the same public concept. The technical details not described in detail in this embodiment can be referred to the above embodiment, and the same technical features have the same beneficial effects in this embodiment and the above embodiment.

[0119] Fig.11 The structure diagram of a texture mapping device provided by an embodiment of the present disclosure is shown in FIG. The texture mapping device provided by this embodiment is suitable for texture mapping.

[0120] like Fig.11 As shown, the texture mapping device provided by the embodiment of the present disclosure may include:

[0121] The data acquisition module 1101 is used to acquire a 3D mesh model, a 3D point cloud model, each sample image, and a camera pose corresponding to each sample image; wherein each sample image and the camera pose corresponding to each sample image are used to construct a 3D mesh model and a 3D point cloud model;

[0122] The point cloud region determination module 1102 is used to determine the corresponding target region in the three-dimensional point cloud model according to the position information of the to-be-mapped face in the three-dimensional mesh model;

[0123] An image determination module 1103 is used to determine a first texture image from each sample image according to a camera pose corresponding to a point cloud in a target area;

[0124] The texture mapping module 1104 is used to perform texture mapping on the to-be-mapped surface according to the first texture image.

[0125] In some optional implementations, the image determination module may also be used to:

[0126] After determining the corresponding target area in the three-dimensional point cloud model, the index item of image selection is set according to the number of point clouds in the target area;

[0127] Based on the set index item, determining the second texture image from the first texture image;

[0128] Correspondingly, the texture mapping module can be used to perform texture mapping on the to-be-mapped surface according to the second texture image.

[0129] In some optional implementations, the image determination module may also be used to:

[0130] When the number of point clouds in the target area is greater than or equal to a preset threshold, the texture gradient integral of the projection area is used as an indicator item; wherein the projection area includes the area after the surface patch to be mapped is projected onto the first texture image;

[0131] When the number of point clouds in the target area is less than a preset threshold, the area of ​​the projection area is used as an indicator item.

[0132] In some optional implementations, the image determination module may also be used to:

[0133] Obtaining a texture atlas; wherein the texture atlas is rendered based on each sample image using a neural radiation field model;

[0134] Determine a third texture image from the first texture image according to the pixel value of each pixel in the projection area and the pixel value of each pixel in the corresponding area in the texture atlas; wherein the projection area includes the area after the surface patch to be mapped is projected onto the first texture image;

[0135] Correspondingly, the texture mapping module can be used to perform texture mapping on the to-be-mapped surface according to the third texture image.

[0136] In some optional implementations, the texture mapping device may further include:

[0137] A face merging module is used to determine a target plane in a three-dimensional mesh model according to normal information of each face in the three-dimensional mesh model;

[0138] Merge the patches in each target plane to obtain a merged patch;

[0139] Accordingly, the point cloud area determination module can be used for:

[0140] According to the position information of the merged facets in the 3D mesh model, the corresponding target area in the 3D point cloud model is determined.

[0141] In some optional implementations, the texture mapping device may further include:

[0142] A hole filling module is used to determine the target surface patch that has been texture mapped inside the texture hole when there are texture holes in the three-dimensional mesh model after texture mapping;

[0143] After removing the texture of the target patch, the texture holes are filled.

[0144] In some optional implementations, after removing the texture of the target patch and before performing the hole filling process, the hole filling module may also be used to:

[0145] Apply Gaussian blur to texture holes from the outside in.

[0146] The texture mapping device provided in the embodiments of the present disclosure can execute the texture mapping method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0147] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0148] Reference below Fig.12 , which shows an electronic device (eg, Fig.12The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.12 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0149] like Fig.12 As shown, the electronic device 1200 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage device 1208 to a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the electronic device 1200 are also stored. The processing device 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0150] Typically, the following devices may be connected to the I / O interface 1205: an input device 1206 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1207 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1208 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1209. The communication device 1209 may allow the electronic device 1200 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.12 The electronic device 1200 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0151] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1209, or installed from the storage device 1208, or installed from the ROM 1202. When the computer program is executed by the processing device 1201, the above-mentioned functions defined in the texture mapping method of the embodiment of the present disclosure are executed.

[0152] The electronic device provided by the embodiment of the present disclosure and the texture mapping method provided by the above embodiment belong to the same disclosed concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0153] The embodiments of the present disclosure provide a computer storage medium on which a computer program is stored. When the program is executed by a processor, the texture mapping method provided by the above embodiments is implemented.

[0154] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or a flash memory (FLASH), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0155] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0156] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0157] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0158] A three-dimensional mesh model, a three-dimensional point cloud model, each sample image and a camera pose corresponding to each sample image are obtained; wherein each sample image and the camera pose corresponding to each sample image are used to construct a three-dimensional mesh model and a three-dimensional point cloud model; according to the position information of the facet to be mapped in the three-dimensional mesh model, the corresponding target area in the three-dimensional point cloud model is determined; according to the camera pose corresponding to the point cloud in the target area, a first texture image is determined from each sample image; according to the first texture image, texture mapping is performed on the facet to be mapped.

[0159] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0160] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0161] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the names of the units and modules do not, in some cases, limit the units and modules themselves.

[0162] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), etc.

[0163] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0164] According to one or more embodiments of the present disclosure, a texture mapping method is provided, the method comprising:

[0165] Acquire a three-dimensional mesh model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image; wherein each sample image and the camera pose corresponding to each sample image are used to construct the three-dimensional mesh model and the three-dimensional point cloud model;

[0166] Determine a corresponding target area in the three-dimensional point cloud model according to position information of the to-be-mapped face in the three-dimensional grid model;

[0167] Determining a first texture image from each of the sample images according to a camera pose corresponding to a point cloud in the target area;

[0168] According to the first texture image, texture mapping is performed on the surface to be mapped.

[0169] According to one or more embodiments of the present disclosure, a texture mapping method is provided, further comprising:

[0170] In some optional implementations, after determining the corresponding target area in the three-dimensional point cloud model, the method further includes:

[0171] According to the number of point clouds in the target area, an index item for image selection is set;

[0172] Correspondingly, performing texture mapping on the to-be-mapped surface according to the first texture image includes:

[0173] Determining a second texture image from the first texture image based on the set indicator item;

[0174] According to the second texture image, texture mapping is performed on the surface to be mapped.

[0175] According to one or more embodiments of the present disclosure, a texture mapping method is provided, further comprising:

[0176] In some optional implementations, setting the index item for image selection according to the number of point clouds in the target area includes:

[0177] When the number of point clouds in the target area is greater than or equal to a preset threshold, the texture gradient integral of the projection area is used as an indicator item; wherein the projection area includes the area after the surface patch to be mapped is projected onto the first texture image;

[0178] When the number of point clouds in the target area is less than the preset threshold, the area of ​​the projection area is used as an indicator item.

[0179] According to one or more embodiments of the present disclosure, a texture mapping method is provided, further comprising:

[0180] In some optional implementations, a texture atlas is obtained; wherein the texture atlas is rendered based on the sample images using a neural radiation field model;

[0181] Correspondingly, performing texture mapping on the to-be-mapped surface according to the first texture image includes:

[0182] Determine a third texture image from the first texture image according to the pixel value of each pixel in the projection area and the pixel value of each pixel in the corresponding area in the texture atlas; wherein the projection area includes the area after the surface patch to be mapped is projected onto the first texture image;

[0183] According to the third texture image, texture mapping is performed on the surface to be mapped.

[0184] According to one or more embodiments of the present disclosure, a texture mapping method is provided, further comprising:

[0185] In some optional implementations, a target plane in the three-dimensional mesh model is determined according to normal information of each facet in the three-dimensional mesh model;

[0186] Merging the face patches in each target plane to obtain a merged face patch;

[0187] Accordingly, according to the position information of the to-be-mapped face in the three-dimensional mesh model, the corresponding target area in the three-dimensional point cloud model is determined, including:

[0188] According to the position information of the merged facets in the three-dimensional mesh model, the corresponding target area in the three-dimensional point cloud model is determined.

[0189] According to one or more embodiments of the present disclosure, a texture mapping method is provided, further comprising:

[0190] In some optional implementations, when there are texture holes in the three-dimensional mesh model after texture mapping, determining a target face patch inside the texture hole that has been texture mapped;

[0191] After removing the texture of the target surface, the texture holes are filled.

[0192] According to one or more embodiments of the present disclosure, a texture mapping method is provided, further comprising:

[0193] In some optional implementations, after removing the texture of the target patch and before performing the hole filling process, the method further includes:

[0194] The texture holes are subjected to Gaussian blur processing from outside to inside.

[0195] According to one or more embodiments of the present disclosure, a texture mapping device is provided, the device comprising:

[0196] A data acquisition module, used to acquire a three-dimensional mesh model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image; wherein each sample image and the camera pose corresponding to each sample image are used to construct the three-dimensional mesh model and the three-dimensional point cloud model;

[0197] A point cloud region determination module, used to determine the corresponding target region in the three-dimensional point cloud model according to the position information of the to-be-mapped face in the three-dimensional grid model;

[0198] An image determination module, configured to determine a first texture image from each of the sample images according to a camera pose corresponding to a point cloud in the target area;

[0199] The texture mapping module is used to perform texture mapping on the surface to be mapped according to the first texture image.

[0200] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0201] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0202] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A texture mapping method, characterized in that: include: Acquire a three-dimensional mesh model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image; wherein each sample image and the camera pose corresponding to each sample image are used to construct the three-dimensional mesh model and the three-dimensional point cloud model; Determining a corresponding target area in the three-dimensional point cloud model according to position information of the to-be-mapped face in the three-dimensional grid model; Determining a first texture image from each of the sample images according to a camera pose corresponding to a point cloud in the target area; According to the first texture image, texture mapping is performed on the surface to be mapped.

2. The method according to claim 1, characterized in that After determining the corresponding target area in the three-dimensional point cloud model, the method further includes: According to the number of point clouds in the target area, an index item for image selection is set; Correspondingly, performing texture mapping on the to-be-mapped surface according to the first texture image includes: Determining a second texture image from the first texture image based on the set indicator item; According to the second texture image, texture mapping is performed on the surface to be mapped.

3. The method according to claim 2, characterized in that The step of setting the index item for image selection according to the number of point clouds in the target area includes: When the number of point clouds in the target area is greater than or equal to a preset threshold, the texture gradient integral of the projection area is used as an indicator item; wherein the projection area includes the area after the surface patch to be mapped is projected onto the first texture image; When the number of point clouds in the target area is less than the preset threshold, the area of ​​the projection area is used as an indicator item.

4. The method according to claim 1, characterized in that Also includes: Acquire a texture atlas; wherein the texture atlas is rendered based on each sample image through a neural radiation field model; Correspondingly, performing texture mapping on the to-be-mapped surface according to the first texture image includes: Determine a third texture image from the first texture image according to the pixel value of each pixel in the projection area and the pixel value of each pixel in the corresponding area in the texture atlas; wherein the projection area includes the area after the surface patch to be mapped is projected onto the first texture image; According to the third texture image, texture mapping is performed on the surface to be mapped.

5. The method according to claim 1, characterized in that Also includes: Determining a target plane in the three-dimensional mesh model according to normal information of each facet in the three-dimensional mesh model; Merging the face patches in each target plane to obtain a merged face patch; Accordingly, according to the position information of the to-be-mapped face in the three-dimensional mesh model, the corresponding target area in the three-dimensional point cloud model is determined, including: According to the position information of the merged facets in the three-dimensional mesh model, the corresponding target area in the three-dimensional point cloud model is determined.

6. The method according to claim 1, characterized in that Also includes: In the case where a texture hole exists in the three-dimensional mesh model after texture mapping, determining a target face patch inside the texture hole that has been texture mapped; After removing the texture of the target surface, the texture holes are filled.

7. The method according to claim 6, characterized in that After removing the texture of the target patch and before performing the hole filling process, the method further includes: The texture holes are subjected to Gaussian blur processing from outside to inside.

8. A texture mapping device, characterized in that: include: A data acquisition module, used to acquire a three-dimensional mesh model, a three-dimensional point cloud model, each sample image, and a camera pose corresponding to each sample image; wherein each sample image and the camera pose corresponding to each sample image are used to construct the three-dimensional mesh model and the three-dimensional point cloud model; A point cloud region determination module, used to determine the corresponding target region in the three-dimensional point cloud model according to the position information of the to-be-mapped face in the three-dimensional grid model; An image determination module, configured to determine a first texture image from each of the sample images according to a camera pose corresponding to a point cloud in the target area; The texture mapping module is used to perform texture mapping on the surface to be mapped according to the first texture image.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the texture mapping method as described in any one of claims 1 to 7.

10. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to perform the texture mapping method according to any one of claims 1 to 7 when executed by a computer processor.

Images