Texture mapping method, system and terminal device of three-dimensional model

Abstract

The application provides a texture mapping method, system and terminal equipment of a three-dimensional model, the method comprising: obtaining an original three-dimensional model, processing the original three-dimensional model to obtain a simplified three-dimensional model; obtaining a camera internal parameter matrix and a camera external parameter matrix, the camera external parameter matrix being pose information of different shooting angles corresponding to the original three-dimensional model; determining a first corresponding relationship matrix between a pixel coordinate system of a physical shooting image corresponding to the original three-dimensional model and a coordinate system of the original three-dimensional model based on the camera external parameter matrix and the camera internal parameter matrix; determining a two-dimensional texture image of each shooting angle based on the first corresponding relationship matrix and the original three-dimensional model, and determining a second corresponding relationship between the two-dimensional texture image and a grid vertex of the simplified three-dimensional model, so as to perform texture mapping on the simplified three-dimensional model based on the second corresponding relationship. The application solves the problem that the image shadow is mapped onto the model in the prior art, resulting in that the model has different colors and is not realistic.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional model texture mapping technology for projectors, and particularly to a three-dimensional model texture mapping method, system, and terminal device. Background Technology

[0002] Generally, 3D scanners can capture the true colors of a model, helping to enhance the realism of the 3D model and further improve the details of the model.

[0003] Currently, 3D model color restoration techniques are mainly divided into two methods: vertex shading and texture mapping. Vertex shading requires a dense mesh to obtain detailed color information, resulting in a large amount of 3D model data, which is detrimental to subsequent editing and design processes. While texture mapping can maintain fine texture details on simplified model data, traditional methods map image shadows onto the model, causing inconsistencies in color brightness and a lack of realism.

[0004] Therefore, it is necessary to propose a method for texture mapping of three-dimensional models to solve the above-mentioned technical problems. Summary of the Invention

[0005] The purpose of this invention is to provide a texture mapping method, system, and terminal device for three-dimensional models, so as to solve the problem in the prior art that the model has inconsistent color brightness and is not realistic when the image shadow is mapped onto the model.

[0006] To achieve the above objectives, in a first aspect, a texture mapping method for a three-dimensional model is provided, the method comprising:

[0007] Obtain the original 3D model, process the original 3D model to obtain a simplified 3D model; obtain the camera intrinsic parameter matrix and the camera extrinsic parameter matrix, wherein the camera extrinsic parameter matrix is ​​the pose information of the original 3D model corresponding to different shooting angles;

[0008] Based on the camera extrinsic matrix and the camera intrinsic matrix, a first correspondence matrix is ​​determined between the pixel coordinate system of the physically captured image corresponding to the original 3D model and the coordinate system of the original 3D model;

[0009] Based on the first correspondence matrix and the original 3D model, determine the 2D texture images of each shooting angle, and determine the second correspondence between the 2D texture images and the mesh vertices of the simplified 3D model, so as to perform texture mapping on the simplified 3D model based on the second correspondence.

[0010] As a further improvement of the present invention, determining the two-dimensional texture image at each shooting angle based on the first correspondence matrix and the original three-dimensional model includes:

[0011] Determine whether the mesh region of the original 3D model is within the corresponding shooting angle based on the mesh normal of the original 3D model and the shooting angle of the physical image.

[0012] If the mesh area of ​​the original 3D model is within the corresponding shooting viewpoint, a mapping relationship between the original 3D model mesh and the pixels of the physical shooting image is established based on the first correspondence matrix; based on the mapping relationship, the pixels of the physical shooting image are assigned values ​​using the centroid interpolation method to form a two-dimensional texture image at the corresponding angle.

[0013] As a further improvement of the present invention, based on the mapping relationship, a centroid interpolation method is used to assign pixel values ​​to the physically captured image to form a two-dimensional texture image at a corresponding angle, including:

[0014] Based on the mapping relationship, the pixel coordinates of the physical image corresponding to the coordinates of the original 3D model are obtained, and the mesh vertices of the original 3D model are determined based on the pixel coordinates of the physical image to determine whether they are within the corresponding shooting view.

[0015] The centroid interpolation method is used to assign color values ​​to the pixels corresponding to the grid located within the shooting viewpoint to form a two-dimensional texture image corresponding to the shooting viewpoint.

[0016] As a further improvement of the present invention, determining whether the mesh vertices of the original 3D model are within the corresponding shooting viewpoint based on the pixel coordinates of the physically captured image includes:

[0017] The minimum bounding rectangle of pixel coordinates is determined based on the pixel coordinates corresponding to the mesh vertices of the original 3D model.

[0018] The centroid interpolation method is used to determine whether all pixels within the minimum bounding rectangle are located within the pixel coordinate bounding box mapped by the grid vertex.

[0019] As a further improvement of the present invention, determining whether all pixels within the minimum bounding rectangle are located within the pixel coordinate bounding box mapped by the grid vertex using the centroid interpolation method includes:

[0020] The weighted sum of the three vertex pixels of the triangle in the grid region is used as the target pixel within the corresponding minimum bounding rectangle;

[0021] Based on the judgment result of whether there is a weighting coefficient less than 0 among the three vertex pixels of the triangle, it is determined whether the target pixel is within the bounding box of the pixel coordinates mapped by the grid vertex.

[0022] As a further improvement of the present invention, determining whether a target pixel is within the bounding box of the pixel coordinates mapped by the grid vertex is based on the judgment result of whether there is a weighting coefficient less than 0 among the three vertex pixels of the triangle, including:

[0023] If there is a weighting coefficient less than 0 among the three vertex pixels of the triangle, then the target pixel is not within the bounding box of the pixel coordinates mapped by the grid vertex.

[0024] If there is no weighting coefficient less than 0 among the three vertex pixels of the triangle, then the target pixel is within the pixel coordinate bounding box of the grid vertex mapping, and the color value obtained by weighted summation of the pixels of the three vertices determined according to the weighting coefficients is assigned to the target pixel.

[0025] As a further improvement of the present invention, determining a first correspondence matrix between the pixel coordinate system of the physically captured image corresponding to the original 3D model and the coordinate system of the original 3D model based on the camera extrinsic parameter matrix and the camera intrinsic parameter matrix includes:

[0026] The product of the camera extrinsic matrix and the camera intrinsic matrix is ​​used as the first correspondence matrix.

[0027] As a further improvement of the present invention, determining a second correspondence between the two-dimensional texture image and the mesh vertices of the simplified three-dimensional model includes:

[0028] Multiplying the coordinates of the simplified 3D model by the transformation matrix corresponding to the shooting angle yields the pixel coordinates of the corresponding physical image.

[0029] The specific color information of the mesh vertices and corresponding pixel coordinates of the simplified 3D model is used as the second correspondence between the 2D texture image and the mesh vertices of the simplified 3D model.

[0030] Secondly, a texture mapping system for three-dimensional models is provided, including:

[0031] The model acquisition unit is used to acquire the original 3D model, and then process the original 3D model to obtain a simplified 3D model.

[0032] The parameter acquisition unit is used to acquire the camera intrinsic parameter matrix and the camera extrinsic parameter matrix, wherein the camera extrinsic parameter matrix is ​​the pose information of the original 3D model corresponding to different shooting angles;

[0033] The first determining unit is used to determine a first correspondence matrix between the pixel coordinate system of the physically captured image corresponding to the original 3D model and the coordinate system of the original 3D model based on the camera extrinsic matrix and the camera intrinsic matrix.

[0034] The second determining unit is configured to determine two-dimensional texture images of each shooting angle based on the first correspondence matrix and the original three-dimensional model, and to determine a second correspondence between the two-dimensional texture images and the mesh vertices of the simplified three-dimensional model; and,

[0035] The data processing unit is used to perform texture mapping on the simplified 3D model based on the second correspondence.

[0036] Thirdly, the present invention provides a terminal device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method described in the first aspect.

[0037] Fourthly, the present invention provides a computer-readable storage medium on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.

[0038] The beneficial effects of this invention are as follows:

[0039] The texture mapping method for 3D models in this invention utilizes the camera extrinsic and intrinsic parameter matrices to determine a first correspondence matrix between the pixel coordinate system of the physically captured image corresponding to the original 3D model and the coordinate system of the original 3D model. Based on this first correspondence matrix and the original 3D model, 2D texture images for each shooting angle are determined. Then, texture mapping is applied to the simplified 3D model based on a second correspondence between the 2D texture images and the vertices of the simplified model mesh. This optimizes the shadow problem inherent in traditional texture mapping, achieving the effect of maintaining fine textures in the simplified 3D model, thereby improving the realism of the 3D model. This solves the problem in existing technologies where image shadows are mapped onto the model, resulting in inconsistent color brightness and unrealistic appearance. Attached Figure Description

[0040] Figure 1 This is a schematic flowchart illustrating a texture mapping method for a three-dimensional model according to an embodiment of the present invention.

[0041] Figure 2 This is a schematic flowchart illustrating a texture mapping method for a three-dimensional model according to another embodiment of the present invention.

[0042] Figure 3 A schematic diagram illustrating the principle of mapping occlusion by mapping points on the original 3D model to points on the corresponding shooting viewpoint image;

[0043] Figure 4 A schematic diagram illustrating the principle of determining whether the mesh area of ​​the original 3D model is within the shooting viewpoint;

[0044] Figure 5This is a schematic flowchart illustrating a texture mapping method for a three-dimensional model according to another embodiment of the present invention.

[0045] Figure 6 A schematic diagram illustrating the principle of determining whether a grid vertex is mapped to the image range;

[0046] Figure 7 This is a schematic flowchart illustrating a texture mapping method for a three-dimensional model according to another embodiment of the present invention.

[0047] Figure 8 This is a schematic flowchart illustrating a texture mapping method for a three-dimensional model according to another embodiment of the present invention.

[0048] Figure 9 A schematic diagram illustrating the principle of determining whether pixel P is within a grid triangle;

[0049] Figure 10 This is a schematic flowchart illustrating a texture mapping method for a three-dimensional model according to another embodiment of the present invention.

[0050] Figure 11 This is a schematic diagram of the texture corresponding to the optimal viewpoint of a mesh according to an embodiment of the present invention;

[0051] Figure 12 This is a schematic structural diagram of a texture mapping system for a three-dimensional model according to an embodiment of the present invention;

[0052] Figure 13 This is a topological diagram of a computer-readable storage medium disclosed in this invention. Detailed Implementation

[0053] The present invention will now be described in detail with reference to the embodiments shown in the accompanying drawings. However, it should be noted that these embodiments are not intended to limit the present invention. Equivalent changes or substitutions in function, method, or structure made by those skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0054] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0055] Example 1:

[0056] Figure 1 This is a texture mapping method for 3D models (hereinafter referred to as "mapping method" or "method") to solve the problem in existing technologies where mapping image shadows onto the model results in inconsistent color and brightness, leading to unrealistic appearances. The method in this embodiment includes:

[0057] Step 102. Obtain the original 3D model, and then process it to obtain a simplified 3D model. The original 3D model is obtained directly from 3D reconstruction; it is an unreduced mesh model with color information attached to its mesh vertices. The simplified 3D model is the original 3D model after mesh simplification and smoothing. The specific degree of simplification in the simplified 3D model is set according to actual needs and is not limited here.

[0058] Step 104. Obtain the camera intrinsic parameter matrix and the camera extrinsic parameter matrix. The camera extrinsic parameter matrix is ​​the pose information of the original 3D model corresponding to different shooting angles.

[0059] It should be noted that the pose information for each shooting angle (i.e., shooting viewpoint) is obtained from the predetermined rotation posture and the distance information between the original 3D model coordinate system and the camera coordinate system, i.e., the camera extrinsic parameter matrix. The camera extrinsic parameter matrix is ​​different for each shooting viewpoint. The camera intrinsic parameter matrix includes the principal point coordinates of the camera image, pixel size parameters, and lens focal length, which are obtained during camera calibration. The camera intrinsic parameter matrix is ​​unique and deterministic.

[0060] Step 106. Determine the first correspondence matrix between the pixel coordinate system of the physically captured image corresponding to the original 3D model and the coordinate system of the original 3D model based on the camera extrinsic matrix and the camera intrinsic matrix.

[0061] Specifically, the camera extrinsic parameter matrix is ​​multiplied by the camera intrinsic parameter matrix to obtain the first correspondence matrix. The first correspondence matrix contains multiple transformation matrices between the original 3D model coordinate system and the pixel coordinate system of the physically captured image. The calculation formula for a certain pixel at a certain angle in the first correspondence matrix is ​​shown in Equation 1.

[0062] P p =K*[R i *T i ]*P w (1)

[0063] Among them, P p Let K be the image coordinates mapped to the corresponding shooting viewpoint, and K be the camera intrinsic parameter matrix. i T i ] is the camera extrinsic parameter matrix for the i-th shooting angle, P w Let R be the second coordinate (x, y, z, 1) of the world coordinates of the grid point, where R i Let T be the rotation matrix for the i-th angle. i Let R be the translation matrix for the i-th angle. i and T i The components are combined to form a 3*4 matrix, which serves as the camera's extrinsic parameter matrix.

[0064] Step 108. Determine the two-dimensional texture images for each shooting angle based on the first correspondence matrix and the original three-dimensional model, and determine the second correspondence between the two-dimensional texture images and the mesh vertices of the simplified three-dimensional model, so as to perform texture mapping on the simplified three-dimensional model based on the second correspondence.

[0065] As Figure 2 shown, the specific operations of "determining the two-dimensional texture images for each shooting angle based on the first correspondence matrix and the original three-dimensional model" in Step 108 include:

[0066] Step 202. Determine whether the mesh area of the original three-dimensional model is within the corresponding shooting view based on the mesh normal of the original three-dimensional model and the shooting angle of the physical shooting image.

[0067] It should be noted that, as Figure 3 shown, when determining the two-dimensional texture images for each shooting view, since at the image point P in the same view, there may be mapping relationships with points on multiple original three-dimensional models, such as P1 and P2. However, in fact, point P2 is not visible in this view, so it is necessary to judge and exclude the occluded points. Specifically, referring Figure 4 to the following for illustration, when judging and excluding the occluded points, it is judged whether the mesh area of the original three-dimensional model is visible within the corresponding shooting view through the mesh normal of the original three-dimensional model and the view direction corresponding to the physical shooting image. If the mesh normal of the original three-dimensional model and the shooting view direction are the same, then the mesh area of the original three-dimensional model is not visible within the corresponding shooting view; otherwise, the mesh area of the original three-dimensional model is within the corresponding shooting view.

[0068] Step 204. If the mesh area of the original three-dimensional model is within the corresponding shooting view, establish a mapping relationship between the mesh of the original three-dimensional model and the pixels of the physical shooting image based on the first correspondence matrix. Otherwise, discard it. In this way, traverse all the meshes of the original three-dimensional model to all the corresponding shooting views, so as to establish a mapping relationship between each mesh area within the corresponding shooting view and the pixels of the corresponding physical shooting image.

[0069] Step 206. Assign values to the pixels of the physical shooting image based on the mapping relationship using the barycentric interpolation method to form the two-dimensional texture image for the corresponding angle.

[0070] Referring Figure 5 to the following for illustration, the specific steps of Step 206 include:

[0071] Step 502. Obtain the pixel coordinates of the physical shooting image corresponding to the coordinates of the original three-dimensional model based on the mapping relationship, so as to determine whether the mesh vertices of the original three-dimensional model are within the corresponding shooting view based on the pixel coordinates of the physical shooting image.

[0072] In other words, based on the first correspondence, the pixel coordinates of the physical shooting viewpoint image corresponding to the original 3D model coordinates can be obtained. Then, based on the pixel coordinates, it is further determined whether the mesh vertices are mapped to the image range. If the mesh vertices are within the image range, the RGB values ​​of the mesh vertices are placed at the corresponding image coordinates, such as... Figure 6 As shown.

[0073] Among them, participants Figure 7 As shown, the specific operation of "determining whether the mesh vertices of the original 3D model are within the corresponding shooting viewpoint based on the pixel coordinates of the physically captured image" in step 502 includes:

[0074] Step 702. Determine the minimum bounding rectangle of pixel coordinates based on the pixel coordinates corresponding to the mesh vertices of the original 3D model.

[0075] Step 704. Use centroid interpolation to determine whether all pixels within the minimum bounding rectangle are located within the bounding box of pixel coordinates mapped from the grid vertices. That is, in this embodiment, the minimum bounding rectangle is determined based on the pixel coordinates mapped from the grid vertices, and centroid interpolation is used to determine whether all pixels inside the rectangle are within the bounding box of pixel coordinates mapped from the grid vertices. The so-called centroid interpolation method for determining whether all pixels within the minimum bounding rectangle are located within the bounding box of pixel coordinates mapped from the grid vertices actually uses the centroid interpolation calculation method to determine whether a point (i.e., a pixel) is within the corresponding triangle in the grid region.

[0076] Among them, participants Figure 8 As shown, step 704 specifically includes:

[0077] Step 7041. The result of the weighted sum of the three vertex pixels of the triangle in the grid region is used as the target pixel point within the corresponding minimum bounding rectangle.

[0078] Step 7042. Based on the judgment result of whether there is a weighting coefficient less than 0 among the three vertex pixels of the triangle, determine whether the target pixel is within the pixel coordinate bounding box (i.e. the minimum bounding rectangle) of the grid vertex mapping.

[0079] Combination Figure 9 To explain, centroid interpolation refers to representing a point inside a triangle using the weighted sum of the pixels of the triangle's three vertices (points A, B, and C). The sum of the weighting coefficients of the pixels at each vertex is 1.

[0080] P(x,y)=αA+βB+γC (2)

[0081] Where α + β + γ = 1. The coordinates of point P are two-dimensional points, and the unique values ​​of α, β, and γ can be obtained by solving formula 2 and the fact that the sum of the weighting coefficients is 1.

[0082] Specifically, if any of the three vertex pixels of the triangle has a weighting coefficient less than 0, then the target pixel (e.g., point D) is not within the bounding box of the pixel coordinates mapped by the grid vertex. If none of the three vertex pixels of the triangle has a weighting coefficient less than 0, then the target pixel (e.g., point P) is within the bounding box of the pixel coordinates mapped by the grid vertex. The color value obtained by weighting and summing the pixels of the three vertices determined by the weighting coefficients is then assigned to the target pixel. That is, if any of the three calculated weights α, β, and γ is less than 0, then the pixel (e.g., point D) is considered not to be inside the triangle and is discarded.

[0083] Step 504. Use centroid interpolation to assign color values ​​to the pixels corresponding to the grid located within the shooting viewpoint to form a two-dimensional texture image corresponding to the shooting viewpoint.

[0084] If point P is inside the triangle, then the color value of the current position is interpolated according to the centroid weight. The color value of point P can be obtained by weighting the RGB values ​​of the three vertices of the triangle (points A, B, and C), and the color value is assigned to the image using formula 3, thereby forming a two-dimensional texture image corresponding to the shooting perspective.

[0085] P(R,G,B)=αA(R,G,B)+βB(R,G,B)+γC(R,G,B) (3)

[0086] like Figure 10 As shown, step 108, "determining the second correspondence between the two-dimensional texture image and the mesh vertices of the simplified three-dimensional model," may specifically include:

[0087] Step 1002. Multiply the coordinates of the simplified 3D model by the transformation matrix corresponding to the shooting angle to obtain the pixel coordinates of the corresponding physical image.

[0088] Step 1004. The specific color information of the simplified 3D model's mesh vertices and corresponding pixel coordinates is used as the second correspondence between the 2D texture image and the simplified 3D model's mesh vertices. The specific color information of the corresponding pixel coordinates is obtained through the simplified 3D model's mesh vertices. Therefore, the specific color information of the simplified 3D model's mesh vertices and corresponding pixel coordinates constitutes the second correspondence in this embodiment, as shown below. Figure 11 As shown, the simplified 3D model mesh's triangular facets (i.e., triangular regions) are mapped to the specific color information (i.e., texture image) of the corresponding pixel coordinates under the optimal viewpoint. The optimal viewpoint refers to the triangular facet with the minimum loss weight under the current viewpoint. The loss weight can be determined based on factors such as the triangular facet normal and the viewpoint angle, similar to traditional mapping problems, and will not be described in detail here.

[0089] It should be understood that the texture mapping method for the 3D model in this embodiment uses the camera extrinsic and intrinsic parameter matrices to determine the first correspondence matrix between the pixel coordinate system of the physically captured image corresponding to the original 3D model and the coordinate system of the original 3D model. Based on the first correspondence matrix and the original 3D model, it determines the 2D texture images for each shooting angle. Then, based on the second correspondence between the 2D texture images and the simplified model mesh vertices of the original 3D model, it maps the simplified 3D model. This optimizes the shadow problem present in traditional texture mapping, achieving the effect of maintaining fine textures under the simplified 3D model, thereby improving the realism of the 3D model. Thus, this embodiment solves the problem in the prior art where mapping image shadows onto the model results in inconsistent color brightness and unrealistic appearance.

[0090] Example 2:

[0091] like Figure 12 As shown, this embodiment also provides a texture mapping system 120 for a three-dimensional model, which includes a model acquisition unit 121 for acquiring an original three-dimensional model and processing the original three-dimensional model to obtain a simplified three-dimensional model; a parameter acquisition unit 122 for acquiring a camera intrinsic parameter matrix and a camera extrinsic parameter matrix, wherein the camera extrinsic parameter matrix represents the pose information of the original three-dimensional model at different shooting angles; a first determination unit 123 for determining a first correspondence matrix between the pixel coordinate system of the physical image corresponding to the original three-dimensional model and the coordinate system of the original three-dimensional model based on the camera extrinsic parameter matrix and the camera intrinsic parameter matrix; a second determination unit 124 for determining two-dimensional texture images at each shooting angle based on the first correspondence matrix and the original three-dimensional model, and determining a second correspondence between the two-dimensional texture images and the mesh vertices of the simplified three-dimensional model; and for performing texture mapping on the simplified three-dimensional model based on the second correspondence.

[0092] The texture mapping system 120 of the 3D model in this embodiment uses the camera extrinsic and intrinsic parameter matrices obtained by the parameter acquisition unit 122 to determine the first correspondence matrix between the physical image pixel coordinate system and the coordinate system of the original 3D model. The second determination unit 124 then determines the 2D texture images for each shooting angle based on the first correspondence matrix and the original 3D model. The data processing unit 125 then applies texture mapping to the simplified 3D model based on the second correspondence between the 2D texture images and the simplified model mesh vertices of the original 3D model. This optimizes the shadow problem inherent in traditional texture mapping, achieving the effect of maintaining fine textures in the simplified 3D model, thereby improving the realism of the 3D model. Thus, this embodiment solves the problem in the prior art where image shadows are mapped onto the model, resulting in inconsistent color brightness and unrealistic appearance.

[0093] It should be noted that the technical solutions of the texture mapping system of the three-dimensional model in this embodiment are the same as those in Embodiment 1. Please refer to Embodiment 1 for the description, and they will not be repeated here.

[0094] Example 3:

[0095] This invention also provides a terminal device, which may include a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the above-described functionality. Figure 1 The various processes of the texture mapping method embodiment for the 3D model shown are all applicable and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0096] Example 4:

[0097] Combination Figure 13 As shown in the illustration, this embodiment also discloses a specific implementation of a computer-readable storage medium 1300. This computer-readable storage medium 1300 can be configured wholly or partially within a physical computer, server, cluster server, or data center.

[0098] In this embodiment, the computer-readable storage medium 1300 stores computer program instructions 1301, which are read and executed by a processor 1302 to perform the steps in the texture mapping method for a three-dimensional model as disclosed in Embodiment 1.

[0099] Optionally, the computer-readable storage medium 1300 can be configured as a server, and the server runs on a physical device used to build a private cloud, hybrid cloud, or public cloud. The computer-readable storage medium 1300 can also be configured as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0100] The computer-readable storage medium 1300 is used to store a program, and the processor 1302, upon receiving an execution instruction, executes the texture mapping method for a three-dimensional model disclosed in Embodiment 1.

[0101] Meanwhile, the processor 1302 disclosed in this embodiment may be an integrated circuit chip with signal processing capabilities. The processor 1302 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0102] The technical solution of the same part in the computer-readable storage medium 1300 disclosed in this embodiment as in Embodiment 1 and / or Embodiment 2 is described in Embodiment 1 and / or Embodiment 2, and will not be repeated here.

[0103] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

[0104] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0105] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A texture mapping method for a three-dimensional model, characterized in that, The method includes: Obtain the original 3D model, and then process the original 3D model to obtain a simplified 3D model; Obtain the camera intrinsic parameter matrix and the camera extrinsic parameter matrix, wherein the camera extrinsic parameter matrix is ​​the pose information of the original 3D model corresponding to different shooting angles; Based on the camera extrinsic matrix and the camera intrinsic matrix, a first correspondence matrix is ​​determined between the pixel coordinate system of the physically captured image corresponding to the original 3D model and the coordinate system of the original 3D model; Based on the first correspondence matrix and the original 3D model, determine the 2D texture images of each shooting angle, and determine the second correspondence between the 2D texture images and the mesh vertices of the simplified 3D model, so as to perform texture mapping on the simplified 3D model based on the second correspondence. The determination of two-dimensional texture images for each shooting angle based on the first correspondence matrix and the original three-dimensional model includes: Determine whether the mesh region of the original 3D model is within the corresponding shooting angle based on the mesh normal of the original 3D model and the shooting angle of the physical image. If the mesh area of ​​the original 3D model is within the corresponding shooting viewpoint, then a mapping relationship between the mesh of the original 3D model and the pixels of the physically captured image is established based on the first correspondence matrix. Based on the mapping relationship, the centroid interpolation method is used to assign pixel values ​​to the physically captured image to form a two-dimensional texture image at the corresponding angle.

2. The method according to claim 1, characterized in that, Based on the mapping relationship, centroid interpolation is used to assign pixel values ​​to physically captured images to form two-dimensional texture images at corresponding angles, including: Based on the mapping relationship, the pixel coordinates of the physical image corresponding to the coordinates of the original 3D model are obtained, and the mesh vertices of the original 3D model are determined based on the pixel coordinates of the physical image to determine whether they are within the corresponding shooting view. The centroid interpolation method is used to assign color values ​​to the pixels corresponding to the grid located within the shooting viewpoint to form a two-dimensional texture image corresponding to the shooting viewpoint.

3. The method according to claim 2, characterized in that, Determining whether the mesh vertices of the original 3D model are within the corresponding shooting viewpoint based on the pixel coordinates of the physically captured image includes: The minimum bounding rectangle of pixel coordinates is determined based on the pixel coordinates corresponding to the mesh vertices of the original 3D model. The centroid interpolation method is used to determine whether all pixels within the minimum bounding rectangle are located within the pixel coordinate bounding box mapped by the grid vertex.

4. The method according to claim 3, characterized in that, Determining whether all pixels within the minimum bounding rectangle are located within the pixel coordinate bounding box mapped by the grid vertex using the centroid interpolation method includes: The weighted sum of the three vertex pixels of the triangle in the grid region is used as the target pixel within the corresponding minimum bounding rectangle; Based on the judgment result of whether there is a weighting coefficient less than 0 among the three vertex pixels of the triangle, it is determined whether the target pixel is within the bounding box of the pixel coordinates mapped by the grid vertex.

5. The method according to claim 4, characterized in that, Based on the determination of whether there are weighting coefficients less than 0 among the three vertex pixels of the triangle, it is determined whether the target pixel is within the bounding box of the pixel coordinates mapped by the grid vertex, including: If there is a weighting coefficient less than 0 among the three vertex pixels of the triangle, then the target pixel is not within the bounding box of the pixel coordinates mapped by the grid vertex. If there is no weighting coefficient less than 0 among the three vertex pixels of the triangle, then the target pixel is within the pixel coordinate bounding box of the grid vertex mapping, and the color value obtained by weighted summation of the pixels of the three vertices determined according to the weighting coefficients is assigned to the target pixel.

6. The method according to claim 1, characterized in that, Based on the camera extrinsic matrix and the camera intrinsic matrix, a first correspondence matrix is ​​determined between the pixel coordinate system of the physically captured image corresponding to the original 3D model and the coordinate system of the original 3D model, including: The product of the camera extrinsic matrix and the camera intrinsic matrix is ​​used as the first correspondence matrix.

7. The method according to claim 1, characterized in that, Determining a second correspondence between the two-dimensional texture image and the mesh vertices of the simplified three-dimensional model includes: Multiplying the coordinates of the simplified 3D model by the transformation matrix corresponding to the shooting angle yields the pixel coordinates of the corresponding physical image. The specific color information of the mesh vertices and corresponding pixel coordinates of the simplified 3D model is used as the second correspondence between the 2D texture image and the mesh vertices of the simplified 3D model.

8. A texture mapping system for a three-dimensional model, characterized in that, include: The model acquisition unit is used to acquire the original 3D model, and then process the original 3D model to obtain a simplified 3D model. The parameter acquisition unit is used to acquire the camera intrinsic parameter matrix and the camera extrinsic parameter matrix, wherein the camera extrinsic parameter matrix is ​​the pose information of the original 3D model corresponding to different shooting angles; The first determining unit is used to determine a first correspondence matrix between the pixel coordinate system of the physically captured image corresponding to the original 3D model and the coordinate system of the original 3D model based on the camera extrinsic matrix and the camera intrinsic matrix. The second determining unit is configured to determine two-dimensional texture images of each shooting angle based on the first correspondence matrix and the original three-dimensional model, and to determine a second correspondence between the two-dimensional texture images and the mesh vertices of the simplified three-dimensional model; and, The data processing unit is used to perform texture mapping on the simplified 3D model based on the second correspondence. The second determining unit is configured as follows: Based on the mesh normal of the original 3D model and the shooting angle of the physical image, it is determined whether the mesh area of ​​the original 3D model is within the corresponding shooting angle. If the mesh area of ​​the original 3D model is within the corresponding shooting angle, a mapping relationship between the mesh of the original 3D model and the pixels of the physical image is established based on the first correspondence matrix. Based on the mapping relationship, the pixels of the physical image are assigned values ​​using the centroid interpolation method to form a two-dimensional texture image at the corresponding angle.

9. A terminal device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method as described in claim 1.

Images