Three-dimensional model texture mapping method, device and system and storage medium

Abstract

The invention discloses a three-dimensional model texture mapping method, device and system and a storage medium. The method comprises the following steps: acquiring oblique photography data of a plurality of measuring areas; performing aerial triangular encryption and three-dimensional reconstruction on the oblique photography data of the plurality of measuring areas, and constructing a texture-free geometric model; performing back rejection and shielding detection on the texture-free geometric model; performing global uniform light and color by using a Laplacian algorithm, and filling the average color of the texture blocks into the texture image; local dodging and color dodging are carried out on the texture blocks by adopting a resolution reduction Poisson dodging method; and performing texture layout on the texture blocks, and outputting a live-action three-dimensional model with consistent texture colors.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a three-dimensional model texture mapping method and device, system, and storage medium. Background Art

[0002] With the rapid development of digital technology, real-life 3D models are playing an increasingly important role in many fields, such as urban planning, geographic information systems, virtual reality, etc. In the construction of real-life 3D models, texture mapping technology is crucial. Early texture mapping methods were relatively simple and rough. After obtaining the survey area data, it was difficult to efficiently perform aerial triangulation and 3D reconstruction to build a high-quality texture-free geometric model. The existing texture mapping processing technology, backface culling and occlusion detection are still not accurate enough, which can easily lead to errors or incomplete texture mapping. At the same time, the effects of global and local uniform light and color are not ideal, and it is difficult to ensure the consistency of the texture color of the output real-life 3D model, which affects the realism and aesthetics of the real-life 3D model and reduces the usability and value of the model. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a 3D model texture mapping method and device, system, and storage medium, firstly obtain multiple oblique photography data of the survey area, then perform aerial triangulation encryption and 3D reconstruction to construct a texture-free geometric model, then perform backface culling and occlusion detection on the geometric model, then perform global light and color uniformity and local light and color uniformity of texture blocks in sequence, and finally perform texture nesting on the texture blocks to output a real-life 3D model with consistent texture and color. The present invention can effectively improve the quality and realism of the real-life 3D model, and has broad application prospects in the fields of urban planning, geographic information systems, virtual reality, etc.

[0004] To achieve the above object, the present invention adopts the following technical solution:

[0005] A three-dimensional model texture mapping method, comprising:

[0006] Step S1, obtaining oblique photography data of multiple survey areas;

[0007] Step S2, performing aerial triangulation and three-dimensional reconstruction on the oblique photography data of multiple survey areas to construct a texture-free geometric model;

[0008] Step S3, performing backface culling and occlusion detection on the texture-free geometric model;

[0009] Step S4, perform global light and color uniformity, and fill the texture image with the average color of the texture block;

[0010] Step S5, performing local uniform light and color uniformity on the texture block;

[0011] Step S6: perform texture nesting on the texture blocks and output a real-scene 3D model with consistent texture color.

[0012] Preferably, in step S2, a Laplace algorithm based on texture edge details is used to perform global light and color uniformity on multiple measurement areas.

[0013] Preferably, in step S3, a reduced-resolution Poisson homogenization method based on edge details is used to perform local homogenization on the texture block.

[0014] The present invention also provides a three-dimensional model texture mapping device, comprising:

[0015] An acquisition module is used to acquire oblique photography data of multiple survey areas;

[0016] The construction module is used to perform aerial triangulation and 3D reconstruction on the oblique photography data of multiple survey areas to construct a texture-free geometric model;

[0017] The first processing module is used to perform backface culling and occlusion detection on the textureless geometric model;

[0018] The second processing module is used to perform global light and color uniformity, and fill the texture image with the average color of the texture block;

[0019] The third processing module is used to perform local uniform light and color of the texture block;

[0020] The mapping module is used to perform texture nesting on texture blocks and output a real-life 3D model with consistent texture color.

[0021] Preferably, the second processing module performs global light and color uniformity on multiple measurement areas based on a Laplace algorithm of texture edge details.

[0022] Preferably, the third processing module performs local homogenization of the texture block in a Poisson homogenization method based on reduced resolution of edge details.

[0023] The present invention also provides a three-dimensional model texture mapping system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes the three-dimensional model texture mapping method when executed by the processor.

[0024] The present invention also provides a storage medium, on which a computer program is stored, and the computer program executes the three-dimensional model texture mapping method when running.

[0025] The beneficial effects of the present invention are as follows:

[0026] The present invention significantly improves the model accuracy and optimizes the real-scene three-dimensional texture mapping effect. By acquiring oblique photography data of multiple survey areas for aerial triangulation encryption and three-dimensional reconstruction, the present invention can construct a more accurate texture-free geometric model, provide a high-quality basic framework for subsequent texture mapping, and thus improve the overall accuracy of the final real-scene three-dimensional model. At the same time, unnecessary texture mapping can be effectively removed through backface culling and occlusion detection steps, avoiding the appearance of erroneous and incomplete textures, making the texture of the model clearer and more reasonable. Combining global uniform light and color with local uniform light and color not only ensures the overall color coordination of the texture image, but also highlights the color reality of local details, making the texture effect of the real-scene three-dimensional model more natural and beautiful. Texture patterning is then performed on the texture blocks to ensure that the texture color of the output real-scene three-dimensional model is consistent. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0028] Figure 1 A flowchart of a three-dimensional model texture mapping method according to an embodiment of the present invention;

[0029] Figure 2 Schematic diagram of the method of uniform color at fixed points at the seam, where (a) is a schematic diagram of the vertex at the seam, and (b) is a schematic diagram of weighted uniform color;

[0030] Figure 3 Schematic diagram of the distribution of 16 pixels of the image;

[0031] Figure 4 Schematic diagram of texture arrangement, where (a) is the first arrangement, (b) is the second arrangement, and (c) is the nth arrangement. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] Embodiment 1:

[0035] like Figure 1 As shown, an embodiment of the present invention provides a three-dimensional model texture mapping method, comprising:

[0036] Step S1, obtaining oblique photography data of multiple survey areas through a drone;

[0037] Step S2, performing aerial triangulation and three-dimensional reconstruction on the oblique photography data of multiple survey areas to construct a texture-free geometric model;

[0038] Step S3, performing backface culling and occlusion detection on the texture-free geometric model;

[0039] Step S4, using the Laplace algorithm to perform global light and color uniformity, and filling the texture image with the average color of the texture block;

[0040] Step S5, using a reduced-resolution Poisson homogenization method to perform local homogenization and color homogenization on the texture block;

[0041] Step S6: perform texture nesting on the texture blocks and output a real-scene 3D model with consistent texture color.

[0042] As an implementation of an embodiment of the present invention, in step S3, the image facing away from the triangle is removed. Specifically, the image is located on the back of the triangle when it is shot. The angle α between the normal of the triangle and the light is less than 90°, and the angle between the normal of the triangle and the light is greater than 90°. Therefore, for the triangle with α less than 90°, the image is invisible. In addition, the angle cannot be too large, because the more inclined the image, the greater the deformation of the imaging. Therefore, it is set to remove the images with an angle greater than 75° from the list.

[0043] When the foreground object occludes the background object, if the occlusion detection is not performed to remove this part of the image, an erroneous texture will be generated on the final model. The ray intersection detection method is adopted. First, the occlusion model is constructed. The model is created by mesh vertices and triangles. The light connecting the center of each triangle and the center of the camera is calculated to determine whether the light intersects with the occlusion model. If the intersection produces more than one intersection point, and the remaining intersection points are located in front of the triangle, the image is considered to be occluded. If the light from the center of the triangle to the center of the camera has two other intersection points with the model, then the image is occluded for the left triangle; if the light from the center of the triangle to the center of the camera has no other intersection points with the model, then the image is not occluded for the right triangle.

[0044] As an implementation method of an embodiment of the present invention, in step S4, in order to improve the display effect of the model and achieve a clear texture effect, an embodiment of the present invention proposes a uniform light method based on texture edge details. First, the input dedistorted image is preprocessed for image enhancement, and the brightness of the target texture is increased by image enhancement technology; then unnecessary obstacles are dimmed to enhance the grayscale contrast to achieve the effect of maximizing the target clarity. The Laplace quadratic operator is used to enhance image details. The purpose of image enhancement is to highlight image details and make the image clearer.

[0045] A compensation value is provided globally for the color discontinuity between all texture fragments to correct the global color value of the model. The constraint equation for global uniform lighting is:

[0046]

[0047] Among them, A is a sparse matrix that marks which texture blocks the vertices belong to; g is the color correction value of each vertex; f is the color sampling difference value of adjacent vertices; τ is a sparse matrix that marks whether the vertices are adjacent.

[0048] Solution The optimization problem can be transformed into a simplified algorithmic form:

[0049]

[0050] According to the optimal solution

[0051]

[0052] This method can effectively solve the color difference problem between textures.

[0053] The vertex at the seam can be divided into two vertices on the left and right. The original color value of the vertex is recorded as f, and the color correction value is recorded as g. The color value of the vertex at the seam is obtained by weighted average of the color of the sampling points of the adjacent edges, such as Figure 2 As shown in (a) and (b), the color of vertex v0 is given by and For two-edge calculation, several points are evenly sampled on the two edges. The farther away from the vertex v0, the lower the weight.

[0054] By minimizing the following formula, the color correction value of each triangle vertex is obtained. To ensure the uniqueness of vertex color, the vertices at the seam are divided into left and right vertices for solution:

[0055]

[0056] The first parameter {v} is the vertex at the seam, ensuring that the color values ​​of the left and right vertices are as consistent as possible after correction. The second parameter {v} is the adjacent vertices in the texture block, minimizing the color value difference of adjacent vertices in the texture block after correction. The above formula can be written in the following matrix form:

[0057]

[0058] Among them, the original color value f, the color correction value g, f corresponds to the above formula A and Γ are sparse matrices containing ±1, which are used to select Using the conjugate gradient method, the RGB three channels are solved to obtain the color correction value g of each vertex. Then, using the barycentric coordinates, the color correction values ​​of the three vertices are interpolated to each pixel of the texture block, and the color correction value is applied to obtain the result after global color correction.

[0059] Use the Poisson image editing algorithm to further transition the color at the seam. Given an image, the Laplacian convolution kernel can be used to solve the divergence of each point. Now, conversely, if the divergence of each pixel is given, the Poisson equation is used to solve the value of each pixel. In order to better understand the reconstruction process, now assume that the image size is a 4*4 16 pixel picture, such as Figure 3 As shown:

[0060] Now given the divergence values ​​div(6), div(7), div(10), div(11) of pixel points 6, 7, 10, and 11, we can list the following four equations:

[0061] [V(2)+V(5)+V(7)+V(10)]-4*V(6)=div(6)

[0062] [V(3)+V(6)+V(8)+V(11)]-4*V(7)=div(7)

[0063] [V(6)+V(9)+V(11)+V(14)]-4*V(10)=div(10)

[0064] [V(7)+V(10)+V(12)+V(15)]-4*V(11)=div(11)

[0065] At this time, there are only four equations, but there are 16 pixels, that is, there are 16 unknowns. Therefore, it is impossible to solve all the pixel values ​​by relying on the above four equations alone, and the equations have countless solutions. Therefore, it is necessary to add constraint equations, which are the constraints of the Poisson reconstruction equation. Assume that the boundary constraint is added, that is, if the value u of each pixel point in the outermost circle of the above image is already known, then 12 constraint equations can be obtained. That is:

[0066] V(1)=u(1)V(2)=u(2)V(3)=u(3)

[0067] V(4)=u(4)V(5)=u(5)V(8)=u(8)

[0068] V(9)=u(9)V(12)=u(12)V(13)=u(13)

[0069] V(14)=u(14)V(15)=u(15)V(16)=u(16)

[0070] There are 12 equations above, plus 4 equations for the given divergence, so there are 16 equations. In this way, the system of equations can be solved, and image reconstruction can be achieved by adding boundary constraints to the divergence.

[0071] As an implementation method of an embodiment of the present invention, in step S5, in order to improve the efficiency of texture uniformity processing while ensuring the display effect of the model, a reduced-resolution Poisson uniformity method based on edge details is proposed. The effect of speeding up and maintaining quality is achieved by reducing the space for Poisson editing solution, reducing the amount of calculation, and paying more attention to the processing of boundary details.

[0072] First, the input image is downsampled by bilinear interpolation to reduce the high-resolution image to a lower-resolution image. Based on the pixel values ​​and gradient field of the down-resolution image, the Poisson equation is constructed. The form of the Poisson equation is:

[0073] Where: f is the image after homogenization to be solved; is the Laplacian operator; div is the divergence operator; v is the vector field constructed based on the original image and the gradient field.

[0074] The vector field v can be constructed by comparing the gradient of the original image with the target gradient (i.e. the gradient after homogenization). If the gradient of a certain area of ​​the original image is large, it means that there may be illumination changes or edge details in the area, and the gradient of the area needs to be kept unchanged during the homogenization process. Therefore, the target gradient can be set to the same value as the gradient of the original image. The Poisson equation is solved using a discretized numerical method to obtain the homogenized image f.

[0075] The reduced-resolution Poisson method is used to limit the Poisson editing range of the texture block to a 20-pixel wide boundary strip. The pixel values ​​of the inner and outer rings of the strip are used as the boundary conditions of the Poisson equation. Taking the texture block with an original scale of 1096x 960 as an example, the number of unknowns in Poisson editing is reduced from nearly 80,000 to 7,500.

[0076] As an implementation method of the embodiment of the present invention, in step S6, the texture blocks are arranged in a texture pattern to facilitate storage and transmission, thereby improving rendering efficiency, which specifically includes:

[0077] 1) Calculate the outer rectangle of each texture block, that is, (x min ,y min ,x max ,y max ) and sort them by area. The largest texture block is placed first. If the length and width of the rectangle are both larger than the optimal texture size, the size of the first texture is set to the maximum texture size; if the length and width of the rectangle are both smaller than half of the optimal texture size, the size of the optimal texture is halved, and so on, until the minimum texture size is reached.

[0078] 2) Define T as the set of areas to be sorted. Initially, it is the entire texture map, with the upper left corner of the texture map as the coordinate origin, the right as the x-axis, and the bottom as the y-axis.

[0079] 3) Arrange the texture blocks from the upper left corner of T, such as Figure 4 As shown in (a), T1 is the first texture block to be arranged. The texture image is divided into 1, 2, and 3 regions by extending the lower right corner of the texture block to the right and downward. 1 and 2∩3 form the upper and lower partitions, and 2 and 1∩3 form the left and right partitions. The absolute value of the difference between the upper and lower partition area ratio and the left and right partition area ratio and 1 is calculated. If the value is larger, it means that the partition area difference is larger, then T is updated and the combined area of ​​2 and 1∩3 is stored.

[0080] 4) Figure 4 As shown in (b), for the second texture block to be arranged, traverse the area to be arranged in T and arrange it into an area whose length and width are smaller than the area to be arranged and whose utilization rate is the highest after arrangement. Then repeat the third step, divide the space and update T, as Figure 4 as shown in (c).

[0081] 5) Repeat steps 3 and 4 until the current texture map cannot accommodate any texture blocks. Then go to step 1, calculate a new texture map, and repeat steps 2 to 5 until all texture blocks are placed in the texture map.

[0082] Finally, after writing, the local model file is replaced with the node information and the saved coordinate system, completing the conversion from computer memory to local. The integration of the textured 3D model is completed.

[0083] All vertices of the triangular mesh model are used as solving units, and the color correction value is solved for each vertex. Then, the vertex color correction value is interpolated to the entire texture block using the centroid coordinates, in order to reduce the large color difference between texture blocks. Then, the image pixel is used as the solving unit, and local Poisson image fusion is performed to further transition the color of the seam. This includes global color adjustment and local color adjustment.

[0084] Embodiment 2:

[0085] The embodiment of the present invention further provides a three-dimensional model texture mapping device, comprising:

[0086] An acquisition module is used to acquire oblique photography data of multiple survey areas;

[0087] The construction module is used to perform aerial triangulation and 3D reconstruction on the oblique photography data of multiple survey areas to construct a texture-free geometric model;

[0088] The first processing module is used to perform backface culling and occlusion detection on the textureless geometric model;

[0089] The second processing module is used to perform global light and color uniformity, and fill the texture image with the average color of the texture block;

[0090] The third processing module is used to perform local uniform light and color of the texture block;

[0091] The mapping module is used to perform texture nesting on texture blocks and output a real-life 3D model with consistent texture color.

[0092] As an implementation method of the embodiment of the present invention, the second processing module performs global light and color uniformity on multiple measurement areas based on the Laplace algorithm of texture edge details.

[0093] As an implementation manner of the embodiment of the present invention, the third processing module performs local homogenization of the texture block in a Poisson homogenization method based on the reduced resolution of edge details.

[0094] Embodiment 3:

[0095] An embodiment of the present invention further provides a three-dimensional model texture mapping system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a three-dimensional model texture mapping method when executed by the processor.

[0096] Embodiment 4:

[0097] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program executes the three-dimensional model texture mapping method when running.

[0098] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A three-dimensional model texture mapping method, characterized in that: include: Step S1, obtaining oblique photography data of multiple survey areas; Step S2, performing aerial triangulation and three-dimensional reconstruction on the oblique photography data of multiple survey areas to construct a texture-free geometric model; Step S3, performing backface culling and occlusion detection on the texture-free geometric model; Step S4, perform global light and color uniformity, and fill the texture image with the average color of the texture block; Step S5, performing local uniform light and color uniformity on the texture block; Step S6: perform texture nesting on the texture blocks and output a real-scene 3D model with consistent texture color.

2. The three-dimensional model texture mapping method according to claim 1, characterized in that: In step S2, the Laplace algorithm based on texture edge details is used to perform global light and color uniformity on multiple measurement areas.

3. The three-dimensional model texture mapping method according to claim 2, characterized in that: In step S3, a reduced-resolution Poisson homogenization method based on edge details is used to perform local homogenization on the texture block.

4. A 3D model texture mapping device, characterized in that: include: An acquisition module is used to acquire oblique photography data of multiple survey areas; The construction module is used to perform aerial triangulation and 3D reconstruction on the oblique photography data of multiple survey areas to construct a texture-free geometric model; The first processing module is used to perform backface culling and occlusion detection on the textureless geometric model; The second processing module is used to perform global light and color uniformity, and fill the texture image with the average color of the texture block; The third processing module is used to perform local uniform light and color of the texture block; The mapping module is used to perform texture nesting on texture blocks and output a real-life 3D model with consistent texture color.

5. The three-dimensional model texture mapping device according to claim 4, characterized in that: The second processing module performs global light and color uniformity on multiple measurement areas based on the Laplace algorithm of texture edge details.

6. The three-dimensional model texture mapping device according to claim 5, characterized in that: The third processing module performs local homogenization of the texture block based on the reduced-resolution Poisson homogenization method of edge details.

7. A three-dimensional model texture mapping system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the three-dimensional model texture mapping method according to any one of claims 1 to 3 is executed.

8. A storage medium, characterized in that: The storage medium stores a computer program, which executes the three-dimensional model texture mapping method according to any one of claims 1 to 3 when running.

Images