A method, medium and system for automatically connecting virtual textures of digital twin 3D models
Through grid simplification, UV expansion and numerical optimization algorithms, a system of texture connection equations was established, which solved the problem of large workload and continuity in the texture mapping process of digital twin 3D models, and realized automated seamless texture mapping, improving efficiency and a sense of reality.
Patent Information
- Application Number
- CN202411034679.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-07-31
AI Technical Summary
The prior art has problems such as high workload and difficulty in ensuring the continuity and reality of texture maps in the process of digital twin 3D models.
Using grid simplification, UV expansion, image segmentation, feature extraction and numerical optimization algorithms to establish equations such as color continuity, texture gradient, geometric constraints and boundary alignment, and solve the texture connection matrix through numerical optimization algorithm to realize the automation of texture maps. Seam connection.
A fully automated texture mapping process is realized, ensuring the visual continuity and geometric consistency of texture mapping, supporting multi-scale detail expression, improving modeling efficiency and reducing labor costs.
Smart Images

Figure CN118570417B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of digital twin technology, and specifically, relates to a method, medium and system for automatically connecting virtual textures of a digital twin 3D model. Background Art
[0002] Digital twin technology is an emerging technology that has emerged in recent years. Its core idea is to use digital models to simulate and represent physical entities in the real world, so as to realize functions such as remote monitoring, analysis and control of entities. Digital twin models usually include a variety of information such as geometry, material, texture, etc., which can accurately restore the appearance and characteristics of physical entities. This technology is widely used in industrial manufacturing, smart cities, cultural heritage protection and other fields, and provides technical support for the deep integration of the physical world and the virtual world. In the process of building digital twin models, how to express the visual characteristics of entities with high fidelity is a key issue. Among them, how to achieve seamless texture mapping on the surface of digital models is a thorny technical challenge. Traditional texture mapping methods usually require manual UV unfolding of the model surface, and then manual adjustment and splicing of textures, which is not only labor-intensive, but also difficult to ensure the continuity and realism of texture mapping. Summary of the invention
[0003] In view of this, the present invention provides a method, medium and system for automatic connection of virtual textures of a digital twin 3D model, which can solve the technical problems of the existing method of manually processing the model surface, which has a large workload and is difficult to ensure the continuity and realism of texture mapping.
[0004] The present invention is achieved in that:
[0005] A first aspect of the present invention provides a method for automatically connecting virtual textures of a digital twin 3D model, which includes the following steps:
[0006] S10. Simplify the mesh of the digital twin 3D model, reduce the number of vertices and faces, and generate a multi-level detailed model;
[0007] S20, using a UV unfolding algorithm, unfolding the simplified multi-level detail model onto a 2D plane to generate a UV mapping;
[0008] S30, based on UV mapping, dividing the multi-level detail model into a plurality of sub-regions, each sub-region corresponding to a texture map;
[0009] S40, using an image segmentation algorithm to perform boundary detection and feature extraction on the texture map of each sub-region;
[0010] S50, using an image matching algorithm to calculate the similarity and position offset between texture maps of adjacent sub-regions;
[0011] S60, establishing a set of equations for connecting textures of adjacent regions, including a color continuity equation, a texture gradient equation, a geometric constraint equation, a boundary alignment equation, and a global optimization equation;
[0012] S70, using a numerical optimization algorithm to solve the texture connection equation group to obtain a texture connection matrix;
[0013] S80, transforming and adjusting the texture maps of adjacent sub-regions according to the texture connection matrix, remapping the adjusted textures into the UV space of the 3D model, and generating a final seamless texture map.
[0014] Wherein, the color continuity equation is specifically expressed as:
[0015] ;
[0016] In the formula, is the number of sub-regions; is the number of color channels; For the The sub-regions are located at the parameter coordinates The color value at and is the parameterized coordinate, and its range is , is the partial derivative operator, To express the integral over s and t, , is a temporary variable used for summation calculation. In this formula, , .
[0017] Color Value Steps to obtain:
[0018] 1) UV coordinates Mapping to parameter coordinates :
[0019] ;
[0020] ;
[0021] 2) Use bilinear interpolation to sample the color value in the texture map.
[0022] Wherein, the texture gradient equation is specifically expressed as:
[0023] ;
[0024] In the formula, For the The sub-regions are located at the parameter coordinates The texture value at is the gradient operator, , is a temporary variable used for summation calculation. In this formula, , , where the texture gradient The calculation steps are:
[0025] 1) Calculation Directional Gradient: ;
[0026] 2) Calculation Directional Gradient: ;
[0027] 3) Combine into a gradient vector: .
[0028] Wherein, the geometric constraint equation is specifically expressed as:
[0029] ;
[0030] In the formula, For the The number of vertices in each sub-region; For the The 3D coordinates of the vertices; For the UV coordinates of vertices; For the The texture mapping function of each sub-region is Indicates Texture mapping function for the UV coordinate area of the vertices; for The Jacobian matrix of ; is the weight coefficient, represents the determinant of the matrix, , is a temporary variable used for summation calculation. In this formula, , and for each ,exist . Among them, the Jacobian matrix The calculation steps are:
[0031] 1) Calculation ;
[0032] 2) Calculation ;
[0033] 3) Combined into Jacobian matrix: .
[0034] Wherein, the boundary alignment equation is specifically expressed as:
[0035] ;
[0036] In the formula, For the The number of sampling points at the sub-region boundaries; For the The sub-region boundary The coordinates of the sampling points are recorded as boundary sampling points; is the curvature smoothing weight, , is a temporary variable used for summation calculation. In this formula, , and for each ,exist Among them, the boundary sampling points Steps to obtain:
[0037] 1) Uniform sampling along sub-region boundaries in UV space points;
[0038] 2) For each sampling point, use bilinear interpolation to calculate its position in the texture.
[0039] Wherein, the global optimization equation is specifically expressed as:
[0040] ;
[0041] In the formula, is the weight coefficient; is a two-dimensional Fourier transform, where The calculation steps are:
[0042] 1) Texture Perform discrete sampling and obtain The matrix
[0043] 2) Calculate the two-dimensional discrete Fourier transform:
[0044] ;
[0045] in .
[0046] The texture connection matrix is specifically expressed as:
[0047] ;
[0048] In the formula, is the height of the texture map (number of pixels); is the width of the texture map (number of pixels); is the first Line The elements of the column are recorded as texture connection vectors.
[0049] Wherein, the texture connection vector is specifically expressed as:
[0050] ;
[0051] In the formula, : Color adjustment coefficient, used to adjust pixel color; : Position offset of direction; : Position offset of direction; : scaling factor; : Rotation angle.
[0052] Specifically, the ranges of the above parameters are described as follows:
[0053] (Color adjustment factor) range is usually between; Indicates no color adjustment; It will darken the color; It will brighten the color;
[0054] and (Position offset) The units of both are pixels; positive values indicate a rightward or downward offset; negative values indicate a leftward or upward offset;
[0055] (scaling factor): Indicates no scaling; Indicates shrinkage; Indicates magnification;
[0056] (Rotation angle): The unit is radians; positive values indicate clockwise rotation; negative values indicate counterclockwise rotation;
[0057] Using this texture concatenation matrix, the original texture can be transformed as follows:
[0058] Color Adjustment:
[0059] ;
[0060] in is the new color value, is the original color value.
[0061] Position Transformation:
[0062] ;
[0063] ;
[0064] in is the new position coordinate, is the original position coordinate.
[0065] Scaling and Rotation:
[0066] The following transformation matrices can be used:
[0067] ;
[0068] Apply transformation: ;
[0069] in are the transformed coordinates.
[0070] Special function description:
[0071] : Cosine function, input is radians.
[0072] : Sine function, input is radians.
[0073] Specifically, the step S10 specifically includes: using a mesh simplification algorithm to geometrically simplify the original high-precision 3D model, and generating a multi-level 3D model containing different levels of geometric details by setting different simplification degrees. The mesh simplification algorithm will gradually delete those vertices and patches that have little effect on the shape of the model to reduce the geometric complexity of the entire model. During the simplification process, the maximum allowable error threshold can be set according to actual needs, usually between 0.01 and 0.1, to balance the degree of simplification and model fidelity. The multi-level 3D model will provide a basis for subsequent texture mapping and connection processing.
[0074] Wherein, the step S20 specifically includes: using a cutting-based UV unfolding algorithm, such as the LSCM algorithm, to map the simplified multi-level 3D model surface to a 2D plane and generate UV coordinate information. The algorithm will find a set of optimal cutting edges, cut the 3D mesh surface into several blocks, and then unfold these blocks to the 2D plane, and finally obtain the UV coordinates of each vertex. During the UV unfolding process, the algorithm will try to maintain the relative position relationship between each block, while also trying to minimize deformation and distortion during the unfolding process. In practical applications, the parameters of the LSCM algorithm can be appropriately adjusted according to the complexity of the model, such as controlling the number of mesh cuts, to balance the unfolding quality and computational efficiency.
[0075] Wherein, the step S30 specifically includes: using an image segmentation algorithm, such as a Watershed transform segmentation algorithm, to segment the texture map on the UV plane and divide it into several smaller sub-regions. The algorithm will divide the texture map into relatively independent regions according to its color, gradient and other characteristics. In the segmentation process, the granularity of the segmentation needs to be reasonably set, which should not be too fine to result in too many regions, nor too coarse to accurately describe the original texture. The UV coordinates can be pre-classified using an adaptive K-Means clustering algorithm, and then the Watershed transform algorithm can be used for refined segmentation, wherein the number of K-Means clustering centers can be set between 50 and 200.
[0076] Wherein, the step S40 specifically includes: first, using the Canny edge detection algorithm to perform edge detection on the texture map of each sub-region to obtain a clear region boundary line. In the Canny algorithm, the high and low thresholds can be set to about 0.1 and 0.3 respectively. Secondly, using the SIFT algorithm to extract key points and descriptors from the texture map to provide valuable information for subsequent texture matching. In the SIFT algorithm, the number of key points can be appropriately adjusted according to the resolution of the texture map, usually between 100 and 1000.
[0077] Wherein, the step S50 specifically includes: using a feature matching-based image registration algorithm, such as the RANSAC algorithm, to calculate the similarity and position offset between the texture maps of adjacent sub-regions. The algorithm will first match the corresponding SIFT key point pairs in the texture maps of adjacent sub-regions, and then estimate the affine transformation relationship between the two sub-regions. By analyzing these transformation parameters, the similarity and position offset information between adjacent regions can be obtained. In the RANSAC algorithm, a higher number of repeated sampling can be set, such as 1000 times, and the matching error threshold can also be appropriately adjusted according to actual conditions, usually set between 1 and 5 pixels.
[0078] Wherein, the step S60 specifically includes: establishing a complete texture connection equation group including a color continuity equation, a texture gradient equation, a geometric constraint equation, a boundary alignment equation and a global optimization equation. The equation group describes various constraints that the texture mapping between adjacent sub-regions should satisfy, and by minimizing the global optimization equation, the overall optimal texture connection effect can be obtained. In the specific implementation, the weight coefficients of each equation need to be set according to the actual situation to balance the relative importance of different constraints.
[0079] Wherein, the step S70 specifically includes: using a numerical optimization algorithm based on the gradient descent method to solve the aforementioned texture connection equations to obtain the optimal texture connection matrix During initialization, you need to set the initial texture connection matrix , and the learning rate , maximum number of iterations and convergence threshold In the iterative optimization process, the current loss function value is calculated first, and then the loss function is calculated. The gradient is finally updated according to the learning rate The matrix is added until the convergence condition is met. In the specific implementation, these parameters can be appropriately adjusted according to the scale and complexity of the problem.
[0080] Wherein, the step S80 specifically comprises: according to the texture connection matrix obtained by the optimization , perform corresponding color adjustment, position offset, scaling and rotation transformations on the texture maps of each sub-area, so that the textures of adjacent sub-areas can be seamlessly connected at the boundary. Then, the transformed texture map is re-pasted back to the surface of the 3D model through inverse UV mapping. In order to ensure the texture connection effect of the entire 3D model surface, some key areas can also be locally optimized and the texture map can be further fine-tuned. The resulting seamless texture map can fit the surface of the 3D model well both visually and geometrically.
[0081] A second aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are run, they are used to execute the above-mentioned method for automatic connection of virtual textures of a digital twin 3D model.
[0082] A third aspect of the present invention provides a digital twin 3D model virtual texture automatic connection system, which includes the above-mentioned computer-readable storage medium.
[0083] Compared with the prior art, the method, medium and system for automatically connecting virtual textures of a digital twin 3D model provided by the present invention have the following beneficial effects:
[0084] 1. A fully automated texture mapping process is achieved. The invented method can automatically generate highly integrated texture maps based on the geometric characteristics of the 3D model without manual intervention. This greatly improves modeling efficiency and reduces labor costs.
[0085] 2. Ensure the visual continuity and geometric consistency of texture mapping. The inventive method can automatically eliminate the obvious boundaries between textures by establishing a complete mathematical model, taking into account factors such as color continuity, texture gradient, and geometric constraints, so that the entire 3D model surface looks natural and unified. This improves the texture discontinuity problem that is prevalent in the prior art.
[0086] 3. Support multi-scale detail expression. The invented method can generate a multi-layer model containing texture maps of different resolutions according to the geometric complexity of the 3D model. This provides flexibility for subsequent rendering, interaction and other applications, and can dynamically adjust the model details according to specific needs.
[0087] In summary, the solution of the present invention solves the technical problems of the existing method of manually processing the model surface, which has a large workload and is difficult to ensure the continuity and realism of the texture mapping. BRIEF DESCRIPTION OF THE DRAWINGS
[0088] Figure 1 A flow chart of the method provided by the present invention. DETAILED DESCRIPTION
[0089] In order to make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0090] like Figure 1 As shown, it is a flow chart of a method for automatically connecting virtual textures of a digital twin 3D model provided by the present invention. The method comprises the following steps:
[0091] S10. Simplify the mesh of the digital twin 3D model, reduce the number of vertices and faces, and generate a multi-level detailed model;
[0092] S20, using a UV unfolding algorithm, unfolding the simplified multi-level detail model onto a 2D plane to generate a UV mapping;
[0093] S30, based on UV mapping, dividing the multi-level detail model into a plurality of sub-regions, each sub-region corresponding to a texture map;
[0094] S40, using an image segmentation algorithm to perform boundary detection and feature extraction on the texture map of each sub-region;
[0095] S50, using an image matching algorithm to calculate the similarity and position offset between texture maps of adjacent sub-regions;
[0096] S60, establishing a set of equations for connecting textures of adjacent regions, including a color continuity equation, a texture gradient equation, a geometric constraint equation, a boundary alignment equation, and a global optimization equation;
[0097] S70, using a numerical optimization algorithm to solve the texture connection equation group to obtain a texture connection matrix;
[0098] S80, transforming and adjusting the texture maps of adjacent sub-regions according to the texture connection matrix, remapping the adjusted textures into the UV space of the 3D model, and generating a final seamless texture map.
[0099] The specific implementation methods of the above steps are described in detail below:
[0100] Step S10: Simplify the mesh of the digital twin 3D model, reduce the number of vertices and faces, and generate a multi-level detailed model
[0101] The purpose of this step is to reduce the detail information in the 3D model and generate a multi-level model that can express different degrees of detail. First, use a mesh simplification algorithm, such as the Progressive Mesh Simplification algorithm, to simplify the original high-precision 3D model. This algorithm will gradually delete those vertices and patches that have little effect on the shape of the model, thereby reducing the geometric complexity of the entire model. By setting different simplification levels, multiple 3D models with different levels of detail can be obtained. These model hierarchies are closely related to subsequent texture mapping and connection. For example, texture mapping and connection on a lower-resolution model can improve computational efficiency, while a finer texture effect can be obtained on a high-resolution model. A commonly used threshold parameter in the mesh simplification algorithm is the maximum allowable error, which can be adjusted according to actual needs, and is usually between 0.01 and 0.1.
[0102] Step S20: Using the UV unfolding algorithm, unfold the simplified multi-level detail model onto a 2D plane to generate a UV mapping.
[0103] The purpose of this step is to map the surface of the 3D model onto a 2D texture plane to prepare for subsequent texture mapping. In specific implementation, a cutting-based UV unfolding algorithm, such as the LSCM (Least Squares Conformal Maps) algorithm, can be used. This type of algorithm will find a set of optimal cutting edges, cut the 3D mesh surface into several blocks, and then unfold these blocks onto a 2D plane to finally obtain a UV coordinate mapping. During the UV unfolding process, it is necessary to try to maintain the relative position relationship between the blocks, while also minimizing deformation and distortion during the unfolding process. The LSCM algorithm achieves this goal by solving a least squares optimization problem. In practical applications, the parameters of the LSCM algorithm can be appropriately adjusted according to the complexity of the model, such as controlling the number of mesh cuts, to balance the unfolding quality and computational efficiency.
[0104] Step S30: Based on UV mapping, the multi-level detail model is divided into several sub-areas, each sub-area corresponds to a texture map
[0105] This step aims to divide the surface of the entire 3D model into several smaller sub-regions, each of which corresponds to an independent texture map. The purpose of this is to facilitate the subsequent texture connection processing. Specifically, image segmentation algorithms, such as the Watershed transform segmentation algorithm, can be used to segment the texture map on the UV plane. This type of algorithm divides the texture map into several relatively independent regions based on its color, gradient and other characteristics. In the segmentation process, the granularity of the segmentation needs to be set reasonably. It should not be too fine that there are too many regions, nor too coarse that the original texture cannot be accurately described. A feasible method is to first use the adaptive K-Means clustering algorithm to pre-classify the UV coordinates, and then use the Watershed transform algorithm for refined segmentation. When performing K-Means clustering, the number of cluster centers can be set between 50 and 200 to balance the segmentation granularity and computational complexity.
[0106] Step S40: Use image segmentation algorithm to perform boundary detection and feature extraction on the texture map of each sub-region
[0107] Based on the texture maps of each sub-region obtained in the previous step, the purpose of this step is to further extract the boundary information and texture features of each sub-region. First, the Canny edge detection algorithm can be used to detect the edges of the texture maps of each sub-region to obtain clear regional boundaries. These boundary line information will play an important role in the subsequent texture connection process. Secondly, in order to obtain the texture features of each sub-region, the SIFT (Scale-Invariant Feature Transform) algorithm can be used to extract key points and descriptors from the texture map. The SIFT algorithm can describe the key points in the texture image with invariant features such as scale, rotation, and brightness, providing valuable information for subsequent texture matching. In the Canny algorithm, the high and low thresholds can be set to about 0.1 and 0.3 respectively; in the SIFT algorithm, the number of key points can be appropriately adjusted according to the resolution of the texture map, usually between 100 and 1000.
[0108] Step S50: Using an image matching algorithm, calculate the similarity and position offset between texture maps of adjacent sub-regions
[0109] After obtaining the boundary information and texture features of each sub-region, the goal of this step is to calculate the similarity and position offset between adjacent sub-regions to provide a basis for subsequent texture connection. Specifically, an image registration algorithm based on feature matching can be used, such as the RANSAC (Random Sample Consensus) algorithm. The algorithm will first match the corresponding SIFT key point pairs in the texture maps of adjacent sub-regions, and then estimate the affine transformation relationship between the two sub-regions, including displacement, rotation, scaling, etc. By analyzing these transformation parameters, the similarity and position offset information between adjacent regions can be obtained. In the RANSAC algorithm, a higher number of repeated sampling can be set, such as 1000 times, to improve the accuracy and robustness of the matching. At the same time, the matching error threshold can also be appropriately adjusted according to the actual situation, usually set between 1 and 5 pixels.
[0110] Step S60: Establishing a set of equations for texture connection between adjacent regions, including color continuity equations, texture gradient equations, geometric constraint equations, boundary alignment equations, and global optimization equations
[0111] The focus of this step is to establish a complete set of texture connection equations to describe the various constraints that texture mapping between adjacent sub-regions should meet. Specifically, they include:
[0112] 1. Color continuity equation: This equation requires that the color values of adjacent sub-regions should remain continuous at the boundaries to avoid obvious color block boundaries.
[0113] 2. Texture gradient equation: This equation requires that the texture gradient of adjacent sub-regions should also remain continuous at the boundary to avoid obvious texture faults.
[0114] 3. Geometric constraint equation: This equation requires that when mapping the texture map back to the 3D model surface, the mapping error should be minimized while keeping the local shape of the texture map unchanged as much as possible.
[0115] 4. Boundary alignment equation: This equation requires that the geometric shapes and positions of adjacent sub-regions on the boundary line should be aligned to avoid obvious boundary misalignment.
[0116] 5. Global optimization equation: This equation is a weighted combination of the first four equations. By minimizing this equation, the overall optimal texture connection effect can be obtained.
[0117] In the specific implementation, the weight coefficients of each equation need to be set according to the actual situation to balance the relative importance of different constraints. For example, color continuity and gradient continuity are usually given higher weights to ensure smooth transition of texture boundaries; geometric constraints and boundary alignment have relatively low weights and play a supporting role. The weight coefficients in the global optimization equation can be adjusted through empirical values or grid search.
[0118] Step S70: Use a numerical optimization algorithm to solve the texture connection equations to obtain the texture connection matrix
[0119] With the above-mentioned texture connection equations, the next key is how to solve this optimization problem to obtain the final texture connection result. Here we can use a numerical optimization algorithm based on the gradient descent method. The specific steps are as follows:
[0120] 1. Initialization: First set the initial texture connection matrix ,in is a The matrix represents the transformation of the texture map. At the same time, the learning rate is set , maximum number of iterations and convergence threshold .
[0121] 2. Discretization equations: Since the aforementioned equations are all defined based on continuous domains, they need to be discretized before they can be solved on a computer. They mainly include:
[0122] Replace continuous color, gradient, geometry, etc. values with discrete values sampled from a grid
[0123] Express the integral term as a finite sum
[0124] Approximate the continuous second-order derivatives using finite differences
[0125] 3. Iterative optimization: Use gradient descent method for iterative optimization. In each iteration, first calculate the current loss function value , and then calculate the loss function Gradient ,Finally, according to the learning rate renew Matrix. Iterate until convergence condition is met.
[0126] 4. Output results: the final result The matrix is the optimal texture connection matrix. It contains information about transforming and adjusting the original texture, and can be used to guide how to remap the texture map to the 3D model surface to achieve seamless connection.
[0127] In specific implementation, the learning rate needs to be adjusted appropriately according to the scale and complexity of the problem. and the maximum number of iterations .generally The value ranges from 0.01 to 0.1. The value is between 100 and 1000. At the same time, an adaptive learning rate strategy can also be used to speed up the convergence speed. In addition, you can also consider using a second-order optimization algorithm such as Newton's method to obtain faster convergence characteristics.
[0128] Step S80: transform and adjust the texture maps of adjacent sub-areas according to the texture connection matrix, remap the adjusted textures to the UV space of the 3D model, and generate the final seamless texture map.
[0129] To obtain the optimal texture connection matrix After that, the last step is to transform and adjust the texture map of each sub-area according to this matrix, and remap the adjusted texture back to the UV space of the 3D model. The specific operation is as follows:
[0130] 1. For each sub-area texture map, according to its corresponding Matrix elements perform corresponding transformations such as color adjustment, position offset, scaling, and rotation. This allows the textures of adjacent sub-areas to be seamlessly connected at the boundary.
[0131] 2. The transformed texture map is pasted back onto the surface of the 3D model by inverse UV mapping. This step utilizes the UV coordinate information obtained in the previous step S20.
[0132] 3. In order to ensure the texture connection effect of the entire 3D model surface, some key areas can be locally optimized, such as further fine-tuning the texture near the boundary line.
[0133] Through the above steps, the final seamless texture map is obtained, which is not only visually smooth and natural, but also geometrically fits the surface of the 3D model well. This method greatly improves the realism and immersion of the digital twin model, and has broad application prospects in virtual reality, game animation, cultural heritage digitization and other fields.
[0134] In general, this method of automatic virtual texture connection of digital twin 3D models is a very complex and powerful technology, involving multiple core algorithms of computer graphics and image processing, such as mesh simplification, UV unfolding, image segmentation, feature matching, and numerical optimization. By cleverly combining these algorithms and building a complete mathematical model, we finally achieved excellent texture mapping effects in both visual and geometric aspects. This provides strong support for the high-fidelity rendering and detailed expression of digital twin models.
[0135] The following is a detailed description of each equation:
[0136] 1. Color continuity equation:
[0137]
[0138] in:
[0139] is the number of sub-regions
[0140] is the number of color channels (usually 3, representing RGB)
[0141] For the The sub-regions are located at the parameter coordinates The color value at
[0142] and is the parameterized coordinate, and its range is
[0143] Color Value Steps to obtain:
[0144] 1. UV coordinates Mapping to parameter coordinates :
[0145]
[0146]
[0147] 2. Use bilinear interpolation to sample the color value in the texture map.
[0148] 2. Texture gradient equation:
[0149]
[0150] in:
[0151] For the The sub-regions are located at the parameter coordinates Texture value at
[0152] is the gradient operator
[0153] Texture Gradient The calculation steps are:
[0154] 1. Calculation Directional Gradient:
[0155] 2. Calculation Directional Gradient:
[0156] 3. Combine into a gradient vector:
[0157] 3. Geometric constraint equations:
[0158]
[0159] in:
[0160] For the The number of vertices in the sub-region
[0161] For the 3D coordinates of the vertices
[0162] For the UV coordinates of vertices
[0163] For the Texture mapping function for each sub-region
[0164] for The Jacobian matrix
[0165] is the weight coefficient
[0166] Jacobian matrix The calculation steps are:
[0167] 1. Calculation
[0168] 2. Calculation
[0169] 3. Combine into Jacobian matrix:
[0170] 4. Boundary alignment equation:
[0171]
[0172] in:
[0173] For the The number of sampling points at the sub-region boundary
[0174] For the The sub-region boundary The coordinates of the sampling points
[0175] Smooth weights for curvature
[0176] Boundary sampling points Steps to obtain:
[0177] 1. Uniform sampling along sub-region boundaries in UV space Points
[0178] 2. For each sampling point, use bilinear interpolation to calculate its position in the texture
[0179] 5. Global optimization equation:
[0180]
[0181] in:
[0182] is the weight coefficient
[0183] is the two-dimensional Fourier transform
[0184] Fourier Transform The calculation steps are:
[0185] 1. Texture Perform discrete sampling and obtain The matrix
[0186] 2. Calculate the two-dimensional discrete Fourier transform:
[0187]
[0188] in
[0189] These equations together form the texture connection equations. By minimizing , the optimal texture connection result can be obtained. To solve this optimization problem, a numerical optimization algorithm such as gradient descent or Newton's method is required.
[0190] S70: Use numerical optimization algorithm to solve the texture connection equations and obtain the texture connection matrix
[0191] 1. Initialization:
[0192] Set the initial texture matrix ,in and The height and width of the texture map respectively.
[0193] Setting the learning rate , maximum number of iterations , and the convergence threshold .
[0194] 2. Discretize the global optimization equation:
[0195]
[0196] Each discretized sub-equation is defined as follows:
[0197] a. Color continuity equation:
[0198]
[0199] in is the second-order difference operator:
[0200]
[0201] b. Texture gradient equation:
[0202]
[0203] in is the discrete gradient operator:
[0204]
[0205] c. Geometric constraint equations:
[0206]
[0207] in is the discrete Jacobian matrix:
[0208]
[0209] d. Boundary alignment equation:
[0210]
[0211] in is the discrete second-order difference:
[0212]
[0213] e. Fourier term:
[0214]
[0215] in is the discrete Fourier transform:
[0216]
[0217] 3. Iterative optimization:
[0218] for arrive :
[0219] a. Calculate the current loss:
[0220]
[0221] b. Calculate the gradient:
[0222]
[0223] The gradient Each element of is calculated as follows:
[0224]
[0225] in is a small disturbance value, is the identity matrix.
[0226] c. Update the texture connection matrix:
[0227]
[0228] d. Check convergence conditions:
[0229] if , the iteration stops.
[0230] 4. Output the final texture connection matrix:
[0231]
[0232] In this process, the texture connection matrix Represents the transformation and adjustment of the original texture. It can contain the following information:
[0233] Color Adjustment: Can be expressed for OK Color adjustment factor for the column pixels.
[0234] Position offset: Some elements of the and The offset of the direction.
[0235] Scaling and Rotation: Certain elements of can form a transformation matrix that represents the scaling and rotation of the texture.
[0236] In this way, an optimized texture connection matrix can be obtained, which can minimize the global optimization equation and thus achieve the best texture connection effect.
[0237] Specifically, the texture connection matrix It can be expressed as:
[0238]
[0239] in:
[0240] : Height of the texture map (number of pixels)
[0241] : Width of the texture map (in pixels)
[0242] : The first Line Elements of a column
[0243] Each element It can be further expressed as a vector, denoted as the texture connection vector:
[0244]
[0245] in:
[0246] : Color adjustment coefficient, used to adjust pixel color
[0247] : Position offset in the direction (pixels)
[0248] : Position offset in the direction (pixels)
[0249] : Scaling factor
[0250] : Rotation angle (radians)
[0251] Parameter and variable description:
[0252] (Color adjustment factor) range is usually between; Indicates no color adjustment; It will darken the color; It will brighten the color;
[0253] and (Position offset) The units of both are pixels; positive values indicate a rightward or downward offset; negative values indicate a leftward or upward offset;
[0254] (scaling factor): Indicates no scaling; Indicates shrinkage; Indicates magnification;
[0255] (Rotation angle): The unit is radians; positive values indicate clockwise rotation; negative values indicate counterclockwise rotation;
[0256] Using this texture concatenation matrix, the original texture can be transformed as follows:
[0257] Color Adjustment:
[0258] ;
[0259] in is the new color value, is the original color value.
[0260] Position Transformation:
[0261] ;
[0262] ;
[0263] in is the new position coordinate, is the original position coordinate.
[0264] Scaling and Rotation:
[0265] The following transformation matrices can be used:
[0266] ;
[0267] Apply transformation: ;
[0268] in are the transformed coordinates.
[0269] Special function description:
[0270] : Cosine function, input is radians.
[0271] : Sine function, input is radians.
[0272] By optimizing this texture connection matrix ,We can find the best parameter combination so that adjacent textures can be smoothly connected while maintaining the continuity and naturalness of the overall texture. In practical applications, the structure and parameters of the matrix may be adjusted according to specific needs to achieve the best connection effect.
[0273] A second aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are run, they are used to execute the above-mentioned method for automatic connection of virtual textures of a digital twin 3D model.
[0274] A third aspect of the present invention provides a digital twin 3D model virtual texture automatic connection system, which includes the above-mentioned computer-readable storage medium.
[0275] Specifically, the principle of the present invention is to achieve this by establishing a complete optimization framework. The framework includes multiple steps such as mesh simplification, UV expansion, region segmentation, feature extraction, region matching, equation establishment and numerical optimization. By cleverly combining and optimizing these core algorithms of computer graphics, a method for automatically generating high-quality seamless texture maps is finally obtained.
[0276] First, in order to reduce the geometric complexity of the 3D model, the inventive method uses a mesh simplification algorithm to geometrically simplify the original model and generate a multi-level model with different levels of detail. This provides a basis for subsequent texture mapping and connection processing. By adjusting the maximum allowable error threshold during simplification, the model detail retention and computational efficiency can be balanced.
[0277] Secondly, the invention method uses a cutting-based UV unfolding algorithm to map the simplified 3D model surface onto a 2D texture plane to obtain the UV coordinate information of each vertex. The key to this step is to find the best cutting edge to minimize deformation and distortion during the unfolding process. By adjusting the cutting parameters, a balance can be achieved between unfolding quality and computational complexity.
[0278] Next, the invention method uses an image segmentation algorithm to divide the texture map into multiple relatively independent sub-regions on the UV plane. This lays the foundation for the subsequent texture connection processing. Specifically, the UV coordinates are pre-classified by a clustering algorithm, and then the Watershed transformation algorithm is used for refinement and segmentation to reasonably control the number and size of the sub-regions.
[0279] After obtaining each sub-region, the invention method uses edge detection and feature extraction algorithms to extract the boundary information and texture features of each sub-region texture map. This information will play a key role in subsequent region matching and texture connection.
[0280] Based on the above analysis, the inventive method establishes an optimization model that includes multiple equations such as color continuity, texture gradient, geometric constraints, boundary alignment, etc. This model describes the various constraints that the texture maps of adjacent sub-regions should meet, and by minimizing the global optimization equation, the overall optimal texture connection effect can be obtained.
[0281] Finally, the invention method uses a numerical optimization algorithm based on gradient descent to solve the above optimization model and obtain the final texture connection matrix. This matrix contains information about the transformation and adjustment of the original texture, which can be used to guide how to re-map the optimized texture map back to the 3D model surface to achieve seamless connection.
[0282] In general, the key innovation of the present invention is to establish a complete mathematical optimization framework, which achieves excellent texture mapping effects in both visual and geometric aspects by scientifically combining multiple core technologies of computer graphics.
[0283] The following is an example of a specific application scenario of the present invention: A well-known cultural relic protection organization is carrying out a digital protection project for ancient stone murals. The goal of this project is to build an accurate digital twin model, which can not only remotely monitor and analyze the original cultural relics, but also provide technical support for virtual display and digital restoration. When building a digital twin model, how to achieve high-fidelity texture mapping is a key issue.
[0284] First, the conservation agency staff used high-precision 3D scanning equipment to conduct a comprehensive three-dimensional digitization of the existing ancient stone murals. The original 3D model obtained after scanning has extremely rich geometric details, with a vertex count of about 20 million and a face count of more than 50 million. This highly complex geometric model is not only difficult to render and interact directly, but also faces huge computational overhead in subsequent texture mapping processing.
[0285] To this end, the staff first used a mesh simplification algorithm to geometrically simplify the original 3D model. Specifically, they used a progressive mesh simplification algorithm to gradually reduce the number of vertices of the model to about 1 million, and the number of faces to less than 3 million. During the simplification process, they set the maximum allowable error threshold to 0.05 to balance the relationship between model detail retention and computational efficiency. Through this step, they obtained a multi-level 3D model with different geometric complexities.
[0286] Next, the staff used the LSCM (Least Squares Conformal Maps) algorithm to UV unfold the simplified multi-layered 3D model. The LSCM algorithm can maximize the preservation of the local shape features of the 3D surface, thereby obtaining a UV mapping with less distortion. In the specific implementation, they set the number of mesh cuts between 80 and 120 to ensure a balance between UV unfolding quality and computational complexity. Through this step, each 3D model vertex is associated with a UV coordinate on the 2D texture plane.
[0287] After obtaining the UV coordinate information, the staff then used the image segmentation algorithm based on the Watershed transform to perform regional segmentation of the texture map on the UV plane. First, they used the adaptive K-Means algorithm to perform preliminary clustering of the UV coordinates, dividing the entire texture area into about 100 sub-regions. Then, the Watershed transform algorithm was used to refine these pre-classified areas, and finally about 200 relatively uniform sub-regions were obtained. The boundary information of these sub-regions will play an important role in the subsequent texture connection.
[0288] In order to further extract the feature information of each sub-region texture map, the staff used the Canny edge detection algorithm and the SIFT feature extraction algorithm. In the Canny algorithm, they set the high and low thresholds to 0.15 and 0.4 respectively, which can well capture the main features of the sub-region boundary. In the SIFT algorithm, due to the high resolution of the texture map, they set 500 key points to obtain a sufficiently rich feature description.
[0289] With the boundary information and texture features of each sub-region mentioned above, the staff then used the RANSAC algorithm to calculate the similarity and position offset between adjacent sub-regions. The RANSAC algorithm first matches the SIFT feature points in adjacent regions, and then estimates the affine transformation relationship between the two regions, including displacement, rotation, and scaling. After 1,000 random sampling iterations, they obtained the transformation parameters between each pair of adjacent sub-regions, reflecting their degree of correlation in texture and position.
[0290] With the results of correlation analysis between sub-regions, the staff began to establish an optimization equation group for texture connection. The equation group includes the following parts:
[0291] 1. Color continuity equation:
[0292]
[0293] in, Indicates The sub-regions are located at the parameter coordinates The equation requires that the colors of adjacent sub-regions at the boundary be continuous.
[0294] 2. Texture gradient equation:
[0295]
[0296] in, Indicates The sub-regions are located at the parameter coordinates The equation requires that the texture gradients of adjacent sub-regions at the boundaries should also be continuous.
[0297] 3. Geometric constraint equations:
[0298]
[0299] in, Indicates The 3D coordinates of the vertices, Represents its UV coordinates, For the The texture mapping function of the sub-region, is its Jacobian matrix. This equation requires that the mapping of texture maps to the surface of 3D models should minimize the geometric error while keeping the local shape unchanged as much as possible.
[0300] 4. Boundary alignment equation:
[0301]
[0302] in, Indicates The sub-region boundary The equation requires that the geometry and positions of adjacent sub-regions at their boundaries be aligned.
[0303] 5. Global Optimization Equation:
[0304]
[0305] This equation is a weighted combination of the first four equations, where each weight coefficient is determined empirically based on actual conditions. By minimizing this global optimization objective, the final texture connection solution can be obtained.
[0306] With the above mathematical model, the staff then used a numerical optimization algorithm based on the gradient descent method to solve it. The specific steps are as follows:
[0307] Initialization: Set the initial texture connection matrix is the unit matrix, the learning rate , maximum number of iterations , convergence threshold .
[0308] Iterative Optimization:
[0309] for to :
[0310] a. Calculate the current loss function value
[0311] b. Calculate the gradient , where each element Computed by numerical differentiation
[0312] c. Update the texture connection matrix:
[0313] d. Check convergence conditions: If , then jump out of the loop
[0314] Output result: Get the final optimized texture connection matrix .
[0315] Finally, the staff obtained the texture connection matrix , transforming and adjusting the texture maps of each sub-area, including color correction, position offset, scaling and rotation, etc. Then, the adjusted texture is pasted back to the simplified 3D model surface through inverse UV mapping to complete the final seamless texture mapping. In order to further optimize certain key areas, they also fine-tuned the local texture.
[0316] Through the above steps, the staff finally obtained a high-fidelity digital twin model of ancient stone murals. The model is not only extremely realistic in geometric details, but also seamless in texture expression, completely eliminating obvious dividing lines and distortion. This provides a solid foundation for subsequent applications such as remote monitoring and virtual display. At the same time, the multi-scale characteristics of the model also provide users with the flexibility to dynamically adjust rendering details.
[0317] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A method for automatically connecting virtual textures of a digital twin 3D model, characterized in that: The following steps are involved: S10. Simplify the mesh of the digital twin 3D model, reduce the number of vertices and faces, and generate a multi-level detailed model; S20, using a UV unfolding algorithm, unfolding the simplified multi-level detail model onto a 2D plane to generate a UV mapping; S30, based on UV mapping, dividing the multi-level detail model into a plurality of sub-regions, each sub-region corresponding to a texture map; S40, using an image segmentation algorithm to perform boundary detection and feature extraction on the texture map of each sub-region; S50, using an image matching algorithm to calculate the similarity and position offset between texture maps of adjacent sub-regions; S60, establishing a set of equations for connecting textures of adjacent regions, including a color continuity equation, a texture gradient equation, a geometric constraint equation, a boundary alignment equation, and a global optimization equation; S70, using a numerical optimization algorithm to solve the texture connection equation group to obtain a texture connection matrix; S80, transforming and adjusting the texture maps of adjacent sub-regions according to the texture connection matrix, remapping the adjusted textures into the UV space of the 3D model, and generating a final seamless texture map; Wherein, the global optimization equation is specifically expressed as: ; In the formula, The color continuity equation is used to require that the color values of adjacent sub-regions should remain continuous at the boundary to avoid obvious color block boundaries; It is the texture gradient equation, which is used to require that the texture gradient of adjacent sub-regions should also remain continuous at the boundary to avoid obvious texture faults; It is a geometric constraint equation, which is used to require that when mapping the texture map back to the 3D model surface, the mapping error should be minimized while keeping the local shape of the texture map unchanged as much as possible; For the boundary alignment equation, it is required that the geometric shapes and positions of adjacent sub-regions on the boundary line should be aligned to avoid obvious boundary misalignment; is the weight coefficient; is the two-dimensional Fourier transform; is the number of sub-regions; For the The sub-regions are located at the parameter coordinates The texture value at The texture connection matrix is specifically expressed as: ; In the formula, is the height of the texture map (number of pixels); is the width of the texture map (number of pixels); is the first Line The elements of the column are recorded as texture connection vectors, and the texture connection vectors are specifically expressed as: ; In the formula, : Color adjustment coefficient, used to adjust pixel color; : Position offset of direction; : Position offset of direction; : scaling factor; : Rotation angle.
2. A method for automatically connecting virtual textures of a digital twin 3D model according to claim 1, characterized in that: The color continuity equation is specifically expressed as: ; In the formula, is the number of sub-regions; is the number of color channels; For the The sub-regions are located at the parameter coordinates The color value at and is the parameterized coordinate, and its range is , is the partial derivative operator, To express the integral over s and t, , is a temporary variable used for summation calculation. In this formula, , .
3. The method for automatically connecting virtual textures of a digital twin 3D model according to claim 2, characterized in that: The texture gradient equation is specifically expressed as: ; In the formula, For the The sub-regions are located at the parameter coordinates The texture value at is the gradient operator, , is a temporary variable used for summation calculation. In this formula, , .
4. The method for automatically connecting virtual textures of a digital twin 3D model according to claim 3, characterized in that: The geometric constraint equation is specifically expressed as: ; In the formula, For the The number of vertices in each sub-region; For the The 3D coordinates of the vertices; For the UV coordinates of vertices; For the The texture mapping function of the sub-region, Indicates Texture mapping function for the UV coordinate area of the vertices; for The Jacobian matrix of ; is the weight coefficient, represents the determinant of the matrix, , is a temporary variable used for summation calculation. In this formula, , and for each ,exist .
5. The method for automatically connecting virtual textures of a digital twin 3D model according to claim 4, characterized in that: The boundary alignment equation is specifically expressed as: ; In the formula, For the The number of sampling points at the sub-region boundaries; For the The sub-region boundary The coordinates of the sampling points; is the curvature smoothing weight, , is a temporary variable used for summation calculation. In this formula, , and for each ,exist .
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, which, when executed, are used to execute the method for automatically connecting virtual textures of a digital twin 3D model according to any one of claims 1 to 5.
7. A digital twin 3D model virtual texture automatic connection system, characterized in that: Contains the computer-readable storage medium of claim 6.
Citation Information
Patent Citations
Method and apparatus for creating surface processing data
CN102792302A
Method for image texture synthesis based on optimum matching
CN103839271A
Multiview large-scale three-dimensional scene reconstruction-based texture mapping method
CN108335352A
Image processing method and device, computer equipment and storage medium
CN111738914A
Fourier transform-based image synthesis using neural networks
CN114331929A
