Data set construction method and equipment for three-dimensional color reconstruction model

Abstract

The invention provides a data set construction method and device for a three-dimensional color reconstruction model, and the method comprises the steps: obtaining a three-dimensional grid model and a two-dimensional texture image library, and randomly selecting two texture images; randomly generating a boundary plane in a three-dimensional space, and calculating a signed distance from a grid vertex to the plane; establishing a UV coordinate system based on the boundary plane and generating a UV coordinate for each vertex; calculating a texture mixing weight according to the signed distance and the mixing bandwidth, sampling colors from the two texture images by using the UV coordinates, and performing color fusion through the mixing weight to obtain a final color of each vertex; and outputting the three-dimensional grid model with the vertex color as training data. According to the method, a complex UV expansion process is not needed, a large amount of three-dimensional texture data with high visual diversity can be automatically and efficiently generated, the generalization ability of a three-dimensional color reconstruction model is effectively improved, and the problem of mode collapse is avoided.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, computer graphics and deep learning, and in particular to a method and apparatus for constructing a dataset for a 3D color reconstruction model. Specifically, it relates to a method for generating texture colors of a 3D model, constructing a dataset, and constructing a dataset for a 3D color reconstruction model to enhance the generalization ability of the 3D color reconstruction model. Background Technology

[0002] 3D color reconstruction aims to recover the color information of a 3D model surface from one or more 2D images, and is a core technology in fields such as digital twins, autonomous driving, and virtual reality. Its performance is highly dependent on the scale and diversity of the training data. However, collecting and labeling real-world 3D color data is costly, cumbersome, and lacks data diversity.

[0003] Existing techniques typically employ data augmentation to expand datasets, such as rotating, scaling, or color jittering input images. However, these methods only transform the data at the two-dimensional level and fail to fundamentally increase the diversity of the appearance of the 3D model itself. Another approach is procedural texture generation, but such methods are often based on finite noise functions or parameters, resulting in monotonous patterns and easily revealing obvious artificial artifacts. This leads to poor generalization ability of the trained model, and when encountering color distributions not present in the training set, the reconstruction effect drops significantly, resulting in "pattern collapse."

[0004] Furthermore, traditional 3D model texture mapping requires a complex UV unwrapping process, which is not only time-consuming but also challenging for complex models. This further limits the efficiency of large-scale, automated generation of textured 3D data. Therefore, there is an urgent need in the field for a dataset construction method that can automatically and efficiently generate large amounts of visually diverse and realistic 3D texture data for 3D color reconstruction models, in order to train robust and highly generalizable 3D color reconstruction models. Summary of the Invention

[0005] The purpose of this invention is to provide a dataset construction method and device for 3D color reconstruction models. This method can automatically and efficiently generate a large amount of 3D texture data with high visual diversity. The method generates textures without UV unwrapping, resulting in high efficiency. Furthermore, the generated 3D textures exhibit high diversity and are not limited by fixed color variation patterns. 3D color reconstruction models trained using this dataset possess high generalization ability, avoiding the pattern collapse problem during color reconstruction.

[0006] According to one objective of the present invention, a method for constructing a dataset for a three-dimensional color reconstruction model is provided, comprising the following steps: Step S1: Obtain the 3D mesh model and the 2D texture image library, load the mesh model and randomly select two texture images from the texture library; Step S2: Randomly generate a boundary plane in three-dimensional space, and calculate the signed distance from each vertex of the mesh model to the plane; Step S3: Based on the boundary plane, establish a UV coordinate system and generate two-dimensional UV coordinates for each vertex; Step S4: Calculate the texture blending weight of each vertex based on the signed distance and the preset blending bandwidth, sample colors from the two texture images based on the UV coordinates, and fuse the sampled colors using the blending weights to obtain the final color of each vertex; Step S5: Output the 3D mesh model after color fusion as training data.

[0007] Furthermore, step S1 specifically includes the following steps: Step S101: Randomly select two images without replacement from the two-dimensional texture image library, and denote them as foreground texture images. Background texture map ; Step S102: ... and Load a digital image in RGB format.

[0008] Furthermore, step S2 specifically includes the following steps: Step S201: Randomly generate a three-dimensional vector And normalize it to obtain the unit normal vector of the boundary plane. ; Step S202: For any vertex in the mesh model Its signed distance to the boundary plane Through formula Obtained through calculation.

[0009] Furthermore, in step S2, a three-dimensional vector is randomly generated. When, the normal vector Set the x-component to zero, then normalize to obtain the unit normal vector. This generates a boundary plane that tends to divide along the YZ plane.

[0010] Furthermore, step S3 specifically includes the following steps: Step S301: Construct the normal vector Orthogonal UV coordinate system basis vectors and ; Step S302: Calculate vertices Projected coordinates in the UV coordinate system ; Step S303: Normalize the projected coordinates of all vertices to obtain the final UV coordinates. .

[0011] Furthermore, step S4 specifically includes the following steps: Step S401: Set the mixed bandwidth b, vertex Mixed weights The calculation method is as follows: like ,but ; like ,but ; like ,but ; Step S402: Based on vertices UV coordinates From the texture map and Foreground texture obtained by mid-sampling and background texture ; Step S403: Vertex final color Through formula Obtained through calculation.

[0012] Furthermore, step S5 specifically includes the following steps: Step S501: Convert each vertex spatial coordinates Its final color value Write together to the file; Step S502: Write the original relationship of the face set F into a file to obtain an OBJ file with vertex colors. The color distribution of the mesh surface is the distribution of the texture map synthesis.

[0013] Furthermore, in step S5, the output 3D mesh model file is in OBJ format containing vertex color attributes, and its vertex data is represented as vxyzrgb.

[0014] According to another objective of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the above-described method for constructing a dataset for a three-dimensional color reconstruction model.

[0015] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for constructing a dataset for a three-dimensional color reconstruction model.

[0016] This invention achieves data augmentation at the 3D spatial level, rather than the traditional 2D image level, by randomly generating boundary planes and combining them with vertex distance calculations. This method eliminates the need for complex and time-consuming UV unwrapping processes, automatically generating an infinite number of texture distributions for the same 3D mesh model. By introducing a hybrid weighting mechanism based on signed distance, the two textures transition naturally in the boundary region, avoiding abrupt seams and resulting in a highly visually realistic output. This method fundamentally solves the problem of insufficient training data diversity. The generated dataset covers a wide range of color and texture distribution patterns, without fixed simple patterns. 3D color reconstruction models trained using this dataset are less prone to overfitting and maintain excellent reconstruction results even when faced with color distributions not present in the training set, significantly improving the model's generalization ability and effectively avoiding the pattern collapse problem in color reconstruction. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the usage of the method in an embodiment of the present invention. Figure 2 This is a flowchart illustrating an embodiment of the present invention. Figure 3 The inference results are for a 3D color reconstruction model trained using the dataset of this embodiment of the invention. Detailed Implementation

[0019] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0021] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified. Furthermore, the terms "installed," "connected," and "linked" should be interpreted broadly; for example, they may refer to a fixed connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0022] Example 1 like Figures 1-3 As shown in the figure, this embodiment provides a method for constructing a dataset for a 3D color reconstruction model, including the following steps: Step 1: Data preparation and random texture sampling: Select the target 3D mesh model M, which consists of a vertex set V and a face set F; at the same time, obtain a texture library T containing N 2D texture images, and randomly select two images from the texture library as the "front and back textures" of the 3D mesh model M.

[0023] Step 2: Random Boundary Plane Generation: In 3D space, a normal vector n is randomly generated to define a boundary plane P in the model space. The signed distance of each vertex in the model along the direction of the normal vector is then calculated. This is used to determine which side of the interface each vertex in the model is on.

[0024] Step 3: Generating UV coordinates for the model: Based on the boundary plane P, for each vertex Calculate the corresponding two-dimensional UV coordinates and normalize the coordinates.

[0025] Step 4: Texture sampling and color blending: For each vertex According to its signed distance Calculate color blending weights And sample the UV coordinates to obtain the foreground texture. and background texture Finally, based on the fusion weight Foreground texture and background texture The colors are weighted and blended to obtain the final texture color value.

[0026] Step 5: Model Output: Write the texture color attributes into the 3D mesh model and output it as an OBJ format mesh file.

[0027] Specifically, step 1 is as follows: Step 1-1: Randomly select two texture images from the texture library T without replacement, and denote them as the foreground texture image. Background texture map .

[0028] Step 1-2: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require and Load the digital image matrix into the RGB color space for subsequent pixel-level sampling.

[0029] Specifically, step 2 is as follows: Step 2-1: Randomly generate a three-dimensional vector The components of this system follow a standard normal distribution. To enhance spatial semantics, we can... Set to zero to generate a plane that tends to be divided along the YZ plane.

[0030] Step 2-2: Normalize the normal vector: , where ||·|| represents the L2 norm of the vector (i.e., the Euclidean length).

[0031] Steps 2-3: For any vertex in the mesh model M Calculate its signed distance to plane P. : , where · represents the dot product operation of vectors. This indicates that the vertex is on the positive side of the plane, and the foreground texture map is used for sampling; This indicates that the background texture map is used during sampling on the negative side.

[0032] Specifically, step 3 is as follows: Step 3-1: Construct a temporary vector orthogonal to the normal vector n. Preferably, if ,but ;otherwise .

[0033] Step 3-2: Calculate the basis vectors in the UV coordinate system: U-axis vector:

[0034] V-axis vector:

[0035] where × represents the cross product operation of vectors.

[0036] Step 3-3: Calculate the projection coordinates of the vertex on the UV plane: . Step 3-4: Normalize the projection coordinates of all vertices to [0, 1] to obtain the final UV coordinates :

[0037]

[0038] where is a very small positive number (such as 1e-8) used to prevent the denominator from being zero.

[0039] Specifically, the said Step 4 is specifically as follows: Step 4-1: Define the mixing bandwidth parameter b (0 < b < 1, usually taking 0.05 - 0.3), and calculate the mixing weight of the vertex : If , then

[0040] If , then

[0041] If , then , smoothly transitioning within the linear region.

[0042] Step 4-2: According to the UV coordinates of the vertex , sample colors from the texture maps and .

[0043] For the texture map (with width and height ):

[0044]

[0045] Color value:

[0046] For texture maps (width is) The height is ):

[0047]

[0048] Color value:

[0049] in, This is a floor function that maps a floating-point number to the largest integer not greater than it, ensuring that the sampled coordinates are discrete pixel indices and preventing out-of-bounds errors. Use This is because the origin of the image coordinate system is usually in the upper left corner, while the texture V-axis points upwards.

[0050] Step 4-3: Calculate vertices final color :

[0051] Right now:

[0052]

[0053]

[0054] This formula is a linear mixing formula for colors.

[0055] Specifically, step 5 is as follows: Step 5-1: Convert each vertex spatial coordinates Its final color value Both are written to the file. Preferably, vertex color format is used, represented in the OBJ file as: , where r, g, b are floating-point numbers normalized to [0,1].

[0056] Step 5-2: Write the original relationship of the face set F into a file to obtain an OBJ file with vertex colors. The color distribution of the mesh surface is the distribution of the texture map synthesis.

[0057] Example 2 Based on Embodiment 1, this embodiment provides a system for implementing the above method, including: one or more processors; and a storage device for storing one or more programs, the programs including instructions that, when executed by the processor, cause the processor to perform the above method.

[0058] Example 3 like Figures 1-3 As shown, this embodiment, based on Embodiment 1, uses a missile model with four tail fins, numbered 002 (002.obj), and a texture library containing images such as zebra stripes, wood grain, and stone textures as examples to demonstrate the implementation process of this method. The process is as follows: Figure 1 As shown.

[0059] 1. Data Preparation (corresponding to step 1): Load the missile model. Randomly select a texture type from the texture library, such as textures / zebra. Then randomly select two textures from the textures / zebra file, for example, a texture image of a single zebra (as...). ) and a textured image of two zebras (as ).

[0060] 2. Generate the boundary plane (corresponding to step 2): Randomly generate normal vectors. For example, it generated The dividing plane is then a plane that is approximately inclined at a 45-degree angle. For any vertex in the missile model M... Calculate its signed distance to plane P. : .

[0061] 3. Generate UV coordinates (corresponding to step 3): ... Given the normal vector, calculate the orthogonal basis vectors of the UV plane. and (For example , ). (Each vertex) Project the coordinates onto the UV plane and normalize them to obtain the UV coordinates.

[0062] 4. Color sampling and blending (corresponding to step 4): Set the blending bandwidth For each vertex According to its Calculate weights In the "front" of the interface ( The vertex of ) It uses entirely black zebra stripes, texture pattern ; behind ( The vertex of ) It uses only white zebra stripes, and the texture pattern is... In the intermediate transition zone ( The vertex of ) The color varies linearly between 0 and 1, and is a smooth blend of two zebra stripes. Then, based on the vertex... UV coordinates From the texture map and Pixel-level sampling is performed to obtain the foreground texture. and background texture Finally, the colors are obtained by weighting the values ​​according to their respective weights.

[0063] 5. Output Result (corresponding to step 5): Save the missile model with vertex colors as 002_color_zebra.obj. This model can be directly displayed in 3D visualization software as a missile with a single zebra on one side, two zebras on the other, and a smooth transition band in the middle of its body. The visualization of the process is shown below. Figure 2 As shown.

[0064] By repeating this process, selecting different textures and missile models each time, a missile shape dataset containing random surface color distributions can be obtained. The 3D color reconstruction model trained using this dataset exhibits excellent generalization performance. The model's inference results for data with different color distribution patterns outside the training set are as follows: Figure 3 As shown, the first column is the input image to be reconstructed, the second column is the result of rendering the reconstructed 3D shape, and the third column is the peak signal-to-noise ratio (PSNR) between the input image and the rendering result.

[0065] This invention is a fully automated process that requires no manual intervention or texture mapping UV unwrapping, and can generate massive amounts of data in batches, greatly improving the efficiency of dataset construction.

[0066] This invention, by randomly selecting textures and boundary planes, can theoretically generate an infinite number of different color distributions for the same type of 3D model, fundamentally solving the problem of insufficient diversity in training data.

[0067] The "smooth blending band" mechanism introduced in the method of this invention allows the two textures to transition naturally at the boundary, avoiding harsh seams. The resulting product looks more natural and conforms to the laws of lighting and material continuity in the physical world.

[0068] The dataset generated using the method of this invention covers an extremely wide range of color and texture distributions and has no fixed, simple patterns. Deep learning models trained using it are less likely to overfit to a specific color distribution pattern, thus possessing stronger generalization ability and being able to better reconstruct the three-dimensional color distribution of an object from an image with arbitrary color distribution.

[0069] The method of this invention does not depend on a specific three-dimensional model structure, can be widely applied to mesh models of various shapes, and can be used in conjunction with any two-dimensional texture image library (such as ImageNet), making it highly versatile.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a dataset for a three-dimensional color reconstruction model, characterized in that, Includes the following steps: Step S1: Obtain the 3D mesh model and the 2D texture image library, load the mesh model and randomly select two texture images from the texture library; Step S2: Randomly generate a boundary plane in three-dimensional space, and calculate the signed distance from each vertex of the mesh model to the plane; Step S3: Based on the boundary plane, establish a UV coordinate system and generate two-dimensional UV coordinates for each vertex; Step S4: Calculate the texture blending weight of each vertex based on the signed distance and the preset blending bandwidth, sample colors from the two texture images based on the UV coordinates, and fuse the sampled colors using the blending weights to obtain the final color of each vertex; Step S5: Output the 3D mesh model after color fusion as training data.

2. The dataset construction method for a three-dimensional color reconstruction model according to claim 1, characterized in that, Step S1 specifically includes the following steps: Step S101: Randomly select two images without replacement from the two-dimensional texture image library, and denote them as foreground texture images. Background texture map ; Step S102: ... and Load a digital image in RGB format.

3. The dataset construction method for a three-dimensional color reconstruction model according to claim 1, characterized in that, Step S2 specifically includes the following steps: Step S201: Randomly generate a three-dimensional vector And normalize it to obtain the unit normal vector of the boundary plane. ; Step S202: For any vertex in the mesh model Its signed distance to the boundary plane Through formula Obtained through calculation.

4. The dataset construction method for a three-dimensional color reconstruction model according to claim 3, characterized in that, In step S2, a three-dimensional vector is randomly generated. When, the normal vector Set the x-component to zero, then normalize to obtain the unit normal vector. This generates a boundary plane that tends to divide along the YZ plane.

5. The dataset construction method for a three-dimensional color reconstruction model according to claim 1, characterized in that, Step S3 specifically includes the following steps: Step S301: Construct the normal vector Orthogonal UV coordinate system basis vectors and ; Step S302: Calculate vertices Projected coordinates in the UV coordinate system ; Step S303: Normalize the projected coordinates of all vertices to obtain the final UV coordinates. .

6. The dataset construction method for a three-dimensional color reconstruction model according to claim 1, characterized in that, Step S4 specifically includes the following steps: Step S401: Set the mixed bandwidth b, vertex Mixed weights The calculation method is as follows: like ,but ; like ,but ; like ,but ; Step S402: Based on vertices UV coordinates From the texture map and Foreground texture obtained by mid-sampling and background texture ; Step S403: Vertex final color Through formula Obtained through calculation.

7. The dataset construction method for a three-dimensional color reconstruction model according to claim 1, characterized in that, Step S5 specifically includes the following steps: Step S501: Convert each vertex spatial coordinates Its final color value Write together to the file; Step S502: Write the original relationship of the face set F into a file to obtain an OBJ file with vertex colors. The color distribution of the mesh surface is the distribution of the texture map synthesis.

8. The dataset construction method for a three-dimensional color reconstruction model according to claim 1, characterized in that, In step S5, the output 3D mesh model file is in OBJ format, which includes vertex color attributes, and its vertex data is represented as vxyzrgb.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the dataset construction method for a three-dimensional color reconstruction model as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the dataset construction method for a three-dimensional color reconstruction model as described in any one of claims 1 to 8.

Images