Three-dimensional model synthesis method based on multiple photos

Abstract

The invention discloses a three-dimensional model synthesis method based on multiple photos. The method comprises the following steps: S1, respectively exporting three-dimensional models generated by the photos as a three-dimensional model A and a three-dimensional model B; s2, performing one-to-one mapping on the feature points on the three-dimensional model A and the corresponding feature points on the three-dimensional model B, and performing connection; s3, cutting and deleting the overlapped part of the three-dimensional model A and the three-dimensional model B, and exporting a three-dimensional model C and a three-dimensional model D; s4, splicing the three-dimensional model A and the three-dimensional model B to form a complete three-dimensional model E, and redistributing uv for the three-dimensional model E; and S5, baking the color maps of the three-dimensional model C and the three-dimensional model D onto the three-dimensional model E to form a three-dimensional model. According to the method, the problems of low model integrity and poor color information accuracy when a special object is processed by an existing photo modeling technology are solved, and the quality and the reality sense of the three-dimensional model are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional models, and in particular to a three-dimensional model synthesis method based on multiple photos. Background Art

[0002] In today's digital age, 3D digital modeling technology is widely used in numerous fields, such as digital collection creation, cultural heritage preservation and display, industrial design, and film, television, and game creation. In these applications, accurate 3D modeling of real-world objects is a key requirement. For example, in the field of digital collections, creating virtual objects with high collectible and ornamental value requires accurate reproduction of the details and appearance of the real objects. In cultural heritage preservation, 3D modeling enables digital archiving of artifacts for better preservation and research, and can also be used in virtual exhibitions, allowing more people to appreciate the beauty of cultural relics. However, existing 3D modeling technology has many limitations, making it difficult to fully meet the needs of these scenarios.

[0003] Traditional 3D modeling methods mainly include scanning modeling using 3D scanners and manual modeling based on polygonal modeling technology. 3D scanners acquire three-dimensional data of an object by emitting and receiving signals such as lasers, and then generate a scanned model. The advantage of this method is that it can quickly obtain the approximate shape of an object and is widely used in some scenarios where precision is not too high. For example, in the preliminary design stage of an industrial product, a 3D scanner can be used to quickly obtain three-dimensional data of the product prototype to provide a reference for subsequent design. However, it also has obvious disadvantages. To ensure scanning accuracy, the object needs to be sprayed with a developer during scanning, which is not only cumbersome to operate but also causes a certain degree of damage to the object. Moreover, the generated model often lacks color information of the actual object and cannot truly present the appearance of the object. It is difficult to meet the needs in application scenarios where high requirements for model realism are required.

[0004] Polygonal modeling technology is a method of constructing models by manually creating polygonal meshes by modelers. The advantage of this method is that modelers can finely shape the model based on their own creativity and understanding of the object, and it is widely used in the fields of film and television character modeling, game scene production, etc. For example, when making virtual creatures for movie special effects, modelers can use polygonal modeling technology to accurately sculpt the muscle texture, skin details, etc. of the creatures. However, the disadvantages of this method are also prominent. It requires extremely high professional skills from the modeler, and the modeling process is very time-consuming and labor-intensive, requiring a lot of manpower costs. Moreover, for the modeling of complex objects, even experienced modelers will need months or even years, which is extremely inefficient and difficult to meet the needs of large-scale rapid modeling.

[0005] Photo-based modeling technology is gaining popularity and becoming a research hotspot in the 3D modeling field. This technology utilizes a large number of photographs and the color information contained in them to generate 3D models within photo-based modeling software. Compared to traditional 3D scanner modeling and manual polygonal modeling, it offers significant advantages. Firstly, it eliminates the need to spray developer onto the object, making it more convenient and significantly reducing the risk of damage to the object itself. Secondly, it can extract color information that closely resembles the actual object, making the generated model appear more realistic. This technology has great potential for application in digital collections and cultural relic displays.

[0006] However, existing photo-based modeling technologies also have many problems. Photo modeling software shows obvious limitations when dealing with translucent objects, special objects with similar or rare feature points. Since it is difficult for the software to accurately identify the features of these special objects, the integrity of the generated model and the accuracy of the color information are not high. For example, when modeling transparent glass products or objects with similar surface textures, the model is prone to problems such as missing parts of the structure and color deviation. In addition, in order to improve the accuracy of modeling, it is often necessary to take a large number of photos, and repeated shooting and repeated recognition are required, which not only wastes time and manpower costs, but also cannot ensure that the ideal modeling effect can be obtained in the end even after multiple attempts, and cannot meet the needs of efficient and accurate modeling in practical applications. Summary of the Invention

[0007] Based on the above, this application discloses a method for synthesizing a 3D model based on multiple photos, which solves the above-mentioned technical problems, including:

[0008] S1. Obtain several photos of the model, and export the three-dimensional models generated from the photos into three-dimensional model A and three-dimensional model B, respectively, where the three-dimensional model A and the three-dimensional model B are complementary;

[0009] S2. Aligning the three-dimensional model A and the three-dimensional model B by obtaining at least three similar feature points, mapping the feature points on the three-dimensional model A with the corresponding feature points on the three-dimensional model B one by one and connecting them;

[0010] S3, cutting and deleting the overlapping parts of the 3D model A and the 3D model B, exporting the 3D model A and the 3D model B and naming them as the 3D model C and the 3D model D;

[0011] S4, combining the 3D model A and the 3D model B to form a complete 3D model E, and reallocating uv to the 3D model E;

[0012] S5. Bake the color maps of the 3D model C and the 3D model D onto the 3D model E, and repair the color map of the joints of the 3D model E to form a 3D model.

[0013] Preferably, the three-dimensional model A and the three-dimensional model B are three-dimensional models generated by photos with a surface integrity greater than or equal to 60%, respectively exported as three-dimensional model A and three-dimensional model B, and the three-dimensional model B fills in the parts of the model surface that are missing, blurred or not accurately constructed in the three-dimensional model A.

[0014] Preferably, the acquisition of similar feature points of the three-dimensional model A and the three-dimensional model B is specifically as follows:

[0015] Mesh the 3D model A and 3D model B separately, discretize their surfaces into multiple small triangular patches, and obtain the similarity between the centroids of the two triangular patches. The formula is: The coordinates of the center of mass of the i-th triangle patch in the three-dimensional model A are The coordinates of the center of mass of the j-th triangle patch in the three-dimensional model B are α and β are the weight coefficients of the center of gravity and normal vector respectively, Δn ij is the difference in the cosine of the angle between the normal vectors of two triangle faces, and the formula is: and are the angles between the normal vectors of the i-th triangle face in the 3D model A and the j-th triangle face in the 3D model B and the reference direction; set the similarity threshold T, when S ij When >T, it is determined to be a similar feature point.

[0016] Preferably, the feature points on the three-dimensional model A are mapped one by one with the corresponding feature points on the three-dimensional model B and connected. By selecting the three-dimensional model A as a reference, the three-dimensional model B is proportionally enlarged and reduced, the spatial coordinate position of the three-dimensional model B is adjusted, and the error between the three-dimensional model and the three-dimensional model B in the three-dimensional coordinates is controlled within the range of ±0.01mm, and the three-dimensional model A and the three-dimensional model B are connected.

[0017] Preferably, the cutting and deleting of the overlapping portion of the three-dimensional model A and the three-dimensional model B in S3 is specifically as follows:

[0018] S3.1. Scan 3D model A and 3D model B point by point in 3D space through spatial scanning, and mark the point sets in the overlapping area;

[0019] S3.2. Constructing a boundary contour of the overlapping region based on the marked overlapping point set, and determining the extent of the overlapping region using a contour tracing algorithm;

[0020] S3.3. Based on the boundary contour, use the difference operation in the Boolean operation to subtract the overlapping parts from 3D model A and 3D model B respectively, and perform topological optimization on the cut models to eliminate redundant vertices and edges generated by cutting.

[0021] Preferably, in S3.1, the three-dimensional model A and the three-dimensional model B are scanned point by point in the three-dimensional space by spatial scanning, and the point sets in the overlapping areas are marked, specifically:

[0022] Divide the three-dimensional space into uniform grid cells. The size of each grid cell is d×d×d. Obtain each vertex P(x, y, z) in the three-dimensional model A and the three-dimensional model B, and calculate the grid cell coordinates (i, j, k) in the three-dimensional model A and the three-dimensional model B. The formula is: Count the number of vertices from 3D model A and 3D model B in each grid cell, denoted as n A and n A , calculate the overlap R, the formula is: When R is greater than the set overlap threshold T R When , there is an overlapping area in the grid unit, and all vertices in the grid unit are marked as the point set of the overlapping area.

[0023] Preferably, the step S3.2 constructs a boundary contour of the overlapping area based on the marked overlapping point set, and determines the range of the overlapping area using a contour tracking algorithm, specifically:

[0024] Choose any point from the marked coincident point set as the starting point P0. For any point P in the coincident point set r(x,y,z) , calculate the adjacent point P adj The connection vector The formula is: Get point P r and its two adjacent points P adj1 、P adj2 , calculate the direction change metric D, the formula is: D = | cosθ 12 |, where θ 12 for and angle, set the direction change threshold T D , when D>T D When r At the edge of the contour, starting from the starting point P0, traverse the adjacent points to form a preliminary contour point sequence and determine the range of the overlapping area.

[0025] Preferably, in S4, the three-dimensional model A and the three-dimensional model B are spliced ​​together to form a complete three-dimensional model E. By adjusting the connection between the three-dimensional model A and the three-dimensional model B, observing the main outline silhouette of the actual object and the structural connection between the three-dimensional model A and the three-dimensional model B, and making fine adjustments by moving the rotation and scaling tools, the three-dimensional model A and the three-dimensional model B are connected and merged to form a new and complete three-dimensional model E.

[0026] Preferably, the uv of the three-dimensional model E is redistributed, and a preliminary uv is automatically and quickly unfolded for the three-dimensional model E through 3D modeling software. The stretching and distortion of the uv block are modified based on the preliminary uv, and the uv is rearranged so that each uv block is evenly and reasonably distributed.

[0027] Preferably, the S5 repairs the color map of the splicing seam of the three-dimensional model E to form a three-dimensional model, and identifies the color and texture mutation areas by performing edge detection on the splicing seam; for color differences, extracts color samples from both sides of the seam through color gradient fusion technology, creates a color transition zone at the seam, and makes the colors naturally connected; for texture discontinuity, uses texture mapping matching technology to analyze the texture features on both sides of the seam, finds similar texture areas, maps the texture on one side to the other side and adjusts the deformation to make its texture continuous, and through fine-tuning of details, manually draws and modifies the unnatural color and texture transitions to complete the color map repair to form a three-dimensional model.

[0028] Compared with the prior art, the technical solution of this application has the following technical effects:

[0029] This invention effectively solves the problem of incomplete models in photo-based modeling by deriving the 3D model generated from a photo into complementary 3D models A and B. Through the design of complementary models, 3D model B can fill in the missing, blurred, or inaccurately constructed parts of 3D model A. After a series of operations such as alignment, cutting overlapping parts, and splicing, the resulting 3D model E combines the advantages of both models, significantly improving their completeness. Compared to models generated by traditional methods, the models obtained by this invention can present a more comprehensive range of object features, from subtle textures to complex structures, providing a solid foundation for subsequent research, demonstration, and utilization.

[0030] The present invention improves modeling efficiency. After obtaining the model photo, the complementary models A and B are automatically derived and the two models are quickly aligned using similar feature points. This process saves time compared to manual adjustment. The steps of cutting overlapping parts and splicing models are also streamlined, reducing repetitive labor in the modeling process. The operations of re-UV division, baking color maps, and repairing seams avoid unnecessary trial and error.

[0031] This invention excels in enhancing the realism of models. In color processing, the color maps of 3D models C and D are baked onto 3D model E, and color differences at the joints are repaired, making the model's colors more closely resemble the actual object. Using color gradient fusion technology, color samples are extracted from both sides of the joint to create a transition zone, allowing the colors to flow naturally and making the color transition of the ceramic model smoother and more realistic. In texture processing, texture mapping and matching technology is used to analyze the texture features on both sides of the joint, identifying similar areas for mapping and deformation adjustment to ensure texture continuity. The processed model, in both visual effects and texture expression, is highly similar to the real object in display and application, providing users with a more realistic experience and enhancing the model's expressiveness and value.

[0032] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application so that it can be implemented in accordance with the contents of the specification, and to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following is a detailed description of the preferred embodiment of the present application in conjunction with the accompanying drawings.

[0033] Based on the detailed description of the specific embodiments of the present application in conjunction with the accompanying drawings below, those skilled in the art will become more aware of the above and other objects, advantages and features of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without inventive work. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0035] Figure 1 This is a structural diagram of a three-dimensional model synthesis method based on multiple photos of the present invention;

[0036] Figure 2 It is a structural model diagram of three-dimensional model A and three-dimensional model B;

[0037] Figure 3 It is a combined structural model diagram of three-dimensional model A and three-dimensional model B;

[0038] Figure 4 It is a structural model diagram of the three-dimensional model C and the three-dimensional model D;

[0039] Figure 5 This is the structural model diagram of the three-dimensional model E. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. In the following description, specific details such as specific configurations and components are provided only to help fully understand the embodiments of the present application. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. In addition, for clarity and brevity, the description of known functions and structures has been omitted in the embodiments.

[0041] It should be understood that references throughout this specification to "one embodiment" or "this embodiment" mean that a particular feature, structure, or characteristic associated with the embodiment is included in at least one embodiment of the present application. Therefore, the appearance of "one embodiment" or "this embodiment" throughout this specification does not necessarily refer to the same embodiment. Furthermore, these particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0042] In addition, the present application may repeat reference numerals and / or letters in different examples. This repetition is for the purpose of simplicity and clarity and does not in itself indicate the relationship between the various embodiments and / or settings discussed.

[0043] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" in this article describes another type of association object relationship, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0044] The term "at least one" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, at least one of A and B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0045] It should also be noted that, in this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include," "comprises," or any other variations thereof are intended to cover non-exclusive inclusion.

[0046] Example 1

[0047] This embodiment mainly describes a method for synthesizing a three-dimensional model based on multiple photos. Figure 1 Shown, including:

[0048] S1. Obtain several photos of the model, and export the three-dimensional models generated from the photos into three-dimensional model A and three-dimensional model B, respectively, where the three-dimensional model A and the three-dimensional model B are complementary;

[0049] S2. Aligning the three-dimensional model A and the three-dimensional model B by obtaining at least three similar feature points, mapping the feature points on the three-dimensional model A with the corresponding feature points on the three-dimensional model B one by one and connecting them;

[0050] S3, cutting and deleting the overlapping parts of the 3D model A and the 3D model B, exporting the 3D model A and the 3D model B and naming them as the 3D model C and the 3D model D;

[0051] S4, combining the 3D model A and the 3D model B to form a complete 3D model E, and reallocating uv to the 3D model E;

[0052] S5. Bake the color maps of the 3D model C and the 3D model D onto the 3D model E, and repair the color map of the joints of the 3D model E to form a 3D model.

[0053] Furthermore, the three-dimensional model A and the three-dimensional model B are three-dimensional models generated by photos with a surface integrity greater than or equal to 60%, respectively exported as three-dimensional model A and three-dimensional model B, and the three-dimensional model B fills in the parts of the model surface that are missing, blurred or not accurately constructed in the three-dimensional model A.

[0054] Furthermore, the acquisition of similar feature points of the three-dimensional model A and the three-dimensional model B is specifically as follows:

[0055] Mesh the 3D model A and 3D model B separately, discretize their surfaces into multiple small triangular patches, and obtain the similarity between the centroids of the two triangular patches. The formula is: The coordinates of the center of mass of the i-th triangle patch in the three-dimensional model A are The coordinates of the center of mass of the j-th triangle patch in the three-dimensional model B are α and β are the weight coefficients of the center of gravity and normal vector respectively, ΔB ij is the difference in the cosine of the angle between the normal vectors of two triangle faces, and the formula is: and are the angles between the normal vectors of the i-th triangle face in the 3D model A and the j-th triangle face in the 3D model B and the reference direction; set the similarity threshold T, when S ij When >T, it is determined to be a similar feature point.

[0056] Furthermore, the feature points on the three-dimensional model A are mapped one by one with the corresponding feature points on the three-dimensional model B and connected. By selecting the three-dimensional model A as a reference, the three-dimensional model B is proportionally enlarged and reduced, the spatial coordinate position of the three-dimensional model B is adjusted, and the error between the three-dimensional model and the three-dimensional model B in the three-dimensional coordinates is controlled within the range of ±0.01mm, and the three-dimensional model A and the three-dimensional model B are connected.

[0057] Furthermore, the overlapping portion of the three-dimensional model A and the three-dimensional model B is cut and deleted in S3, specifically:

[0058] S3.1. Scan 3D model A and 3D model B point by point in 3D space through spatial scanning, and mark the point sets in the overlapping area;

[0059] S3.2. Constructing a boundary contour of the overlapping region based on the marked overlapping point set, and determining the extent of the overlapping region using a contour tracing algorithm;

[0060] S3.3. Based on the boundary contour, use the difference operation in the Boolean operation to subtract the overlapping parts from 3D model A and 3D model B respectively, and perform topological optimization on the cut models to eliminate redundant vertices and edges generated by cutting.

[0061] Furthermore, in S3.1, the three-dimensional model A and the three-dimensional model B are scanned point by point in the three-dimensional space by spatial scanning, and the point sets in the overlapping area are marked, specifically:

[0062] Divide the three-dimensional space into uniform grid cells. The size of each grid cell is d×d×d. Obtain each vertex P (x, y, z) in the three-dimensional model A and the three-dimensional model B, and calculate the grid cell coordinates (i, j, k) in the three-dimensional model A and the three-dimensional model B. The formula is: Count the number of vertices from 3D model A and 3D model B in each grid cell, denoted as n A and n A , calculate the overlap R, the formula is: When R is greater than the set overlap threshold T R When , there is an overlapping area in the grid unit, and all vertices in the grid unit are marked as the point set of the overlapping area.

[0063] Furthermore, the S3.2 constructs a boundary contour of the overlapping area based on the marked overlapping point set, and uses a contour tracing algorithm to determine the range of the overlapping area, specifically:

[0064] Choose any point from the marked coincident point set as the starting point P0. For any point P in the coincident point set r(x,y,z) , calculate the adjacent point P adjThe connection vector The formula is: Get point P r and its two adjacent points P adj1 、P adj2 , calculate the direction change metric D, the formula is: D = | cosθ 12 |, where θ 12 for and angle, set the direction change threshold T D , when D>T D When r At the edge of the contour, starting from the starting point P0, traverse the adjacent points to form a preliminary contour point sequence and determine the range of the overlapping area.

[0065] Furthermore, in S4, the three-dimensional model A and the three-dimensional model B are spliced ​​together to form a complete three-dimensional model E. By adjusting the connection between the three-dimensional model A and the three-dimensional model B, observing the main outline silhouette of the actual object and the structural connection between the three-dimensional model A and the three-dimensional model B, and making fine adjustments by moving, rotating and scaling tools, the three-dimensional model A and the three-dimensional model B are connected and merged to form a new and complete three-dimensional model E.

[0066] Furthermore, the uv of the three-dimensional model E is redistributed, and a preliminary uv is automatically and quickly unfolded for the three-dimensional model E through the 3D modeling software. The stretching and distortion of the uv block are modified based on the preliminary uv, and the uv is rearranged so that each uv block is evenly and reasonably distributed.

[0067] Furthermore, the S5 repairs the color map of the splicing seam of the three-dimensional model E to form a three-dimensional model, and identifies the color and texture mutation areas by performing edge detection on the splicing seam; for color differences, extracts color samples from both sides of the seam through color gradient fusion technology, creates a color transition zone at the seam, and makes the colors naturally connected; for texture discontinuity, uses texture mapping matching technology to analyze the texture features on both sides of the seam, finds similar texture areas, maps the texture on one side to the other side and adjusts the deformation to make the texture continuous, and through fine-tuning of details, manually paints and modifies the unnatural color and texture transitions to complete the color map repair to form a three-dimensional model.

[0068] This embodiment describes in detail that the present application can effectively solve the problems of incomplete models and difficult color information extraction in photo modeling. By deriving complementary models, aligning feature points, cutting and splicing, and processing color maps, the model integrity and realism are greatly improved, the modeling process is simplified, the modeling efficiency is improved, the operation difficulty and labor costs are reduced, and in the fields of digital collections and cultural relics display, high-precision and high-fidelity three-dimensional models can be quickly constructed, which has great application value.

[0069] Based on Example 1, this example describes in detail the optimization technical solution for obtaining similar feature points of the present application, specifically:

[0070] Mesh the 3D model A and 3D model B separately, discretize their surfaces into multiple small triangular patches, and obtain the similarity between the centroids of the two triangular patches. The formula is: The coordinates of the center of mass of the i-th triangle patch in the three-dimensional model A are The coordinates of the center of mass of the j-th triangle patch in the three-dimensional model B are α and β are the weight coefficients of the center of gravity and normal vector respectively, Δn ij is the difference in the cosine of the angle between the normal vectors of two triangle faces, and the formula is: and are the angles between the normal vectors of the i-th triangle face in the 3D model A and the j-th triangle face in the 3D model B and the reference direction; set the similarity threshold T, when S ij When >T, it is determined to be a similar feature point;

[0071] For all satisfying S ij >T point pairs, according to similarity S ij Sort from largest to smallest and select the first n point pairs as the best similar feature points. The value of n is adjusted according to the complexity of the 3D model and actual needs. The best similar feature points of 3D model A and 3D model B are obtained through the above formula, providing a basis for subsequent alignment and connection operations.

[0072] This embodiment describes in detail how similar areas of three-dimensional models A and B can be accurately located by calculating similar feature points, and the best similar feature points are determined by arrangement, thereby improving the accuracy of model alignment, making subsequent connections tighter, and ensuring the accuracy of model splicing. The synthesized three-dimensional model is structurally closer to the real object, reducing model defects, improving model quality, and providing more reliable technical support for reverse modeling.

[0073] Based on Example 1, this example describes the implementation process of this application in detail, specifically:

[0074] From the current state of technology, existing realistic modeling methods mainly include 3D scanner scanning and photo modeling software. While 3D scanner scanning can capture the shape of an object, it requires spraying a developer and the model lacks color information. While photo modeling software can extract color information, it suffers from issues with model integrity and color accuracy when dealing with special objects. The 3D model synthesis method based on multiple photos proposed in this application aims to address these technical challenges and improve modeling quality and efficiency.

[0075] Experiments were conducted on objects with varying materials, textures, and structural characteristics. Examples included a translucent glass artifact, a unique stone with a complex surface texture and similar feature points, and a cultural relic sculpture. The glass artifact's high light transmittance and smooth surface made it difficult to accurately identify its feature points; the unique stone had a unique texture but many similar features; and the cultural relic sculpture had a complex structure and rich details, requiring extremely high standards for color and texture reproduction. Modeling experiments were conducted for each experimental object using both the proposed method and existing technologies.

[0076] The experiment used a professional digital camera to capture photos, ensuring a resolution of 5000×3750 pixels to ensure clear image details. Mainstream photo modeling software, such as RealityCapture, was used as the basic modeling tool, combined with 3D modeling and processing software such as Maya and ZBrush to assist in model post-processing.

[0077] Existing technology uses existing photo-based modeling software to directly process photographs. For example, 300 photos of a glass craft were taken from different angles and imported into the modeling software. However, due to the translucency and smooth surface of glass, the software struggled to accurately identify feature points, resulting in a model with numerous cracks and holes. The model's integrity was only 35.27%, with significant color deviations, including an average RGB value deviation of 25.43. Texture details were severely lost, and the glass craft's texture and form could not be accurately captured.

[0078] When modeling the unusual stone, 300 photos were also taken. Due to the similar surface textures of the unusual stones, the software's recognition was confused, resulting in a distorted model structure and a completeness of 42.56%. Regarding color, due to the inability to accurately match the color areas corresponding to the textures, color reproduction was low. In the HSV color space, the mean hue deviation was 18.32, the mean saturation deviation was 11.54, and the mean lightness deviation was 9.27.

[0079] In an experiment modeling a cultural relic sculpture, 400 photographs were taken. While the model generated by existing technology roughly captures the sculpture's outline, details are blurred, with a completeness of 55.18%. Color information is severely lost. In the LAB color space, the mean deviation for L* (lightness) is 12.65, the mean deviation for a* (red-green) is 8.42, and the mean deviation for b* (yellow-blue) is 7.18. These results fail to meet the requirements for high-precision digital restoration of cultural relics.

[0080] The experimental process of this application method is as follows: Figure 2As shown, the photographs were initially screened to eliminate blurry, overexposed, or poorly angled images. For example, 256 photos of glass artifacts were retained after this screening. Modeling software was used to generate 3D models from these photos. Models with surface integrity greater than or equal to 60% were selected and exported as 3D Model A and its complementary 3D Model B. Testing revealed that 3D Model A had a completeness of 68.34% and 3D Model B had a completeness of 66.72%. The complementary areas of these two models covered most of the missing structures of the glass artifact.

[0081] For the strange stones, 238 photos were retained after screening, and the generated 3D model A had a completeness of 63.57%, and the 3D model B had a completeness of 61.89%. The complementary parts can effectively fill the missing texture details of each other; 320 photos of cultural relics and sculptures were retained after screening, and the 3D model A had a completeness of 70.21%, and the 3D model B had a completeness of 67.85%, laying the foundation for subsequent precise modeling.

[0082] The algorithm automatically searches for no less than three similar feature points on the three-dimensional model A and the three-dimensional model B. In the glass handicraft model, five similar feature points are found through calculation. These feature points are used to map the feature points on the three-dimensional model A to the corresponding feature points on the three-dimensional model B one by one. Figure 3 As shown in the figure, 3D model A is used as a reference, and 3D model B is scaled up or down proportionally to adjust its spatial coordinate position. After multiple fine-tuning, the error in the 3D coordinates of the two models is controlled within ±0.01mm, ensuring that the initial model alignment accuracy reaches 99.87%.

[0083] During the alignment process for the stone model, four similar feature points were found. Adjustments were made to keep the model coordinate error within the specified range, resulting in an alignment accuracy of 99.75%. Six similar feature points were found for the cultural relic sculpture model, and the alignment accuracy reached 99.92%, ensuring the accuracy of subsequent splicing.

[0084] A spatial scanning algorithm was used to scan the aligned 3D models A and B point by point, dividing the 3D space into grid cells of 0.5×0.5×0.5 mm. The coordinates of each vertex in the grid cell were calculated. The number of vertices from 3D models A and B within each grid cell of the glass artifact model was counted, and the degree of overlap was calculated. A threshold for the overlap was set at 0.5. When the overlap exceeded the threshold, the vertices within that grid cell were marked as overlapping points. The calculated overlap percentage of the glass artifact model was 12.36%.

[0085] Based on the set of marked points, the boundary outline of the overlapping area was constructed, and the range was determined using a contour tracing algorithm. Then, using the Boolean difference operation, the overlapping areas were subtracted from the two models. The resulting models were then topologically optimized. The overlapping areas of the stone model accounted for 15.28%, and the overlapping areas of the cultural relic sculpture model accounted for 10.45%. Both models achieved precise cutting and topological optimization, eliminating redundant structures.

[0086] In the modeling software, the cut 3D model A and 3D model B are combined, as shown in the following example: Figure 4 As shown in the figure, using a glass craft as an example, the silhouette of the actual object and the connection between the model structure were observed, and fine-tuned using the move, rotate, and scale tools. After more than 20 fine-tuning steps, the two models were perfectly connected and merged, generating a complete 3D model E. Measurements showed that the surface flatness error of Model E was within 0.05mm, and the deviation from the actual object shape was within 0.12mm.

[0087] When the stone model was assembled, after 15 adjustments, the surface flatness error of Model E was 0.06mm, and the shape deviation was 0.15mm; after 30 fine-tunings, the surface flatness error of the cultural relic sculpture model was 0.03mm, and the shape deviation was 0.08mm, highly restoring the details and shape of the cultural relic sculpture.

[0088] like Figure 5 As shown, the 3D modeling software is used to automatically unfold the preliminary UV of the generated three-dimensional model E; taking the glass handicraft model as an example, after the preliminary UV unfolding, the stretching and distortion of the UV block are manually modified; through adjustment, the UV block stretching ratio deviation is controlled within 5%, the distortion angle deviation is controlled within 3°, and the UV is rearranged so that each UV block is evenly and reasonably distributed, reducing deformation during texture baking.

[0089] After UV adjustment, the stretching ratio deviation of the stone model was 4.5%, and the distortion angle deviation was 2.8°. The stretching ratio deviation of the cultural relic sculpture model was 3.8%, and the distortion angle deviation was 2.5°, laying the foundation for high-quality texture mapping.

[0090] The color maps of the cut and renamed 3D models C and D were baked onto 3D model E. Appropriate parameters were set in the baking software, such as a sampling rate of 2048, a lighting intensity of 1200 lux, and a baking time of 15 minutes. After baking, the joints of the glassware model were inspected, revealing color deviation and texture discontinuities.

[0091] For color inconsistencies, use the Brush tool to pick color samples on both sides of the seam and paint along the seam with a soft brushstroke at an opacity of 0.3, keeping the color deviation within 3 in RGB value. For texture discontinuity, use the Clone tool, using the complete texture near the seam as the source, to cover the seam, ensuring a natural texture transition and a texture continuity error of less than 0.1 pixel.

[0092] In the HSV color space, the color deviation of the stone model is controlled within 2 for hue, 1.5 for saturation, and 1 for lightness. The texture continuity error is less than 0.12 pixels. In the LAB color space, the L* deviation of the cultural relic sculpture model is controlled within 2, the a* deviation is controlled within 1.2, and the b* deviation is controlled within 1. The texture continuity error is less than 0.08 pixels, greatly enhancing the realism of the model.

[0093] The experimental results are shown in the following table:

[0094]

[0095] From the comparison of experimental results, the method of the present application is significantly superior to the existing technology in terms of model completeness, color restoration and texture continuity. In terms of model completeness, the model processed by the method of the present application is close to the real object, while the model generated by the existing technology has a large number of missing and incomplete parts. In terms of color restoration, the method of the present application controls the color deviation within a very small range and highly restores the true color of the object, while the existing technology has a larger deviation. In terms of texture continuity, the method of the present application ensures the natural connection of textures, while the existing technology has problems with texture loss and distortion.

[0096] This embodiment fully demonstrates the superiority of this application's method for synthesizing 3D models based on multiple photos through modeling experiments on various object types. This method effectively addresses the issues of low model integrity and poor color accuracy associated with existing photo-based modeling techniques when processing specialized objects, significantly improving the quality and realism of 3D models. In practical applications, whether in the production of digital collections, the digital preservation of cultural relics, or other fields requiring high-precision 3D modeling, this method possesses significant practical value and promotional significance, providing new insights and effective solutions for the development of 3D digital modeling technology.

[0097] The above are only preferred embodiments of the present invention, which do not limit the scope of protection of the present invention. For those skilled in the art, the present invention can be modified and varied in various ways. Any changes, modifications, replacements, integrations and parameter changes to these embodiments through conventional substitutions or that can achieve the same functions without departing from the principles and spirit of the present invention fall within the scope of protection of the present invention.

Claims

1. A method for synthesizing a three-dimensional model based on multiple photos, characterized in that: include: S1. Obtain several photos of the model, and export the three-dimensional models generated from the photos into three-dimensional model A and three-dimensional model B, respectively, where the three-dimensional model A and the three-dimensional model B are complementary; S2. Aligning the three-dimensional model A and the three-dimensional model B by obtaining at least three similar feature points, mapping the feature points on the three-dimensional model A with the corresponding feature points on the three-dimensional model B one by one and connecting them; S3, cutting and deleting the overlapping parts of the 3D model A and the 3D model B, exporting the 3D model A and the 3D model B and naming them as the 3D model C and the 3D model D; S4, combining the 3D model A and the 3D model B to form a complete 3D model E, and reallocating uv to the 3D model E; S5. Bake the color maps of the 3D model C and the 3D model D onto the 3D model E, and repair the color map of the joints of the 3D model E to form a 3D model.

2. The method for synthesizing a three-dimensional model based on multiple photos according to claim 1, characterized in that: The three-dimensional model A and the three-dimensional model B are three-dimensional models generated by photos with a surface integrity greater than or equal to 60%, respectively exported as three-dimensional model A and three-dimensional model B, and the three-dimensional model B fills in the parts of the model surface that are missing, blurred or not accurately constructed in the three-dimensional model A.

3. The method for synthesizing a three-dimensional model based on multiple photos according to claim 1, characterized in that: The acquisition of similar feature points of the three-dimensional model A and the three-dimensional model B is specifically as follows: Mesh the 3D model A and 3D model B separately, discretize their surfaces into multiple small triangular patches, and obtain the similarity between the centroids of the two triangular patches. The formula is: The coordinates of the center of mass of the i-th triangle patch in the three-dimensional model A are The coordinates of the center of mass of the j-th triangle patch in the three-dimensional model B are α and β are the weight coefficients of the center of gravity and normal vector respectively, ΔN ij is the difference in the cosine of the angle between the normal vectors of two triangle faces, and the formula is: and are the angles between the normal vectors of the i-th triangle face in the 3D model A and the j-th triangle face in the 3D model B and the reference direction; set the similarity threshold T, when S ij When >T, it is determined to be a similar feature point.

4. The method for synthesizing a three-dimensional model based on multiple photos according to claim 1, wherein: The characteristic points on the three-dimensional model A are mapped one by one with the corresponding characteristic points on the three-dimensional model B and connected. By selecting the three-dimensional model A as a reference, the three-dimensional model B is proportionally enlarged and reduced, the spatial coordinate position of the three-dimensional model B is adjusted, and the error between the three-dimensional model and the three-dimensional model B in the three-dimensional coordinates is controlled within the range of ±0.01mm, and the three-dimensional model A and the three-dimensional model B are connected.

5. The method for synthesizing a three-dimensional model based on multiple photos according to claim 1, characterized in that: The overlapping parts of the 3D model A and the 3D model B are cut and deleted in S3, specifically: S3.

1. Scan 3D model A and 3D model B point by point in 3D space through spatial scanning, and mark the point sets in the overlapping area; S3.

2. Constructing a boundary contour of the overlapping region based on the marked overlapping point set, and determining the extent of the overlapping region using a contour tracing algorithm; S3.

3. Based on the boundary contour, use the difference operation in the Boolean operation to subtract the overlapping parts from 3D model A and 3D model B respectively, and perform topological optimization on the cut models to eliminate redundant vertices and edges generated by cutting.

6. The method for synthesizing a three-dimensional model based on multiple photos according to claim 1, characterized in that: In S3.1, the three-dimensional model A and the three-dimensional model B are scanned point by point in the three-dimensional space by spatial scanning, and the point sets in the overlapping area are marked, specifically: Divide the three-dimensional space into uniform grid cells. The size of each grid cell is d×d×d. Obtain each vertex P(x, y, z) in the three-dimensional model A and the three-dimensional model B, and calculate the grid cell coordinates (i, j, k) in the three-dimensional model A and the three-dimensional model B. The formula is: Count the number of vertices from 3D model A and 3D model B in each grid cell, denoted as n A and n A , calculate the overlap R, the formula is: When R is greater than the set overlap threshold T R When , there is an overlapping area in the grid unit, and all vertices in the grid unit are marked as the point set of the overlapping area.

7. The method for synthesizing a three-dimensional model based on multiple photos according to claim 1, characterized in that: S3.2 constructs the boundary contour of the overlapping area based on the marked overlapping point set, and uses the contour tracking algorithm to determine the range of the overlapping area, specifically: Choose any point from the marked coincident point set as the starting point P0. For any point P in the coincident point set r(x,y,z) , calculate the adjacent point P adj The connection vector The formula is: Get point P r and its two adjacent points P adj1 、P adj2 , calculate the direction change metric D, the formula is: D = | cosθ 12 |, where θ 12 for and angle, set the direction change threshold T D , when D>T D When r At the edge of the contour, starting from the starting point P0, traverse the adjacent points to form a preliminary contour point sequence and determine the range of the overlapping area.

8. The method for synthesizing a three-dimensional model based on multiple photos according to claim 1, wherein: In the above-mentioned S4, the three-dimensional model A and the three-dimensional model B are spliced ​​together to form a complete three-dimensional model E. By adjusting the connection between the three-dimensional model A and the three-dimensional model B, observing the main outline silhouette of the actual object and the structural connection between the three-dimensional model A and the three-dimensional model B, and making fine adjustments by moving, rotating and scaling tools, the three-dimensional model A and the three-dimensional model B are connected and merged to form a new and complete three-dimensional model E.

9. The method for synthesizing a three-dimensional model based on multiple photos according to claim 1, characterized in that: The uv of the three-dimensional model E is redistributed, and a preliminary uv is automatically and quickly unfolded for the three-dimensional model E through 3D modeling software. The stretching and distortion of the uv block are modified based on the preliminary uv, and the uv is rearranged so that each uv block is evenly and reasonably distributed.

10. The method for synthesizing a three-dimensional model based on multiple photos according to claim 1, characterized in that: The S5 repairs the color map of the splicing seam of the three-dimensional model E to form a three-dimensional model, and identifies the color and texture mutation areas by performing edge detection on the splicing seam; for color differences, extracts color samples from both sides of the seam through color gradient fusion technology, creates a color transition zone at the seam, and makes the colors naturally connected; for texture discontinuity, uses texture mapping matching technology to analyze the texture features on both sides of the seam, finds similar texture areas, maps the texture on one side to the other side and adjusts the deformation to make the texture continuous, and through fine-tuning of details, manually paints and modifies the unnatural color and texture transitions to complete the color map repair to form a three-dimensional model.

Images