Top-level reconstruction method of 3D model using oblique photography

Abstract

This invention discloses a method for reconstructing the top layer of a 3D model using oblique photography, belonging to the field of 3D data processing technology. The method comprises the following steps: S1, calculating the similarity scores of spatially adjacent tile pairs using a dynamically adjusted weighted composite of curvature, normal, and texture similarities, and determining whether to merge the top layer tiles based on the similarity scores; S2, performing a merge quality check: dynamically adjusting the weights based on the merged quality; and S3, performing mesh fusion. The advantages of this method are that it significantly reduces the number of top layer tiles while minimizing the loss of model accuracy, effectively improving loading efficiency, and significantly enhancing the user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional data processing, and in particular to a method for reconstructing the top layer of a three-dimensional model using oblique photography. Background Art

[0002] Oblique photography 3D models (oblique models) are 3D data models organized based on Level of Detail (LOD). This model achieves high-precision 3D reconstruction while adapting to multiple levels of detail. However, oblique models often have a high number of tiles at the highest level (top layer), resulting in excessive data loading, hindering loading efficiency and real-time rendering performance. Therefore, it is necessary to develop a top-level reconstruction method for existing oblique models that significantly reduces the number of tiles while maintaining model accuracy and improving loading speed. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a top-level reconstruction method for an oblique photography three-dimensional model while ensuring the accuracy of the model and significantly reducing the number of tiles.

[0004] In order to solve the above technical problems, the technical solution of the present invention is a method for reconstructing the top layer of a 3D model using oblique photography, comprising the following steps:

[0005] S1. Merge top-level tiles: Merge tiles in the top level that have the same semantics, are spatially adjacent, and meet similar geometric characteristics to form a new top-level tile.

[0006] S11. Semantic Classification: Use a pre-trained 3D semantic segmentation network to perform pixel-level semantic annotation on each original tile and generate a semantic label vector for each original tile. The semantic label vector includes the category and the category confidence; the categories include buildings, vegetation, roads, and mountains;

[0007] S12. Calculate similarity: Calculate the curvature similarity Sc, normal similarity Sn, and texture similarity St of adjacent tiles in the original tile;

[0008] S13. Calculate the similarity score S of adjacent tiles:

[0009] When at least one of the calculated curvature similarity Sc, normal similarity Sn, and texture similarity St of two adjacent tiles is less than a preset threshold, the two tiles are determined not to be merged;

[0010] When the calculated curvature similarity Sc, normal similarity Sn, and texture similarity St of two adjacent tiles are greater than or equal to the corresponding preset thresholds, the total similarity score S is calculated and S is added to the threshold S. Tar Compare; if S is greater than or equal to STar , then the two tiles are determined to be merged;

[0011] The calculation method of S is:

[0012] S=Wn*Sn+Wc*Sc+Wt*St; where Wn+Wc+Wt=1;

[0013] Where Wn is the normal weight, Wt is the texture weight; Wc is the curvature weight;

[0014] S14, merging tiles: merging the two tiles determined to be merged;

[0015] S2. Merge quality check: Check whether there are quality issues after the merge. If so, roll back to the state before the merge and update Wn, Wt and S Tar , recalculate the similarity score S; the quality issues include checking whether the merged tiles have boundary geometric cracks, overlaps, or holes; checking whether there are color discontinuities or misalignments at the texture splicing;

[0016] S3, grid fusion: grid fusion of spatially adjacent top-level tiles, merging the vertices, repeated edges and adjacent faces of adjacent boundaries of adjacent top-level tiles in turn.

[0017] Furthermore, in step S2, the number of rollbacks n is recorded. If , then record the Tile pair as a prohibited merge list. If , then recalculate Wn, Wt, S Tar The numerical value of S Tar The update method is:

[0018] ,

[0019] in represents the updated similarity score threshold, S Tar Represents the similarity score threshold before updating, is the adjustment coefficient, n represents the number of rollbacks, and N represents the preset maximum rollback threshold.

[0020] Furthermore, the updating method of Wn and Wt is:

[0021] When geometric cracks appear, increase Wn: ; and set the latest normal weight , the texture weight Wt and curvature weight Wc are normalized to ensure that the sum of the three is 1;

[0022] When overlap occurs, increase Wn: ; and set the latest normal weight , the texture weight Wt and curvature weight Wc are normalized to ensure that the sum of the three is 1;

[0023] When cavitation occurs, increase Wn: ; and set the latest normal weight , the texture weight Wt and curvature weight Wc are normalized to ensure that the sum of the three is 1;

[0024] in, Represents the updated normal weight; 1. 2. 3 are all unit coefficients, D1 is the ratio of the crack length to the preset crack length threshold, D2 is the ratio of the overlapping area to the preset overlapping area threshold, and D3 is the ratio of the void area to the preset void area threshold;

[0025] When color discontinuity or misalignment occurs at the texture joints, increase Wt: ; and apply the latest texture weights , the normal weight Wn and curvature weight Wc are normalized to ensure that the sum of the three is 1;

[0026] in, Represents the updated texture weight; 4 is the unit coefficient, and D4 is the ratio of the color variance to the color variance threshold.

[0027] Furthermore, in the step S13, the value of Wn is dynamically adjusted and is related to the curvature feature Cg of the tile. The calculation method of Wn is:

[0028] First, calculate the curvature features Cg1 and Cg2 of two adjacent tiles respectively. , ; Where H1 and H2 are the average curvatures of the two tiles, and K1 and K2 are the Gaussian curvatures of the two tiles;

[0029] Then, calculate the average value Cg of Cg1 and Cg2; ;

[0030] Finally, update Wn:

[0031] ;

[0032] Among them, Cgt is the preset threshold, Wn is the initial normal weight, is the updated normal weight; are preset parameters; after adjusting Wn, the latest normal weight, texture weight, and curvature weight are normalized to ensure that the sum of the three is 1.

[0033] Furthermore, the value of Wt is also dynamically adjusted and is related to the texture entropy Et of the tile. The calculation method of Wt is:

[0034] First, calculate the texture entropy Et1 and Et2 of two adjacent tiles respectively. , ; Where Pi1 and Pi2 are the ratios of the number of pixels in the i-th bin of the two tiles to the total number of pixels in the entire texture map; n1 and n2 are the total number of bins where the texture features of the two tiles are quantized;

[0035] Then, calculate the average value Et of Et1 and Et2; ;

[0036] Finally, update Wt:

[0037] ;

[0038] Among them, Ett is the preset threshold; Wt is the initial texture weight, is the updated texture weight, These are the preset parameters. After adjusting Wt, the latest normal weight, texture weight, and curvature weight are normalized to ensure that the sum of the three is 1.

[0039] The curvature weight Wc only changes when the normal weight Wn and / or texture weight Wt are adjusted and normalized.

[0040] Furthermore, the setting values ​​of the initial texture weight Wt and the initial normal weight Wn are different according to the different semantic types of different tiles: if the category of the tile is a building, the initial normal weight is greater than the normal weight of the tile whose semantic category is vegetation, road, or mountain; if the category of the tile is vegetation, road, or mountain, the initial texture weight is greater than the texture weight of the tile whose semantic category is a building.

[0041] Furthermore, the specific method of the grid fusion in step S3 is as follows:

[0042] S31, Vertex Merging: Use the ICP algorithm to perform rigid alignment on the adjacent tile boundary vertices;

[0043] S32, Edge Merging: Build a topological graph, identify shared edges, merge duplicate edges, and update the topological relationship between adjacent patches;

[0044] S33, Face Merging: Detect overlapping faces and fuse adjacent faces using union or stitching algorithms;

[0045] Furthermore, after step S3, there are steps S4, geometry correction: performing Laplace smoothing on the fused mesh to eliminate local deformities; S5, texture mapping: unifying the UV coordinate system and performing color blending on overlapping areas; and S6, constructing a new tileset.json file.

[0046] Furthermore, the curvature similarity Sc is calculated by: calculating the mean curvature of all vertices in the tile and variance ,

[0047] .

[0048] Furthermore, the normal similarity Sn is calculated as follows:

[0049] ;

[0050] in, and They are the normal change rates of the two tiles respectively, and Rmax is the global maximum normal change rate.

[0051] Furthermore, the method for calculating the texture similarity St is as follows: for two tiles, the mean of the RGB channels of the texture blocks is calculated respectively. and standard deviation ,

[0052] .

[0053] Beneficial effects of the present invention:

[0054] The present invention proposes a top-level reconstruction method for oblique photography 3D models. Through a unique tile merging method, it can adapt to complex models, significantly reduce the number of top-level tiles, and at the same time, have very low loss of model accuracy, effectively improve loading efficiency, and significantly improve user experience. DETAILED DESCRIPTION

[0055] The following is a further description of specific embodiments of the present invention. It should be noted that the description of these embodiments is intended to facilitate understanding of the present invention and does not constitute a limitation of the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0056] Example

[0057] The top layer reconstruction method of the oblique photography 3D model includes the following steps:

[0058] S1. Merge top-level tiles: Merge tiles in the top level that have the same semantics, are spatially adjacent, and meet similar geometric characteristics to form a new top-level tile.

[0059] S11. Semantic Classification: Use a pre-trained 3D semantic segmentation network to perform pixel-level semantic annotation on each original tile and generate a semantic label vector for each original tile. The semantic label vector includes the category and the category confidence; the categories include buildings, vegetation, roads, and mountains;

[0060] S12. Calculate similarity: Calculate the curvature similarity Sc, normal similarity Sn, and texture similarity St of adjacent tiles in the original tile;

[0061] In this embodiment, the curvature similarity Sc is calculated by: calculating the mean curvature of all vertices in the tile and variance ,

[0062] .

[0063] The calculation method of the normal similarity Sn is:

[0064] ;

[0065] in, and They are the normal change rates of the two tiles respectively, and Rmax is the global maximum normal change rate.

[0066] The method for calculating the texture similarity St is as follows: for two tiles, the mean of the RGB channels of the texture blocks is calculated respectively. and standard deviation ,

[0067] .

[0068] In another embodiment, other known methods for calculating curvature similarity Sc, normal similarity Sn, and texture similarity St may be selected based on the model type and other characteristics. For example, the curvature similarity Sc may be calculated by discretizing the curvature values ​​into a histogram and evaluating the consistency of the overall curvature features by comparing the similarity of the histograms; the normal similarity Sn may be calculated using methods such as the angle cosine method and the dot product method; and the texture similarity St may be calculated using the texture image histogram method.

[0069] S13. Calculate the similarity score S of adjacent tiles:

[0070] When at least one of the calculated curvature similarity Sc, normal similarity Sn, and texture similarity St of two adjacent tiles is less than a preset threshold, the two tiles are determined not to be merged;

[0071] When the calculated curvature similarity Sc, normal similarity Sn, and texture similarity St of two adjacent tiles are greater than or equal to the preset threshold, the total similarity score S is calculated and S is added to the threshold S. Tar Compare; if S is greater than or equal to S Tar , then the two tiles are determined to be merged; the calculation method of S is:

[0072] S=Wn*Sn+Wc*Sc+Wt*St; Wn+Wc+Wt=1;

[0073] Where Wn is the normal weight, Wt is the texture weight; Wc is the curvature weight;

[0074] The value of Wn is dynamically adjusted and is related to the curvature characteristic Cg of the tile. The calculation method of Wn is:

[0075] First, calculate the curvature features Cg1 and Cg2 of two adjacent tiles respectively. , ; Where H1 and H2 are the average curvatures of the two tiles, and K1 and K2 are the Gaussian curvatures of the two tiles;

[0076] Then, calculate the average value Cg of Cg1 and Cg2; ;

[0077] Finally, update Wn:

[0078] ;

[0079] Among them, Cgt is the preset threshold, Wn is the initial normal weight, is the updated normal weight; is the preset parameter; after adjusting Wn, the latest normal weight, texture weight, and curvature weight are normalized to ensure that the sum of the three is 1. The specific method of normalization is illustrated by an example: if the updated normal weight Wn, texture weight Wt, and curvature weight Wc are 0.43, 0.2, and 0.4 respectively, then the normalized normal weight =0.42, =0.19; =0.39. The value can be adjusted based on multiple experiments, the quality of the merged model, and the number of tiles.

[0080] The value of Wt is also dynamically adjusted and is related to the texture entropy Et of the tile. The calculation method of Wt is:

[0081] First, calculate the texture entropy Et1 and Et2 of two adjacent tiles respectively. , ; Where Pi1 and Pi2 are the ratios of the number of pixels in the i-th bin of the two tiles to the total number of pixels in the entire texture map; n1 and n2 are the total number of bins where the texture features of the two tiles are quantized;

[0082] Then, calculate the average value Et of Et1 and Et2; ;

[0083] Finally, update Wt:

[0084] ;

[0085] Among them, Ett is the preset threshold; Wt is the initial texture weight, is the updated texture weight, These are the preset parameters. After adjusting Wt, the latest normal weight, texture weight, and curvature weight are normalized to ensure that the sum of the three is 1. The value can be adjusted based on multiple experiments, the quality of the merged model, and the number of tiles.

[0086] The curvature weight Wc only changes when the normal weight Wn and / or texture weight Wt are adjusted and normalized.

[0087] Among them, the setting values ​​of the above-mentioned initial texture weight Wt and initial normal weight Wn are different according to the different semantic types of different tiles: if the category of the tile is a building, the initial normal weight is greater than the normal weight of the tile whose semantic category is vegetation, road, or mountain; if the category of the tile is vegetation, road, or mountain, the initial texture weight is greater than the texture weight of the tile whose semantic category is a building. In this embodiment, for the tile whose category is a building, the initial normal weight Wn is 0.4, the initial texture weight Wt is 0.2, and the initial curvature weight Wc is 0.4; for the tile whose category is vegetation, road, or mountain, the initial normal weight Wn is 0.2, the initial texture weight Wt is 0.6, and the initial curvature weight Wc is 0.2. Reasonable adjustments can also be made.

[0088] S14, merging tiles: merging the two tiles determined to be merged;

[0089] S2. Merge quality check: Check whether there are quality problems after merging. The quality problems include checking whether there are boundary geometric cracks, overlaps, and holes in the merged tiles; checking whether there are color discontinuities or misalignments at the texture joints; if so, roll back to the state before merging and record the number of rollbacks n. If , then record the Tile pair as a prohibited merge list. If , then recalculate Wn, Wt, S Tar The value of , and then recalculate the similarity score S;

[0090] wherein the S Tar The update method is:

[0091] ,

[0092] in represents the updated similarity score threshold, S Tar Represents the similarity score threshold before updating, is the adjustment coefficient, n represents the number of rollbacks, and N represents the preset maximum rollback threshold. The value of can be adjusted based on multiple experiments, the quality of the merged model and the number of tiles. The value is set to 0.1 and can be adjusted reasonably according to actual conditions.

[0093] The updating method of Wn and Wt is:

[0094] When geometric cracks appear, increase Wn: ; and set the latest normal weight , the texture weight Wt and curvature weight Wc are normalized to ensure that the sum of the three is 1;

[0095] When overlap occurs, increase Wn: ; and set the latest normal weight , the texture weight Wt and curvature weight Wc are normalized to ensure that the sum of the three is 1;

[0096] When cavitation occurs, increase Wn: ; and set the latest normal weight , the texture weight Wt and curvature weight Wc are normalized to ensure that the sum of the three is 1;

[0097] in, Represents the updated normal weight; 1. 2. 3 are all unit coefficients, D1 is the ratio of the crack length to the preset crack length threshold, D2 is the ratio of the overlapping area to the preset overlapping area threshold, and D3 is the ratio of the void area to the preset void area threshold;

[0098] When color discontinuity or misalignment occurs at the texture joints, increase Wt: ; and apply the latest texture weights , the normal weight Wn and curvature weight Wc are normalized to ensure that the sum of the three is 1;

[0099] in, Represents the updated texture weight; 4 is the unit coefficient, and D4 is the ratio of the color variance to the color variance threshold.

[0100] 1. 2. 3. The value of 4 can be adjusted based on multiple experiments, the quality of the merged model and the number of tiles. 1. 2. 3. The values ​​of 4 are all set to 0.1, which can be adjusted reasonably according to actual conditions.

[0101] S3, Grid Fusion: Grid fusion of spatially adjacent top-level tiles, including the following steps:

[0102] S31, Vertex Merging: Use the ICP algorithm to perform rigid alignment on the adjacent tile boundary vertices;

[0103] S32, Edge Merging: Build a topological graph, identify shared edges, merge duplicate edges, and update the topological relationship between adjacent patches;

[0104] S33, Face Merging: Detect overlapping faces and fuse adjacent faces using union or stitching algorithms;

[0105] S4, geometric correction: perform Laplace smoothing on the fused mesh to eliminate local deformities;

[0106] S5. Texture mapping: Unify the UV coordinate system and perform color blending on overlapping areas; color blending can be achieved through a variety of methods, such as linear interpolation, multi-band blending, etc.

[0107] S6. Build a new tileset.json file, including the following steps:

[0108] S61. Collect merged tile information: traverse all merged and optimized tiles, and collect the geometric information, texture information, and position and orientation of each tile in three-dimensional space.

[0109] S62. Update model structure: Update the overall structure of the 3D model based on the merged tile information, including determining the hierarchical relationship, spatial index, and boundary information between tiles to ensure efficient spatial division and visibility judgment during model loading and rendering.

[0110] S63. Save texture mapping: Save the merged texture information to the tileset.json file.

[0111] The embodiments of the present invention are described in detail above, but the present invention is not limited to the described embodiments. It is apparent to those skilled in the art that various changes, modifications, substitutions, and variations of these embodiments may be made without departing from the principles and spirit of the present invention, and the changes still fall within the scope of protection of the present invention.

Claims

1. A method for top-level reconstruction of a 3D model using oblique photography, characterized by: The steps include: S1. Merge top-level tiles: Merge tiles in the top level that have the same semantics, are spatially adjacent, and meet similar geometric characteristics to form a new top-level tile. S11. Semantic classification: Use a pre-trained 3D semantic segmentation network to perform pixel-level semantic annotation on each original tile and generate a semantic label vector for each original tile. The semantic label vector includes the category and the category confidence. S12. Calculate similarity: Calculate the curvature similarity Sc, normal similarity Sn, and texture similarity St of adjacent tiles in the original tile; S13. Calculate the similarity score S of adjacent tiles: When at least one of the calculated curvature similarity Sc, normal similarity Sn, and texture similarity St of two adjacent tiles is less than the corresponding preset threshold, it is determined that the two tiles are not merged; When the calculated curvature similarity Sc, normal similarity Sn, and texture similarity St of two adjacent tiles are greater than or equal to the corresponding preset thresholds, the total similarity score S is calculated and S is added to the threshold S. Tar Compare; if S is greater than or equal to S Tar , then the two tiles are determined to be merged; The similarity score S is calculated as follows: S=Wn*Sn+Wc*Sc+Wt*St; where Wn+Wc+Wt=1; Where Wn is the normal weight, Wt is the texture weight; Wc is the curvature weight; S14, merging tiles: merging the two tiles determined to be merged; S2. Merge quality check: Check whether there are quality issues after the merge. If so, roll back to the state before the merge and update Wn, Wt and S Tar , recalculate the similarity score S; the quality issues include checking whether the merged tiles have boundary geometric cracks, overlaps, or holes; checking whether there are color discontinuities or misalignments at the texture splicing; S3, grid fusion: grid fusion of spatially adjacent top-level tiles, merging the vertices, repeated edges and adjacent faces of adjacent boundaries of adjacent top-level tiles in turn.

2. The method for reconstructing the top layer of a 3D model using oblique photography according to claim 1, wherein: In step S2, the number of rollbacks n is recorded. If , then record the Tile pair as a prohibited merge list. If , then recalculate Wn, Wt, S Tar The numerical value of S Tar The update method is: ， in represents the updated similarity score threshold, S Tar Represents the similarity score threshold before updating, is the adjustment coefficient, n represents the number of rollbacks, and N represents the preset maximum rollback threshold.

3. The method for reconstructing the top layer of a 3D model using oblique photography according to claim 2, wherein: The updating method of Wn and Wt is: When geometric cracks appear, increase Wn: ; And the latest normal weight , the texture weight Wt and curvature weight Wc are normalized to ensure that the sum of the three is 1; When overlap occurs, increase Wn: ; And the latest normal weight , the texture weight Wt and curvature weight Wc are normalized to ensure that the sum of the three is 1; When cavitation occurs, increase Wn: ; And the latest normal weight , the texture weight Wt and curvature weight Wc are normalized to ensure that the sum of the three is 1; in, Represents the updated normal weight; 1.

2. 3 are all unit coefficients, D1 is the ratio of the crack length to the preset crack length threshold, D2 is the ratio of the overlapping area to the preset overlapping area threshold, and D3 is the ratio of the void area to the preset void area threshold; When color discontinuity or misalignment occurs at the texture joints, increase Wt: ; and apply the latest texture weights , the normal weight Wn and curvature weight Wc are normalized to ensure that the sum of the three is 1; in, Represents the updated texture weight; 4 is the unit coefficient, and D4 is the ratio of the color variance to the color variance threshold.

4. The method for reconstructing the top layer of a 3D model using oblique photography according to claim 1, wherein: In step S13, the value of Wn is dynamically adjusted and is related to the curvature feature Cg of the tile. The calculation method of Wn is: First, calculate the curvature features Cg1 and Cg2 of two adjacent tiles respectively. , ; Where H1 and H2 are the average curvatures of the two tiles, and K1 and K2 are the Gaussian curvatures of the two tiles; Then, calculate the average value Cg of Cg1 and Cg2; ; Finally, update Wn: ； Among them, Cgt is the preset threshold, Wn is the initial normal weight, is the updated normal weight; are preset parameters; after adjusting Wn, normalize the latest normal weight, texture weight, and curvature weight to ensure that the sum of the three is 1; The value of Wt is also dynamically adjusted and is related to the texture entropy Et of the tile. The calculation method of Wt is: First, calculate the texture entropy Et1 and Et2 of two adjacent tiles respectively. , ; Where Pi1 and Pi2 are the ratios of the number of pixels in the i-th bin of the two tiles to the total number of pixels in the entire texture map; n1 and n2 are the total number of bins where the texture features of the two tiles are quantized; Then, calculate the average value Et of Et1 and Et2; ; Finally, update Wt: ； Among them, Ett is the preset threshold; Wt is the initial texture weight, is the updated texture weight, These are the preset parameters. After adjusting Wt, the latest normal weight, texture weight, and curvature weight are normalized to ensure that the sum of the three is 1.

5. The method for reconstructing the top layer of a 3D model using oblique photography according to claim 4, wherein: The setting values ​​of the initial texture weight Wt and the initial normal weight Wn are different according to the different semantic types of different tiles: if the category of the tile is building, the initial normal weight is greater than the normal weight of the tile with the semantic category of vegetation, road, or mountain; If the category of the tile is vegetation, road, or mountain, the initial texture weight is greater than the texture weight of the tile whose semantic category is building.

6. The method for reconstructing the top layer of a 3D model using oblique photography according to claim 1, wherein: The specific method of grid fusion in step S3 is: S31, Vertex Merging: Use the ICP algorithm to perform rigid alignment on the adjacent tile boundary vertices; S32, Edge Merging: Build a topological graph, identify shared edges, merge duplicate edges, and update the topological relationship between adjacent patches; S33, Face merging: Detect overlapping faces and fuse adjacent faces using union or stitching algorithms.

7. The method for reconstructing the top layer of a 3D model using oblique photography according to claim 1, wherein: After step S3, there are steps S4, geometry correction: performing Laplace smoothing on the fused mesh to eliminate local deformities; S5, texture mapping: unifying the UV coordinate system and performing color blending on overlapping areas; and S6, constructing a new tileset.json file.

8. The method for reconstructing the top layer of a 3D model using oblique photography according to claim 1, wherein: The calculation method of the curvature similarity Sc is: calculate the mean curvature of all vertices in the tile and variance , 。 9. The method for reconstructing the top layer of a 3D model using oblique photography according to claim 1, wherein: The calculation method of the normal similarity Sn is: ； in, and They are the normal change rates of the two tiles respectively, and Rmax is the global maximum normal change rate.

10. The method for reconstructing the top layer of a 3D model using oblique photography according to claim 1, wherein: The method for calculating the texture similarity St is as follows: for two tiles, the mean of the RGB channels of the texture blocks is calculated respectively. and standard deviation , 。