A method for dynamic merging of three-dimensional scene models and multi-level detail level optimization

Abstract

The application discloses a three-dimensional scene model dynamic merging and multi-level detail level optimization method, which converts original objects of multi-source three-dimensional data into a unified intermediate representation, and divides the original objects into forced independent objects, conditionally merged objects and mergeable objects according to interactive requirements; constraint screening and merging scoring are performed on the conditionally merged objects and the mergeable objects to form a merging set, geometric, material and texture merging is performed on the merging set, geometric merging mapping and business attribute mapping are established, and multi-level detail level resources are generated; in runtime, resources are selected according to a camera state and a screen error, when an interactive request of the conditionally merged object is responded, object attributes are returned by using the mapping and a geometric sub-region of the object is determined, and when the current resource does not reserve the sub-region, high-precision resources are loaded to execute local highlighting. The application can reduce drawing calling and resource occupation, and maintain object-level interactive ability of the original objects after merging.

Description

Technical Field

[0001] This invention relates to the field of 3D model data processing and computer graphics rendering technology, specifically to a method for dynamic merging and multi-level detail optimization of 3D scene models. Background Technology

[0002] With the development of technologies such as digital twins, 3D geographic information systems, smart cities, and building operation and maintenance, the number of models and the scale of data carried by 3D scenes are constantly increasing. The data in 3D scenes typically comes from geographic information data, building information models, oblique photogrammetry models, terrain data, and ordinary mesh models. Because data from different sources differ in coordinate references, units of measurement, hierarchical structure, geometric representation methods, and material texture organization methods, it is usually necessary to convert and reorganize the above data in order to load and display it in the same 3D scene.

[0003] For 3D scenes containing a large number of model objects, loading and rendering each model object one by one according to the original model structure will generate a large number of scene nodes and drawing calls, accompanied by frequent material switching, texture loading, and data submission, thereby increasing the workload of the processor and graphics processor. At the same time, rendering models that are far from the viewing position or occupy a small area in the display screen with the original precision will also result in the invalid use of geometric data and texture resources.

[0004] To improve the loading and rendering efficiency of 3D scenes, existing technologies typically employ methods such as spatial partitioning, model merging, material deduplication, texture atlases, and level of detail processing to process 3D models. Among these, model merging reduces scene nodes and drawing calls, material deduplication and texture atlases reduce rendering state switching and the number of texture resources, and level of detail processing selects model resources of appropriate precision for rendering based on the viewing distance or the size of the model's projection on the screen.

[0005] However, when multiple original objects are merged into a single model object, the identifiers, attributes, and geometric positions of the original objects within the merged model can easily lose their clear correspondence. When a query, pickup, local highlighting, or independent update of a particular original object is needed, it is usually necessary to reload the original model or process the entire merged model. Conversely, if each independent object is always retained and rendered, it increases the number of scene nodes and drawing calls, making it difficult to fully leverage the performance advantages of model merging.

[0006] Furthermore, model merging and detail processing are typically performed separately. During the progressive simplification of models, the local geometric regions corresponding to the original objects may be simplified or reorganized with other geometric regions, making it difficult for low-precision models to continue supporting accurate object-level queries and local highlighting. At the same time, factors such as spatial relationships between models, the compatibility of materials and textures, object attributes and update requirements, and the resource scale of the merged model all affect the merging result. Merging based solely on spatial distance or material type makes it difficult to simultaneously consider rendering efficiency, resource scale, and object-level interaction requirements.

[0007] The invention patent application with publication number CN106570922A, entitled "A Method for Improving the Rendering Performance of 3D Models", discloses a method for performing multi-level detail processing on the original texture of a model object, forming a texture set by combining the textures corresponding to models with shared textures, merging multiple small textures in the texture set into a large texture, and merging the models corresponding to the large texture, thereby reducing the number of rendering objects and the number of times data is submitted to the graphics processor. Summary of the Invention

[0008] The technical problem to be solved by this invention is: when merging and detailing model objects in a 3D scene, how to maintain the geometric and attribute correspondence between the original objects and the merged objects while reducing the number of scene nodes and rendering resource overhead, and how to ensure that the merged model can still support object-level querying and local highlighting when switching between different detail level resources.

[0009] To address the aforementioned technical problems, this invention provides a method for dynamic merging and multi-level detail optimization of 3D scene models, comprising:

[0010] The original objects of multi-source 3D data are transformed into a unified intermediate representation that includes object identifiers, geometric data, material data, texture data, spatial bounding volume, object attributes, interaction flags, and object resource costs.

[0011] Based on the interaction flags, the original objects that need to be updated, edited, or displayed with alarms independently are classified as mandatory independent objects; the original objects that do not have the aforementioned requirements but need to be queried at the object level or highlighted locally are classified as conditional mergeable objects; and the rest are mergeable objects.

[0012] Using the object identifier as the primary key, establish a business attribute mapping for the merged objects, including record object attributes, the original bounding body, and high-precision resource references.

[0013] Candidate combinations are generated for conditional merging objects and merging objects. Candidate combinations that do not meet coordinate transformation relationships, rendering status or resource limit constraints are excluded. A merging score is calculated based on spatial proximity, material similarity, texture compatibility, attribute compatibility, query independence penalty and merging resource cost. A merging set is formed based on the score threshold and cumulative resource limit.

[0014] Perform geometry, material, and texture merging on the merged collection. During merging, record the geometric range and material region of each original object in the merged object, generate a geometry merge mapping, and write the geometry merge mapping identifier and the merged object identifier into the business attribute mapping.

[0015] Generate multi-level detail resources for the merged object and write resource references into the corresponding business attribute mapping;

[0016] At runtime, the level of detail resource is selected based on the camera status and screen error; when responding to an interaction request of a conditional merge object, the object attributes are returned based on the business attribute mapping, and its geometric sub-region is determined based on the geometric merge mapping; when the interaction request includes local highlighting and the current level of detail resource does not retain a geometric sub-region, the corresponding resource is loaded based on the high-precision resource reference, and local highlighting is performed using the geometric sub-region or the high-precision resource.

[0017] Preferably, the original objects of multi-source 3D data are converted into a unified intermediate representation, including:

[0018] Analyze multi-source 3D data according to the data source type to obtain the geometric data, material data, texture data and hierarchical path of the original object;

[0019] Generate a coordinate transformation matrix based on the source coordinate reference, source coordinate origin, and source units. Based on the coordinate transformation matrix, transform the geometric data of the original object to the project's local coordinate system using metric units.

[0020] The stable identifier of the original object is used as the object identifier; if the original object does not have a stable identifier, the object identifier is generated based on the data source identifier, the normalized hierarchical path, and the local identifier of the source object.

[0021] Calculate the spatial bounding volume in the project's local coordinate system, and determine the object resource cost based on the number of vertices, indexes, texture data volume, and data package size.

[0022] Preferably, the interactive flags include object-level query flags, local highlight flags, edit flags, alarm display flags, and independent update flags;

[0023] Object classification is performed in the order of mandatory independent objects, conditionally merged objects, and mergeable objects. Original objects that have been classified as mandatory independent objects do not participate in candidate combination generation and merging scoring.

[0024] For conditional merge objects, a selection proxy for object picking is generated before the merge is executed, and an initial record of business attribute mapping containing object identifier, object attributes, original bounding body and object-level high-precision resource references is established.

[0025] Preferably, generating candidate combinations includes:

[0026] A spatial index is established based on the spatial bounding volume of the original object, the original object is written into the corresponding spatial cell, and the original object in the spatial cell and its adjacent spatial cells is retrieved.

[0027] When the spatial bounding volume distance between two original objects is not greater than the candidate radius, the two original objects are combined into a candidate combination.

[0028] Perform compatibility constraint checks on candidate combinations. Compatibility constraints include valid coordinate transformation relationships, compatible rendering queues and depth states, compatible object update boundaries, the ability of geometric data of each original object to be converted to the same coordinate system, and that the number of vertices, indexes, data volume, texture atlas area, and spatial span of the merged combination do not exceed the corresponding resource limits.

[0029] Preferably, spatial proximity, material similarity, texture compatibility, attribute compatibility, query independence penalty, and merging resource cost are normalized to [0,1], and the merging score between two original objects in the candidate combination is calculated according to the following formula:

[0030]

[0031] Among them, M ij The combined score between original object i and original object j; p ij m represents spatial proximity. ij For material similarity; u ij For texture compatibility; a ij For attribute compatibility; q ij In order to query independence penalty; z ij To consolidate resource costs, this is used to characterize the expected resource quantity of the merged two original objects relative to their corresponding resource caps; w p w m w u w a w q and w z These are the normalized weights of the corresponding factors, and they satisfy:

[0032] w p +w m +w u +w a +w q +w z =1

[0033] Spatial proximity is determined based on the spatial bounding volume distance between the two original objects and the candidate radius; material similarity is determined based on normalized material parameters; texture compatibility is determined based on texture format, channel semantics, color space, and sampling method; attribute compatibility is determined based on attribute structure and update method; query independence penalty is determined based on object-level query flags and local highlight flags.

[0034] The combined resource cost is the maximum value among vertex utilization, index utilization, data utilization, texture atlas area utilization, and spatial span utilization. Each resource utilization rate is the ratio of the expected resource amount after the combination to the corresponding resource limit.

[0035] Preferably, the merged set includes:

[0036] Construct a candidate graph using the original object as nodes, candidate combinations as edges, and merge scores as edge weights;

[0037] Select the unassigned original object as the seed object and create the current merge set containing the seed object;

[0038] From candidate edges whose nodes at one end belong to the current merge set and whose nodes at the other end are not assigned, select the candidate edge with the highest merge score and determine the unassigned node of the candidate edge as the object to be added.

[0039] Calculate the cumulative number of vertices, cumulative number of indices, cumulative data volume, texture atlas area, and spatial span of the objects to be added after they are added to the current merge set;

[0040] When the merging score of the candidate edge reaches the scoring threshold and the cumulative resource amount of each item does not exceed the corresponding resource limit, the object to be added is added to the current merging set, and the next candidate edge is selected.

[0041] When the merge score of a candidate edge is lower than the score threshold, or when the cumulative resource amount of any object to be added exceeds the corresponding resource limit, the candidate edge is marked as processed, and the next candidate edge is selected.

[0042] When there are no candidate edges that meet the scoring threshold and the cumulative resource limit, output the current merge set and reselect seed objects from the remaining unallocated original objects.

[0043] Preferably, geometry, material, and texture merging is performed on the merged set, including:

[0044] Based on the coordinate transformation relationship between the local coordinate system of the original object and the local coordinate system of the merged object, the vertices of the original object are transformed to the local coordinate system of the merged object, and the normals of the original object are transformed using the inverse of the linear transformation matrix corresponding to the coordinate transformation relationship.

[0045] Before writing the geometric data of each original object, record the vertex start position and index start position; write the vertices of the original objects into the vertex sequence of the merged object, and add the vertex start position to each index value of the original object; after writing, record the vertex interval, index interval, triangle interval and submesh interval corresponding to the original object.

[0046] Material feature identifiers are generated based on the shading model, transparency state, double-sided state, material parameters, and texture semantics. Materials with the same material feature identifiers are merged into the same material instance.

[0047] Write textures with compatible texture formats, channel semantics, color spaces, and sampling methods into a texture atlas, and offset and scale the texture coordinates of the original object according to the region of the texture in the texture atlas to obtain the merged texture coordinates;

[0048] Generate a geometric merge map, which records the object identifier, merged object identifier, vertex interval, index interval, triangle interval, submesh interval, material slot, texture atlas region, and bounding volume in the local coordinate system of the merged object; for conditional merged objects, the geometric merge map also records the selection proxy identifier; when the geometric data of the original object is not continuous in the merged object, the geometric merge map records multiple geometric intervals or selection mask identifiers.

[0049] Preferably, the business attribute mapping establishes an initial record after object classification is completed, writes a geometry merging mapping identifier after geometry, material and texture merging is completed, and writes resource references at each level after multi-level detail layer resource generation is completed.

[0050] Business attribute mapping and geometry merging mapping are associated through object identifiers and geometry merging mapping identifiers;

[0051] The business attribute mapping, geometric merge mapping, and resource files of the merged object are each set with matching version identifiers and verification values. When a version identifier or verification value mismatch is detected, the location of the geometric range based on the geometric merge mapping is stopped, and the business attribute mapping, geometric merge mapping, and resource files with matching versions are loaded.

[0052] Preferably, generating multi-level detail resources for the merged object includes:

[0053] The geometric data of the merged object is simplified step by step by using an edge folding method based on a quadratic error metric, and the texture data is downsampled step by step to determine the geometric and texture errors of the resources at each level of detail.

[0054] Within the preset query level corresponding to the condition merging object, retain the geometric range or selection proxy of the condition merging object, and retain the high-precision resource reference of the condition merging object in the business attribute mapping;

[0055] The geometric screen error of each level of detail resource is determined based on the geometric error of the resource at each level of detail, the viewport pixel height, the distance from the camera to the bounding volume of the merged object space, and the vertical field of view of the camera.

[0056] The texture screen error of each level of detail resource is determined based on the texture error of each level of detail resource, the projected area of ​​the merged object in the viewport, and the texture resolution.

[0057] Among the detail level resources where both geometric screen error and texture screen error do not exceed the corresponding error threshold, the detail level resource with the smallest data volume is selected, and different entry and exit thresholds are set for adjacent levels.

[0058] Preferably, the interaction request of the response condition merging object includes:

[0059] The interaction locations are initially screened by the spatial bounding volume, and the object identifiers of the target condition merged objects are determined by the intersection results of the object identifier buffer or the selected agent's ray.

[0060] The business attribute mapping is queried based on the object identifier, the object attributes of the target condition merged object are returned, the location of the target condition merged object is determined based on the original bounding volume, and the geometric merge mapping is queried based on the geometric merge mapping identifier.

[0061] When the current level of detail resource preserves the index range, submesh range, or selection mask of the target condition merged object, additional drawing is performed only on the index range, submesh range, or selection mask;

[0062] When the current level-of-detail resource does not preserve the geometric sub-region of the target conditional merged object, load the object-level high-precision resource of the target conditional merged object and perform additional drawing on the object-level high-precision resource;

[0063] When the object-level high-precision resource does not exist or cannot be loaded independently, the high-precision detail level resource of the merged object where the target condition merged object is located is loaded, and the geometric sub-region of the target condition merged object is determined according to the geometric merge mapping, and additional drawing is performed on the determined geometric sub-region.

[0064] After the interaction request ends, undo the additional drawing and restore the level of detail resource selected based on camera status and screen error.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] (1) Classify the original objects according to their interaction requirements, exclude the original objects that need to be updated, edited or alarmed from the scope of merging, and combine spatial relationships, material and texture compatibility, attribute structure, object-level interaction requirements and resource scale to form a merge set, which can reduce scene nodes and drawing calls while controlling the resource scale of merged objects.

[0067] (2) When performing geometry, material and texture merging, the geometry range and material area of ​​each original object in the merged object are recorded synchronously, and the association between the original object, geometry merging mapping, merged object and detail level resources is established through business attribute mapping, so that the original object still has traceable geometry and attribute correspondence after the model merging is completed.

[0068] (3) Select detail level resources based on screen error, and select the corresponding geometric sub-region or load high-precision resources on demand based on the current detail level resources for the retention of geometric sub-regions of the target condition merged object, thereby reducing the resource overhead of 3D scene rendering while maintaining the accuracy of object-level query and local highlighting.

[0069] (4) By mapping business attributes, geometric merging mapping and version correspondence and verification between resource files, the original geometric interval is stopped when the mapping version and the resource version do not match, which can reduce the error location caused by the inconsistency between the mapping data and the model resources. Attached Figure Description

[0070] Figure 1 A flowchart of a method for dynamic merging and multi-level detail optimization of 3D scene models provided in an embodiment of the present invention;

[0071] Figure 2 A flowchart illustrating the process of classifying and merging objects into a set, provided in an embodiment of the present invention;

[0072] Figure 3 This is a schematic diagram illustrating the correspondence between the original object, business attribute mapping, geometric merging mapping, merged object, and multi-level detail resources provided in the embodiments of the present invention.

[0073] Figure 4 This is a flowchart illustrating the interaction process of resource selection and condition merging objects at a level of detail, as provided in an embodiment of the present invention. Detailed Implementation

[0074] The technical solution of the present invention will be further described below with reference to the accompanying drawings. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Conventional adjustments made by those skilled in the art to parameter values, spatial indexing forms, or specific algorithms without departing from the concept of the present invention are all feasible implementations of the technical solution of the present invention.

[0075] Example 1

[0076] This embodiment provides a method for dynamic merging and multi-level detail optimization of 3D scene models. This method can be executed jointly by a model preprocessor and a runtime rendering program. The model preprocessor is used to complete multi-source data parsing, object classification, determination of the merge set, model merging, and generation of multi-level detail resources. The runtime rendering program is used to complete resource loading, detail selection, object-level querying, and local highlighting.

[0077] This embodiment uses a 3D scene of a park as an example. The 3D scene of the park includes multi-source 3D data such as building information model, geographic information model, and mesh model. Among them, the building information model includes components such as walls, beams, columns, pipelines, and equipment; the geographic information model includes objects such as terrain and roads; and the mesh model includes greening, decorative components, and other ancillary facilities. Data from different sources have their own coordinate references, units of measurement, hierarchical structure, material textures, and object attributes.

[0078] like Figure 1 As shown, the method in this embodiment includes steps S101 to S107.

[0079] S101. Establish a unified intermediate representation for the original object.

[0080] Based on the data source type of the multi-source 3D data, the corresponding parser is called to read the geometric data, material data, texture data, hierarchical path, and object attributes of the original object from each data source.

[0081] Geometric data should include at least vertex positions, vertex normals, texture coordinates, and index data. Material data should include at least the shading model, base color, transparency state, two-sided state, and material parameters. Texture data should include at least the texture format, channel semantics, color space, sampling method, and texture resolution. Object attributes can be stored in a format where field names correspond to field values.

[0082] For original objects with stable component or stable node identifiers, these stable identifiers are directly used as object identifiers. For original objects without stable identifiers, the data source identifier, normalized hierarchical path, and source object local identifier are sequentially concatenated to generate a summary value, which is then used as the object identifier. A duplicate check is performed when generating object identifiers; if duplicate object identifiers exist, a data version or geometric feature summary is added when generating the summary value to ensure the uniqueness of object identifiers within the same project scope.

[0083] A coordinate transformation matrix is ​​generated based on the source coordinate reference, source coordinate origin, and source units of the original object. Using this transformation matrix, the vertex positions of the original object are transformed to the project's local coordinate system using metric units. The origin of the project's local coordinate system can be set near the center of the park or the center of the project's bounding volume to reduce the impact of large coordinate values ​​on the accuracy of floating-point operations.

[0084] Calculate the axial bounding box of each original object in the local coordinate system of the project, and use the axial bounding box as the spatial bounding volume of the original object. For original objects with narrow or elongated shapes or strong orientation, oriented bounding boxes can also be recorded at the same time, but the candidate object retrieval is still based on the axial bounding box.

[0085] The number of vertices, indexes, texture data volume, and data packet size of the original object are calculated, and an object resource cost record for the original object is created. This object resource cost record is not used as a single storage capacity value; instead, each of these resource quantities is stored separately for comparison with the corresponding resource limits when forming the merged set later.

[0086] Based on the object's business configuration, the object type in the data source, and runtime interaction requirements, interaction flags are set for each original object. These interaction flags include object-level query flags, local highlight flags, edit flags, alarm display flags, and independent update flags. After completing the above processing, a unified intermediate representation of each original object is obtained.

[0087] S102. Classify the original object according to the interaction flag.

[0088] like Figure 2 As shown, the original objects are classified in the order of forced independent objects, conditional mergeable objects, and mergeable objects.

[0089] First, check the edit flag, alarm display flag, and independent update flag. If any of these flags is valid, the corresponding original object will be classified as a forced independent object. For example, turnstiles that need to change position or orientation independently, alarm devices that need to change display status in real time, and movable components that need to update their geometric data independently are all treated as forced independent objects. Forced independent objects maintain independent geometric resources and rendering nodes and do not participate in subsequent candidate combination generation and merging scoring.

[0090] For original objects not classified as mandatory independent objects, continue checking object-level query flags and local highlight flags. When at least one flag is valid, the corresponding original object is classified as a conditionally merged object. For example, wall, beam, column, and pipeline components that need to be queried based on object identifiers or need to be highlighted individually after selection can be classified as conditionally merged objects.

[0091] For original objects that do not have the aforementioned interaction requirements, they are classified as mergeable objects. For example, decorative components or static auxiliary models that are only displayed as scene backgrounds and do not require object-level operations can be classified as mergeable objects.

[0092] For conditional merging objects, a selection proxy is generated before performing model merging. The selection proxy uses the original bounding volume of the conditional merging object, or a low-facet closed mesh generated from the original geometry data. The selection proxy stores the object identifier and is used for ray intersection and object localization at runtime. Simultaneously, the object-level high-precision resources of the conditional merging object are preserved, and an initial record of the business attribute mapping is established. At this point, the business attribute mapping contains the object identifier, object attributes, original bounding volume, selection proxy identifier, and high-precision resource references; the geometry merging mapping identifier, merged object identifier, and level-of-detail resource references are reserved for later processing and will be written later.

[0093] S103. Generate candidate combinations and form a merged set.

[0094] Continue to refer to Figure 2 A spatial index is created based on the spatial bounding volumes of mergeable objects and mergeable objects according to the conditions.

[0095] This embodiment uses a three-dimensional uniform grid as a spatial index to divide the project's local coordinate system into several spatial units. The side length of each spatial unit is determined based on the candidate radius and can be set to be equal to the candidate radius. Each original object is written into the spatial unit covered by its bounding volume. When retrieving candidate objects for a given original object, the original objects in the spatial unit containing that original object and its adjacent spatial units are read.

[0096] Calculate the minimum distance between the spatial bounding volumes of two original objects. When the minimum distance is not greater than the candidate radius, the two original objects are grouped into a candidate combination. In this embodiment, the candidate radius can be set to 20 meters; for indoor components or scenarios with high object density, the candidate radius can be reduced according to the object density in the spatial unit.

[0097] Perform a compatibility constraint check on each candidate combination. Exclude the corresponding candidate combination if any of the following conditions exist:

[0098] (1) The original object's coordinate transformation matrix is ​​invalid, or the transformed vertex coordinates exceed the preset valid range;

[0099] (2) The rendering queues, depth write states, or depth test states of the two original objects are incompatible;

[0100] (3) The two original objects have different update boundaries, and after merging, they cannot be processed according to their respective data update cycles;

[0101] (4) The original object contains skeletal animation, vertex animation, or other data that is not suitable for static geometry merging;

[0102] (5) The number of vertices, indexes, data volume, texture atlas area or spatial span after merging exceed the corresponding resource limit.

[0103] In this embodiment, the maximum number of vertices in a single merged object can be set to 500,000, the maximum number of indices to 1,500,000, the maximum resource data size to 64MB, the maximum texture atlas resolution to 4096×4096, and the maximum spatial span to 200 meters. These values ​​are adjustable based on the memory, video memory, and graphics interface capabilities of the running device and are not fixed limitations.

[0104] For candidate combinations that pass the compatibility constraint check, spatial proximity, material similarity, texture compatibility, attribute compatibility, query independence penalty, and merging resource cost are determined respectively, and each parameter is normalized to [0, 1].

[0105] Spatial proximity is determined based on the minimum distance between two spatial bounding volumes. When the two spatial bounding volumes intersect, the spatial proximity is 1; when they do not intersect, the spatial proximity decreases as the minimum distance increases; when the minimum distance reaches the candidate radius, the spatial proximity is 0.

[0106] Material similarity is determined based on normalized material parameters. The shading model, base color, metallicity, roughness, transparency, and self-illumination parameters are compared. Material similarity is high when the shading model and transparency are identical and the differences in other parameters are small.

[0107] Texture compatibility is determined based on texture format, channel semantics, color space, and sampling method. When the texture format and usage can be directly unified, a higher texture compatibility value is used; when textures need to be format converted before being written to the same texture atlas, an intermediate value is used; when the channel semantics or color space are incompatible, candidate combinations are not included in texture merging, and the texture compatibility is set to 0.

[0108] Attribute compatibility is determined based on the attribute structure and update method. When two original objects have the same or a unified attribute field structure, and attribute updates do not cause independent replacement of geometric resources, the attribute compatibility value is higher.

[0109] The query independence penalty is determined based on object-level query flags and local highlighting flags. The more objects the candidate combination includes, or the more stringent the object-level operation requirements, the greater the query independence penalty.

[0110] The resource cost of merging is calculated by evaluating the utilization rate of the number of vertices, indexes, data volume, texture atlas area, and spatial span relative to the corresponding resource limits, and then taking the maximum value among the utilization rates.

[0111] The combined score between the i-th original object and the j-th original object in the candidate combination is calculated according to the following formula:

[0112]

[0113] Among them, M ij The combined score between the i-th original object and the j-th original object; p ij m represents spatial proximity. ij For material similarity; u ij For texture compatibility; a ij For attribute compatibility; q ij In order to query independence penalty; z ij To consolidate resource costs; w p w m w u w a w q and w z These are the normalized weights for the corresponding parameters.

[0114] In this embodiment, w p w m w u w a w q and w z The values ​​can be set sequentially to 0.25, 0.20, 0.15, 0.15, 0.15, and 0.10, and the scoring threshold can be set to 0.45. For scenarios that emphasize object-level interaction, the weight corresponding to the query independence penalty can be increased; for running devices with small video memory capacity, the weight corresponding to the merged resource cost can be increased.

[0115] Construct a candidate graph using the original objects as nodes, candidate combinations that pass compatibility constraint checks as edges, and merging scores as edge weights. Arrange the candidate edges in descending order of merging scores, select a seed object from the unassigned original objects, and then try to add the candidate original objects adjacent to the seed object to the current merging set in turn.

[0116] For each candidate original object added, the cumulative number of vertices, cumulative number of indices, cumulative data volume, texture atlas area, and spatial span of the current merged set are recalculated.

[0117] After each candidate edge is selected, the cumulative number of vertices, cumulative indexes, cumulative data volume, texture atlas area, and spatial span of the object to be added to the current merge set are calculated. If the merging score of the candidate edge is not lower than the score threshold, and the cumulative resource volume of each item does not exceed the corresponding resource limit, the object to be added is added to the current merge set; otherwise, the candidate edge is marked as processed, and the next candidate edge is selected. This process is repeated until no candidate edge meets the score threshold and cumulative resource limit, at which point the current merge set is output.

[0118] S104, Perform geometry, material, and texture merging.

[0119] For each merged set, a merge object is created, and the coordinate system of the merge object is determined. The origin of the merge object coordinate system is set near the center of the union bounding volume of the merged set.

[0120] Let the transformation matrix from the original object's coordinate system to the target coordinate system be the original object transformation matrix, and the transformation matrix from the merged object's coordinate system to the target coordinate system be the merged object transformation matrix. Invert the merged object transformation matrix and combine it with the original object transformation matrix to obtain the transformation relationship from the original object's coordinate system to the merged object's coordinate system. Use this transformation relationship to transform the vertex positions of the original object. For vertex normals, use the inverse transpose matrix corresponding to the linear part of the transformation relationship for transformation, and then normalize the transformed normals.

[0121] The geometric data of each original object in the merge set is written to the vertex buffer and index buffer of the merged object in a predetermined order. Before writing the geometric data of each original object, the current vertex buffer position is recorded as the vertex starting position, and the current index buffer position is recorded as the index starting position.

[0122] When writing index data to the original object, the corresponding vertex start position is added to each index. After writing is complete, the vertex interval, index interval, triangle interval, and sub-mesh interval corresponding to the original object are determined based on the amount of data written. All intervals are recorded using their start positions and the number of elements to avoid discrepancies arising from different interpretations of closed or open intervals.

[0123] Material feature identifiers are generated based on the shading model, transparency state, double-sided state, material parameters, and texture semantics. Materials with the same material feature identifier share the same material slot; materials with different material feature identifiers are assigned separate material slots.

[0124] For textures compatible in format, channel semantics, color space, and sampling method, a rectangular region binning method is used to write them into the texture atlas. After the texture is written, its starting coordinates, width, and height in the texture atlas are recorded. Based on the texture atlas region, the original texture coordinates are scaled and offset, so that the merged vertices continue to sample the image region corresponding to the original texture. For textures that cannot be written into the same texture atlas, separate texture atlases are created or independent texture resources are retained.

[0125] During the geometry, material, and texture merging process, a geometry merging map is generated for each original object. This geometry merging map records the object identifier, merged object identifier, vertex intervals, index intervals, triangle intervals, sub-mesh intervals, material slots, texture atlas regions, the bounding volume in the merged object's local coordinate system, and the selection proxy identifier. When the geometric data of the same original object forms multiple discontinuous intervals in the merged object, each geometric interval is recorded separately; alternatively, a selection mask is generated based on the object identifiers corresponding to each triangle in the merged object, and the selection mask identifier is recorded in the geometry merging map. Thus, the geometric data position and material / texture position of the original object within the merged object can be determined based on the object identifier.

[0126] S105. Establish and maintain business attribute mapping and geometric merge mapping.

[0127] like Figure 3 As shown, the business attribute mapping uses the object identifier as the primary key, with one end associated with the object attributes and the original bounding body of the original object, and the other end associated with the geometric merge mapping identifier, the merged object identifier, resource references at all levels of detail, and high-precision resource references.

[0128] Business attribute mappings and geometry merge mappings are stored separately to avoid regenerating geometric resources when object attributes are updated. Business attribute mappings are used to find the attributes and resources of a business object, while geometry merge mappings are used to determine the specific geometric region of the object within the merged resource.

[0129] After object classification is completed, an initial record for business attribute mapping is first established; after geometry, material, and texture merging are completed, the geometry merging mapping identifier and the merged object identifier are written into the corresponding business attribute mapping; after multi-level detail layer resource generation is completed, the references of each level of detail layer resource are written into the business attribute mapping.

[0130] For a conditional merge object, its business attribute mapping can record the object identifier "BuildingA-F2-Wall0031", wall type and floor level, original bounding volume, geometric merge mapping identifier "Map-0086", merge object identifier "Merge-0012", resource references at various levels of detail, and object-level high-precision resource references. The geometric merge mapping "Map-0086" records the vertex range, index range, material slot, and texture atlas area of ​​the wall in the merge object "Merge-0012".

[0131] Version identifiers and checksums are set for business attribute mappings, geometric merging mappings, and resource files of merged objects, respectively. Version identifiers within the same resource package have a pre-defined correspondence. After resource generation is complete, checksums for business attribute mappings, geometric merging mappings, and resource files are calculated, and version identifiers and checksums are written to the corresponding files.

[0132] When reading the above data at runtime, the correspondence between each version identifier and the corresponding check value are verified. If the version identifier or check value does not match, the location of the geometric range based on the geometric merge mapping is stopped, and the business attribute mapping, geometric merge mapping and resource files that match the version are reloaded to prevent the location of the wrong geometric area based on the expired geometric merge mapping.

[0133] When the geometric data, material data, interaction flags, or update method of the original object changes, the affected range is determined based on the spatial unit where the original object is located. Only the original objects within the affected range are reclassified, candidate combination generated, and merged into a set. After the new resource is generated and verified, the new resource version replaces the original resource version. The old version is retained during the replacement process until the new version switch is completed at runtime to avoid inconsistencies between resource files and mapping files.

[0134] S106, Generate multi-level detail resources

[0135] Continue to refer to Figure 3 For each merged object, multi-level detail resources are generated. In this embodiment, the merged original precision resources are used as the level 0 detail resource, and the geometric data is simplified step by step using an edge folding method based on a quadratic error metric to obtain level 1, level 2 and level 3 detail resources.

[0136] For example, the number of target triangles for detail levels 0 to 3 can be set to 100%, 50%, 20%, and 8% of the original number of triangles, respectively. Texture resources at each level can be set to the original resolution, half, a quarter, and an eighth of the original resolution, respectively. The specific number of levels and simplification ratios are determined based on the scene size and the performance of the running device.

[0137] When performing edge collapse, the object identifier of the source object is attached to the triangle or vertex. For conditional merge objects within the preset query level, cross-object edge collapse is not performed on vertices belonging to different object identifiers to preserve the geometric boundaries between conditional merge objects. After completing a certain level of geometric simplification, the triangle indices are reordered according to the object identifier, and the index range and submesh range of the conditional merge object in the resource of that level of detail are recorded.

[0138] When a certain level of detail resource no longer requires the retention of object-level geometric ranges, cross-object edge collapsing is allowed, but the selection proxy and high-precision resource references of the conditionally merged object are still retained. In this embodiment, levels 0 to 2 are set as preset query levels, and the geometric ranges or selection proxies of the conditionally merged object are retained in levels 0 to 2; level 3, as a remote display resource, may not retain the complete object-level geometric range, but the corresponding high-precision resource references are still saved in the business attribute mapping.

[0139] Geometric error is determined based on the maximum spatial deviation of the simplified surface relative to the original surface, while texture error is determined based on the color difference between the downsampled texture restored to its original resolution and the original texture. The geometric error, texture error, data size, and resource references of each level of detail resource are written into the corresponding resource description record.

[0140] In this embodiment, the geometric screen error of the k-th level of detail resource is calculated according to the following formula:

[0141]

[0142] Among them, SSE k The geometric screen error of the k-th level of detail resource is expressed in pixels. denoted as the geometric error of the k-th level of detail resource; H is the viewport pixel height; d is the distance from the camera to the bounding volume of the merged object space; and θ is the camera's vertical field of view.

[0143] The texture error of the k-th level texture resource is determined based on the color difference between the texture at this level, after being upsampled and restored to its original resolution, and the original texture. This texture error is normalized to [0, 1]. The product of the width and height of the k-th level texture is determined as the number of texture pixels. The square root of the ratio of the projected area of ​​the merged object in the viewport to the number of texture pixels is used to obtain the average screen coverage scale of a single texture pixel. This average screen coverage scale is multiplied by the texture error of the k-th level texture resource to obtain the texture screen error of the k-th level detail resource. The lower the texture resolution, the larger the projected area, or the larger the texture error, the larger the texture screen error.

[0144] Among the level-of-detail resources where both geometric screen error and texture screen error do not exceed their respective error thresholds, the level-of-detail resource with the smallest data size is selected. In this embodiment, the geometric screen error threshold can be set to 1.5 pixels, and the texture screen error threshold can be set to 1 pixel.

[0145] To avoid repeated resource switching when the camera moves near the layer transition location, different entry and exit thresholds are set for adjacent layers. For example, the entry threshold for switching from a high-precision layer to a low-precision layer is set to 0.8 pixels, and the exit threshold for returning from a low-precision layer to a high-precision layer is set to 1.2 pixels. Between the two thresholds, the current level of detail resource is kept unchanged.

[0146] S107. Select the level of detail resource and respond to object interaction requests.

[0147] like Figure 4 As shown, the runtime renderer loads the spatial bounding volume, business attribute mapping, geometric merge mapping, and resource description information at each level of detail of the merged object.

[0148] When the camera position, field of view, or viewport size changes, calculate the distance between each merged object and the camera, the projected area of ​​the merged object in the viewport, and the geometric screen error and texture screen error of each level of detail resources, and select the level of detail resources for rendering in the manner described in step S106.

[0149] When a user issues an object query or a local highlight request, the original bounding volumes of each object are first used to coarsely filter the interaction location, excluding objects irrelevant to the interaction location. Subsequently, the object identifiers of the target condition-merged objects are obtained through the object identifier buffer or the intersection results of the selected proxy's rays.

[0150] When using an object identifier buffer, an encoded value corresponding to the object identifier is assigned to the conditionally merged object. During the drawing of the object identifier buffer, this encoded value is written based on the index range of the conditionally merged object, a selection mask, or a selection proxy. After reading the encoded value at the interaction location, the corresponding object identifier can be obtained.

[0151] When using a selection agent to perform ray intersection, a picking ray is generated based on the camera position and interaction position. The intersection point between the picking ray and the original bounding volume is first calculated. Then, an exact intersection is performed on the selection agent that has passed the coarse screening, and the object corresponding to the intersection point closest to the camera is determined as the target condition merge object.

[0152] Based on the object identifier of the target condition merged object, query the business attribute mapping to obtain the object attributes, original bounding volume, merged object identifier, and geometric merge mapping identifier. Then, read the corresponding geometric merge mapping based on the geometric merge mapping identifier, and after version verification, obtain the geometric sub-region of the target condition merged object within the merged object.

[0153] When the current level-of-detail resource retains the index range, sub-mesh range, or selection mask of the target conditional merged object, the additional draw call only draws the aforementioned index range, sub-mesh range, or selection mask. The additional draw call uses highlight materials, outline materials, or stencil buffers to distinguish the target conditional merged object from other objects in the merged object. Object query requests can also directly return object attributes stored in the business attribute mapping.

[0154] When the current level-of-detail resource does not retain the geometric sub-region of the target conditional merging object, the object-level high-precision resource of the target conditional merging object is loaded according to the high-precision resource reference in the business attribute mapping, and superimposed onto the current scene according to its original coordinate transformation relationship. When the object-level high-precision resource does not exist or cannot be loaded independently, the level 0 or other high-precision level-of-detail resource of the merging object containing the target conditional merging object is loaded, and then local highlighting is performed according to the geometric interval in the high-precision resource.

[0155] During high-precision resource loading, a temporary highlighted outline can be displayed using the original bounding volume or by selecting a proxy; after the high-precision resource loading is complete, the temporary highlighted outline is replaced with the actual geometric region of the target condition merged object.

[0156] After the interaction request ends, the additional drawing for the target condition merged object is canceled. High-precision resources loaded during the interaction can be released immediately or retained in the cache for a preset cache duration. Then, based on the camera state, geometric screen error, and texture screen error, detail level resources are selected again to restore normal rendering of the merged object.

[0157] Through the above processing, this embodiment reduces the number of scene nodes, drawing calls, and the amount of high-precision resources that remain in the environment, while preserving the geometric and attribute correspondence between the original and merged objects. When the low-precision resource still retains the geometric sub-region of the target object, the geometric sub-region can be directly queried or locally highlighted; when the low-precision resource no longer retains the geometric sub-region, the corresponding high-precision resources can be loaded on demand according to the business attribute mapping, thereby balancing the rendering efficiency of the 3D scene and the object-level interaction capability.

[0158] The above description is merely a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Equivalent substitutions or conventional changes made by those skilled in the art based on the disclosure of the present invention should all fall within the scope of protection of the present invention.

Claims

1. A method for dynamic merging and multi-level detail optimization of 3D scene models, characterized in that, include: The original objects of multi-source 3D data are transformed into a unified intermediate representation that includes object identifiers, geometric data, material data, texture data, spatial bounding volume, object attributes, interaction flags, and object resource costs. Based on the interaction flags, the original objects that need to be updated, edited, or displayed with alarms independently are classified as mandatory independent objects; the original objects that do not have the aforementioned requirements but need to be queried at the object level or highlighted locally are classified as conditional mergeable objects; and the rest are mergeable objects. Using the object identifier as the primary key, establish a business attribute mapping for the merged objects, including record object attributes, the original bounding body, and high-precision resource references. Candidate combinations are generated for conditional merging objects and merging objects. Candidate combinations that do not meet coordinate transformation relationships, rendering status or resource limit constraints are excluded. A merging score is calculated based on spatial proximity, material similarity, texture compatibility, attribute compatibility, query independence penalty and merging resource cost. A merging set is formed based on the score threshold and cumulative resource limit. Perform geometry, material, and texture merging on the merged collection. During merging, record the geometric range and material region of each original object in the merged object, generate a geometry merge mapping, and write the geometry merge mapping identifier and the merged object identifier into the business attribute mapping. Generate multi-level detail resources for the merged object and write resource references into the corresponding business attribute mapping; At runtime, the system selects the level of detail resource based on the camera status and screen error; when responding to an interaction request of a conditional merge object, it returns the object attributes based on the business attribute mapping and determines its geometric sub-region based on the geometric merge mapping; when the interaction request includes local highlighting and the current level of detail resource does not retain the geometric sub-region, it loads the corresponding resource based on the high-precision resource reference and performs local highlighting using the geometric sub-region or the high-precision resource.

2. The method for dynamic merging and multi-level detail optimization of a 3D scene model according to claim 1, characterized in that, Converting the original objects of multi-source 3D data into a unified intermediate representation includes: Analyze multi-source 3D data according to the data source type to obtain the geometric data, material data, texture data and hierarchical path of the original object; Generate a coordinate transformation matrix based on the source coordinate reference, source coordinate origin, and source units. Based on the coordinate transformation matrix, transform the geometric data of the original object to the project's local coordinate system using metric units. The stable identifier of the original object is used as the object identifier; if the original object does not have a stable identifier, the object identifier is generated based on the data source identifier, the normalized hierarchical path, and the local identifier of the source object. Calculate the spatial bounding volume in the project's local coordinate system, and determine the object resource cost based on the number of vertices, indexes, texture data volume, and data package size.

3. The method for dynamic merging and multi-level detail optimization of a 3D scene model according to claim 1, characterized in that, The interactive flags include object-level query flag, local highlight flag, edit flag, alarm display flag, and independent update flag; Object classification is performed in the order of mandatory independent objects, conditionally merged objects, and mergeable objects. Original objects that have been classified as mandatory independent objects do not participate in candidate combination generation and merging scoring. For conditional merge objects, a selection proxy for object picking is generated before the merge is executed, and an initial record of business attribute mapping containing object identifier, object attributes, original bounding body and object-level high-precision resource references is established.

4. The method for dynamic merging and multi-level detail optimization of a 3D scene model according to claim 1, characterized in that, Generate candidate combinations, including: A spatial index is established based on the spatial bounding volume of the original object, the original object is written into the corresponding spatial cell, and the original object in the spatial cell and its adjacent spatial cells is retrieved. When the spatial bounding volume distance between two original objects is not greater than the candidate radius, the two original objects are combined into a candidate combination. Perform compatibility constraint checks on candidate combinations. The compatibility constraints include valid coordinate transformation relationships, compatible rendering queues and depth states, compatible object update boundaries, the ability of the geometric data of each original object to be converted to the same coordinate system, and that the number of vertices, indexes, data volume, texture atlas area, and spatial span of the merged combination do not exceed the corresponding resource limits.

5. The method for dynamic merging and multi-level detail optimization of a 3D scene model according to claim 4, characterized in that, Spatial proximity, material similarity, texture compatibility, attribute compatibility, query independence penalty, and merging resource cost are normalized to [0,1], and the merging score between two original objects in the candidate combination is calculated according to the following formula: Among them, M ij The combined score between original object i and original object j; p ij m represents spatial proximity. ij For material similarity; u ij For texture compatibility; a ij For attribute compatibility; q ij In order to query independence penalty; z ij To consolidate resource costs, this is used to characterize the expected resource quantity of the merged two original objects relative to their corresponding resource caps; w p w m w u w a w q and w z These are the normalized weights of the corresponding factors, and they satisfy: In p +in m +in u +in a +in q +in z =1 Spatial proximity is determined based on the spatial bounding volume distance between the two original objects and the candidate radius; material similarity is determined based on normalized material parameters; texture compatibility is determined based on texture format, channel semantics, color space, and sampling method; attribute compatibility is determined based on attribute structure and update method; query independence penalty is determined based on object-level query flags and local highlight flags. The merged resource cost is the maximum value among vertex utilization rate, index utilization rate, data volume utilization rate, texture atlas area utilization rate, and spatial span utilization rate. Each resource utilization rate is the ratio of the expected resource volume after merging to the corresponding resource limit.

6. The method for dynamic merging and multi-level detail optimization of a three-dimensional scene model according to claim 5, characterized in that, The merged set includes: Construct a candidate graph using the original objects as nodes, candidate combinations as edges, and merge scores as edge weights; Select the unassigned original object as the seed object, and establish the current merge set containing the seed object; From candidate edges whose one end node belongs to the current merge set and whose other end node is not assigned, select the candidate edge with the highest merge score, and determine the unassigned node of the candidate edge as the object to be added; Calculate the cumulative number of vertices, cumulative number of indices, cumulative data volume, texture atlas area, and spatial span of the objects to be added after they are added to the current merge set; When the merging score of the candidate edge reaches the scoring threshold and the cumulative resource amount of each item does not exceed the corresponding resource limit, the object to be added is added to the current merging set, and the next candidate edge is selected. When the merged score of the candidate edge is lower than the score threshold, or when the cumulative resource amount of any object to be added exceeds the corresponding resource limit, the candidate edge is marked as processed, and the next candidate edge is selected. When there are no candidate edges that meet the scoring threshold and the cumulative resource limit, output the current merge set and reselect seed objects from the remaining unallocated original objects.

7. The method for dynamic merging and multi-level detail optimization of a 3D scene model according to claim 1, characterized in that, Perform geometry, material, and texture merging on the merged collection, including: Based on the coordinate transformation relationship between the local coordinate system of the original object and the local coordinate system of the merged object, the vertices of the original object are transformed to the local coordinate system of the merged object, and the inverse of the linear transformation matrix corresponding to the coordinate transformation relationship is used to transform the normal of the original object. Before writing the geometric data of each original object, record the vertex start position and index start position; write the vertices of the original objects into the vertex sequence of the merged object, and add the vertex start position to each index value of the original object; after writing, record the vertex interval, index interval, triangle interval and submesh interval corresponding to the original object. Material feature identifiers are generated based on the shading model, transparency state, double-sided state, material parameters, and texture semantics. Materials with the same material feature identifiers are merged into the same material instance. Write textures with compatible texture formats, channel semantics, color spaces, and sampling methods into a texture atlas, and offset and scale the texture coordinates of the original object according to the region of the texture in the texture atlas to obtain the merged texture coordinates; A geometric merge mapping is generated, which records the object identifier, merged object identifier, vertex interval, index interval, triangle interval, submesh interval, material slot, texture atlas region, and bounding volume in the local coordinate system of the merged object; for conditional merged objects, the geometric merge mapping also records the selection proxy identifier; when the geometric data of the original object is not continuous in the merged object, the geometric merge mapping records multiple geometric intervals or selection mask identifiers.

8. The method for dynamic merging and multi-level detail optimization of a 3D scene model according to claim 7, characterized in that, The business attribute mapping establishes an initial record after object classification is completed, writes a geometry merging mapping identifier after geometry, material and texture merging is completed, and writes resource references at each level after multi-level detail layer resource generation is completed. The business attribute mapping and geometric merging mapping are associated through object identifiers and geometric merging mapping identifiers; The business attribute mapping, geometry merging mapping, and resource files of the merged objects are each set with matching version identifiers and verification values; When a version identifier or checksum mismatch is detected, stop locating the geometric range based on the geometric merge mapping, and load the business attribute mapping, geometric merge mapping, and resource files that match the version.

9. The method for dynamic merging and multi-level detail optimization of a 3D scene model according to claim 1, characterized in that, Generate multi-level detail resources for the merged objects, including: The geometric data of the merged object is simplified step by step by using an edge folding method based on a quadratic error metric, and the texture data is downsampled step by step to determine the geometric and texture errors of the resources at each level of detail. Within the preset query level corresponding to the condition merging object, retain the geometric range or selection proxy of the condition merging object, and retain the high-precision resource reference of the condition merging object in the business attribute mapping; The geometric screen error of each level of detail resource is determined based on the geometric error of the resource at each level of detail, the viewport pixel height, the distance from the camera to the bounding volume of the merged object space, and the vertical field of view of the camera. The texture screen error of each level of detail resource is determined based on the texture error of each level of detail resource, the projected area of ​​the merged object in the viewport, and the texture resolution. Among the detail level resources where both geometric screen error and texture screen error do not exceed the corresponding error threshold, the detail level resource with the smallest data volume is selected, and different entry and exit thresholds are set for adjacent levels.

10. The method for dynamic merging and multi-level detail optimization of a three-dimensional scene model according to claim 8, characterized in that, The response condition merge object's interactive request includes: The interaction locations are initially screened by the spatial bounding volume, and the object identifiers of the target condition merged objects are determined by the intersection results of the object identifier buffer or the selected agent's ray. The business attribute mapping is queried based on the object identifier, the object attributes of the target condition merged object are returned, the location of the target condition merged object is determined based on the original bounding volume, and the geometric merge mapping is queried based on the geometric merge mapping identifier. When the current level of detail resource retains the index range, sub-mesh range, or selection mask of the target condition merged object, additional drawing is performed only on the index range, sub-mesh range, or selection mask; When the current level-of-detail resource does not retain the geometric sub-region of the target conditional merged object, load the object-level high-precision resource of the target conditional merged object and perform additional drawing on the object-level high-precision resource; When the object-level high-precision resource does not exist or cannot be loaded independently, the high-precision detail level resource of the merged object where the target condition merged object is located is loaded, and the geometric sub-region of the target condition merged object is determined according to the geometric merge mapping, and additional drawing is performed on the determined geometric sub-region. After the interaction request ends, undo the additional drawing and restore the level of detail resource selected based on camera status and screen error.

Images