A document online rendering method and system based on a 3D lightweight model

Abstract

The application discloses a kind of based on 3D light weight model's document online rendering method and system, it is related to three-dimensional modeling technical field;The method comprises: obtaining original 3D model data;Original 3D model data is carried out global structured progressive simplification processing and obtains initial simplified 3D model data;Initial simplified 3D model data is carried out vertex data simplification and obtains simplified 3D model data;Simplified 3D model data is carried out isomorphism and is merged and obtains light weight 3D model data;The association index of online document structure and light weight 3D model data is established and light weight 3D model is embedded in online document, and light weight 3D model is loaded in response to the access request of client to online document, and light weight 3D model is rendered and displayed in document according to association index.The application guarantees the structural integrity and rendering effect of model, reduces data storage and transmission cost, improves the access efficiency and user experience of online document containing 3D model.

Description

Technical Field

[0001] This invention belongs to the field of 3D modeling technology, specifically relating to an online document rendering method and system based on a lightweight 3D model. Background Technology

[0002] 3D models are widely used in industrial design, digital twins, online document interaction, digital display and other scenarios. Their lightweight and efficient rendering technology can significantly reduce data transmission and loading costs, allowing complex 3D models to be presented smoothly on various terminal devices. This not only helps product design, engineering education and other fields to achieve more intuitive visual interaction, but also promotes real-time data synchronization and scene restoration in digital twin scenarios, providing technical support for online collaboration, virtual display and other applications, and improving user experience and work efficiency.

[0003] Patent application CN121708228A discloses a lightweight hierarchical digital twin modeling and rendering method for large-scale scenes. The method includes: analyzing the surface geometry of the original high-precision 3D model, calculating and extracting the curvature features of each local region corresponding to the high-precision 3D model, and constructing a curvature response distribution based on the curvature features; filtering and determining each model region corresponding to the original high-precision 3D model based on the obtained curvature response distribution and dynamic curvature threshold to obtain a second key detail region; generating independent multi-resolution detail enhancement data blocks for the second key detail region, and generating multiple data versions with decreasing detail levels for the same second key detail region to form a directly indexable multi-resolution detail library; and in the real-time rendering stage, seamlessly integrating the detail data blocks with the basic main model to achieve a balance between overall scene lightweighting and high-fidelity local details.

[0004] However, this method relies on a simplification approach based on local geometry, focusing only on the analysis and processing of the surface geometric features of the 3D model. It has a low utilization rate of the overall structural information of the model, making it difficult to maintain the fidelity of the geometric structure and topological validity during the model simplification process. This can easily lead to problems such as model structural distortion and broken topological connections. Summary of the Invention

[0005] The purpose of this invention is to solve the problem that existing lightweight 3D model processing methods based on local geometric features cannot effectively balance data compression efficiency and structural fidelity during model simplification. Therefore, this invention proposes an online document rendering method and system based on lightweight 3D models.

[0006] In a first aspect of this invention, a method for online document rendering based on a 3D lightweight model is first proposed, the method comprising:

[0007] Obtain the original 3D model data, convert all faces of the original 3D model data into a triangular mesh, and then extract the planar primitives to obtain the initial segmentation plane set;

[0008] The initial segmented plane set is optimized and clustered using a region growing algorithm, and the plane region set is obtained by fitting and optimizing the parameters of each plane using principal component analysis.

[0009] Based on the set of planar regions, spatial topological associations between planes are identified, and a global attribute connection graph is constructed with planes as nodes and topological association types as edges; the topological association types include adjacency type and intersection type;

[0010] The original 3D model data is structurally and hierarchically simplified based on the global attribute connection graph to obtain initial simplified 3D model data;

[0011] The initial simplified 3D model data is simplified by reducing vertex data to obtain simplified 3D model data; the initial simplified 3D model data includes texture primitives and corresponding UV coordinates and normal data;

[0012] The simplified 3D model data is homogeneously merged to obtain lightweight 3D model data;

[0013] An association index is established between the online document structure and the lightweight 3D model data, and the lightweight 3D model is embedded in the online document. Then, in response to the client's access request to the online document, the lightweight 3D model is loaded, and the lightweight 3D model is rendered and displayed in the document according to the association index.

[0014] Optionally, the initial simplified 3D model data is obtained by performing structured hierarchical simplification on the original 3D model data based on the global attribute connection graph, including:

[0015] The global attribute connection graph is decomposed into substructure sets.

[0016] The original 3D model data is used to perform multi-view contour fusion modeling to obtain a visual 3D mesh;

[0017] The initial simplified 3D model data is obtained by hierarchically simplifying the visual 3D mesh based on the substructure set.

[0018] Optionally, the structured decomposition of the global attribute connection graph to obtain a set of substructures includes:

[0019] Remove the intersecting edges from the global attribute connection graph to obtain the first substructure set and the substructure set to be processed;

[0020] The second substructure set is obtained by splitting each substructure in the set of substructures to be processed.

[0021] The first substructure set and the second substructure set are merged to obtain a substructure set.

[0022] This scheme achieves initial structural separation by removing intersecting edges from the global attribute connection graph. It then refines the remaining structures to be processed, and finally merges the two sets of results to obtain a substructure set. This achieves ordered decomposition and classification integration of complex model structures, effectively eliminating intersecting interference factors between structures and further subdividing complex substructures, making the substructure division more closely reflect the geometric and attribute characteristics of the actual model.

[0023] Optionally, splitting each substructure in the set of substructures to be processed to obtain the second substructure set includes:

[0024] Step 1: Construct a second substructure set with an initial empty state;

[0025] Step 2: Select a substructure from the set of substructures to be processed as the target substructure, and remove the target substructure from the set of substructures to be processed;

[0026] Step 3: Starting from a preset cut point threshold, increase the number of cut points sequentially to perform cut point segmentation on the target substructure.

[0027] Step 4: If the current number of cut points can split the target substructure, then stop increasing the number of cut points, obtain an intermediate substructure set, and update all substructures of the intermediate substructure set to the substructure set to be processed.

[0028] Step 5: If increasing the number of cut points to the preset cut point threshold still fails to split the target substructure, then add the target substructure to the second substructure set;

[0029] Step 6: Return to Step 2 until there are no more substructures in the set of substructures to be processed, and finally obtain the second set of substructures.

[0030] This solution iteratively selects target substructures from the set of substructures to be processed, and attempts to divide them step by step with a preset number of cut points as the upper limit. After splitting the divisible substructures, the solution updates them back to the set of substructures to be processed, and classifies the indivisible substructures into the second set of substructures. This achieves hierarchical processing and classification of complex substructures, which can not only perform fine decomposition of divisible substructures, but also retain the indivisible key structures, thus improving the flexibility and targeting of substructure processing.

[0031] Optionally, performing multi-view contour fusion modeling on the original 3D model data to obtain a visual 3D mesh includes:

[0032] Step 1: Determine multiple viewpoint directions based on the normal directions of each plane in the original 3D model to obtain a set of viewpoint directions;

[0033] Step 2: Project the original 3D model onto each viewpoint direction in the viewpoint direction set and simplify it to obtain a two-dimensional simplified contour set;

[0034] Step 3: Using the viewpoint directions in the viewpoint direction set as the stretching direction, extend each two-dimensional simplified contour in the two-dimensional simplified contour set into three dimensions along the stretching direction to obtain the original three-dimensional mesh set.

[0035] Step 4: Perform a Boolean cross operation on each original 3D mesh in the original 3D mesh set to obtain a merged 3D mesh;

[0036] Step 5: Render the fused 3D mesh according to each viewpoint direction in the viewpoint direction set to obtain a mask image set;

[0037] Step 6: Calculate the visual similarity between each mask image in the mask image set and the original 3D model image under the corresponding viewpoint direction to obtain a visual similarity set;

[0038] Step 7: Determine whether each visual similarity in the visual similarity set reaches the preset similarity threshold. If not, use the fused 3D mesh as the new original 3D model and return to step 1 until all visual similarities reach the preset threshold, and finally obtain the visual 3D mesh.

[0039] This solution starts with the multi-view projection contours of the original 3D model, and obtains an initial 3D mesh through stretching and Boolean cross-fusion. Then, iterative optimization is performed based on the visual similarity between the rendered mask images of each view and the original model. This can quickly construct a 3D mesh that is highly consistent with the visual effect of the original model in each view without the need for complex 3D reconstruction algorithms, which greatly reduces the computational complexity of 3D model reconstruction, while ensuring the visual fidelity of the reconstruction results in different viewpoints.

[0040] Optionally, the initial simplified 3D model data is obtained by hierarchically simplifying the visual 3D mesh according to the substructure set, including:

[0041] Calculate the depth loss between the substructures in the substructure set and the visual 3D mesh, and filter the substructure set according to the depth loss to obtain the substructure set to be processed;

[0042] Spatial geometric matching is performed between each substructure to be processed in the set of substructures to be processed and the visual 3D mesh to obtain the set of primitives to be processed;

[0043] A corrected 3D mesh is obtained by iterative geometric correction of the visual 3D mesh based on the set of primitives to be processed;

[0044] Extract the planes of the corrected 3D mesh, calculate the number of adjacent planes for each vertex in the corrected 3D mesh based on the extracted planes, and assign simplified weights to each vertex according to the number of adjacent planes to obtain a set of simplified vertex weights;

[0045] A quadratic error metric algorithm based on the optimized vertex simplified weight set is used to perform edge folding simplification on the corrected 3D mesh to obtain the initial simplified 3D model data.

[0046] This scheme achieves accurate correction of the visual 3D mesh through depth loss filtering, spatial geometry matching, and iterative correction. Then, it combines the simplified weighting of the number of adjacent planes of vertices to optimize the secondary error measurement algorithm and fold the edges of the 3D mesh to simplify it. This not only effectively corrects the geometric deviation of the original mesh, but also retains the key features of the model during the simplification process, avoiding the oversimplification of key structures. It achieves a balance between the geometric accuracy and lightweight effect of the 3D mesh, and provides high-quality initial simplified data for subsequent 3D model processing.

[0047] Optionally, simplifying the initial simplified 3D model data by reducing vertex data includes:

[0048] The texture primitives in the initial simplified 3D model data are classified to identify solid color texture types and gradient texture types, thus obtaining an initial set of texture primitives to be replaced.

[0049] Calculate the visual saliency of each texture primitive in the original 3D model in the initial set of texture primitives to be replaced, and filter out texture primitives with visual saliency lower than a preset threshold to obtain the set of texture primitives to be replaced;

[0050] The vertex color attribute set is obtained by generating differentiated colors for the texture primitives in the set of texture primitives to be replaced according to their types.

[0051] The original texture mapping of the texture primitive to be replaced is replaced according to the vertex color attribute set, and the UV coordinates and normal data of the corresponding vertex of the texture primitive to be replaced are deleted to obtain simplified 3D model data.

[0052] This solution classifies and filters texture primitives in 3D models based on their visual saliency. For solid color and gradient texture primitives with low saliency, it replaces the original texture mapping with a differentiated vertex color generation method and deletes the corresponding UV coordinates and normal data. Without affecting the core visual effect of the model, it significantly reduces the storage overhead of textures and related data, achieving efficient and lightweight processing of 3D models. At the same time, it reduces the data reading and calculation burden during model rendering and improves the loading and rendering efficiency of the model.

[0053] Optionally, the vertex color attribute set is obtained by generating differentiated colors for the texture primitives in the set of texture primitives to be replaced according to their types, including:

[0054] For the texture primitives in the set of texture primitives to be replaced, which are of the solid color texture type, the primary color value is extracted as the color value of each vertex in the corresponding primitive to obtain the solid color vertex attribute value set;

[0055] For the texture primitives in the set of texture primitives to be replaced, which are of the gradient texture type, the gradient color vertex attribute value set is obtained by interpolating the color value of each vertex based on the spatial mapping relationship between the gradient direction of the texture image and the vertex normal.

[0056] The vertex color attribute set is obtained by integrating the solid color vertex attribute value set and the gradient color vertex attribute value set.

[0057] This solution employs differentiated vertex color generation strategies for different types of texture primitives. For solid-color textures, the primary color value is extracted and directly assigned to the vertices. For gradient textures, the color is generated by interpolating the gradient direction and vertex normal. This not only accurately reproduces the visual characteristics of different texture types but also avoids the redundancy caused by directly storing a large amount of texture primitive data. By converting texture information into vertex color attributes, the amount of model data is effectively reduced, while ensuring the realism and detail of the 3D model in texture rendering and improving the visual quality of the lightweight model.

[0058] Optionally, isomorphically merging the simplified 3D model data to obtain lightweight 3D model data includes:

[0059] Traverse all triangular faces in the simplified 3D model data and cluster them based on the spatial connectivity between the triangular faces to obtain a set of connected components.

[0060] Calculate the bounding box size and number of vertices of each component in the connected component set, and group components with the same bounding box size and number of vertices into the isomorphic candidate group to obtain the isomorphic candidate group set;

[0061] For each candidate group in the isomorphic candidate group set, the relative distance matrix between the vertices of each component in the group is compared one by one, and the components with the same relative distance matrix are determined to be isomorphic components to obtain the isomorphic component set;

[0062] Select one component from each isomorphic component group in the isomorphic component group set as the prototype component, calculate the spatial transformation matrix between the remaining components in the isomorphic component group and the prototype component, and delete the triangular facet data of the remaining components in the isomorphic component group to obtain the prototype component set and the transformation matrix set.

[0063] The prototype component set and the transformation matrix set are combined and stored to generate lightweight 3D model data.

[0064] This solution performs connectivity clustering on triangular faces in a simplified 3D model, then selects isomorphic components based on bounding box size, number of vertices, and relative vertex distance matrix. Finally, it replaces the repetitive isomorphic component data with prototype components and spatial transformation matrices. This approach can significantly reduce redundant data storage while preserving the key geometric information and structural features of the model, effectively achieving lightweight processing of 3D models.

[0065] In a second aspect of this invention, an online document rendering system based on a 3D lightweight model is proposed, comprising:

[0066] The acquisition module is used to acquire the original 3D model data, convert all faces of the original 3D model data into a triangular mesh, and then extract the planar primitives to obtain the initial segmentation plane set.

[0067] The planar region generation module is used to optimize and cluster the initial segmented planar set using a region growing algorithm, and combine principal component analysis to fit and optimize the parameters of each planar region to obtain a planar region set.

[0068] The global graph generation module is used to identify spatial topological associations between planes based on the set of planar regions and construct a global attribute connection graph with planes as nodes and topological association types as edges; the topological association types include adjacency type and intersection type;

[0069] The structure simplification module is used to perform structured hierarchical simplification of the original 3D model data according to the global attribute connection graph to obtain initial simplified 3D model data;

[0070] The vertex simplification module is used to simplify the vertex data of the initial simplified 3D model data to obtain simplified 3D model data; the initial simplified 3D model data includes texture primitives and corresponding UV coordinates and normal data.

[0071] The isomorphic merging module is used to perform isomorphic merging on the simplified 3D model data to obtain lightweight 3D model data;

[0072] The document rendering module is used to establish an association index between the online document structure and the lightweight 3D model data, embed the lightweight 3D model into the online document, load the lightweight 3D model in response to the client's access request for the online document, and render and display the lightweight 3D model in the document according to the association index.

[0073] The beneficial effects of this invention are as follows: This solution effectively reduces the size of the model data by performing multi-stage structured simplification and isomorphic merging on the 3D model. Combined with the association indexing mechanism with online documents, it achieves efficient loading and rendering of lightweight models, which not only ensures the structural integrity and display effect of the model, but also reduces the data transmission and rendering costs, and significantly improves the access efficiency and user experience of online documents containing 3D models. Attached Figure Description

[0074] The present invention will now be further described with reference to the accompanying drawings.

[0075] Figure 1 A flowchart illustrating an online document rendering method based on a 3D lightweight model, provided as an embodiment of the present invention;

[0076] Figure 2 A flowchart of the substructure splitting process provided in an embodiment of the present invention. Detailed Implementation

[0077] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0078] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0079] This invention provides an online document rendering method based on a lightweight 3D model. See also... Figure 1 , Figure 1 A flowchart illustrating an online document rendering method based on a 3D lightweight model, provided as an embodiment of the present invention. The method includes the following steps:

[0080] S101, Obtain the original 3D model data, convert all faces of the original 3D model data into triangular meshes, and then extract the planar primitives to obtain the initial segmentation plane set;

[0081] S102, the initial segmented plane set is optimized and clustered by the region growing algorithm, and the plane region set is obtained by fitting and optimizing the parameters of each plane by combining principal component analysis;

[0082] S103, Identify spatial topological associations between planes based on the planar region set and construct a global attribute connection graph with planes as nodes and topological association types as edges;

[0083] S104, The original 3D model data is structurally simplified according to the global attribute connection graph to obtain the initial simplified 3D model data;

[0084] S105, simplify the vertex data of the initial simplified 3D model data to obtain simplified 3D model data;

[0085] S106, Simplified 3D model data is homogeneously merged to obtain lightweight 3D model data;

[0086] S107: Establish an association index between the online document structure and the lightweight 3D model data, embed the lightweight 3D model into the online document, and then load the lightweight 3D model in response to the client's access request to the online document. Render and display the lightweight 3D model in the document according to the association index.

[0087] Among them, topological association types include adjacency type and intersection type;

[0088] The initial simplified 3D model data includes texture primitives and their corresponding UV coordinates and normal data.

[0089] This invention provides an online document rendering method based on a lightweight 3D model. By sequentially performing global structured progressive simplification, vertex data simplification, and isomorphic merging on the original 3D model data, efficient lightweighting of the model data is achieved. Then, an association index between the lightweight 3D model data and the online document structure is established and embedded in the document. When accessed by the client, it can be quickly loaded and rendered. This method not only ensures the structural integrity and rendering effect of the model, but also reduces data storage and transmission costs, thereby improving the access efficiency and user experience of online documents containing 3D models.

[0090] In one implementation, the generation process of the planar region set involves first using a single planar primitive from the initial segmented planar set as the initial primitive. Based on geometric similarity criteria such as the angle between the plane normal vectors, position offset, and adjacency relationship of triangular facets, region-growing clustering is performed on the discrete and fragmented initial planar primitives. Principal component analysis is then performed on all spatial coordinate points within each planar clustering region to solve for the three principal component directions of the point set, which determine the two-dimensional extension direction and normal vector direction of the plane. Combining the spatial center coordinates of the point set, the standard plane equation of the plane is fitted using the least squares method to obtain the optimal plane parameters for each clustering region. Finally, a set of planar regions with regular parameters and well-defined geometric characteristics is obtained. This process solves the fragmentation and noise problems of the initial segmentation.

[0091] In one implementation, the global attribute connection graph is generated by using each plane in the planar region set as an independent node of the global attribute connection graph. First, an AABB tree is constructed for the triangulated mesh of the original model. Through this tree, spatially adjacent triangular faces belonging to different planes are matched for each planar region, thereby determining the spatial neighbor planes of each plane. Then, based on the spatial interaction characteristics between planes, the topological association types are divided into two categories: adjacency and intersection. Adjacency refers to two non-coplanar planes sharing only a common edge, while intersection includes two cases: coplanar intersection and non-coplanar intersection without a common edge. Finally, corresponding edges are inserted for plane pairs that meet the characteristics of various topological associations to represent their topological attributes, and a global attribute connection graph with planes as nodes and topological association types as edges is finally constructed.

[0092] In one embodiment, the initial simplified 3D model data is obtained by structurally simplifying the original 3D model data according to the global attribute connectivity graph, including:

[0093] The global attribute connection graph is decomposed into sub-structure sets through structured decomposition;

[0094] A visual 3D mesh is obtained by multi-view contour fusion modeling of the original 3D model data;

[0095] The initial simplified 3D model data is obtained by hierarchically simplifying the visual 3D mesh based on the substructure set.

[0096] In one embodiment, the structured decomposition of the global attribute connectivity graph to obtain a set of substructures includes:

[0097] Remove intersecting edges from the global attribute connection graph to obtain the first substructure set and the substructure set to be processed;

[0098] The second substructure set is obtained by splitting each substructure in the set of substructures to be processed.

[0099] The first substructure set and the second substructure set are merged to obtain the substructure set.

[0100] In one embodiment, see Figure 2 , Figure 2 This is a flowchart of the substructure splitting process provided in an embodiment of the present invention; the splitting operation of each substructure in the set of substructures to be processed to finally obtain the second substructure set includes:

[0101] S201, Construct the second substructure set with an initial empty state;

[0102] S202, Select a substructure from the set of substructures to be processed as the target substructure, and remove the target substructure from the set of substructures to be processed;

[0103] S203, with a preset cut point number threshold as the upper limit, starting from 1 cut point, the number of cut points is increased sequentially to perform cut point segmentation operation on the target substructure;

[0104] S204, if the current number of cut points can split the target substructure, then stop increasing the number of cut points, obtain the intermediate substructure set, and update all substructures of the intermediate substructure set to the substructure set to be processed.

[0105] S205, if the target substructure cannot be split even after the number of cut points is increased to the preset cut point number threshold, then the target substructure is added to the second substructure set;

[0106] S206, return to S202 until there are no more substructures in the set of substructures to be processed, and finally obtain the second set of substructures.

[0107] In one implementation, the preset cut point number threshold is set by the technician and can be 4.

[0108] In one implementation, the target substructure is the processing object. Starting from a preset cut vertex number threshold, the number of cut vertices is increased sequentially. Each attempt generates a candidate set of cut vertices of a specified number, sorted in descending order of the area of ​​the corresponding planar region. The candidate cut vertex set is traversed and a graph partitioning operation is performed, i.e., after removing the group of cut vertices and their associated edges, it is checked whether the number of connected components obtained is not less than 2. If the number of connected components is not less than 2, it is determined that the current number of cut vertices can split the target substructure, and the increase in the number of cut vertices is stopped, resulting in an intermediate substructure set. If the number of cut vertices is increased to the preset cut vertex number threshold but still cannot obtain not less than 2 connected components, it is determined that the target substructure cannot be effectively split by cut vertices, and it is directly retained in the second substructure set.

[0109] In one embodiment, obtaining a visual 3D mesh by performing multi-view contour fusion modeling on the original 3D model data includes:

[0110] Step 1: Determine multiple viewpoint directions based on the normal directions of each plane in the original 3D model to obtain a set of viewpoint directions;

[0111] Step 2: Project the original 3D model onto each viewpoint direction using the viewpoint direction set and simplify it to obtain a two-dimensional simplified contour set;

[0112] Step 3: Using the view directions in the view direction set as the stretching direction, extend the corresponding two-dimensional simplified contours in the two-dimensional simplified contour set into three dimensions along the stretching direction to obtain the original three-dimensional mesh set.

[0113] Step 4: Perform a Boolean cross operation on each original 3D mesh in the original 3D mesh set to obtain a merged 3D mesh;

[0114] Step 5: Render and fuse the 3D mesh according to the viewpoint direction set to obtain the mask image set;

[0115] Step 6: Calculate the visual similarity between each mask image in the mask image set and the original 3D model image at the corresponding viewpoint direction to obtain the visual similarity set;

[0116] Step 7: Determine whether each visual similarity in the visual similarity set reaches the preset similarity threshold. If not, use the fused 3D mesh as the new original 3D model and return to Step 1 until all visual similarities reach the preset threshold, and finally obtain the visual 3D mesh.

[0117] In one implementation, the preset similarity threshold is set by a technician, specifically 0.8.

[0118] In one implementation, step 2 specifically involves performing an orthogonal projection transformation on the original 3D model for each viewpoint direction in the viewpoint direction set. This maps all planar nodes, topological edges, and patch structures in the model space to a two-dimensional projection plane perpendicular to that viewpoint direction, forming an initial two-dimensional projection contour. Subsequently, the projection contour undergoes topological simplification processing. By removing short edges, merging collinear vertices, and filtering out isolated microstructures, redundant geometric features and insignificant details in the projection contour are eliminated, while retaining the core boundary contours that can characterize the overall shape of the model. After traversing all viewpoint directions to complete the above projection and simplification operations, a set of two-dimensional simplified contours corresponding one-to-one with each viewpoint direction is finally formed.

[0119] In one implementation, step 3 specifically involves using the viewpoint directions in the viewpoint direction set as the stretching directions of the corresponding two-dimensional simplified contours, performing linear stretching on each two-dimensional simplified contour in the set. Specifically, linear stretching involves uniformly translating all vertices and boundary edges on the two-dimensional closed contour along the specified stretching direction to form a continuous side mesh in space, while retaining the two end faces at the start and end positions of the contour. Through vertex connection, facet partitioning, and mesh reconstruction, the two-dimensional contour is expanded into a three-dimensional prism mesh with spatial thickness and closed volume. Each two-dimensional simplified contour generates a corresponding original three-dimensional mesh in this manner. After all contours are processed, the original three-dimensional mesh set is formed.

[0120] In one implementation, step 5 specifically involves first constructing a camera coordinate system and projection matrix that match the viewpoint direction, transforming the vertex coordinates of the fused 3D mesh from world space to clip space and screen space, completing vertex rasterization interpolation, enabling depth testing during rendering and recording the minimum depth value of each pixel position, retaining only the mesh surface area visible from the current viewpoint, and then binarizing the rendering result by assigning a value of 1 to visible pixels and a value of 0 to invisible background pixels, removing irrelevant information such as color, texture, and lighting, and retaining only the binary mask image representing the visible area of ​​the fused 3D mesh. After traversing all viewpoint directions to complete the above rendering and binarization process, a mask image set corresponding to each viewpoint direction is obtained.

[0121] In one implementation, step 6 specifically involves using image similarity algorithms such as structural similarity index or intersection-union ratio to calculate the degree of matching between the two in terms of contour consistency, region coverage, and spatial distribution. The calculation results under each viewpoint are used as the visual similarity in that direction, and the similarity values ​​under all viewpoints together constitute a visual similarity set.

[0122] In one embodiment, the initial simplified 3D model data obtained by hierarchically simplifying the visual 3D mesh based on the substructure set includes:

[0123] Calculate the depth loss between the substructures in the substructure set and the visual 3D mesh, and filter the substructure set according to the depth loss to obtain the substructure set to be processed;

[0124] The set of primitives to be processed is obtained by performing spatial geometric matching between each substructure to be processed in the set of substructures to be processed and the visual 3D mesh;

[0125] The corrected 3D mesh is obtained by iterative geometric correction of the visual 3D mesh based on the set of primitives to be processed;

[0126] Extract the planes of the corrected 3D mesh, calculate the number of neighboring planes for each vertex in the corrected 3D mesh based on the extracted planes, and assign simplified weights to each vertex according to the number of neighboring planes to obtain the vertex simplified weight set;

[0127] A quadratic error metric algorithm based on vertex simplification weight set optimization is used to perform edge folding simplification on the corrected 3D mesh to obtain the initial simplified 3D model data.

[0128] In one implementation, the screening process involves first performing depth rendering on each substructure and the visual 3D mesh from a unified perspective to generate a depth map. The absolute error and mean square error of the depth values ​​of the two substructures are calculated pixel by pixel and accumulated to obtain the depth loss corresponding to the substructure. The depth loss is used to measure the degree of deviation between the substructure and the visual 3D mesh in spatial position. Then, a depth loss threshold is set, and substructures with loss values ​​exceeding the threshold or excessive geometric deviations are removed, while substructures with smaller loss values ​​and higher spatial consistency are retained, thus forming a set of substructures to be processed.

[0129] In one implementation, the spatial geometric matching process involves extracting planar patches, normal vectors, boundary contours, and bounding box features from each substructure in the set of substructures to be processed and the visual 3D mesh. Multi-dimensional geometric matching is achieved by calculating the Euclidean distance, normal angle, contour intersection-union ratio, and spatial overlap between the features. Geometric primitives that are highly aligned with the visual 3D mesh in terms of pose, position, and shape are selected, and geometric units with low matching degree, redundancy, or abnormality are removed. The matched geometric primitives are then integrated to form a set of primitives to be processed.

[0130] In one implementation, the iterative geometric correction process uses the vertices, faces, and topology of the visual 3D mesh as the initial reference. Each geometric primitive in the primitive set to be processed is used as a rigid geometric constraint, and the visual 3D mesh is constrained and optimized region by region: the position of vertices with deviations is adjusted by fitting, the normals of faces with inconsistent normals are redirected and smoothed, and the faces of topological breaks or contour misalignments are reconstructed and the boundaries are stitched. After each round of correction, the spatial consistency and error level of the mesh are checked based on the constraint conditions of the geometric primitives. If the error does not meet the requirements, iterative adjustment is continued based on the current primitive constraints until the mesh geometric accuracy, topological consistency and primitive constraints are completely matched, and finally the corrected 3D mesh is output.

[0131] In one implementation, the edge folding simplification operation involves weighting the error metric by combining vertex simplification weights when calculating the folding cost of each edge, reducing the folding priority of edges containing high-weight vertices, and prioritizing the folding of non-critical edges composed of low-weight vertices. During the edge folding process, the overall geometric contour, planar structure, and topological connectivity of the model are maintained, and the number of vertices and faces is gradually reduced until the preset simplification accuracy is reached, ultimately obtaining the initial simplified 3D model data that retains the key structures.

[0132] In one embodiment, reducing vertex data in the initial simplified 3D model data to obtain simplified 3D model data includes:

[0133] The texture primitives in the initial simplified 3D model data are classified to identify solid color texture types and gradient texture types, thus obtaining an initial set of texture primitives to be replaced.

[0134] Calculate the visual saliency of each texture primitive in the initial set of texture primitives to be replaced in the original 3D model, and filter out texture primitives with visual saliency lower than a preset threshold to obtain the set of texture primitives to be replaced.

[0135] The vertex color attribute set is obtained by generating differential colors for the texture primitives in the texture primitive set to be replaced according to their types.

[0136] The original texture mapping of the texture primitive to be replaced is replaced based on the vertex color attribute set, and the UV coordinates and normal data of the corresponding vertex of the texture primitive to be replaced are deleted to obtain simplified 3D model data.

[0137] In one implementation, the preset threshold is set by a technician.

[0138] In one implementation, the texture primitive classification and filtering process involves analyzing the texture images associated with each texture primitive, calculating the color histogram of the texture images, and determining the texture type as a solid color texture if the color histogram shows a single main peak and the color variance is lower than a preset threshold. If the color histogram shows a continuous gradient distribution along a certain direction, it is determined as a gradient texture type. Texture primitives determined to be solid color or gradient types are included in the initial set of texture primitives to be replaced. Then, each texture primitive in the initial set of texture primitives to be replaced is mapped to the screen space, and the average saliency value of each texture primitive in the multi-view projection of the original 3D model is calculated using a visual attention algorithm based on image gradients. Texture primitives with an average saliency value lower than a preset threshold are filtered out to obtain the set of texture primitives to be replaced.

[0139] In one implementation, the texture replacement simplification process is as follows: First, the color values ​​in the vertex color attribute set are written to the vertex attribute buffer of each vertex in the texture primitive to be replaced according to the vertex index, so that each vertex has two sets of color information sources: texture sampling data and vertex color. Then, the material rendering state of the texture primitive to be replaced is modified, the texture mapping switch is turned off and the shading mode is switched to vertex shading mode, so that the rendering engine directly reads the vertex color buffer for shading, thereby completing the replacement of texture mapping. Next, all vertices in the texture primitive to be replaced are traversed, and UV coordinate data and normal data are removed from their vertex attribute buffers to eliminate redundant data related to texture sampling. Finally, the processed texture primitive to be replaced is merged with the unprocessed texture primitive in the initial simplified 3D model data to generate simplified 3D model data, in which the texture primitive to be replaced no longer depends on external texture images, the amount of vertex data is significantly reduced, and the visual appearance is presented independently through vertex color.

[0140] In one embodiment, the vertex color attribute set is obtained by differentially generating colors for the texture primitives in the set of texture primitives to be replaced according to their types, including:

[0141] If the texture primitives in the texture primitive set to be replaced are solid color textures, extract the primary color value as the color value of each vertex in the corresponding primitive to obtain the solid color vertex attribute value set.

[0142] If the texture primitives in the texture primitive set to be replaced are of the gradient texture type, the gradient color vertex attribute value set is obtained by interpolating the color values ​​of each vertex based on the spatial mapping relationship between the gradient direction of the texture image and the vertex normal.

[0143] The vertex color attribute set is obtained by integrating the attribute value set of solid color vertices and the attribute value set of gradient color vertices.

[0144] In one implementation, the vertex color determination process for a solid color texture primitive involves: acquiring the texture image associated with the solid color texture primitive; traversing all pixels in the texture image and statistically analyzing the color value distribution of each pixel; then, using a clustering algorithm or color histogram peak detection, identifying the most frequently occurring color value as the primary color value, which is typically a combination of the values ​​of the three RGB channels; next, acquiring the vertex indices of all vertices contained in the solid color texture primitive; copying the primary color value to the color value of each vertex to form a color value sequence equal to the number of vertices; and finally, organizing this color value sequence into a solid color vertex attribute value set according to the vertex index order.

[0145] In one implementation, the vertex color determination process for a gradient texture primitive is as follows: First, the texture image associated with the gradient texture primitive is acquired, and color sampling is performed along the gradient direction of the texture image to obtain a color sequence from the start point to the end point. This sequence records the color values ​​at different positions along the gradient direction. Then, the angle between the normal direction of each vertex in the gradient texture primitive and the gradient direction of the texture image is calculated. The relative position of the vertex in the gradient direction is determined based on the angle. The closer the angle is to 0 degrees, the more likely the vertex is near the start point of the gradient. The closer the angle is to 180 degrees, the more likely the vertex is near the end point of the gradient. Next, based on the relative position of the vertex in the gradient direction, the color value corresponding to the vertex is calculated from the color sequence through linear interpolation or spline interpolation, so that the color changes continuously with the angle between the vertex normal and the gradient direction. Finally, the interpolated color values ​​of all vertices are organized into a gradient vertex attribute value set according to the vertex index order.

[0146] In one implementation, for cases where a shared vertex belongs to a combination of different types of texture primitives, a unified processing strategy combining hierarchical priority and weighted fusion is used to determine its final vertex color value: First, all texture primitive types to which the shared vertex belongs are identified, and they are judged in order of visual complexity from high to low priority, with gradient textures having the highest priority and solid color textures second. If a gradient texture primitive exists, the color value generated by gradient interpolation is used as the base color; Second, if multiple gradient texture primitives exist simultaneously, a weighted average is performed based on the proportion of the triangular facet area of ​​each gradient primitive at the vertex to obtain the base color value; Then, if a solid color texture primitive exists and there is no gradient texture primitive... For each solid color primitive, a color consistency check is performed based on its primary color value. If the difference between all primary color values ​​is less than a preset threshold, any primary color value is used. If the difference exceeds the threshold, a base color value is obtained by area-weighted averaging. Next, if the vertex belongs to both gradient texture primitives and solid color texture primitives, the color value generated by the gradient texture primitive is used, ignoring the primary color value of the solid color texture primitive, because gradient textures contain richer color transition information. Finally, the determined base color value is used as the final assignment of the vertex in the vertex color attribute set, ensuring that all shared vertices, regardless of the primitive types they belong to, can obtain a unique and smooth vertex color value, eliminating color jump artifacts across primitive boundaries.

[0147] In one embodiment, isomorphic merging of simplified 3D model data to obtain lightweight 3D model data includes:

[0148] Traverse all triangular faces in the simplified 3D model data and cluster them based on the spatial connectivity between the triangular faces to obtain a set of connected components.

[0149] Calculate the bounding box size and number of vertices of each component in the connected component set, and group components with the same bounding box size and number of vertices into the isomorphic candidate group to obtain the isomorphic candidate group set;

[0150] For each candidate group in the isomorphic candidate group set, compare the relative distance matrix between the vertices of each component in the group one by one, and determine the components with the same relative distance matrix as isomorphic components to obtain the isomorphic component set.

[0151] Select one component from each isomorphic component group in the isomorphic component group set as the prototype component, calculate the spatial transformation matrix between the remaining components in the isomorphic component group and the prototype component, and delete the triangular facet data of the remaining components in the isomorphic component group to obtain the prototype component set and the transformation matrix set.

[0152] The prototype component set and transformation matrix set are combined and stored to generate lightweight 3D model data.

[0153] In one implementation, clustering is based on the spatial connectivity between triangular faces. Specifically, the connectivity between two triangular faces is determined by whether they share vertices or edges. All triangular faces are traversed using a disjoint-set data structure or a breadth-first search algorithm, and faces that are directly or indirectly connected are grouped into the same component.

[0154] The foregoing has described one embodiment of the present invention in detail, but this content is merely a preferred embodiment and should not be considered as limiting the scope of the present invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the scope of the claims of this invention.

Claims

1. A method for online document rendering based on a 3D lightweight model, characterized in that, The method includes: Obtain the original 3D model data, convert all faces of the original 3D model data into a triangular mesh, and then extract the planar primitives to obtain the initial segmentation plane set; The initial segmented plane set is optimized and clustered using a region growing algorithm, and the plane region set is obtained by fitting and optimizing the parameters of each plane using principal component analysis. Based on the set of planar regions, spatial topological associations between planes are identified, and a global attribute connection graph is constructed with planes as nodes and topological association types as edges; the topological association types include adjacency type and intersection type; The original 3D model data is structurally and hierarchically simplified based on the global attribute connection graph to obtain initial simplified 3D model data; The initial simplified 3D model data is simplified by reducing vertex data to obtain simplified 3D model data; the initial simplified 3D model data includes texture primitives and corresponding UV coordinates and normal data; The simplified 3D model data is homogeneously merged to obtain lightweight 3D model data; An association index is established between the online document structure and the lightweight 3D model data, and the lightweight 3D model is embedded in the online document. Then, in response to the client's access request to the online document, the lightweight 3D model is loaded, and the lightweight 3D model is rendered and displayed in the document according to the association index.

2. The online document rendering method based on a 3D lightweight model according to claim 1, characterized in that, include: The global attribute connection graph is decomposed into substructure sets. The original 3D model data is used to perform multi-view contour fusion modeling to obtain a visual 3D mesh; The initial simplified 3D model data is obtained by hierarchically simplifying the visual 3D mesh based on the substructure set.

3. The online document rendering method based on a 3D lightweight model according to claim 2, characterized in that, The structured decomposition of the global attribute connection graph yields a set of substructures, including: Remove the intersecting edges from the global attribute connection graph to obtain the first substructure set and the substructure set to be processed; The second substructure set is obtained by splitting each substructure in the set of substructures to be processed. The first substructure set and the second substructure set are merged to obtain a substructure set.

4. The online document rendering method based on a 3D lightweight model according to claim 3, characterized in that, The second substructure set is obtained by splitting each substructure in the set of substructures to be processed, including: Step 1: Construct a second substructure set with an initial empty state; Step 2: Select a substructure from the set of substructures to be processed as the target substructure, and remove the target substructure from the set of substructures to be processed; Step 3: Starting from a preset cut point threshold, increase the number of cut points sequentially to perform cut point segmentation on the target substructure. Step 4: If the current number of cut points can split the target substructure, then stop increasing the number of cut points, obtain an intermediate substructure set, and update all substructures of the intermediate substructure set to the substructure set to be processed. Step 5: If increasing the number of cut points to the preset cut point threshold still fails to split the target substructure, then add the target substructure to the second substructure set; Step 6: Return to Step 2 until there are no more substructures in the set of substructures to be processed, and finally obtain the second set of substructures.

5. The online document rendering method based on a 3D lightweight model according to claim 2, characterized in that, The visual 3D mesh obtained by performing multi-view contour fusion modeling on the original 3D model data includes: Step 1: Determine multiple viewpoint directions based on the normal directions of each plane in the original 3D model to obtain a set of viewpoint directions; Step 2: Project the original 3D model onto each viewpoint direction in the viewpoint direction set and simplify it to obtain a two-dimensional simplified contour set; Step 3: Using the viewpoint directions in the viewpoint direction set as the stretching direction, extend each two-dimensional simplified contour in the two-dimensional simplified contour set into three dimensions along the stretching direction to obtain the original three-dimensional mesh set. Step 4: Perform a Boolean cross operation on each original 3D mesh in the original 3D mesh set to obtain a merged 3D mesh; Step 5: Render the fused 3D mesh according to each viewpoint direction in the viewpoint direction set to obtain a mask image set; Step 6: Calculate the visual similarity between each mask image in the mask image set and the original 3D model image under the corresponding viewpoint direction to obtain a visual similarity set; Step 7: Determine whether each visual similarity in the visual similarity set reaches the preset similarity threshold. If not, use the fused 3D mesh as the new original 3D model and return to step 1 until all visual similarities reach the preset threshold, and finally obtain the visual 3D mesh.

6. The online document rendering method based on a 3D lightweight model according to claim 2, characterized in that, The initial simplified 3D model data is obtained by hierarchically simplifying the visual 3D mesh based on the substructure set, including: Calculate the depth loss between the substructures in the substructure set and the visual 3D mesh, and filter the substructure set according to the depth loss to obtain the substructure set to be processed; Spatial geometric matching is performed between each substructure to be processed in the set of substructures to be processed and the visual 3D mesh to obtain the set of primitives to be processed; A corrected 3D mesh is obtained by iterative geometric correction of the visual 3D mesh based on the set of primitives to be processed; Extract the planes of the corrected 3D mesh, calculate the number of adjacent planes for each vertex in the corrected 3D mesh based on the extracted planes, and assign simplified weights to each vertex according to the number of adjacent planes to obtain a set of simplified vertex weights; A quadratic error metric algorithm based on the optimized vertex simplified weight set is used to perform edge folding simplification on the corrected 3D mesh to obtain the initial simplified 3D model data.

7. The online document rendering method based on a 3D lightweight model according to claim 1, characterized in that, The simplified 3D model data obtained by reducing the vertex data of the initial simplified 3D model data includes: The texture primitives in the initial simplified 3D model data are classified to identify solid color texture types and gradient texture types, thus obtaining an initial set of texture primitives to be replaced. Calculate the visual saliency of each texture primitive in the original 3D model in the initial set of texture primitives to be replaced, and filter out texture primitives with visual saliency lower than a preset threshold to obtain the set of texture primitives to be replaced; The vertex color attribute set is obtained by generating differentiated colors for the texture primitives in the set of texture primitives to be replaced according to their types. The original texture mapping of the texture primitive to be replaced is replaced according to the vertex color attribute set, and the UV coordinates and normal data of the corresponding vertex of the texture primitive to be replaced are deleted to obtain simplified 3D model data.

8. The online document rendering method based on a 3D lightweight model according to claim 7, characterized in that, The vertex color attribute set is obtained by generating differentiated colors for the texture primitives in the set of texture primitives to be replaced according to their types, including: For the texture primitives in the set of texture primitives to be replaced, which are of the solid color texture type, the primary color value is extracted as the color value of each vertex in the corresponding primitive to obtain the solid color vertex attribute value set; For the texture primitives in the set of texture primitives to be replaced, which are of the gradient texture type, the gradient color vertex attribute value set is obtained by interpolating the color value of each vertex based on the spatial mapping relationship between the gradient direction of the texture image and the vertex normal. The vertex color attribute set is obtained by integrating the solid color vertex attribute value set and the gradient color vertex attribute value set.

9. The online document rendering method based on a 3D lightweight model according to claim 1, characterized in that, The simplified 3D model data is homogeneously merged to obtain lightweight 3D model data, including: Traverse all triangular faces in the simplified 3D model data and cluster them based on the spatial connectivity between the triangular faces to obtain a set of connected components. Calculate the bounding box size and number of vertices of each component in the connected component set, and group components with the same bounding box size and number of vertices into the isomorphic candidate group to obtain the isomorphic candidate group set; For each candidate group in the isomorphic candidate group set, the relative distance matrix between the vertices of each component in the group is compared one by one, and the components with the same relative distance matrix are determined to be isomorphic components to obtain the isomorphic component set; Select one component from each isomorphic component group in the isomorphic component group set as the prototype component, calculate the spatial transformation matrix between the remaining components in the isomorphic component group and the prototype component, and delete the triangular facet data of the remaining components in the isomorphic component group to obtain the prototype component set and the transformation matrix set. The prototype component set and the transformation matrix set are combined and stored to generate lightweight 3D model data.

10. An online document rendering system based on a 3D lightweight model, characterized in that, The system includes: The acquisition module is used to acquire the original 3D model data, convert all faces of the original 3D model data into a triangular mesh, and then extract the planar primitives to obtain the initial segmentation plane set. The planar region generation module is used to optimize and cluster the initial segmented planar set using a region growing algorithm, and combine principal component analysis to fit and optimize the parameters of each planar region to obtain a planar region set. The global graph generation module is used to identify spatial topological associations between planes based on the set of planar regions and construct a global attribute connection graph with planes as nodes and topological association types as edges; the topological association types include adjacency type and intersection type; The structure simplification module is used to perform structured hierarchical simplification of the original 3D model data according to the global attribute connection graph to obtain initial simplified 3D model data; The vertex simplification module is used to simplify the vertex data of the initial simplified 3D model data to obtain simplified 3D model data; the initial simplified 3D model data includes texture primitives and corresponding UV coordinates and normal data. The isomorphic merging module is used to perform isomorphic merging on the simplified 3D model data to obtain lightweight 3D model data; The document rendering module is used to establish an association index between the online document structure and the lightweight 3D model data, embed the lightweight 3D model into the online document, load the lightweight 3D model in response to the client's access request for the online document, and render and display the lightweight 3D model in the document according to the association index.

Images