Hybrid editing, fusion rendering and 3D printing method, system, device and medium of multi-modal three-dimensional assets

Abstract

The application provides a mixed editing, fusion rendering and 3D printing method, system, device and storage medium of a multi-modal three-dimensional asset, and the method comprises the following steps: S1, importing a plurality of three-dimensional scenes into a unified scene and establishing a scene graph; S2, performing mixed editing on each three-dimensional asset; S3, acquiring a shell mesh model and constructing shell constraint information; S4, converting the unified scene into a target volume rendering representation or constructing a samplable mixed field function; S5, based on the shell constraint information and the target volume rendering representation / mixed field function, voxelizing and sampling a space to obtain a voxel attribute set; and S6, generating corresponding voxel label functions according to the inside-outside relationship between voxels and shells, and outputting layer-by-layer printing data. The application realizes unified editing and printable conversion of various three-dimensional representations, improves the controllability of printing boundaries, the stability of support generation and the processing efficiency of mixed assets to physical products.

Description

Technical Field

[0001] This invention relates to the fields of three-dimensional digital content processing, computer graphics and full-color three-dimensional printing technology, and in particular to a method, system, device and medium for unified editing, fusion rendering and conversion of multimodal / hybrid three-dimensional assets into printable layer-by-layer manufacturing data. Background Technology

[0002] With the development of manufacturing processes such as full-color inkjet 3D printing and resin photopolymerization, users' demand for "what you see is what you get" 3D printing is constantly increasing. At the same time, the forms of expression for 3D digital assets are becoming increasingly diversified: in addition to traditional mesh models, new expressions such as 3D Gaussian Splatting, radiation field representation, sparse volumetric meshes (e.g., representations based on voxel / sparse volume data structures), and procedural volume data have emerged. Different expressions have their own advantages in terms of geometry, materials, transparency, and detail representation, but they also bring the following problems: (1) Mixed assets are difficult to edit uniformly: different types of assets have different coordinate systems, scales, material / color meanings and rendering methods. Existing tools can often only edit or export a single representation, making it difficult to achieve cross-representation combination, alignment, clipping and color adjustment in the same scene;

[0003] (2) Uncontrollable printing boundaries: For “volume rendering assets” such as 3DGS, radiation fields or volume data, their geometric boundaries are often unclear or contain floating noise. Direct slicing is prone to producing broken structures, abnormal support or difficulty in meeting the molding strength requirements.

[0004] (3) Inconsistent manufacturing formats: Industrial software or printing links usually prefer grid / general manufacturing formats, while when hybrid assets contain non-grid representations, there is a lack of unified, controllable and efficient conversion and export methods.

[0005] (4) Lack of a universal workflow for co-printing of shell and content: In practical applications, users often need to add a transparent shell or protective shell to the internal content and want the gaps inside the shell to be filled with transparent material, and the shell to automatically generate support or skip strategies. The existing workflow requires repeated processing by multiple software programs, which is inefficient and prone to introducing errors.

[0006] Therefore, there is an urgent need for a new hybrid asset processing and printing method that can perform mixed editing of multiple types of 3D assets within the same tool; can achieve controllable transparent filling, trimming and support generation through shell constraints; and can stably convert hybrid scenes into printing data suitable for voxel slicing and layer-by-layer manufacturing. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes a method, system, device, and medium for hybrid editing, fusion rendering, and 3D printing of multimodal 3D assets.

[0008] The technical solution of this invention provides a method for hybrid editing, fusion rendering, and 3D printing of multimodal 3D assets, comprising the following steps: S1: Import several different types of 3D scenes into a unified scene and create a scene graph; S2 allows for mixed editing of 3D assets from various 3D scenes within a unified scene, recording the pose, hierarchical relationship, and cross-representation material / color descriptions of each 3D asset in the scene graph; S3, Obtain the shell mesh model and construct shell constraint information to characterize the internal / external relationship / distance between any point in space and the shell; S4, according to the preset transformation rules, converts the unified scene into the target volume rendering representation, or constructs a sampleable hybrid field function; S5, based on the shell constraint information obtained in S3 and the target volume rendering representation / hybrid field function obtained in S4, voxel sampling is performed in the preset three-dimensional real coordinate space to obtain the voxel attribute set; S6 generates corresponding voxel tag functions based on the internal and external relationship between the voxel and the shell, and outputs layer-by-layer printing data to the printing device based on the voxel tag functions.

[0009] In some embodiments of S1, the three-dimensional scene includes any one or more of 3DGS, mesh models, GLB / OBJ / FBX models, radiation fields, sparse volumetric meshes, volume data, or point clouds.

[0010] In S1 of some embodiments, after importing the 3D scene, a unified coordinate system, scale and unit are established, and the assets are organized into a scene graph structure.

[0011] In some embodiments of S2, mixed editing includes: translation, rotation, scaling, alignment, grouping, cropping, color grading, material parameter editing, visibility control, history rollback, etc.

[0012] In some embodiments of S2, the scene graph records the pose, hierarchy, rendering pass, and cross-representation material / color description of each asset node, so that different representations can be operated under the same editing semantics.

[0013] In some embodiments of S3, the shell mesh model is represented as: M_s=(V_s,F_s) Where V_s={v_i∈R^3} is a set of three-dimensional vertices, R^3 represents a three-dimensional real coordinate space, v_i is the coordinate of a vertex in the three-dimensional real coordinate space, and F_s is a set of triangular faces.

[0014] In some embodiments of S3, the shell model is a closed mesh with or without thickness.

[0015] In some embodiments of S3, when the shell mesh model is a closed mesh or can be regarded as a closed mesh after tolerance repair, its internal region is denoted as Ω_s and the boundary surface is denoted as dΩ_s.

[0016] In some embodiments of S3, the shell constraint information is obtained by at least one of the following methods: ray parity method, generalized winding number method, mesh model voxelization method; and further forms a symbolic distance field SDF, sparse volume grid VDB, inner and outer masks, shell thickness, and default labels inside / outside the shell.

[0017] In S3 of some embodiments, an indicative function χ_s(p) is defined based on the shell mesh model, where χ_s(p)=1 indicates that point p is inside Ω_s, and χ_s(p)=0 indicates that point p is outside Ω_s.

[0018] Define the distance to the outer shell surface as d_s(p) = min_{q∈dΩ_s}||pq||_2, and the symbolic distance field as φ_s(p) = (1-2χ_s(p))·d_s(p); In some embodiments, S3, χ_s(p) is obtained by the ray parity method, including selecting a direction r for the point p to be measured, and counting the number of intersections N(p,r) between the ray p+λr (λ>0) and the set of outer shell patches F_s; when N(p,r) is odd, let χ_s(p)=1, and when N(p,r) is even, let χ_s(p)=0; In S3 of some embodiments, χ_s(p) is obtained through the generalized wrap number method, and then calculated as follows: W_s(p)=1 / (4π)·Σ_{f∈F_s}Ω_f(p) Where W_s(p) represents the generalized wrap number of point p relative to the shell mesh model M_s, which is used to characterize the degree to which point p is surrounded by the shell mesh model; Ω_f(p) is the directed solid angle spanned by the directed surface f relative to point p; When |W_s(p)|≥τ_w, let χ_s(p)=1; When |W_s(p)| < τ_w, let χ_s(p) = 0; τ_w is a preset threshold.

[0019] In some embodiments, S3, the shell mesh model is converted into a volume mesh by a mesh voxelization method, including using p_ijk as the voxel center point and calculating χ_s(p_ijk) or φ_s(p_ijk) for p_ijk, thereby obtaining the shell occupancy grid, the symbolic distance field grid, the sparse volume mesh, or the inner and outer masks.

[0020] In some embodiments, S4, when the unified scene only contains a mesh model, the mesh model is converted into a unified mesh format that supports color or texture information, and further converted into a general manufacturing format that can be recognized by the printing software.

[0021] In some embodiments, S4, the general manufacturing format includes a mesh file with color information or a general model file with texture; In S4 of some embodiments, for scenarios containing non-mesh content or requiring voxel slicing, one or a combination of the following paths can be selected: When converting a mesh model into a 3DGS collection, the process includes: acquiring the geometric and texture / material information of the mesh model; sampling the mesh surface in screen space or orthographic projection space based on rasterization interpolation; determining the scale and rotation of the 3DGS based on the Jacobian matrix from the local two-dimensional coordinates of the mesh triangles to the three-dimensional surface; generating the 3DGS center at the sampling point and writing the color and material parameters obtained from the texture sampling into the 3DGS attributes; and generating a renderable 3DGS data file from the 3DGS attributes.

[0022] Furthermore, for the mesh triangle f=(v_1,v_2,v_3) and its local two-dimensional coordinates u_1,u_2,u_3, construct the three-dimensional edge matrix V_f=[v_2-v_1,v_3-v_1] and the two-dimensional coordinate edge matrix U_f=[u_2-u_1,u_3-u_1], and calculate the Jacobian matrix J_f=V_f·U_f^{-1} from the two-dimensional local coordinates to the three-dimensional surface; determine the scale and rotation of 3DGS in the tangential direction of the surface based on the column vector of J_f, and determine the normal direction and normal thickness of 3DGS based on the triangle normal.

[0023] In some embodiments, S4, when converting volume data or point clouds into a 3DGS set, includes: identifying adjacent point sets with the same or similar colors in a voxel grid or point set, and aggregating them into a 3DGS; the center of the 3DGS is determined by the position of the aggregated point set, and the covariance of the 3DGS is determined by the spatial distribution of the aggregated point set.

[0024] In some embodiments, S4, converting a unified scene into a target volume rendering representation includes: performing multi-view offline or real-time rendering on the unified scene to obtain multi-view images and camera parameters; and reconstructing a 3DGS set or radiation field based on the multi-view images and camera parameters as a target volume rendering representation.

[0025] In some embodiments of S5, the voxel properties include at least: color, transparency (or equivalent absorption / attenuation), and material label or material index.

[0026] In S6 of some embodiments, voxels located inside the housing are labeled as content voxels, and voxels located outside the housing are labeled as support voxels.

[0027] The voxel labeling function L_ijk satisfies: When χ_s(p_ijk)=1, L_ijk=content; When χ_s(p_ijk)=0, L_ijk=support; The content voxel is used to carry the color, transparency, or material properties of the content asset, while the support voxel is used to fill the support material.

[0028] Furthermore, when a voxel in the content domain is occupied by a content asset, the color, transparency, or material parameters of the content asset at that voxel are preserved. Furthermore, when a voxel in the content domain is not occupied by a content asset, it is assigned transparent material or blank content material according to a preset strategy.

[0029] This invention also provides a system for hybrid editing, fusion rendering, and 3D printing of multimodal 3D assets, comprising: The asset import module is used to acquire and import various types of 3D assets; The hybrid editing module is used to uniformly edit multiple types of 3D assets and generate scene diagrams; The shell processing module is used to acquire the shell mesh model and construct the shell constraint information; the shell constraint information can be obtained at least by the ray parity method, the generalized wrap number method, or the mesh to volume voxelization method. The transformation and reconstruction module is used to convert a uniform scene into a target volume rendering representation or to construct a hybrid field function; The voxelization and slicing module is used to generate voxel attribute sets and voxel label functions based on shell constraint information, and output layer-by-layer print data.

[0030] The present invention also provides a 3D printing device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the methods described.

[0031] The present invention also provides a chip including one or more processors for calling and running a computer program from a memory, causing a device having the chip mounted to perform any of the methods described.

[0032] The present invention also provides a computer-readable storage medium having a computer program stored thereon, the computer program implementing any of the methods described when executed by a processor.

[0033] Compared with the prior art, the present invention has the following advantages: (1) Unified Hybrid Editing: Supports unified editing and printable conversion of multiple asset types such as mesh, volume data, 3DGS, and radiation fields in the same tool; (2) Controllable shell printing: The printing range is made explicit by shell constraints, which suppresses floating noise, improves the controllability of printing boundaries, supports the stability of generation and the efficiency of processing hybrid assets into solid products, and supports applications such as transparent shells / protective shells; (3) Multi-path printable conversion: Provides multiple conversion paths for different asset types, balancing speed and quality; (4) Automatic transparent filling and support strategy: Based on the relationship between the inside and outside of the shell and the rules of gravity direction, transparent filling and support / skipping voxels are generated to improve printing stability and reduce manual operation. Attached Figure Description

[0034] Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the module structure of the system according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the hybrid editing software interface according to an embodiment of the present invention; Figure 4 A schematic diagram of voxel classification and support determination under shell constraints; Figure 5 A schematic diagram illustrating the transformation path from different asset types to the target volume rendering representation; Figure 6 The first set of embodiments includes screenshots of Studio editing and corresponding finished product images; Figure 7 The images shown are screenshots of the Studio editing function and the corresponding finished product images for the second set of embodiments. Detailed Implementation

[0035] The following is in conjunction with the appendix Figure 1-7The embodiments of the present invention will be further described below. The following examples are used to explain the present invention, but not to limit the present invention.

[0036] S1: Importing Hybrid Assets and Building a Unified Scene.

[0037] Several different types of 3D scenes are imported into a unified scene to create a scene graph. The 3D assets of each scene can include, but are not limited to: mesh models, models in formats such as GLB / OBJ / FBX, 3DGS ensemble representations, radiation field representations, sparse volume data (e.g., VDB), point clouds, etc. After importing, a unified coordinate system, scale, and units are established, and the assets are organized into a scene graph structure.

[0038] S2, hybrid editing and cross-representation attribute management.

[0039] Within a unified scene, various assets are mixed and edited, and the poses, hierarchical relationships, and cross-representation material / color descriptions of each 3D asset are recorded in the scene graph.

[0040] Hybrid editing includes: translation, rotation, scaling, alignment, grouping, clipping, color grading, material parameter editing, visibility control, and history rollback. The scene graph records the pose, hierarchy, rendering pass, and cross-representation material / color descriptions of each asset node, allowing different representations to be manipulated under the same editing semantics.

[0041] S3: Shell model specification and shell constraint information construction.

[0042] Obtain the shell mesh model and construct shell constraint information to characterize the internal / external relationship / distance between any point in space and the shell.

[0043] The shell model is preferably a closed mesh (with or without thickness). See also Figure 4 In some embodiments, the outer shell is defined using a grid. Let the outer shell grid be: M_s=(V_s,F_s) Where V_s={v_i∈R^3} is the set of vertices, R^3 represents the three-dimensional real coordinate space, v_i is the coordinate of a vertex in the three-dimensional real coordinate space, and F_s is the set of triangular faces.

[0044] If M_s is a closed mesh, or can be considered a closed mesh after tolerance repair, then the enclosed internal space is denoted as Ω_s, and the boundary surface is denoted as dΩ_s.

[0045] The shell constraint information can be obtained through at least one of the following methods: ray parity method, generalized winding number method, mesh model voxelization method; and can further form a symbolic distance field (SDF), sparse volume grid (VDB), inner and outer masks, shell thickness, and default labels inside / outside the shell.

[0046] For any spatial point p, define an indicator function χ_s(p) indicating whether the point is inside or outside the Ω_s. When χ_s(p) = 1, it means that point p is inside Ω_s; when χ_s(p) = 0, it means that point p is outside Ω_s. That is: χ_s(p) = 1, p ∈ Ω_s; χ_s(p) = 0, p ∉ does not belong to Ω_s.

[0047] Furthermore, the distance to the outer shell surface is defined as d_s(p) = min_{q∈dΩ_s}||pq||_2, and the symbolic distance field φ_s(p) = (1-2χ_s(p))·d_s(p). When p is inside the outer shell, φ_s(p) < 0, and when p is outside the outer shell, φ_s(p) > 0.

[0048] In the first conversion method, the ray parity method is used to obtain χ_s(p). A direction r that does not intersect with the degenerate outer shell mesh is selected for the test point p. The number of intersections N(p,r) between the ray p+λr (λ>0) and the set of outer shell patches F_s is counted. If N(p,r) is odd, then χ_s(p)=1; if N(p,r) is even, then χ_s(p)=0.

[0049] Furthermore, for degenerate cases such as rays passing through vertices, edges, or coplanar with faces, the calculation can be recalculated by changing r or adding a small perturbation.

[0050] In the second transformation method, the generalized wrap number method is used to obtain χ_s(p). For the directed shell mesh, the calculation is as follows: W_s(p)=1 / (4π)·Σ_{f∈F_s} Ω_f(p) Where W_s(p) represents the generalized wrap number of point p relative to the shell mesh model M_s, which is used to characterize the degree to which point p is surrounded by the shell mesh model; Ω_f(p) is the directed solid angle spanned by the facet f relative to point p.

[0051] When |W_s(p)|≥τ_w, χ_s(p) is determined to be 1; when |W_s(p)|<τ_w, χ_s(p) is determined to be 0. Here, τ_w is a preset threshold, preferably close to 0.5. This method is robust to shell meshes with localized small gaps or imperfect closure.

[0052] In the third conversion method, the shell mesh is converted into a volume representation using a mesh voxelization method. Let the minimum corner point of the target printed voxel mesh be b_min, the voxel size be (h_x, h_y, h_z), and the voxel index be (i, j, k).

[0053] For each voxel center point p_ijk, calculate χ_s(p_ijk) or φ_s(p_ijk) to obtain the outer shell volume mesh.

[0054] Furthermore, voxels satisfying χ_s(p_ijk)=1 can be used as in-shell voxels, and voxels satisfying χ_s(p_ijk)=0 can be used as out-shell voxels; alternatively, narrow-band voxels satisfying |φ_s(p_ijk)|≤β can be retained as active voxels of the sparse volumetric mesh, where β is a preset bandwidth. In this way, occupied grids, SDF grids, VDB-type sparse volumetric meshes, or inner and outer masks can be generated from the mesh shell.

[0055] S4, a fusion of rendering and representation transformation.

[0056] The unified scene is converted into a target volume rendering representation according to a preset conversion rule, or a sampleable hybrid field function is constructed.

[0057] See Figure 5 In some embodiments, for scenarios with only mesh content, any mesh format is first converted into a unified mesh format that supports color or texture information (such as GLB, OBJ with textures, etc.), and then exported as a common manufacturing format supported by the printing software (such as most full-color 3D printers that support importing triangular mesh formats with color and texture).

[0058] For scenarios containing non-mesh content or requiring voxel slices, one or a combination of the following paths can be selected: (1) Direct conversion path from mesh to 3DGS set (fast conversion).

[0059] This approach employs a direct conversion method based on rasterization interpolation, directly converting the geometry, texture, and material information of the mesh model into 3DGS attributes without first generating multi-view training images. Let the mesh triangle to be converted be f=(v_1,v_2,v_3), corresponding to local 2D coordinates or texture coordinates u_1,u_2,u_3. For this triangle, construct the 3D edge matrix V_f=[v_2-v_1, v_3-v_1] and the 2D coordinate edge matrix U_f=[u_2-u_1, u_3-u_1], and calculate the Jacobian matrix from the 2D local coordinates to the 3D surface: J_f=V_f·U_f^{-1} Let j_u and j_v be the two column vectors of J_f, and n_f be the triangle normal. Initialize the two-dimensional Gaussian covariance Σ_2D=diag(σ_u^2,σ_v^2), where σ_u and σ_v are adjustable sampling scales. The corresponding three-dimensional scale can be set as s_u=σ_u||j_u||, s_v=σ_v||j_v||, s_n=ε, where ε is the minimum thickness along the normal. The three-dimensional rotation matrix R_f consists of the orthogonalized directions of the normalized j_u and j_v, and n_f.

[0060] Triangles are sampled in screen space or orthographic projection space using a rasterization pipeline. For any rasterized sampling point with centroid coordinates λ=(λ_1,λ_2,λ_3), the 3DGS center is generated. μ=λ_1v_1+λ_2v_2+λ_3v_3 The texture coordinates or material lookup coordinates of the sampling point are u = λ_1u_1 + λ_2u_2 + λ_3u_3, and the color c and optional material parameters m are obtained from the texture sampling or material sampling. The final generated 3DGS can be represented as: G=(μ,R_f,log s_u,log s_v,log s_n,α,c,m) Where α is the Gaussian opacity or a transparency-related parameter. Multiple triangles generate a set of sampling points during rasterization, which together form the 3DGS set. This method utilizes the interpolation capabilities of the graphics pipeline to achieve dense sampling, and allows the center, scale, rotation, and color of the 3DGS to directly inherit the geometric and textural information of the mesh surface.

[0061] (2) Aggregation path from point cloud / volume data to 3DGS collection.

[0062] Neighborhood aggregation is performed on point sets or voxel meshes: if adjacent points have the same or similar colors, they are merged into a single 3DGS; the center of the 3DGS is the weighted mean of the aggregated points, and the covariance is obtained from the spatial distribution statistics of the points. This method can reduce the number of 3DGS and improve rendering and tiling efficiency.

[0063] (3) Rendering-Reconstruction Path.

[0064] A unified scene is rendered from multiple perspectives to obtain multi-view images and camera parameters; based on the data, a 3DGS set or radiation field is reconstructed. This approach is suitable for scenarios requiring global consistency or those requiring the unification of hybrid representations into a single volume rendering representation.

[0065] S5: Voxelization slicing and material allocation under shell constraints.

[0066] See Figure 1 and Figure 4Under the combined effect of shell constraint information and target volume rendering representation / blending field function, the space is voxelized and sampled to obtain a set of voxel attributes. Voxel attributes may include at least: color, transparency (or equivalent absorption / attenuation), and material tags or material indexes.

[0067] S6: Transparent fill, support generation, and manufacturing data output.

[0068] Based on the relationship between the voxel and the shell, a voxel labeling function is generated, and layer-by-layer manufacturing data is output. For each voxel center point p_ijk, it is first determined whether it is located inside or outside the shell based on χ_s(p_ijk) or φ_s(p_ijk), and then a basic voxel labeling function L_ijk is generated. L_ijk=content, χ_s(p_ijk)=1; L_ijk=support, χ_s(p_ijk)=0.

[0069] The voxels located inside the shell are labeled as content voxels, which are used to carry the color, transparency, or transparent fill material of the content assets; the voxels located outside the shell are labeled as support voxels, which are used to fill support material or for further processing according to the manufacturing strategy.

[0070] In some embodiments, when a voxel in the content field is occupied by a content asset, the color, transparency, or material parameters of the content asset at that voxel are preserved; when a voxel in the content field is not occupied by a content asset, transparent material or blank content material can be assigned according to a preset strategy. Since voxels outside the shell are classified as support fields by the basic tag function, support material can be filled in this area during printing, thereby ensuring the structural stability of the shell and internal content during the layer-by-layer manufacturing process.

[0071] See Figure 6 and Figure 7 The resulting images from this example can be displayed in the form of "Studio final edit state + corresponding finished product image". Each example only shows the user's final edit screenshot in Studio and the corresponding finished entity image, excluding intermediate voxelization, density optimization, support generation, or slice images. Studio screenshots are used to illustrate the final configuration of multimodal assets, shell models, print size, and spatial pose in a unified scene; finished product images are used to illustrate the entity effect obtained after shell constraints, voxel label assignment, and layer-by-layer manufacturing output.

[0072] A second embodiment of the present invention provides a system corresponding to the above method, comprising: The asset import module is used to acquire and import various types of 3D assets; The hybrid editing module is used to uniformly edit multiple types of 3D assets and generate scene diagrams; The shell processing module is used to acquire the shell mesh model and construct the shell constraint information; The transformation and reconstruction module is used to convert a uniform scene into a target volume rendering representation or to construct a hybrid field function; The voxelization and slicing module is used to generate voxel attribute sets and voxel label functions based on shell constraint information, and output layer-by-layer print data.

[0073] See Figure 3 In some embodiments, the system interface includes at least: an asset collection panel, a shell settings panel, a transformation parameter panel, a viewport rendering area, a history panel, and an export button. Users can import various types of assets and perform unified transformations, alignment, and visual previews on them. Shell models are imported in mesh form and designated as "shells." Shells can be managed separately from content assets for subsequent independent voxelization and material allocation.

[0074] A third embodiment of the present invention provides an apparatus corresponding to the above-described method, including a processor and a memory. The processor executes a program in the memory to implement the method of the present invention. A storage medium stores executable instructions that, when executed by the processor, implement the steps of the method of the present invention.

[0075] Memory may include random access memory, flash memory, read-only memory, programmable read-only memory, non-volatile memory, or registers, etc. The processor may be a central processing unit (CPU), or a graphics processing unit (GPU). Memory can store executable instructions. The processor can execute the executable instructions stored in memory to implement the various processes described herein.

[0076] It is understood that the memory in this embodiment can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be ROM (Read-Only Memory), PROM (Programmable ROM), EPROM (Erasable PROM), EEPROM (Electrically Erasable EPROM), or flash memory. The volatile memory can be RAM (Random Access Memory), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as SRAM (Static RAM), DRAM (Dynamic RAM), SDRAM (Synchronous DRAM), DDR SDRAM (Double Data Rate SDRAM), ESDRAM (Enhanced SDRAM), SLDRAM (Synchlink DRAM), and DRRAM (Direct Rambus RAM). The memories described herein are intended to include, but are not limited to, these and any other suitable types of memory.

[0077] In some implementations, the memory stores elements such as upgrade packages, executable units, or data structures, or subsets thereof, or extended sets thereof: operating systems and applications.

[0078] The operating system includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application programs include various applications used to implement various application functions. Programs implementing the methods of this invention can be included within these application programs.

[0079] In this embodiment of the invention, the processor executes the rendering method steps provided in the first embodiment by calling a program or instruction stored in the memory, specifically a program or instruction stored in an application program.

[0080] A fourth embodiment of the present invention also provides a chip for executing the rendering method in the first aspect described above. Specifically, the chip includes a processor for calling and running a computer program from a memory, such that a device equipped with the chip is used to execute the method in the first embodiment described above.

[0081] Furthermore, a fifth embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the rendering method in the first embodiment of the present invention. For example, the machine-readable storage medium may include, but is not limited to, various known and unknown types of non-volatile memory.

[0082] A sixth embodiment of the present invention also provides a computer program product, including computer program instructions that cause a computer to perform the above-described method.

[0083] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for hybrid editing, fused rendering and 3D printing of multi-modal three-dimensional assets, characterized in that include: S1: Import several different types of 3D scenes into a unified scene and create a scene graph; S2 allows for mixed editing of 3D assets from various 3D scenes within a unified scene, recording the pose, hierarchical relationship, and cross-representation material / color descriptions of each 3D asset in the scene graph; S3, Obtain the shell mesh model and construct shell constraint information to characterize the internal / external relationship / distance between any point in space and the shell; S4 converts the unified scene into a target volume rendering representation, or constructs a sampleable blending field function; S5, based on the shell constraint information and the target volume rendering representation / hybrid field function, voxel sampling is performed in the preset three-dimensional real coordinate space to obtain a set of voxel attributes; S6 generates corresponding voxel label functions based on the internal and external relationships between the voxels and the shell, and outputs the layer-by-layer printing data.

2. The method of claim 1, wherein: In S1, the three-dimensional scene includes any one or more of the following: 3DGS, mesh model, GLB / OBJ / FBX model, radiation field, sparse volume mesh, volume data or point cloud. And / or, after importing the 3D scene, establish a unified coordinate system, scale, and units, and organize each asset into a scene graph structure.

3. The method of claim 1, wherein: In S3, the shell mesh model is represented as: M_s=(V_s,F_s) Where V_s={v_i∈R^3} is the set of three-dimensional vertices, R^3 represents the three-dimensional real coordinate space, v_i is the vertex coordinate in the three-dimensional real coordinate space, and F_s is the set of triangular facets; And / or, in S3, the shell model is a closed mesh with or without thickness; And / or, in S3, when the shell mesh model is a closed mesh or can be regarded as a closed mesh after tolerance repair, its internal region is denoted as Ω_s and the boundary surface is denoted as dΩ_s; And / or, in S3, the shell constraint information is obtained through at least one of the following methods: ray parity method, generalized wrap number method, and mesh model voxelization method; Furthermore, it forms the symbolic distance field (SDF), sparse volumetric grid (VDB), inner and outer masks, shell thickness, and default labels inside / outside the shell.

4. The method according to claim 1, characterized in that: In S3, the shell mesh model defines the shell inside and outside indicator function χ_s(p). When χ_s(p)=1, it means that point p is inside Ω_s, and when χ_s(p)=0, it means that point p is outside Ω_s. Define the distance to the outer shell surface as d_s(p) = min_{q∈dΩ_s}||pq||_2, and the symbolic distance field as φ_s(p) = (1-2χ_s(p))·d_s(p); And / or, in S3, χ_s(p) is obtained by the ray parity method, including selecting the direction r of the point to be measured p, and counting the number N(p,r) of the intersection points of the ray p+λr (λ>0) and the set of outer shell patches F_s; when N(p,r) is odd, let χ_s(p)=1, and when N(p,r) is even, let χ_s(p)=0; And / or, in S3, χ_s(p) is obtained through the generalized wrap number method, and calculated as follows: W_s(p)=1 / (4π)·Σ_{f∈F_s}Ω_f(p) Where W_s(p) is the generalized wrap number of point p relative to the shell mesh model M_s; Ω_f(p) is the directed solid angle spanned by the directed surface f relative to point p; When |W_s(p)|≥τ_w, let χ_s(p)=1; when |W_s(p)|<τ_w, let χ_s(p)=0; τ_w is a preset threshold. And / or, in S3, the shell mesh model is converted into a volume mesh by a mesh voxelization method, including using p_ijk as the voxel center point and calculating χ_s(p_ijk) or φ_s(p_ijk) for p_ijk, thereby obtaining the shell occupancy grid, the symbolic distance field grid, the sparse volume mesh, or the inner and outer masks.

5. The method according to claim 1, characterized in that: In S4, when the unified scene only contains a mesh model, the mesh model is converted into a unified mesh format that supports color or texture information, and further converted into a general manufacturing format that can be recognized by the printing software. And / or, the general manufacturing format includes a mesh file with color information or a general model file with texture; And / or, in S4, when converting the mesh model into a 3DGS set, the process includes: acquiring the geometric and texture / material information of the mesh model; sampling the mesh surface in screen space or orthographic projection space based on rasterization interpolation; determining the scale and rotation of the 3DGS based on the Jacobian matrix from the local two-dimensional coordinates of the mesh triangles to the three-dimensional surface; generating the 3DGS center at the sampling point and writing the color and material parameters obtained from the texture sampling into the 3DGS attributes; and generating a renderable 3DGS data file from the 3DGS attributes. And / or, in S4, when converting volume data or point cloud into a 3DGS set, the following is included: identifying adjacent point sets with the same or similar colors in the voxel grid or point set, and aggregating them into a 3DGS; the center of the 3DGS is determined by the position of the aggregated point set, and the covariance of the 3DGS is determined by the spatial distribution of the aggregated point set; And / or, in S4, when converting the unified scene into a target volume rendering representation, the process includes: performing multi-view offline or real-time rendering on the unified scene to obtain multi-view images and camera parameters; and reconstructing a 3DGS set or radiation field based on the multi-view images and camera parameters as a target volume rendering representation.

6. The method according to claim 1, characterized in that: In S6, voxels located inside the shell are labeled as content voxels, and voxels located outside the shell are labeled as support voxels. The voxel labeling function L_ijk satisfies: When χ_s(p_ijk)=1, L_ijk=content; When χ_s(p_ijk)=0, L_ijk=support; The content voxel is used to carry the color, transparency, or material properties of the content asset, while the support voxel is used to fill the support material. And / or, when a voxel in the content field is occupied by a content asset, preserve the color, transparency, or material parameters of the content asset at that voxel; And / or, when a voxel in the content field is not occupied by a content asset, assign transparent material or blank content material according to a preset strategy.

7. A system for hybrid editing, fusion rendering, and 3D printing of multimodal 3D assets, characterized in that, include: The asset import module is used to acquire and import various types of 3D assets; The hybrid editing module is used to uniformly edit multiple types of 3D assets and generate scene diagrams; The shell processing module is used to acquire the shell mesh model and construct the shell constraint information; The transformation and reconstruction module is used to convert a uniform scene into a target volume rendering representation or to construct a hybrid field function; The voxelization and slicing module is used to generate voxel attribute sets and voxel label functions based on shell constraint information, and output layer-by-layer print data.

8. A 3D printing apparatus, comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the method of any one of claims 1-6.

9. A chip, characterized in that, It includes one or more processors for calling and running a computer program from memory, causing a device on which the chip is mounted to perform the method of any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images