Multi-source heterogeneous 3D model-oriented rendering and animation processing system and method, electronic equipment and storage medium

Abstract

The invention discloses a rendering and animation processing system and method for a multi-source heterogeneous 3D model, electronic equipment and a storage medium. The rendering and animation processing system comprises a unified abstraction layer module, a material mapping engine module, a skeleton animation processing module and a rendering module. The unified abstraction layer module executes standardized conversion from original data to internal scene data; the material mapping engine module maps an original material into an internal material with complete PBR parameters through a material processor with a built-in rule base; the skeletal animation processing module realizes a smooth animation effect by executing skeletal matrix calculation and an animation state machine; and the rendering module transmits the 3D model data processed by the unified abstraction layer module, the material mapping engine module and the skeletal animation processing module to the GPU in real time, and executes rendering calculation of the 3D data so as to output an accurate and vivid rendering result. According to the invention, processing and rendering display of 3D data in various formats can be efficiently, accurately and visually completed.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of 3D model rendering, and relates to a 3D model rendering and animation processing system, in particular to a rendering and animation processing system and method for multi-source heterogeneous 3D models, an electronic device and a storage medium. BACKGROUND

[0002] Current 3D model rendering faces multiple pain points: firstly, format heterogeneity, modern three-dimensional content pipeline must support dozens of formats such as FBX, glTF, OBJ, DAE, etc. However, these formats have fundamental differences in coordinate system, unit, and animation organization. The coordinate systems used by different formats are often different and difficult to unify; industrial models are often in centimeters, while game assets use meters, and lack of explicit labeling can easily lead to scale distortion. Therefore, the existing scheme must design an independent loader for each format, resulting in a linear expansion of the maintenance cost of the 3D loading and parsing system with the number of formats.

[0003] Secondly, in terms of material, there are huge differences in material description of different formats, and the material attribute naming is out of line with the physical rendering (PBR) requirements, leading to non-physical and unrealistic lighting abnormalities in rendering. In addition, the traditional scheme generally uses linear interpolation (LERP) to process rotation, which can easily lead to model rotation distortion; the bone level matrix calculation is not optimized, resulting in low rendering efficiency of bone animation and obvious stuttering; animation mixing relies on hard-coded jumping, making it difficult to achieve smooth transition of complex states such as walking, running, and jumping.

[0004] Current industry practice mainly relies on monomer parsing library or engine built-in converter to realize the loading and parsing of 3D models. The former can parse multiple formats, but still requires secondary development of coordinate system conversion and material remapping; the latter is optimized for a specific engine, but is closed source and has poor cross-platform adaptability; and neither of them builds an intermediate representation layer independent of specific formats, and the rendering pipeline is strongly coupled with the format, violating the high cohesion design principle.

[0005] Therefore, there is an urgent need to develop a system for processing 3D model data with different data structures of multiple formats to solve the problems of complex rendering pipeline adaptation, inconsistent material effects, and low animation performance caused by the differences in the data structure of multi-format 3D models in the prior art, in order to overcome at least some of the above-mentioned defects of existing 3D model visualization systems. SUMMARY

[0006] The present application provides a rendering and animation processing system and method for multi-source heterogeneous 3D models, an electronic device and a storage medium, which can efficiently, accurately and visually complete the processing and rendering display of 3D data in multiple formats.

[0007] To solve the above technical problems, according to one aspect of the present application, the following technical solutions are adopted:

[0008] A rendering and animation processing system for multi-source heterogeneous 3D models, the rendering and animation processing system comprising:

[0009] A unified abstraction layer module for performing standardized conversion of raw data to internal scene data; through automatic coordinate system conversion, mesh data optimization, material mapping preprocessing, and skeleton animation preprocessing, the unified abstraction layer module is configured to convert raw data output by different parsers into standardized internal scene data;

[0010] A material mapping engine module for mapping raw materials to internal materials with complete PBR parameters through a material processor with an embedded rule base; the material mapping engine module is configured to map raw material properties to internal materials with complete PBR parameters based on rule base and heuristic matching;

[0011] A skeleton animation processing module for achieving smooth animation effects by performing skeleton matrix calculation and animation state machines;

[0012] A rendering module for real-time transmission of 3D model data processed by the unified abstraction layer module, the material mapping engine module, and the skeleton animation processing module to a GPU, performing rendering calculation of 3D data to output accurate and realistic rendering results.

[0013] As an embodiment of the present application, the unified abstraction layer module is configured to convert raw data output by different parsers into standardized internal scene data through automatic coordinate system conversion, mesh data optimization, material mapping preprocessing, and skeleton animation preprocessing;

[0014] The unified abstraction layer module first traverses the nodes of the input raw data and performs automatic conversion of coordinate systems and unit scaling; second, optimizes the mesh data of the nodes, extracts mesh vertex data and performs normal / tangent calculation optimization to generate optimized meshes; then maps the raw materials to internal materials with PBR parameters through the material mapping engine; finally, pre-processes the skeleton and animation data, compresses redundant keyframes, and reconstructs the skeleton data into a matrix structure to finally output standardized internal scene data;

[0015] The automatic coordinate system conversion is configured to automatically convert the original coordinate system of the input data to the target coordinate system through a set unified conversion matrix; unit scaling refers to converting the input source format unit to centimeters through a set scaling factor, which unifies the scale of the model in the internal scene data, specifically: defining a conversion matrix M conversion , a scaling matrix S, and converting the original node local transformation matrix M local to a matrix M final under the target coordinate system:

[0016] M final = M conversion x M local x S

[0017] The grid data is optimized to create a separate internal grid for each grid of input data, and the position, normal, texture coordinate, tangent, etc. of the vertex data in the grid are uniformly extracted and indexed to improve rendering efficiency.

[0018] As an embodiment of the application, the material mapping engine module is composed of a mapping rule engine composed of rule configuration and intelligent mapping. First, internal materials are created for the input original data, and the numerical color, texture type, etc. of the original material are mapped to standard internal materials in the form of direct mapping, heuristic file name matching, channel separation combination, constant value filling, etc. according to the rule configuration file. In addition, the missing material parameters are filled with default values, the texture path is absolute, and embedded texture reading is supported, and finally the input original material properties are output as standardized internal materials meeting the requirements of the target rendering pipeline.

[0019] The rendering module first performs initialization setting of the graphics pipeline state such as depth test, transparent mixing and face culling to build a basic rendering framework. Then, based on the bone animation characteristics, normal map state and PBR material properties of the current rendering node, the corresponding shader program is selected and bound in real time. Then, the hierarchical model transformation matrix is calculated node by node by depth-first traversal of the standardized scene data, and the bone transformation matrix is transmitted to the GPU synchronously when the bone animation data is detected. Finally, for each grid unit in the scene graph, the vertex data and material resources are associated, and after the PBR texture set and scalar parameters are uniformly bound, the index drawing instruction is executed in a loop to complete the rendering output of the 3D data.

[0020] As an embodiment of the application, the bone animation processing module uses an optimized spherical linear interpolation SLERP algorithm to realize rotation interpolation, and combines hierarchical bone matrix calculation and an animation state machine to realize efficient and smooth animation blending effect.

[0021] The bone animation processing module first performs grid traversal on the input data to identify grids containing bone weights, and creates internal bone matrices for each bone node to build a unified bone hierarchy. At the same time, for the animation data corresponding to each bone node, an internal animation clip is created, and redundant frames are removed to ensure the continuity of the animation. Then, the animation state machine performs SLERP interpolation calculation on each bone node according to the play, mix and stop instructions at each frame to realize smooth transition of the animation. For position keyframes, linear interpolation is used, and the formula is as follows:

[0022]

[0023] Wherein, p(t) is the position at time t, p0, p1 are positions of adjacent two key frames respectively, t0, t1 are times of adjacent two key frames respectively;

[0024] For the rotation key frame, the improved quaternion spherical linear interpolation SLERP algorithm is adopted, the quaternion q1, q2 of adjacent two key frames, interpolation factor t, the included angle θ is calculated first:

[0025] θ=cos -1 (q1·q2)

[0026] When cosθ>0.9995:

[0027] Slerp(q1,q2,t)=normalize(q1+t(q2-q1))

[0028] Otherwise:

[0029]

[0030] The calculation result is concatenated with the global bone matrix to generate the final bone transform matrix, wherein the bone matrix calculation satisfies:

[0031] M final_bone =M global ×M offset

[0032] Wherein, M offset is a bone offset matrix, M global is a global bone matrix;

[0033] Finally, according to the bone hierarchy and the interpolated key frame data, it is applied to the corresponding vertex, and the playing of the bone animation is realized.

[0034] According to another aspect of the application, the following technical scheme is adopted: a rendering and animation processing method for multi-source heterogeneous 3D models, the rendering and animation processing method comprising:

[0035] The uniform abstraction layer step: performing standardized conversion of original data to internal scene data; through automatic coordinate system conversion, mesh data optimization, material mapping preprocessing, and skeleton animation preprocessing, the original data output by different parsers is converted into standardized internal scene data;

[0036] The material mapping step: through the material processor of the built-in rule library, the original material is mapped to the internal material of complete PBR parameters; based on the rule library and heuristic matching, the original material properties are mapped to the internal material containing complete PBR parameters;

[0037] Skeleton animation processing step: by performing skeleton matrix calculation and animation state machine to achieve smooth animation effect;

[0038] Rendering step: real-time transmission of 3D model data processed by the uniform abstract layer module, material mapping engine module and skeleton animation processing module to GPU, performing rendering calculation of 3D data to output accurate and realistic rendering results.

[0039] As an embodiment of the present application, in the uniform abstract layer step, the original data output by different parsers is converted into standardized internal scene data through automatic coordinate system conversion, mesh data optimization, material mapping preprocessing and skeleton animation preprocessing;

[0040] First, the nodes of the input original data are traversed, and automatic conversion and unit scaling of the coordinate system are performed; second, the mesh data of the nodes is optimized, the mesh vertex data is extracted and normal / tangent calculation optimization is performed to generate an optimized mesh; then the original material is mapped to internal material containing PBR parameters through the material mapping engine; finally, the skeleton and animation data are preprocessed, redundant key frames are compressed, and the skeleton data is reconstructed into a matrix structure, and finally the standardized internal scene data is output;

[0041] The uniform abstract layer step includes a coordinate system automatic conversion step and a mesh data optimization step.

[0042] In the coordinate system automatic conversion step, the original coordinate system of the input data is automatically converted into the target coordinate system through the set uniform conversion matrix; unit scaling refers to converting the input source format unit into centimeters through the set scaling factor, which unifies the scale of the model in the internal scene data, specifically: defining the conversion matrix M conversion , the scaling matrix S, and converting the original node local transformation matrix M local into the matrix M final under the target coordinate system:

[0043] M final =M conversion ×M local ×S

[0044] In the mesh data optimization step, a separate internal mesh is created for each mesh of the input data, and the position, normal, texture coordinates and tangent of the vertex data in the mesh are uniformly extracted and indexed optimized to improve rendering efficiency.

[0045] As an embodiment of the present application, in the material mapping step, the mapping rule engine composed of rule configuration and intelligent mapping is used to first create internal materials for the input raw data, and then map the numerical color, texture type, etc. of the raw material to the standard internal material in the form of direct mapping, heuristic file name matching, channel separation combination, constant value filling, etc. according to the rule configuration file. In addition, the default value is filled for the missing material parameters, the texture path is absolute, and the embedded texture reading is supported, and finally the input raw material properties are output as standardized internal materials meeting the requirements of the target rendering pipeline.

[0046] In the rendering step, first, the initialization setting of the graphics pipeline state such as depth test, transparent mixing and surface culling is performed to build a basic rendering framework; then, based on the bone animation features, normal map state and PBR material properties of the current rendering node, the corresponding shader program is selected and bound in real time; then, the hierarchical model transformation matrix is calculated node by node through the depth-first traversal of the standardized scene data, and the bone transformation matrix is transmitted to the GPU in synchronization when the bone animation data is detected; finally, for each mesh unit in the scene graph, the vertex data and material resources are associated, and after the PBR texture set and scalar parameters are uniformly bound, the index drawing instruction is executed in a loop to complete the rendering output of the 3D data.

[0047] As an embodiment of the present application, in the bone animation processing step, the optimized spherical linear interpolation SLERP algorithm is used to realize rotation interpolation, and the hierarchical bone matrix calculation and animation state machine are combined to realize efficient and smooth animation mixing effect.

[0048] First, the input data is traversed to identify the mesh containing the bone weight, and an internal bone matrix is created for each bone node to build a unified skeletal hierarchy; at the same time, for the animation data corresponding to each bone node, an internal animation clip is created, and redundant frames are removed to ensure the continuity of the animation; then, the animation state machine performs SLERP interpolation calculation for each bone node at each frame according to the play, mix and stop instructions to realize smooth transition of the animation. For position keyframes, linear interpolation is used, and the formula is as follows:

[0049]

[0050] Where p(t) is the position at time t, p0 and p1 are the positions of the adjacent two keyframes, and t0 and t1 are the times of the adjacent two keyframes.

[0051] For rotation keyframes, the improved quaternion spherical linear interpolation SLERP algorithm is used to interpolate the quaternions q1 and q2 of the adjacent two keyframes with the interpolation factor t. First, the included angle θ is calculated:

[0052] cos -1 (q1·q2)

[0053] When cos 0 > 0.9995:

[0054] Slerp(q1, q2, t) = normalize(q1 + t(q2 - q1))

[0055] Otherwise:

[0056]

[0057] The calculation result is concatenated with a global bone matrix to generate a final bone transform matrix, wherein the bone matrix calculation satisfies:

[0058] M final_bone = M global × M offset

[0059] Wherein, M offset is a bone offset matrix, and M global is a global bone matrix.

[0060] Finally, according to the bone hierarchy and the interpolated keyframe data, the corresponding vertices are applied to realize the playing of the bone animation.

[0061] According to another aspect of the present application, the following technical solution is adopted: an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0062] According to another aspect of the present application, the following technical solution is adopted: a storage medium having computer program instructions stored thereon, wherein the computer program instructions are executed by a processor to implement the steps of the above method.

[0063] The rendering and animation processing system and method for multi-source heterogeneous 3D models according to the present application can efficiently, accurately and visually process and render display various formats of 3D data. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 The figure is a component diagram of the rendering and animation processing system for multi-source heterogeneous 3D models according to an embodiment of the present application.

[0065] Figure 2 The figure is a flowchart of the rendering and animation method for multi-source heterogeneous 3D models according to an embodiment of the present application.

[0066] Figure 3A unified abstract layer data processing flowchart in an embodiment of the present application.

[0067] Figure 4 A material mapping engine working principle schematic diagram in an embodiment of the present application.

[0068] Figure 5 A skeletal animation processing flowchart in an embodiment of the present application.

[0069] Figure 6 A rendering pipeline schematic diagram in an embodiment of the present application.

[0070] Figure 7 A schematic diagram of the composition of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The preferred embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0072] In order to further understand the present application, the preferred embodiments of the present application are described below in conjunction with the embodiments, but it should be understood that the description is only for further illustrating the features and advantages of the present application, and is not a limitation on the claims of the present application.

[0073] The description in this part is only for several typical embodiments, and the present application is not limited to the scope of the embodiment description. The same or similar prior art means and some technical features in the embodiments are replaced with each other, which is within the description and protection scope of the present application.

[0074] The description of the steps in each embodiment in the specification is only for convenience of description, and the implementation mode of the present application is not limited by the order of the step implementation.

[0075] In the specification, "connection" includes direct connection and indirect connection, such as connection through some active devices, passive devices or electrically conductive media; and can also include the connection of other active devices or passive devices known to those skilled in the art on the basis of achieving the same or similar functional purposes, such as the connection of circuits or components such as switches, follower circuits, etc.

[0076] The present application discloses a rendering and animation processing system for multi-source heterogeneous 3D models, Figure 1 A schematic diagram of the composition of a rendering and animation processing system for multi-source heterogeneous 3D models in an embodiment of the present application; please refer to Figure 1 The rendering and animation processing system comprises a unified abstract layer module 1, a material mapping engine module 2, a skeletal animation processing module 3 and a rendering module 4.

[0077] The unified abstract layer module 1 is used to perform standardized conversion of raw data to internal scene data; through automatic coordinate system conversion, mesh data optimization, material mapping preprocessing, and skeleton animation preprocessing, the raw data output by different parsers is converted into standardized internal scene data.

[0078] The material mapping engine module 2 is used to map the original material to the internal material with complete PBR parameters through the material processor of the built-in rule library; the original material properties are mapped to the internal material with complete PBR parameters through rule library and heuristic matching.

[0079] The skeleton animation processing module 3 is used to achieve smooth animation effect by performing skeleton matrix calculation and animation state machine.

[0080] The rendering module 4 is used to transmit the 3D model data processed by the unified abstract layer module, the material mapping engine module and the skeleton animation processing module to the GPU in real time, perform rendering calculation of 3D data, and output accurate and realistic rendering results.

[0081] In an embodiment of the present application, the unified abstract layer module 1 is used to convert the raw data output by different parsers into standardized internal scene data through automatic coordinate system conversion, mesh data optimization, material mapping preprocessing, and skeleton animation preprocessing.

[0082] The unified abstract layer module 1 first traverses the nodes of the input raw data and performs automatic conversion and unit scaling of the coordinate system; secondly, the mesh data of the nodes is optimized, the mesh vertex data is extracted and the normal / tangent calculation optimization is performed, and the optimized mesh is generated; then the original material is mapped to the internal material with PBR parameters through the material mapping engine; finally, the skeleton and animation data are preprocessed, the redundant key frames are compressed, and the skeleton data is reconstructed into a matrix structure, and finally the standardized internal scene data is output.

[0083] The unified abstract layer module 1 can include a coordinate system automatic conversion unit and a mesh data optimization unit. The coordinate system automatic conversion unit is used to automatically convert the original coordinate system of the input data to the target coordinate system through a set of unified conversion matrices; unit scaling refers to converting the input source format unit to centimeters through a set of scaling factors, and the scale of the unified model in the internal scene data is specific: defining the conversion matrix M conversion , the scaling matrix S, and converting the original node local transformation matrix M local to the matrix M final under the target coordinate system:

[0084] M final =M conversion ×M local ×S

[0085] The mesh data optimization unit is used to create a separate internal mesh for each mesh of the input data, and uniformly extract and index optimize the position, normal, texture coordinate, and tangent of the vertex data in the mesh, so as to improve the rendering efficiency.

[0086] In an embodiment of the present application, the material mapping engine module 2 is composed of a mapping rule engine of rule configuration and intelligent mapping. First, the internal material is created for the input original data, and the numerical color, texture type, and the like of the original material are mapped to the standard internal material in the manner of direct mapping, heuristic file name matching, channel separation combination, constant value filling, and the like according to the rule configuration file. In addition, the default value is filled for the missing material parameters, the texture path is absolutized, and embedded texture reading is supported, and finally the input original material attribute is output as the standardized internal material meeting the requirements of the target rendering pipeline.

[0087] The skeleton animation processing module 3 is used to realize rotation interpolation by using an optimized spherical linear interpolation SLERP algorithm, and realize efficient and smooth animation mixing effect by combining hierarchical skeleton matrix calculation and an animation state machine.

[0088] The skeleton animation processing module first performs mesh traversal on the input data to identify the mesh containing the skeleton weight, and creates an internal skeleton matrix for each skeleton node to build a unified skeleton hierarchy. At the same time, an internal animation clip is created for the animation data corresponding to each skeleton node, and redundant frames are removed to ensure the continuity of the animation. Then, the animation state machine performs SLERP interpolation calculation on each skeleton node according to the play, mix, and stop instructions at each frame to realize smooth transition of the animation. For position keyframes, linear interpolation is used, and the formula is as follows:

[0089]

[0090] Wherein, p(t) is the position at time t, p0 and p1 are the positions of the adjacent two keyframes, and t0 and t1 are the times of the adjacent two keyframes.

[0091] For rotation keyframes, an improved quaternion spherical linear interpolation SLERP algorithm is used to interpolate the quaternion q1 and q2 of the adjacent two keyframes and the interpolation factor t. First, the included angle θ is calculated as follows:

[0092] θ = cos -1 (q1 · q2)

[0093] When cosθ>0.9995:

[0094] Slerp(q1, q2, t) = normalize(q1 + t(q2 - q1))

[0095] Otherwise:

[0096]

[0097] The calculation result is cascaded with a global bone matrix to generate a final bone transform matrix, wherein the bone matrix calculation satisfies:

[0098] M final_bone = M global x M offset

[0099] Wherein, M offset is a bone offset matrix, M global is a global bone matrix;

[0100] Finally, according to the bone hierarchy and the interpolated keyframe data, it is applied to the corresponding vertex to realize the playing of the bone animation.

[0101] The rendering module 4 first carries out initialization setting of graphic pipeline states such as depth test, transparent mixing and surface culling to construct a basic rendering framework; then, based on the bone animation characteristics, normal map state and PBR material properties of the current rendering node, corresponding shader programs are selected and bound in real time; then, through depth-first traversal of standardized scene data, hierarchical model transformation matrices are calculated node by node, and when bone animation data is detected, bone transformation matrices are transmitted to the GPU synchronously; finally, for each mesh unit in the scene graph, the vertex data and material resources thereof are associated, and after the PBR texture set and scalar parameters are uniformly bound, the index drawing instruction is executed in a loop to complete the rendering output of 3D data.

[0102] The application further discloses a rendering and animation processing method for multi-source heterogeneous 3D models, which comprises the following steps:

[0103] Step 1: uniform abstract layer step: performing standardized conversion of original data to internal scene data; through automatic coordinate system conversion, mesh data optimization, material mapping preprocessing and bone animation preprocessing, the original data output by different parsers is converted into standardized internal scene data.

[0104] In an embodiment of the application, in the uniform abstract layer step, through automatic coordinate system conversion, mesh data optimization, material mapping preprocessing and bone animation preprocessing, the original data output by different parsers is converted into standardized internal scene data.

[0105] First, the nodes of the input original data are traversed, and automatic conversion of coordinate system and unit scaling are performed; second, the mesh data of the nodes are optimized, mesh vertex data are extracted and normal / tangent calculation optimization is performed to generate an optimized mesh; then the original material is mapped to an internal material containing PBR parameters through a material mapping engine; finally, the skeleton and animation data are preprocessed, redundant key frames are compressed, and the skeleton data are reconstructed into a matrix structure, and finally standardized internal scene data are output.

[0106] In an embodiment, the uniform abstract layer step comprises a coordinate system automatic conversion step and a mesh data optimization step.

[0107] In the coordinate system automatic conversion step, the original coordinate system of the input data is automatically converted into a target coordinate system through a set uniform conversion matrix; unit scaling refers to converting the input source format unit into centimeters through a set scaling factor, which is the scale of the uniform model in the internal scene data, and specifically, a conversion matrix M conversion is defined, a scaling matrix S is defined, and the original node local transformation matrix M local is converted into a matrix M final under the target coordinate system:

[0108] M final =M conversion *M local *S

[0109] In the mesh data optimization step, a separate internal mesh is created for each mesh of the input data, and the position, normal, texture coordinate and tangent of the vertex data in the mesh are uniformly extracted and indexed to optimize the rendering efficiency.

[0110]

Step 2

[0111] In an embodiment of the present application, in the material mapping step, a mapping rule engine composed of rule configuration and intelligent mapping is used to first create an internal material for the input original data, and according to the rule configuration file, the numerical color and texture type of the original material are mapped to a standard internal material in the form of direct mapping, heuristic file name matching, channel separation combination, constant value filling, etc. In addition, the missing material parameters are filled with default values, the texture path is absolute, and embedded texture reading is supported, and finally the input original material properties are output as standardized internal materials meeting the requirements of the target rendering pipeline.

[0112]

Step 3

[0113] In an embodiment of the present application, in the skeleton animation processing step, an optimized spherical linear interpolation SLERP algorithm is used to realize rotation interpolation, and a hierarchical skeleton matrix calculation and an animation state machine are combined to realize efficient and smooth animation blending effect.

[0114] First, the input data is mesh traversed to identify the mesh containing the bone weight, and an internal bone matrix is created for each bone node to build a unified bone hierarchy. At the same time, for the animation data corresponding to each bone node, an internal animation clip is created, and redundant frames are removed to ensure the continuity of the animation. Then, the animation state machine performs SLERP interpolation calculation on each bone node at each frame according to the play, blend, and stop instructions to realize smooth transition of the animation. For position keyframes, linear interpolation is used, and the formula is as follows:

[0115]

[0116] Where p(t) is the position at time t, p0 and p1 are the positions of the adjacent two keyframes, and t0 and t1 are the times of the adjacent two keyframes.

[0117] For rotation keyframes, an improved quaternion spherical linear interpolation SLERP algorithm is used to interpolate the quaternions q1 and q2 of the adjacent two keyframes with the interpolation factor t. First, the included angle θ is calculated:

[0118] θ=cos -1 (q1·q2)

[0119] When cosθ>0.9995:

[0120] Slerp(q1,q2,t)=normalize(q1+t(q2-q1))

[0121] Otherwise:

[0122]

[0123] The calculation result is concatenated with the global skeleton matrix to generate the final skeleton transformation matrix, where the skeleton matrix calculation satisfies:

[0124] M final_bone =M global ×M offset

[0125] Where M offset is the bone offset matrix, and M global is the global skeleton matrix.

[0126] Finally, based on the skeletal hierarchy and the interpolated keyframe data, the data is applied to the corresponding vertices to achieve the playback of the skeletal animation.

[0127]

Step 4

[0128] In one embodiment of the present invention, the rendering step first initializes the graphics pipeline states such as depth testing, transparency blending, and face culling to build a basic rendering framework; then, based on the skeletal animation features, normal map state, and PBR material properties of the current rendering node, the corresponding shader program is selected and bound in real time; then, the normal map state is calculated node by node through depth-first traversal of the standardized scene data, and the skeletal transformation matrix is ​​synchronously transmitted to the GPU when skeletal animation data is detected; finally, for each mesh unit in the scene graph, its vertex data and material resources are associated, the PBR texture set and scalar parameters are uniformly bound, and the indexed drawing instructions are executed in a loop to complete the rendering output of 3D data.

[0129] In one embodiment of the present invention, when the system starts, the resource manager receives a list of file paths input by the user, in formats including FBX, gLTF, OBJ, etc. In one embodiment of the present invention, taking input data FBX as an example, such as... Figure 3 As shown, in step S1, the unified abstraction layer module 1 first receives and parses the raw data into a unified structure SourceData, which contains all nodes, meshes, materials, and animation data of the raw data. Then, coordinate system transformation and unit scaling are performed on the SourceData to generate preprocessed data. Specifically, the transformation matrix M is first defined. conversion :

[0130]

[0131] And the scaling matrix S that converts units to meters:

[0132]

[0133] Finally, the original node local transformation matrix M local Transform the matrix M into the target coordinate system final .

[0134] M final =M conversion ×M local ×S

[0135] In step S2, the normal and tangent calculation optimization is performed on the preprocessed data, and the MikkTSpace algorithm is called to generate the optimized mesh data matrix TBN:

[0136]

[0137] The position, normal, texture coordinate, tangent and the like of the vertex data in the mesh are uniformly extracted and indexed optimized through the above conversion, and the data processing efficiency is improved.

[0138] In step S3, as shown in Figure 4 , the original material of the data is intelligently mapped by the material mapping engine module 2. First, the rule configuration file MaterialRules.json is loaded:

[0139]

[0140] The numerical color and texture type of the original material are mapped into the standard internal material in the form of direct mapping, heuristic file name matching, channel separation combination, constant value filling and the like. In addition, the missing material parameters are filled with default values, wherein the metallic degree metallic is set to 0, the roughness is set to 0.5, the texture path is absolute, and embedded texture reading is supported. Finally, the input original material properties are output as standardized internal materials that meet the requirements of the target rendering pipeline.

[0141] In step S4, the bone and animation data are processed by the bone animation processing module 3. Specifically, as shown in Figure 5 , first, the input data is mesh traversed to identify the mesh containing the bone weight, and an internal bone matrix is created for each bone node to build a unified bone hierarchy structure. At the same time, the animation duration and key frame data corresponding to each bone node are created, and redundant frames are removed to ensure the continuity of the animation.

[0142] Then, according to the play, mix and stop instructions, SLERP interpolation calculation is performed on each bone node at each frame to realize smooth transition of the animation. For position key frames, linear interpolation is adopted, and the formula is as follows:

[0143]

[0144] Where p(t) is the position at time t, p0 and p1 are the positions of the adjacent two key frames, and t0 and t1 are the times of the adjacent two key frames.

[0145] For the rotation key frame, an improved quaternion spherical linear interpolation SLERP algorithm is adopted, and the included angle θ is calculated for the adjacent two key frames of quaternion q1, q2, interpolation factor t:

[0146] θ=cos -1 (q1·q2)

[0147] When cosθ>0.9995:

[0148] Slerp(q1,q2,t)=normalize(q1+t(q2-q1))

[0149] Otherwise:

[0150]

[0151] The calculation result is concatenated with the global bone matrix to generate a final bone transformation matrix, wherein the bone matrix calculation satisfies:

[0152] M final_bone =N gkibak ×N offset

[0153] Wherein N offset is a bone offset matrix, and M global is a global bone matrix.

[0154] Then, according to the bone hierarchy and the interpolated key frame data, it is applied to the corresponding vertex to realize the playing of the bone animation.

[0155] The data processed through the above steps is finally rendered and displayed by the rendering module 4, as shown in Figure 6 , first, the initialization setting of the graphics pipeline state such as depth test, transparent mixing and surface culling is carried out, so as to construct the basic rendering framework; then, based on the bone animation characteristics, normal map state and PBR material properties of the current rendering node, the corresponding shader program is selected and bound in real time; then, the hierarchical model transformation matrix is calculated node by node through the depth-first traversal of the standardized scene data, and when the bone animation data is detected, the bone transformation matrix is transmitted to the GPU; finally, for each mesh unit in the scene graph, the vertex data and material resources are associated, the PBR texture set and scalar parameters are uniformly bound, and the index drawing instruction is executed in a loop to complete the rendering output of the 3D data.

[0156] The application also discloses an electronic device, Figure 7 is a component schematic diagram of the electronic device in an embodiment of the application; please refer to Figure 7The electronic device comprises a memory, a processor and at least one communication interface at the hardware level; the processor can be a microprocessor, and the memory can comprise an internal memory, such as a random access memory (RAM) and a non-volatile memory. Of course, other hardware can also be provided according to requirements.

[0157] The processor, the communication interface and the memory can be connected with each other through an internal bus. The memory is used for storing programs (which can comprise an operating system program and an application program); the programs can comprise program codes, and the program codes can comprise computer operation instructions. The memory can comprise an internal memory and a non-volatile memory, and provide instructions and data for the processor.

[0158] In an embodiment, the processor can read corresponding programs from the non-volatile memory into the internal memory, and then run; the processor can execute the programs stored in the memory, and specifically used for executing the following operations (as shown in the Figure 2

[0159] Step 1: Unified abstract layer step: performing standardized conversion of original data to internal scene data; through automatic coordinate system conversion, grid data optimization, material mapping preprocessing, skeleton animation preprocessing, the original data output by different parsers is converted into standardized internal scene data.

[0160] Step 2: Material mapping step: through the material processor of the built-in rule library, the original material is mapped to the internal material of complete PBR parameters; based on the rule library and heuristic matching, the original material properties are mapped to the internal material containing complete PBR parameters.

[0161] Step 3: Skeleton animation processing step: through the execution of skeleton matrix calculation and animation state machine, smooth animation effect is realized.

[0162] Step 4: Rendering step: real-time transmission of the 3D model data processed by the unified abstract layer module, the material mapping engine module and the skeleton animation processing module to the GPU, performing rendering calculation of the 3D data, so as to output accurate and realistic rendering results.

[0163] The application further discloses a storage medium, which stores computer program instructions, and the computer program instructions are executed by a processor to realize the following steps (as shown in the Figure 2

[0164] ​​[Step 1] Unified abstraction layer step: performing standardized conversion of raw data to internal scene data; through automatic coordinate system conversion, mesh data optimization, material mapping preprocessing, skeleton animation preprocessing, converting raw data output by different parsers into standardized internal scene data.

[0165] [Step 2] Material mapping step: through a material processor with a built-in rule library, mapping the original material to an internal material with complete PBR parameters; based on rule library and heuristic matching, mapping the original material attributes to an internal material with complete PBR parameters.

[0166] [Step 3] Skeleton animation processing step: through performing skeleton matrix calculation and animation state machine to achieve smooth animation effect.

[0167] [Step 4] Rendering step: real-time transmission of 3D model data processed by the unified abstraction layer module, material mapping engine module and skeleton animation processing module to GPU, performing rendering calculation of 3D data to output accurate and realistic rendering results.

[0168] In summary, the rendering and animation processing system and method for multi-source heterogeneous 3D models, the electronic device and the storage medium can efficiently, accurately and visually complete the processing and rendering display of 3D data in multiple formats.

[0169] It should be noted that the present application can be implemented in software and / or a combination of software and hardware; for example, an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device can be used. In some embodiments, the software program of the present application can be executed by a processor to implement the above steps or functions. Similarly, the software program of the present application (including related data structures) can be stored in a computer readable recording medium; for example, RAM memory, magnetic or optical drive or soft disk and similar devices. In addition, some steps or functions of the present application can be implemented by hardware; for example, as a circuit cooperating with the processor to perform each step or function.

[0170] The technical features of the above-described embodiments can be combined in any manner, and to make the description concise, not all possible combinations of the technical features in the above-described embodiments are described, however, as long as the combinations of the technical features do not exist contradictions, it should be considered as the scope of the present disclosure.

[0171] The description and applications of the present application are illustrative, and not intended to limit the scope of the present application. The effects or advantages of the embodiments can not be achieved in the embodiments due to various factors, and the description of the effects or advantages is not intended to limit the embodiments. Variations and modifications of the embodiments disclosed herein can be possible, and various components of the embodiments can be substituted and equivalents can be used. It should be clear to those skilled in the art that the present application can be implemented in other forms, structures, arrangements, proportions, and with other components, materials, and parts, without departing from the spirit or essential characteristics of the present application. Other variations and modifications of the embodiments disclosed herein can be made without departing from the scope and spirit of the present application.

Claims

1. A rendering and animation processing system for multi-source heterogeneous 3D models, characterized in that, The rendering and animation processing system comprises: a unified abstract layer module for performing standardized conversion of original data to internal scene data; through automatic coordinate system conversion, mesh data optimization, material mapping preprocessing, and skeleton animation preprocessing, the original data output by different parsers is converted into standardized internal scene data; a material mapping engine module for mapping original materials to internal materials with complete PBR parameters through a material processor with a built-in rule library; through rule library and heuristic matching, the original material properties are mapped to internal materials with complete PBR parameters; a skeleton animation processing module for achieving smooth animation effects by performing skeleton matrix calculation and animation state machines; a rendering module for transmitting 3D model data processed by the unified abstract layer module, the material mapping engine module, and the skeleton animation processing module to a GPU in real time, performing rendering calculation of 3D data, and outputting accurate and realistic rendering results.

2. The rendering and animation processing system for multi-source heterogeneous 3D models according to claim 1, wherein: the unified abstract layer module is used to convert the original data output by different parsers into standardized internal scene data through automatic coordinate system conversion, mesh data optimization, material mapping preprocessing, and skeleton animation preprocessing; the unified abstract layer module first traverses the nodes of the input original data and performs automatic conversion and unit scaling of the coordinate system; secondly, optimizes the mesh data of the nodes, extracts mesh vertex data, and performs normal / tangent calculation optimization to generate an optimized mesh; then maps the original material to an internal material with PBR parameters through a material mapping engine; finally, pre-processes the skeleton and animation data, compresses redundant keyframes, and reconstructs the skeleton data into a matrix structure to finally output standardized internal scene data; The unified abstract layer module comprises a coordinate system automatic conversion unit and a grid data optimization unit; the coordinate system automatic conversion unit is used to automatically convert the original coordinate system of input data into a target coordinate system through a set unified conversion matrix; unit scaling refers to converting the input source format unit into centimeters through a set scaling factor, and the scale of the unified model in the internal scene data is specifically defined as follows: defining a conversion matrix M conversion , a scaling matrix S, and converting the original node local transformation matrix M local into a matrix M final under the target coordinate system: M final = M conversion x M local x S the mesh data optimization unit is used to create a separate internal mesh for each mesh of the input data, and uniformly extract and index the position, normal, texture coordinate, and tangent of the vertex data in the mesh to improve rendering efficiency.

3. The rendering and animation processing system for multi-source heterogeneous 3D models according to claim 1, wherein: the material mapping engine module comprises a mapping rule engine composed of rule configuration and intelligent mapping, first creates an internal material for the input original data, and according to the rule configuration file, maps the numerical color and texture type of the original material to a standard internal material in direct mapping, heuristic file name matching, channel separation combination, and constant value filling; additionally, fills in default values for missing material parameters, performs absolute processing on texture paths, supports embedded texture reading, and finally outputs the input original material properties as standardized internal materials meeting the requirements of the target rendering pipeline; The rendering module first performs initialization setting of graphics pipeline states such as depth test, transparent mixing and face culling to construct a basic rendering framework; then, based on the skeletal animation features, normal map state and PBR material properties of the current rendering node, corresponding shader programs are selected and bound in real time; then, through depth-first traversal of standardized scene data, hierarchical model transformation matrices are calculated node by node, and when skeletal animation data is detected, the skeletal transformation matrix is transmitted to the GPU in synchronization; finally, for each mesh element in the scene graph, the vertex data and material resources thereof are associated, and after the PBR texture set and scalar parameters are uniformly bound, the index drawing instruction is executed in a loop to complete the rendering output of the 3D data.

4. The rendering and animation processing system for multi-source heterogeneous 3D models according to claim 1, characterized in that: The skeletal animation processing module is used to realize rotation interpolation by using an optimized spherical linear interpolation (SLERP) algorithm, and to realize efficient and smooth animation mixing effect by combining hierarchical skeletal matrix calculation and an animation state machine; The skeletal animation processing module first performs mesh traversal on the input data to identify meshes containing skeletal weights, and creates an internal skeletal matrix for each skeletal node to construct a unified skeletal hierarchy; at the same time, for the animation data corresponding to each skeletal node, an internal animation clip is created, and redundant frames are removed to ensure the continuity of the animation; then, the animation state machine performs SLERP interpolation calculation on each skeletal node at each frame according to the play, mix and stop instructions to realize smooth transition of the animation; for position keyframes, linear interpolation is used, and the formula is as follows: Where p(t) is the position at time t, p0 and p1 are the positions of the adjacent two keyframes, and t0 and t1 are the times of the adjacent two keyframes; For rotation keyframes, an improved quaternion spherical linear interpolation (SLERP) algorithm is used to interpolate the quaternions q1 and q2 of the adjacent two keyframes with the interpolation factor t. First, the included angle θ is calculated: cos -1 (q1·q2) When cosθ>0.9995: Slerp(q1,q2,t)=normalize(q1+t(q2-q1)) Otherwise: The calculation result is concatenated with the global skeletal matrix to generate the final skeletal transformation matrix, wherein the skeletal matrix calculation satisfies: M final_bone = M global x M offset where M offset is a bone offset matrix, M global is a global bone matrix; Finally, according to the skeletal hierarchy and the interpolated keyframe data, the corresponding vertices are applied to realize the playback of the skeletal animation.

5. A rendering and animation processing method for multi-source heterogeneous 3D models, characterized in that, The rendering and animation processing method comprises: A uniform abstraction layer step: performing standardized conversion of original data to internal scene data; through automatic coordinate system conversion, mesh data optimization, material mapping preprocessing and skeletal animation preprocessing, the original data output by different parsers is converted into standardized internal scene data; A material mapping step: through the material processor of the built-in rule library, the original material is mapped to the internal material with complete PBR parameters; based on the rule library and heuristic matching, the original material properties are mapped to the internal material with complete PBR parameters; A skeletal animation processing step: through the execution of skeletal matrix calculation and animation state machine, smooth animation effect is realized. The rendering step: real-time transmission of the 3D model data processed by the uniform abstract layer module, the material mapping engine module and the skeletal animation processing module to the GPU, performing rendering calculation of the 3D data to output accurate and realistic rendering results.

6. The rendering and animation processing method for multi-source heterogeneous 3D models according to claim 5, characterized in that: In the uniform abstract layer step, the original data output by different parsers is converted into standardized internal scene data through automatic coordinate system conversion, mesh data optimization, material mapping preprocessing, and skeletal animation preprocessing. First, the nodes of the input original data are traversed, and automatic conversion and unit scaling of the coordinate system are performed; second, the mesh data of the nodes are optimized, the mesh vertex data is extracted and normal / tangent calculation optimization is performed to generate optimized meshes; then the original material is mapped to internal material containing PBR parameters through the material mapping engine; finally, the skeletal and animation data are preprocessed, redundant key frames are compressed, and the skeletal data are reconstructed into a matrix structure, and finally the standardized internal scene data are output. In the coordinate system automatic conversion step, the original coordinate system of the input data is automatically converted into the target coordinate system through a set unified conversion matrix; unit scaling refers to converting the input source format unit into centimeters through a set scaling factor, and unifying the scale of the model in the internal scene data, specifically: defining a conversion matrix M conversion , a scaling matrix S, and converting the original node local transformation matrix M local into a matrix M final under the target coordinate system: M final = M conversion x M local x S In the mesh data optimization step, a separate internal mesh is created for each mesh of the input data, and the position, normal, texture coordinate and tangent of the vertex data in the mesh are uniformly extracted and indexed optimized to improve the rendering efficiency.

7. The rendering and animation processing method for multi-source heterogeneous 3D models according to claim 5, characterized in that: In the material mapping step, the mapping rule engine composed of rule configuration and intelligent mapping is used to create internal material for the input original data, and the numerical color and texture type of the original material are mapped to standard internal material according to the rule configuration file in the form of direct mapping, heuristic file name matching, channel separation combination, and constant value filling. In addition, the missing material parameters are filled with default values, the texture path is absolute, and embedded texture reading is supported, and finally the input original material properties are output as standardized internal material meeting the requirements of the target rendering pipeline. In the rendering step, the initialization setting of the graphics pipeline state such as depth test, transparent mixing and face culling is performed to build a basic rendering framework; then based on the skeletal animation features, normal map state and PBR material properties of the current rendering node, the corresponding shader program is selected and bound in real time; then the hierarchical model transformation matrix is calculated node by node through depth-first traversal of the standardized scene data, and the skeletal transformation matrix is transmitted to the GPU in synchronization when the skeletal animation data is detected; finally, for each mesh unit in the scene graph, its vertex data and material resources are associated, the PBR texture set and scalar parameters are uniformly bound, and the index drawing instruction is executed in a loop to complete the rendering output of the 3D data.

8. The rendering and animation processing method for multi-source heterogeneous 3D models according to claim 5, characterized in that: In the skeletal animation processing step, the optimized spherical linear interpolation SLERP algorithm is used to realize rotation interpolation, and the hierarchical skeletal matrix calculation and animation state machine are used to realize efficient and smooth animation blending effect. Firstly, the input data is traversed to identify the mesh containing the bone weights and create an internal bone matrix for each bone node to build a unified bone hierarchy; at the same time, for each bone node corresponding animation data, create an internal animation clip and perform redundant frame elimination to ensure the continuity of the animation; then the animation state machine executes SLERP interpolation calculation for each bone node according to the play, mix, stop instructions at each frame to realize the smooth transition of the animation; for the position key frame, linear interpolation is adopted, and the formula is as follows: Wherein, p(t) is the position at time t, p0, p1 are the positions of the adjacent two key frames, t0, t1 are the times of the adjacent two key frames; For the rotation key frame, the improved quaternion spherical linear interpolation SLERP algorithm is adopted, and for the quaternion q1, q2 of the adjacent two key frames, the interpolation factor t is calculated first: cos -1 (q1·q2) When cosθ>0.9995: Slerp(q1,q2,t)=normalize(q1+t(q2-q1)) Otherwise: The calculation result is concatenated with the global bone matrix to generate the final bone transformation matrix, wherein the bone matrix calculation satisfies: M final_bone = M global x M offset where M offset is a bone offset matrix, M global is a global bone matrix; Finally, according to the bone hierarchy and the interpolated key frame data, it is applied to the corresponding vertex to realize the playing of the bone animation.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the steps of the method of any one of claims 5 to 8.

10. A storage medium having stored thereon computer program instructions, characterized in that, The computer program instructions are executed by the processor to realize the steps of the method of any one of claims 5 to 8.

Images