A virtual pet 3D scene rendering system
By combining dual quaternions and three-dimensional directed distance fields, the problem of separating skin deformation and physical collision in the rendering of virtual pet 3D scenes is solved, achieving stable joint mesh volume and smooth hair collision, thus improving rendering efficiency and stability.
Patent Information
- Application Number
- CN202611295000.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-25
- Publication Date
- 2026-11-10
AI Technical Summary
Existing 3D scene rendering technologies suffer from several drawbacks when handling interactions between virtual pets and complex indoor spaces. These include the problem of repeated data updates caused by processing skin deformation and physical collisions separately, and the difficulty in maintaining stable joint mesh volumes and smooth, continuous hair collisions, resulting in low rendering efficiency.
Dual quaternions are used to uniformly represent the rotation and translation of bone nodes. Hybrid dual quaternions are calculated by combining the skinning weights of multiple dominant bones. Hair collisions are handled by combining physical proxy meshes and three-dimensional directional distance fields. Dual quaternion skinning deformation, three-dimensional directional distance field collision deflection, and subsurface scattering shading are completed through a unified graphics rendering pipeline.
While maintaining stable joint mesh volume and smooth hair collisions, the rendering efficiency and stability of virtual pets in complex indoor scenes are improved by reducing redundant data transfers and memory state switching.
Smart Images

Figure CN123071510A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of 3D modeling technology, and more specifically, to a virtual pet 3D scene rendering system. Background Technology
[0002] Real-time rendering and display of virtual pets in indoor 3D scenes is used to assist in the construction of 3D digital interactive environments. Existing 3D scene rendering technologies typically use mesh models bound to skeletal animations and generate images in conjunction with preset graphics rendering workflows. When virtual pets are in indoor spaces containing complex geometries such as furniture, walls, and floors, the rendering process requires real-time handling of deformation of high-density hair mesh vertices, pose switching of complex skeletal chains, and dynamic physical collisions and spatial occlusion relationships between the pet mesh and indoor geometry. When handling dynamic behaviors driven by cross-modal instructions, existing graphics rendering pipelines typically handle skeletal skin deformation, hair physical collisions, and geometric occlusion in separate rendering batches and computation channels. When virtual pets frequently switch poses such as standing, eating, and walking, vertex data needs to be repeatedly transferred and updated between different computational stages, causing the vertex buffer of the graphics processor to be frequently rewritten. This also leads to frequent switching of rendering states. In addition, traditional linear skinning weighted algorithms are prone to causing significant volume changes in the mesh near the joints when handling large-angle bending of skeletal joints, resulting in mesh collapse and compression deformation.
[0003] Besides the limitations of traditional mesh processing based on physical or geometric deformation, real-time simulation algorithms using data-driven and machine learning methods also have shortcomings in solving such high-degree-of-freedom deformations and collisions. For example, Chinese invention patent CN111192367B discloses a real-time simulation method and device for collision deformation of deformable objects. It extracts image state features and uses principal component analysis for dimensionality reduction, combined with a neural network model to predict the subspace state of deformable objects to simulate collision effects. However, this type of solution highly relies on global linear subspace dimensionality reduction. When dealing with the interaction of high-density fur of virtual pets and complex indoor space topology, subspace projection is prone to losing high-frequency local details, causing distortion of high-dimensional fur geometric deformation. Moreover, after the subspace predicted state output by the neural network is decompressed and restored to three-dimensional space, due to the lack of explicit spatial continuous distance field and dynamic damping smoothing constraints, position correction jumps and flicker artifacts are easily generated at the collision isosurface boundary, making it impossible to maintain the stability of rendering frame rate and geometric topology at the same time.
[0004] Therefore, existing 3D scene rendering technologies still need to address the problem of repeated data updates caused by handling skin deformation and physical collision separately, and reduce the overhead caused by buffer rewriting and memory state switching while maintaining the stability of joint mesh volume and the continuous smoothness of hair collision. Summary of the Invention
[0005] To address the problems in the background art, the present invention provides a virtual pet 3D scene rendering system, comprising: The real pet data acquisition and mesh reconstruction module is used to construct the dominant skeleton and its skinning weight matrix based on the acquired virtual pet mesh vertices and skeletal nodes. The transformation of each dominant skeleton is represented by dual quaternions. The cross-modal interaction command parsing and attitude timing synthesis module is used to generate rotation quaternion timing tensors based on the received interaction commands, and to derive continuous skeletal motion trajectories using quaternion interpolation. The indoor scene spatial topology and dynamic collision proxy construction module is used to extract the three-dimensional directional distance field scalar data of the surface of the indoor scene geometry and construct a physical proxy mesh that wraps the hair growth point. The real-time graphics rendering pipeline and lighting and material calculation module, located in the computational shading channel of the graphics processor, is used to calculate the normalized hybrid dual quaternion using the skin weight matrix and the rotation quaternion time tensor, and apply it to the mesh vertex coordinates to calculate the deformed vertex coordinates. When the sampled 3D directed distance field scalar value is less than the preset hair collision cross-section radius, the gradient vector at the current sampling point is calculated based on the central difference. The difference between the hair collision cross-section radius and the 3D directed distance field scalar value is used as the intrusion depth. The position correction amount is calculated by multiplying the intrusion depth, the preset damping coefficient and the unit vector of the gradient vector. The position correction amount is superimposed on the deformed vertex coordinates to update the dynamic vertex buffer.
[0006] Preferably, in the real pet data acquisition and mesh reconstruction module, each mesh vertex is bound to 4 dominant bones, and the sum of the skin weights corresponding to each dominant bone is 1; in the skin weight matrix, the transformation of the dominant bone at the current moment is represented by a dual quaternion containing rotation quaternions and translation quaternions, and the dual unit square of the dual quaternion is 0; the real-time graphics rendering pipeline and the lighting and material calculation module are used to calculate the mixed dual quaternion acting on the mesh vertex by weighted summation and normalization of the product of the dual quaternion of each dominant bone and its corresponding skin weight.
[0007] Preferably, the cross-modal interaction command parsing and posture temporal synthesis module includes a temporal attention solver and a temporal decoding network. The temporal attention solver is used to map the received interaction commands containing text, speech, or posture markers into latent space feature vectors. The temporal decoding network is used to calculate and output the target rotation quaternion temporal tensor based on the latent space feature vectors. During posture switching, the rotation quaternion is derived by performing quaternion spherical linear interpolation on the temporal tensors of the current skeleton posture quaternion and the target rotation quaternion, where the frame time interval is from 0.016s to 0.033s and the posture transition smoothing attenuation coefficient is 12.0.
[0008] Preferably, the indoor scene spatial topology and dynamic collision proxy construction module extracts a three-dimensional directed distance field as a three-dimensional texture scalar field covering the indoor physical space. The numerical values in the scalar field represent the signed Euclidean distance from the spatial sampling point to the nearest geometric surface in the room. The scalar value is positive when the sampling point is outside the geometry and negative when it is inside the geometry. The physical proxy mesh is a low-polygon mesh structure that wraps around the hair growth point. The world space coordinates of each hair strand node extending from the hair growth point are driven by the deformable vertex coordinates derived from the hybrid dual quaternion calculation.
[0009] Preferably, the real-time graphics rendering pipeline and lighting and material calculation module includes a collision deflection calculation unit; during the graphics processor's calculation and shading stage, the collision deflection calculation unit is used to sample the three-dimensional oriented distance field and obtain hair nodes. Distance scalar value at And when the distance scalar value is less than the preset hair collision cross-section radius. At that time, the gradient vector of the three-dimensional directed range field at the hair node is calculated by sampling neighboring voxels using the central difference method. And calculate the position correction amount of the hair node. The position correction amount is equal to the preset overshoot damping coefficient. Depth of Intrusion , and the normalized direction vector of the gradient vector The product of the three, where the radius of the hair collision cross section is... The value ranges from 0.002m to 0.005m, and the overshoot damping coefficient is... The value ranges from 0.85 to 0.95; the collision deflection calculation unit is used to superimpose the position correction amount onto the predicted coordinates of the hair node, and write the corrected hair node coordinates and updated normals into the dynamic vertex buffer of the graphics processor.
[0010] Preferably, the real-time graphics rendering pipeline and lighting and material calculation module also includes a temporal momentum smoothing unit; the temporal momentum smoothing unit is used to calculate the instantaneous velocity vector of the hair node based on the corrected coordinates of the previous frame and the predicted coordinates of the current frame; when the hair node crosses the collision boundary isosurface of the three-dimensional directional distance field, the temporal momentum smoothing unit extracts the spatial rate of change of the gradient vector between adjacent frames and generates a weighted momentum smoothing factor; the collision deflection calculation unit is used to superimpose the weighted momentum smoothing factor on the position correction amount to smooth the acceleration step of the position correction vector, so that the hair strands maintain continuous acceleration when sliding along the surface of the scene geometry, thereby eliminating hair jitter and flicker artifacts.
[0011] Preferably, the real-time graphics rendering pipeline and lighting and material calculation module adopt a multi-channel rendering architecture, including a frustum culling channel, a GPU computational shading channel, a subsurface scattering shading channel, and a shadow blending channel. The frustum culling channel is used to filter scene primitives and mesh primitives falling within the field of view based on the hierarchical bounding box tree. The GPU computational shading channel is used to calculate the dual quaternion skin deformation and hair collision offset per vertex, and update the vertex buffer and normal buffer. The subsurface scattering shading channel is used to generate a shallow scattering effect of light penetrating a semi-transparent material by using a Gaussian blur kernel to weighted sample the hair and skin mesh based on the cached screen space depth map and normal map. The shadow blending channel is used to perform alpha blending of the output layer of the subsurface scattering shading channel and the scene shadow map and write it to the frame buffer.
[0012] Preferably, in the real-time graphics rendering pipeline and the lighting and material calculation module, the GPU computing shading channel allocates a number of computing threads equal to the number of hair growth points on the main hair shaft to perform parallel collision resolution when rendering the current frame; in the subsurface scattering shading channel, the image space diffusion radius of the Gaussian blur kernel is set to 0.015m; the shadow map combined by the shadow blending channel is generated by drawing the scene depth map and has a resolution of 2048×2048; the system's rendering output frame rate on the display device is higher than 60FPS.
[0013] Preferably, the mesh vertex topology matrix and skinning weight matrix constructed by the real pet data acquisition and mesh reconstruction module are stored in the shader storage buffer of the graphics processor; the real-time graphics rendering pipeline and the lighting and material calculation module update the dynamic vertex buffer through the calculation shader channel within a single rendering frame cycle, without rewriting vertex data or repeatedly allocating video memory between different calculation channels, so as to maintain stable video memory bandwidth overhead.
[0014] Preferably, the real-time graphics rendering pipeline and the lighting and material calculation module input the motion trajectory generated by the pose temporal synthesis module, the three-dimensional directional distance field scalar data extracted by the indoor scene spatial topology and the dynamic collision proxy construction module into the same computational shading pipeline of the graphics processor. At the video memory data level, the closed-loop calculation of pose interpolation, skin deformation, hair collision correction and subsurface scattering lighting and shading is completed, and the rendered image is output.
[0015] The embodiments of the present invention have at least the following beneficial effects: 1. In the rendering of virtual pet 3D scenes, by using dual quaternions to uniformly represent the rotation and translation of bone nodes, and combining the skinning weights of multiple dominant bones to calculate hybrid dual quaternions, the deformation process of mesh vertices can be made more stable when the bone chain undergoes large-angle bending or frequent posture switching. Compared with traditional linear hybrid skinning, this method can better maintain the mesh volume and geometry near the joints, reduce mesh collapse, compression and clipping during joint bending, and enable the bone and muscle areas of the virtual pet to maintain a natural and continuous 3D geometry during movement.
[0016] 2. By combining physical proxy meshes with 3D directed distance fields to handle hair collisions, the complex intersecting calculations between high-density hair and indoor geometry can be transformed into queries of spatial distance and gradient direction. This reduces the processing overhead of hair collision calculations. When a hair node enters the boundary region of a geometry, its position is continuously corrected based on the intrusion depth and gradient direction. This reduces sudden position jumps during collisions, making the movement of hair along geometric surfaces such as furniture and walls smoother and reducing clipping, jitter, and flickering. At the same time, it alleviates the memory bandwidth pressure and rendering state switching overhead caused by frequent collision calculations.
[0017] 3. By completing dual quaternion skinning deformation, 3D directional distance field collision deflection, and subsurface scattering shading in a unified graphics rendering pipeline, the redundant data transfer caused by separate processing of skeletal animation, physical collision, and material rendering can be reduced. Vertex positions and normal buffers can be updated directly within a single rendering frame, which can reduce repeated writing of vertex buffers and state switching between different computing channels, reduce memory data transfer and rendering latency. Therefore, when virtual pets move in complex indoor 3D scenes, they can maintain relatively stable rendering efficiency while maintaining geometric shape and material lighting performance. Attached Figure Description
[0018] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings, in which several embodiments of the invention are illustrated by way of example and not limitation, wherein: Figure 1 This is a structural diagram of the dynamic vertex buffer update module of the present invention; Figure 2 This is a diagram of the cross-modal driving and lighting rendering pipeline of the present invention. Detailed Implementation
[0019] The principles and spirit of the present invention will now be described with reference to several exemplary embodiments in conjunction with the accompanying drawings. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.
[0020] A virtual pet 3D scene rendering system, comprising: The real pet data acquisition and mesh reconstruction module is used to construct the dominant skeleton and its skinning weight matrix based on the acquired virtual pet mesh vertices and skeletal nodes. The transformation of each dominant skeleton is represented by dual quaternions. The cross-modal interaction command parsing and attitude timing synthesis module is used to generate rotation quaternion timing tensors based on the received interaction commands, and to derive continuous skeletal motion trajectories using quaternion interpolation. The indoor scene spatial topology and dynamic collision proxy construction module is used to extract the three-dimensional directional distance field scalar data of the surface of the indoor scene geometry and construct a physical proxy mesh that wraps the hair growth point. The real-time graphics rendering pipeline and lighting and material calculation module, located in the computational shading channel of the graphics processor, is used to calculate the normalized hybrid dual quaternion using the skin weight matrix and the rotation quaternion time tensor, and apply it to the mesh vertex coordinates to calculate the deformed vertex coordinates. When the sampled 3D directed distance field scalar value is less than the preset hair collision cross-section radius, the gradient vector at the current sampling point is calculated based on the central difference. The difference between the hair collision cross-section radius and the 3D directed distance field scalar value is used as the intrusion depth. The position correction amount is calculated by multiplying the intrusion depth, the preset damping coefficient and the unit vector of the gradient vector. The position correction amount is superimposed on the deformed vertex coordinates to update the dynamic vertex buffer.
[0021] Preferably, in the real pet data acquisition and mesh reconstruction module, each mesh vertex is bound to 4 dominant bones, and the sum of the skin weights corresponding to each dominant bone is 1; in the skin weight matrix, the transformation of the dominant bone at the current moment is represented by a dual quaternion containing rotation quaternions and translation quaternions, and the dual unit square of the dual quaternion is 0; the real-time graphics rendering pipeline and the lighting and material calculation module are used to calculate the mixed dual quaternion acting on the mesh vertex by weighted summation and normalization of the product of the dual quaternion of each dominant bone and its corresponding skin weight.
[0022] Preferably, the cross-modal interaction command parsing and posture temporal synthesis module includes a temporal attention solver and a temporal decoding network. The temporal attention solver is used to map the received interaction commands containing text, speech, or posture markers into latent space feature vectors. The temporal decoding network is used to calculate and output the target rotation quaternion temporal tensor based on the latent space feature vectors. During posture switching, the rotation quaternion is derived by performing quaternion spherical linear interpolation on the temporal tensors of the current skeleton posture quaternion and the target rotation quaternion, where the frame time interval is from 0.016s to 0.033s and the posture transition smoothing attenuation coefficient is 12.0.
[0023] Preferably, the indoor scene spatial topology and dynamic collision proxy construction module extracts a three-dimensional directed distance field as a three-dimensional texture scalar field covering the indoor physical space. The numerical values in the scalar field represent the signed Euclidean distance from the spatial sampling point to the nearest geometric surface in the room. The scalar value is positive when the sampling point is outside the geometry and negative when it is inside the geometry. The physical proxy mesh is a low-polygon mesh structure that wraps around the hair growth point. The world space coordinates of each hair strand node extending from the hair growth point are driven by the deformable vertex coordinates derived from the hybrid dual quaternion calculation.
[0024] Preferably, the real-time graphics rendering pipeline and lighting and material calculation module includes a collision deflection calculation unit; during the graphics processor's calculation and shading stage, the collision deflection calculation unit is used to sample the three-dimensional oriented distance field and obtain hair nodes. Distance scalar value at And when the distance scalar value is less than the preset hair collision cross-section radius. At that time, the gradient vector of the three-dimensional directed range field at the hair node is calculated by sampling neighboring voxels using the central difference method. And calculate the position correction amount of the hair node. The position correction amount is equal to the preset overshoot damping coefficient. Depth of Intrusion , and the normalized direction vector of the gradient vector The product of the three, where the radius of the hair collision cross section is... The value ranges from 0.002m to 0.005m, and the overshoot damping coefficient is... The value ranges from 0.85 to 0.95; the collision deflection calculation unit is used to superimpose the position correction amount onto the predicted coordinates of the hair node, and write the corrected hair node coordinates and updated normals into the dynamic vertex buffer of the graphics processor.
[0025] Preferably, the real-time graphics rendering pipeline and lighting and material calculation module also includes a temporal momentum smoothing unit; the temporal momentum smoothing unit is used to calculate the instantaneous velocity vector of the hair node based on the corrected coordinates of the previous frame and the predicted coordinates of the current frame; when the hair node crosses the collision boundary isosurface of the three-dimensional directional distance field, the temporal momentum smoothing unit extracts the spatial rate of change of the gradient vector between adjacent frames and generates a weighted momentum smoothing factor; the collision deflection calculation unit is used to superimpose the weighted momentum smoothing factor on the position correction amount to smooth the acceleration step of the position correction vector, so that the hair strands maintain continuous acceleration when sliding along the surface of the scene geometry, thereby eliminating hair jitter and flicker artifacts.
[0026] Preferably, the real-time graphics rendering pipeline and lighting and material calculation module adopt a multi-channel rendering architecture, including a frustum culling channel, a GPU computational shading channel, a subsurface scattering shading channel, and a shadow blending channel. The frustum culling channel is used to filter scene primitives and mesh primitives falling within the field of view based on the hierarchical bounding box tree. The GPU computational shading channel is used to calculate the dual quaternion skin deformation and hair collision offset per vertex, and update the vertex buffer and normal buffer. The subsurface scattering shading channel is used to generate a shallow scattering effect of light penetrating a semi-transparent material by using a Gaussian blur kernel to weighted sample the hair and skin mesh based on the cached screen space depth map and normal map. The shadow blending channel is used to perform alpha blending of the output layer of the subsurface scattering shading channel and the scene shadow map and write it to the frame buffer.
[0027] Preferably, in the real-time graphics rendering pipeline and the lighting and material calculation module, the GPU computing shading channel allocates a number of computing threads equal to the number of hair growth points on the main hair shaft to perform parallel collision resolution when rendering the current frame; in the subsurface scattering shading channel, the image space diffusion radius of the Gaussian blur kernel is set to 0.015m; the shadow map combined by the shadow blending channel is generated by drawing the scene depth map and has a resolution of 2048×2048; the system's rendering output frame rate on the display device is higher than 60FPS.
[0028] Preferably, the mesh vertex topology matrix and skinning weight matrix constructed by the real pet data acquisition and mesh reconstruction module are stored in the shader storage buffer of the graphics processor; the real-time graphics rendering pipeline and the lighting and material calculation module update the dynamic vertex buffer through the calculation shader channel within a single rendering frame cycle, without rewriting vertex data or repeatedly allocating video memory between different calculation channels, so as to maintain stable video memory bandwidth overhead.
[0029] Preferably, the real-time graphics rendering pipeline and the lighting and material calculation module input the motion trajectory generated by the pose temporal synthesis module, the three-dimensional directional distance field scalar data extracted by the indoor scene spatial topology and the dynamic collision proxy construction module into the same computational shading pipeline of the graphics processor. At the video memory data level, the closed-loop calculation of pose interpolation, skin deformation, hair collision correction and subsurface scattering lighting and shading is completed, and the rendered image is output.
[0030] Example 1: In the deployment of high-frequency interactive indoor virtual pets, the issue of dynamic clipping and rendering frame rate drop caused by the high-density fur mesh and complex indoor 3D geometry of the virtual pet under cross-modal command drive is addressed. The real pet data acquisition and mesh reconstruction module acquires multi-view geometric topology and material texture data of the real pet, reconstructing a 3D digital model of the virtual pet that includes a bone layer, muscle mesh layer, and hair growth points. The model contains a set of 50,000 mesh vertices. Set of topological relationships with patches The skeleton layer contains 128 skeleton nodes, each vertex Bound to Each dominant skeleton has a weight greater than 0 and satisfies the following: ,in, For grid vertices, As vertex Binding the Skin weight of each dominant skeleton, To determine the dominant bone count, a skinning weight matrix is generated and stored in the shader storage buffer. A skeleton in The transformation of time is determined by dual quaternions express: ,in, For the first A skeleton in The dual quaternion of time, For rotation quaternions, For translation quaternions, As a dual unit, satisfying Specifically, the real pet data acquisition and mesh reconstruction module acquires the pet's 3D point cloud and texture map through a multi-view camera array, and generates a 3D surface mesh containing 50,000 mesh vertices using the Poisson reconstruction algorithm. Then, a voxelization thinning operator based on topological centerline extraction is used to automatically fit and extract the motion hierarchy tree of 128 bone nodes inside the pet mesh, and the mesh is divided into muscle mesh layers and epidermal hair growth points according to the muscle physical anatomy structure. When calculating skin weights, for each mesh vertex, the inverse of its Euclidean distance to the four nearest bone nodes is calculated, and the inverse is normalized to determine the skin weights corresponding to the four dominant bones bound to each vertex, thereby constructing a skin weight matrix containing 50,000 rows and 128 columns and writing it into the shader storage buffer.
[0031] When the cross-modal interaction command parsing and gesture timing synthesis module receives an interaction command containing text and control gesture signals, the timing attention solver maps the interaction command into a latent space feature vector. Output through a timing decoding network Target rotation quaternion temporal tensor of the time-series skeletal chain The system performs pose interpolation for each bone node. When calculating the current bone node, it extracts the four quaternion components corresponding to that bone node from the target rotation quaternion temporal tensor as the target input for this interpolation, and extracts the current bone pose quaternion. ; during frame interval The attitude transition smoothness decay coefficient is 0.016s. When the value is 12.0, the rotation quaternion for the next frame is calculated using the quaternion spherical linear interpolation operator: ,in, The rotated quaternion after interpolation transformation. The current skeletal pose quaternion. Given a temporal tensor representing the target rotation quaternion, when calculating a single bone node, the four components of this tensor corresponding to the current bone node are used for interpolation. The attitude transition smoothing attenuation coefficient, The current frame time interval; translate the interpolation result into dual quaternion form. In the real-time graphics rendering pipeline and the lighting and material calculation module, within the graphics processor's computational shading channel, the normalized mixed dual quaternion acting on the mesh vertices is calculated by weighting and normalizing the product of the dual quaternion of each dominant bone and its corresponding skinning weight. In actual operation, the temporal attention solver employs a three-layer bidirectional long short-term memory network combined with a multi-head self-attention mechanism. It concatenates the input 512-dimensional text embedding vector with 128-dimensional speech features into a 640-dimensional cross-modal input. The scaling dot product attention operator calculates the correlation weight distribution between different temporal frames, outputting a 256-dimensional latent space feature vector. The temporal decoding network consists of four fully connected layers and residual connections, sequentially mapping the 256-dimensional latent space feature vector into a quaternion sequence containing 128 bone nodes at future time points. The output is a 128x4 target rotation quaternion temporal tensor, completing the numerical mapping between discrete interactive text and continuous bone rotation states. ,in, For normalized mixed dual quaternions, The norm normalization operator is applied to the grid vertex coordinates. Calculate the coordinates of the deformed vertex This ensures that the muscle-bone junction maintains rigid body algebraic constraints and mesh volume stability when the skeletal chain bends, eliminating volume collapse and compression clipping at joints. In specific feature representation, the aforementioned latent space feature vector is a single-precision floating-point one-dimensional continuous numerical vector with a fixed length of 256, used to match heterogeneous input signals. Its generation process is as follows: a 32-dimensional semantic vector is extracted from the input text string using a language encoder; 32-dimensional acoustic Mel-spectrum features are extracted from the speech signal; and 64-dimensional spatial skeleton action features are extracted from gestures and posture markers. Subsequently, the above heterogeneous features are input into a cross-attention fusion layer for multimodal spatiotemporal alignment and modal weighting, and finally concatenated and compressed into a 256-dimensional unified latent space feature vector and passed to the temporal decoding network.
[0032] In the indoor scene spatial topology and dynamic collision proxy construction module, the system extracts 256×256×256 voxel resolution 3D directed distance field scalar data covering a 10m×10m×4m indoor physical space, stores it in a 3D texture, and constructs a low-polygon physical proxy mesh that wraps around the hair growth points. This mesh marks 20,000 main hair growth points, each corresponding to a main hair strand. The world space coordinates of the hair strand nodes extending from this main hair strand are driven by the deformable vertex coordinates derived from normalized hybrid dual quaternion calculations. In the computational shading channel of the graphics processor, the real-time graphics rendering pipeline and the lighting and material calculation module start 20,000 computation threads, so that each computation thread corresponds to a main hair growth point and performs parallel collision calculations on the hair strand nodes extending from that growth point. Sample 3D texture to obtain distance scalar value When the sampling distance scalar value is less than the radius of the hair collision cross section When the value is 0.003m, the sampling step size is adopted. , , The gradient vector at the current sampling point is calculated using the central difference method with a depth of 0.005m. ,in, For a three-dimensional directed distance field at a point The gradient vector at that point, It is a distance scalar function. , , The sampling step size is the three-dimensional axial step size; the radius of the hair collision cross section is... and distance scalar value The difference is used as the intrusion depth, combined with the overshoot damping coefficient. Calculate the position correction amount as 0.90: ,in, This is the position correction amount. Let the radius of the cross-section of the hair collision be denoted as . The sampling distance scalar value. The overshoot damping coefficient is... Represents the vector magnitude, and the current frame predicted coordinates of the hair node. Driven by the deformable vertex coordinates derived from normalized hybrid dual quaternion calculations, the collision deflection calculation unit will adjust the position. The predicted coordinates are superimposed onto the vertex coordinates driven by the deformation, and the corrected hair node coordinates and updated normals are written to the dynamic vertex buffer of the graphics processor; the temporal momentum smoothing unit adjusts the coordinates according to the previous frame. With the predicted coordinates of the current frame The instantaneous velocity vector of the hair node is calculated. When the hair node crosses the collision boundary isosurface of the 3D directed distance field, the spatial rate of change of the gradient vector between adjacent frames is extracted to generate a weighted momentum smoothing factor, which is then superimposed on the position correction amount. This smooths the acceleration step of the position correction vector, ensuring that the hair maintains continuous acceleration as it slides along the surface of the scene geometry, thus eliminating hair clipping and flicker artifacts during pose switching. Specifically, the generation process of the weighted momentum smoothing factor is as follows: First, the gradient vector sampled in the current frame is subtracted from the gradient vector saved in the previous frame, and then the velocity vector is calculated using the hair... The spatial displacement magnitude of the node from the corrected coordinates of the previous frame to the predicted coordinates of the current frame is normalized to obtain the first-order rate of change of the gradient vector space along the displacement direction of the node in the adjacent frame. Then, the magnitude of the spatial rate of change is multiplied by a time smoothing coefficient of 0.016s and the dot product is performed with the corrected position vector of the previous frame to derive the scalar momentum factor. Finally, the scalar momentum factor is multiplied with the unit vector of the gradient vector of the current frame to obtain the three-dimensional weighted momentum smoothing factor vector, which is directly summed and superimposed on the position correction to complete the second-order acceleration smoothing of the position correction vector.
[0033] The real-time graphics rendering pipeline and lighting / material calculation module operate through a multi-channel rendering architecture. The frustum culling channel filters scene primitives and mesh primitives falling within the field of view based on a hierarchical bounding box tree. The GPU-computed shading channel solves dual quaternion skin deformation and hair collision offset vertex-by-vertex, directly updating the dynamic vertex buffer and normal buffer. The subsurface scattering shading channel uses cached screen-space depth and normal maps, employing a Gaussian blur kernel with an image space diffusion radius of 0.015m for weighted sampling to generate a shallow scattering effect of light penetrating a semi-transparent material. During the specific dimensional transformation and sampling process, the subsurface scattering shading channel... The surface scattering shading channel first extracts the camera perspective projection matrix parameters corresponding to the current pixel and the linear depth value in the depth buffer. It divides the image space diffusion radius, which is set to 0.015m, by the measured physical depth from the current sampling point to the camera, and then multiplies it by the focal length projection coefficient of the camera's field of view. This is converted into a pixel-level diffusion radius that varies with distance in the two-dimensional screen space. Subsequently, kernel-weighted sampling is performed on the screen space depth map based on the converted pixel radius. The shadow blending channel performs alpha blending of the output layer of the subsurface scattering shading channel and the scene shadow map with a resolution of 2048×2048 and writes it into the frame buffer.
[0034] The motion trajectory generated by the cross-modal interaction instruction parsing and posture timing synthesis module, and the 3D directional distance field scalar data extracted by the indoor scene spatial topology and dynamic collision proxy construction module are uniformly input into the same computational shading pipeline of the graphics processor. Within a single rendering frame cycle, this computational shading pipeline uses dynamic vertex buffers and normal buffers to receive data from each stage. The GPU computational shading channel completes posture interpolation, skin deformation, and hair collision correction, and writes the updated vertex positions and normals in place into the corresponding buffers. The subsurface scattering shading channel directly reads the updated buffers and screen space depth and normal maps to complete lighting and shading. The shadow blending channel then writes the output into the frame buffer. Each stage uses the same set of vertex and normal data in the same rendering frame, without rewriting vertex data or repeatedly allocating video memory between different computational channels. This enables the system to stably output a 3D rendered image with no clipping, stable geometry and topology, and a semi-transparent texture at a frame rate higher than 60 FPS on the display device.
[0035] Example 2: The current test environment uses a graphics workstation equipped with a graphics processing unit. The indoor scene covers a physical space of 10m×10m×4m, and the voxel resolution is set to 256×256×256. The 3D directed distance field scalar data is extracted offline and written into the 3D texture. The virtual pet model used in the test contains a set of 50,000 mesh vertices. It has 128 bone nodes and 20,000 main hair growth points, with each mesh vertex bound to... To simulate signal interference and high-frequency dynamic interaction at cross-modal command input points, a dominant skeleton is constructed by superimposing 25dB Gaussian white noise and 50Hz power frequency interference onto the gesture control signal and posture marker data. The current frame time interval... Set to 0.016s to balance attitude synthesis response and graphics processor computational load; attitude transition smoothing attenuation coefficient. Set to 12.0, when the target rotation quaternion time series tensor calculated by the cross-modal interaction command parsing and attitude timing synthesis module is... When high-frequency abrupt changes occur, the rotation quaternion for the next frame is calculated according to the quaternion spherical linear interpolation relationship described in Example 1. This corresponds to 60 FPS rendering at a frame interval of 0.016s. When the coefficient is 12.0, the pose interpolation weight exponent reaches approximately 43.6% transition completion within 3 rendering frame cycles (0.048s) and 68.3% convergence within 6 rendering frame cycles (0.096s), with a motion response time of less than 0.1s. When the coefficient is below 8.0, the pose convergence time increases to over 0.2s, resulting in visual lag and trailing during motion transitions. When the coefficient is above 16.0, the decay rate increases, leading to step changes in joint angular velocity and screen stuttering during high-frequency interpolation, and affecting the hair collision cross-sectional radius. Set to 0.003m, overshoot damping coefficient Set to 0.90; when a hair node intrudes into the three-dimensional oriented distance field boundary protection layer, if the overshoot damping coefficient is too small, the position correction will be insufficient; if it is too large, it will cause reverse push-away overshoot. The value range of the hair collision section radius is 0.002m to 0.005m. When the radius is less than 0.002m, the voxel grid sampling spatial resolution is less than 0.005m, and the high-speed moving hair node may directly cross the isosurface. When the radius is greater than 0.005m, the collision enclosure range increases, and the hair deflects before contacting the obstacle surface. The value range of the overshoot damping coefficient is 0.85 to 0.95. When it is less than 0.85, the single-frame position correction is insufficient to offset the intrusion depth, and the hair node has residual intrusion. When it is greater than 0.95, the hair node repeatedly pushes away from both sides of the isosurface.
[0036] The experiment was set up with five independent test groups. The embodiment of this invention employs dual quaternion skin deformation, three-dimensional directional distance field collision deflection, and time-series momentum smoothing elements, wherein the overshoot damping coefficient... Set to 0.90, the radius of the hair collision cross section. The value is set to 0.003m; Control group 1 uses the traditional linear hybrid skin algorithm and geometric patch intersection collision calculation; Control group 2 maintains the same skin deformation and collision deflection processing as the sample group in this embodiment, but removes the temporal momentum smoothing unit; Control group 3 adjusts the overshoot damping coefficient. The value was lowered to 0.70, which is below the lower limit of the range of 0.85; the control group 4 adjusted it to 0.99, which is above the upper limit of the range of 0.95. Each test group used the same three-dimensional indoor geometric scene and drove the virtual pet to complete the smooth switching of posture, jumping to the sofa, extreme posture of large-angle joint bending and high-frequency turning movements in sequence. The rendering frame rate, the rewrite frequency of the video memory vertex buffer, the joint mesh volume collapse rate, the number of hair strand jitter and flicker, the vertex rendering delay and the collision position correction acceleration residual were collected in the GPU computing shading channel.
[0037] During the smooth attitude transition, the sample group in this embodiment performed skinning deformation on the mesh vertex coordinates according to the normalized hybrid dual quaternion, and completed vertex updates in the GPU compute shading channel. The measured rendering frame rate was 62.4 FPS, the VRAM vertex buffer rewrite frequency was 60.0 times / s, the joint mesh volume collapse rate was 0.12%, the hair strand jitter flicker count was 0 times / min, the vertex rendering latency was 1.15ms, and the collision position correction acceleration residual was 0.08m / s². The control group adopted the same working conditions. Traditional linear hybrid skinning and geometric patch intersection collision calculation, with a rendering frame rate of 31.5 FPS, a VRAM vertex buffer rewrite frequency of 360.0 times / s, a joint mesh volume collapse rate of 18.40%, hair flicker count of 12 times / min, vertex rendering latency of 4.85ms, and collision position correction acceleration residual of 1.85m / s², resulted in corresponding differences in vertex buffer rewrite frequency, joint mesh deformation, and collision position correction state after using different skinning and collision calculation methods.
[0038] During the high-frequency jumping of the virtual pet onto the sofa, the sample group in this embodiment continued to use dual quaternion skinning deformation, three-dimensional directional distance field collision deflection, and temporal momentum smoothing. The rendering frame rate was 61.2 FPS, the memory vertex buffer rewrite frequency was 60.0 times / s, the joint mesh volume collapse rate was 0.18%, the hair flickering frequency was 0 times / min, the vertex rendering latency was 1.22ms, and the collision position correction acceleration residual was 0.12m / s². The control group used traditional linear hybrid skinning and geometric patch intersection collision calculation. The rendering frame rate was 24.3 FPS, the memory vertex buffer rewrite frequency was 480.0 times / s, the joint mesh volume collapse rate was 22.60%, the hair flickering frequency was 35 times / min, the vertex rendering latency was 6.20ms, and the collision position correction acceleration residual was 3.20m / s².
[0039] Under the same high-frequency jump to sofa condition, control group 2 retained the same skin deformation and 3D directional distance field collision deflection processing as the sample group in this embodiment, only removing the temporal momentum smoothing unit. The measured rendering frame rate was 58.5 FPS, the memory vertex buffer rewrite frequency was 60.0 times / s, the joint mesh volume collapse rate was 0.19%, the number of hair strand jittering and flickering increased to 48 times / min, the vertex rendering latency was 1.35ms, and the collision position correction acceleration residual was 2.15m / s². When the hair strand node crossed the 3D directional distance field boundary, the position correction amount changed directly with the current sampling result without undergoing temporal momentum smoothing processing. Therefore, a large collision position correction acceleration residual was formed between adjacent frames, accompanied by hair strand jittering and flickering.
[0040] Control group three under this operating condition will have an overshoot damping coefficient When the overshoot damping coefficient is lowered to 0.70, the rendering frame rate is 59.2 FPS, the VRAM vertex buffer rewrite frequency is 60.0 times / s, the joint mesh volume collapse rate is 0.18%, the hair flicker frequency is 15 times / min, the vertex rendering latency is 1.30ms, and the collision position correction acceleration residual is 0.95m / s². When the overshoot damping coefficient is lower than the set range, the position correction calculated based on the intrusion depth and gradient direction decreases, and residual intrusion appears at the hair nodes. Control group four lowers the overshoot damping coefficient... When the value was increased to 0.99, the measured rendering frame rate was 55.4 FPS, the video memory vertex buffer rewrite frequency was 60.0 times / s, the joint mesh volume collapse rate was 0.20%, the hair strand jitter flicker count increased to 85 times / min, the vertex rendering latency was 1.55ms, and the collision position correction acceleration residual was 3.85m / s². After the overshoot damping coefficient exceeded the set range, the position correction amplitude increased, and the hair strand node exhibited reverse push-away oscillation near the collision boundary. The above two sets of data correspond to the results of the sample group in this embodiment under the same working conditions, reflecting the influence of the overshoot damping coefficient value change on the collision correction state of the hair strand node.
[0041] During the extreme posture of a large-angle joint bend, the sample group in this embodiment uses normalized hybrid dual quaternions to update the mesh vertex coordinates. The joint mesh volume collapse rate is 0.25%, the memory vertex buffer rewrite frequency is 60.0 times / s, the rendering frame rate is 60.8 FPS, the hair flicker frequency is 0 times / min, the vertex rendering latency is 1.28ms, and the collision position correction acceleration residual is 0.11m / s². In the control group, after using traditional linear hybrid skinning and independent collision calculation, the joint mesh volume collapse rate is 26.70%, and the memory vertex buffer rewrite frequency is... The rendering frame rate was 22.1 FPS, the number of hair flickering times was 42 times / min, the vertex rendering latency was 7.10ms, and the collision position correction acceleration residual was 3.80m / s². When traditional linear blend skinning directly performs matrix weighting, the mesh volume change in the joint region is large, and the independent collision calculation process increases the number of vertex buffer rewrites. In this embodiment, the sample group completes the vertex update based on normalized blend dual quaternions in the GPU computing shader channel, and continues to perform 3D directed distance field collision deflection and temporal momentum smoothing processing.
[0042] After superimposing Gaussian white noise and power frequency interference at the input end, the rendering frame rate of this sample group is 61.0 FPS, the rewrite frequency of the video memory vertex buffer is 60.0 times / s, the joint mesh volume collapse rate is 0.21%, the number of hair strand jitter flicker times / min is 0 times, the vertex rendering latency is 1.25ms, and the collision position correction acceleration residual is 0.14m / s². The input of this group is still generated by the cross-modal interactive command parsing and attitude temporal synthesis module to generate the target rotation quaternion temporal tensor, and after attitude interpolation, dual quaternion skin deformation, three-dimensional directional distance field collision deflection and temporal momentum smoothing, it is written into the dynamic vertex buffer. Based on the above acquisition results, the rendering and collision states under noise interference conditions are determined.
[0043] The data correspondence for each working condition shows that dual quaternion skin deformation and vertex updates within the GPU computed shading channel can reduce mesh volume changes and buffer rewrite overhead during large-angle joint bending. In hair collision processing, the 3D directional distance field provides the collision position correction direction and intrusion depth, the overshoot damping coefficient controls the position correction amplitude of a single frame, and the temporal momentum smoothing unit further processes the position correction changes of adjacent frames. The differences in rendering frame rate, joint mesh volume collapse rate, hair flicker, vertex rendering latency, and collision position correction acceleration residuals formed by each test group under the same scene and action conditions correspond to the settings changes of the above processing steps.
[0044] Example 3: When a virtual pet performs a large-scale pose change across its skeletal chain, the real-time graphics rendering pipeline and the lighting and material calculation module process the pose quaternion node by node in the graphics processor's computational shading channel. For the ... Each skeletal node is first rotated from the target quaternion temporal tensor. Extract the target rotation quaternion corresponding to the bone node and compare it with the current bone pose quaternion. Calculate the dot product; when both are in the dual hemisphere and the dot product is less than 0, invert the extracted target rotation quaternion as a whole, and then calculate the dot product within the frame interval. The attitude transition smoothness decay coefficient is 0.016s. Under the condition of 12.0, quaternion spherical linear interpolation is performed, such that the interpolation is carried out along the shortest path on the quaternion manifold, and the resulting rotated quaternion is obtained. Translated into the first The dual quaternions corresponding to each skeletal node Then based on the grid vertices Bound Each dominant skeleton and its corresponding skin weight The normalized mixed dual quaternion is calculated and applied to the grid vertex coordinates. Obtain the coordinates of the deformed vertex It is used to handle skin deformation and tearing, joint mesh voxel collapse and reverse flipping through the mold that occur during large-angle bending across the hemisphere.
[0045] The indoor scene spatial topology and dynamic collision proxy construction module extracts 3D directed distance field scalar data covering a 10m×10m×4m indoor physical space with a voxel resolution of 256×256×256, and stores it in a 3D texture. Simultaneously, it constructs a physical proxy mesh encompassing 20,000 main hair growth points. The real-time graphics rendering pipeline and lighting and material calculation module launch 20,000 computation threads in the computational shading channel. Each computation thread corresponds to a main hair growth point and handles collisions of hair nodes extending from that point. The computation thread samples distance scalar values from the 3D texture. When the distance scalar value is less than the radius of the hair collision cross section. ,and When the sampling step size is 0.003m, the gradient vector is calculated using the central difference method with a sampling step size of 0.005m. When the magnitude of the gradient vector is not less than a preset lower threshold At that time, according to the penetration depth and overshoot damping coefficient The position correction is calculated using the normalized gradient direction, where Take 0.90; when the magnitude of the gradient vector is less than At that time, predict coordinates using the current frame. Corrected coordinates from the previous frame The instantaneous velocity direction is obtained by replacing the normalized gradient direction, and its instantaneous velocity direction is: When the gradient magnitude is less than Within the region, the normal component of the instantaneous velocity direction with respect to the nearest geometric plane is first projected and its value is reversed to point towards the outside of the geometry. Then, this direction is linearly fused with the effective historical gradient direction of the previous frame with a weight of 0.5 and normalized. The fused direction replaces the normalized gradient direction in the calculation of the position correction. At the same time, a weighted momentum smoothing factor generated by the spatial rate of change of the gradient vector is superimposed to smooth the changes of the deflection correction vector in adjacent frames, so that the hair node maintains continuous acceleration when sliding along the surface of the scene geometry, and avoids the instantaneous velocity direction pointing directly into the geometry to produce secondary clipping.
[0046] The real-time graphics rendering pipeline and lighting and material calculation module adopt a multi-channel rendering architecture. The view frustum culling channel filters scene primitives and mesh primitives falling into the field of view based on the hierarchical bounding box tree; the GPU computational shading channel calculates dual quaternion skin deformation and hair collision offset vertex by vertex, and writes the corrected hair node coordinates and updated normals directly into the dynamic vertex buffer through the shader storage buffer, reducing vertex data transfer between the main processor and the graphics processor memory; the subsurface scattering shading channel is based on cached screen space depth map and normal map, and uses a Gaussian blur kernel with an image space diffusion radius of 0.015m for weighted sampling to generate a shallow scattering effect for semi-transparent materials; shadows The blending channel alpha-blends the output of the subsurface scattering shading channel with a 2048×2048 resolution shadow map and writes it to the frame buffer. The motion trajectory generated by the cross-modal interaction instruction parsing and pose temporal synthesis module, as well as the 3D directional distance field scalar data extracted by the indoor scene spatial topology and dynamic collision proxy construction module, are all uniformly input into the same computational shading pipeline of the graphics processor. Within a single rendering frame cycle, each processing stage uses the dynamic vertex buffer and normal buffer data in the video memory to complete pose interpolation, skin deformation, hair collision correction, and subsurface scattering shading calculation, keeping the video memory bandwidth overhead stable and outputting a rendered image with a frame rate higher than 60 FPS.
[0047] Example 4: This example combines Figures 1 to 2 This paper describes a virtual pet 3D scene rendering system, such as... Figure 1 As shown, the mesh reconstruction module constructs the skin weight matrix, the pose synthesis module exports the skeleton motion trajectory, the material calculation module receives the skin weight matrix constructed by the mesh reconstruction module and the skeleton motion trajectory exported by the pose synthesis module and derives the coordinates of the deformed vertices, the collision construction module constructs the physical proxy mesh, the temporal momentum smoothing unit generates the smoothing factor, the collision deflection calculation unit combines the physical proxy mesh constructed by the collision construction module and the smoothing factor generated by the temporal momentum smoothing unit to calculate the position correction amount, the material calculation module transmits the derived deformed vertex coordinates to the dynamic vertex buffer, the collision deflection calculation unit inputs the calculated position correction amount into the dynamic vertex buffer, and the dynamic vertex buffer updates the deformed coordinates.
[0048] like Figure 2As shown, the system receives input cross-modal interaction commands to map latent space feature vectors. After outputting the feature vectors, it derives the skeletal motion trajectory and generates a target rotation quaternion temporal tensor. Based on the input real pet 3D point cloud, it obtains a 3D point cloud mesh to construct the dominant skeleton skinning weight matrix. The skinning weight matrix is written into the shader storage buffer. When calculating skinning deformation vertex by vertex, it reads the quaternions and weights in the shader storage buffer and combines them with the target rotation quaternion temporal tensor to calculate the deformed vertex coordinates driving the hair. It extracts indoor scene geometry to obtain indoor scene spatial geometry to extract distance field scalar data. The obtained 3D directed distance field scalar is stored in the 3D texture scalar field. When calculating the hair position correction, it samples the gradient vectors of neighboring voxels in the 3D texture scalar field and calculates them in combination with the derivation results of the deformed vertex coordinates driving the hair. The corrected position and updated normal are written into the dynamic vertex buffer. When rendering subsurface scattering light and shadow, it reads the depth and normal maps in the dynamic vertex buffer. The obtained graphics are mixed with light and shadow and written into the frame buffer. Finally, the display device outputs the screen.
[0049] Example 5: When the indoor scene space topology and dynamic collision proxy construction module initializes the 3D directed distance field scalar data, it performs offline 3D voxel scanning on the surfaces of furniture and floor geometry covering a 10m×10m×4m indoor physical space at a voxel resolution of 256×256×256, generating a signed Euclidean distance scalar field. Subsequently, a 3D Gaussian convolution kernel smoothing filter is applied to the distance scalar field to reduce the discontinuity of the distance field gradient caused by voxelized discrete sampling, resulting in a continuously distributed scalar field function. The real pet data acquisition and mesh reconstruction module reconstructs a virtual pet 3D digital model containing 50,000 mesh vertices, and extracts 20,000 main hair growth points based on a multi-level hair density map; the collision cross-sectional radius of the hair is calibrated at the physical proxy mesh and each main hair growth point. Its value is determined based on the hair thickness and density gradient, ranging from 0.002m to 0.005m. In this embodiment, it is taken as 0.003m. The overshoot damping coefficient... A value of 0.90 is selected within the range of 0.85 to 0.95 as the position correction benchmark when hair nodes invade the geometric boundary protection layer. The physical proxy mesh uses a quadratic error metric mesh simplification algorithm to downsample the pet high-precision skin mesh with 50,000 vertices, reducing the number of vertices to 2,000. Each vertex is bound to a triangular facet of the high-precision skin mesh through nearest neighbor local centroid coordinate interpolation. When the skeleton drives the high-precision skin mesh to perform dual quaternion skin deformation, the vertices of the physical proxy mesh are updated synchronously with the coordinates of the deformed vertices according to this binding relationship. The 20,000 main hair growth points are fixed at the centroid positions of the corresponding triangles of the physical proxy mesh, and the world space coordinates of the hair nodes extending from each growth point are updated with the coordinates of the deformed vertices.
[0050] Before the graphics processor's computational shading pipeline starts the rendering pipeline, the system allocates corresponding data storage locations for 20,000 main hair growth points in the shader storage buffer. It also loads the initialized 3D directed distance field scalar data into video memory in single-channel floating-point texture format. As the real-time graphics rendering pipeline and the lighting and material calculation module run frame-by-frame, the computational shading pipeline applies normalized hybrid dual quaternion skinning deformation to the mesh vertices based on the bone motion trajectory, obtaining the deformed vertex coordinates. It then starts a number of computation threads equal to the 20,000 main hair growth points. Each computation thread processes the collisions of hair nodes extending from the corresponding growth point in parallel. Within each thread, the 3D texture is sampled, and the directed distance scalar values at the hair nodes are read. When a hair node intrudes into the geometry protection layer and the directed distance scalar value is... Smaller than the preset hair collision cross-sectional radius At that time, the shader channel is computed by sampling adjacent voxels using the center difference method to calculate the gradient vector as the collision normal, and then combining it with the corrected coordinates from the previous frame. With the predicted coordinates of the current frame The collision correction displacement is calculated based on the intrusion depth, overshoot damping coefficient, and collision deflection direction. This is then superimposed onto the current frame's predicted coordinates, which are driven by the deformed vertex coordinates, to obtain the corrected coordinates of the current frame. The corrected hair node coordinates and updated normal vectors are written back to the dynamic vertex buffer in place for subsequent direct retrieval by each rendering channel.
[0051] In frame time interval Within a dynamic interaction cycle of 0.016s to 0.033s, the computational shading channel continuously completes skin deformation, hair node collision deflection, and dynamic vertex buffer update in the video memory. Each processing stage uses the same set of vertex and normal data, and vertex data is not transferred between different rendering channels, so that the display device maintains a 3D image output of more than 60FPS.
[0052] Example 6: Before system deployment, offline calibration is performed. The indoor scene spatial topology and dynamic collision proxy construction module establishes gradient tolerance and boundary calibration procedures for the 3D directed distance field with a resolution of 256×256×256 voxels. For the decrease in distance field gradient continuity caused by sharp edges of indoor furniture, the offline calibration process uses a sampling step size of 0.005m to obtain the central difference gradient vector of the voxel mesh. And set a lower limit threshold for determining the gradient magnitude. When the gradient vector magnitude Less than At that time, the gradient vector of the voxel node is replaced with the weighted average of the gradients of its neighboring voxels. Based on this, a continuous three-dimensional directed distance field scalar and gradient lookup table are constructed and written into the three-dimensional texture. The real pet data acquisition and mesh reconstruction module reconstructs a virtual pet three-dimensional digital model containing 50,000 mesh vertices. Under static preset conditions, the geometric coverage thickness of the hair growth density is measured, and the radius of the hair collision cross section is determined. The initial calibration value was set at 0.003m, and the overshoot damping coefficient was set... The initial calibration value was set to 0.90; when the hair node... Intrusion into the geometric protective layer and sampling distance scalar value less than At that time, the position correction amount for the shaded channel is calculated according to the following formula: ,in, This is the correction amount for the position of the hair node. The overshoot damping coefficient is... Let the radius of the cross-section of the hair collision be denoted as . The sampling distance scalar value. The gradient vector, Denotes the magnitude of the vector; where, Take 0.003m, Take 0.001m, Taking 0.90, the corrected modulus length is 0.90×(0.003-0.001)=0.0018m, thus completing the initial alignment of the physical proxy mesh and the collision scalar field.
[0053] During the online dynamic rendering process, the real-time graphics rendering pipeline and the lighting and material calculation module adjust the load based on the graphics processor's single-frame rendering time. When the number of primitives in the indoor 3D scene increases, causing the single-frame rendering time to exceed the frame interval threshold of 0.033s, the graphics processor's computational shading channel initiates adaptive downsampling, reducing the number of hair growth points participating in collision resolution from 20,000 to 10,000, while retaining the corresponding hair nodes for each growth point to participate in collision resolution. Simultaneously, the hair collision cross-section radius is adjusted according to the area conservation relationship. The adjusted hair collision cross-section radius is: ,in, The adjusted radius of the hair collision cross section. The initial radius of the hair collision cross-section is 0.003m; substituting this into the formula 0.003 × 1.414 = 0.004242m, Taking 0.0042m, the physical enclosure volume of the hair nodes after downsampling remains consistent with the initial state. Each main hair node forms a fixed-length cylindrical segment along the spatial tangent direction. The physical enclosure volume of a single hair node is equal to the product of its two-dimensional cylindrical cross-sectional area and the fixed hair segment length. When the number of main hair growth points is downsampled from 20,000 to 10,000, that is, reduced to 0.5 times the original number, in order to keep the total physical enclosure volume of the overall hair layer in three-dimensional space unchanged, the cylindrical cross-sectional area corresponding to a single hair node is expanded to twice the original size, and the radius of the cylindrical cross-section is correspondingly multiplied by the square root of 2, that is, approximately 1.414.
[0054] The computational shading pass writes the updated vertex positions and normal data to the dynamic vertex buffer within a single rendering frame cycle. Under conditions of limited computing resources, it maintains hair volume expansion constraints and collision de-clipping effects, and keeps the rendering frame rate above 60 FPS, outputting a 3D image with stable geometry, topology, materials, and lighting.
[0055] The above description is only a few preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, technical solutions formed by replacing the above-mentioned features with, but not limited to, the technical features disclosed in the embodiments of the present invention that have similar functions.
Claims
1. A virtual pet 3D scene rendering system, characterized in that, include: The real pet data acquisition and mesh reconstruction module is used to construct the dominant skeleton and its skinning weight matrix based on the acquired virtual pet mesh vertices and skeletal nodes. The transformation of each dominant skeleton is represented by dual quaternions. The cross-modal interaction command parsing and attitude timing synthesis module is used to generate rotation quaternion timing tensors based on the received interaction commands, and to derive continuous skeletal motion trajectories using quaternion interpolation. The indoor scene spatial topology and dynamic collision proxy construction module is used to extract the three-dimensional directional distance field scalar data of the surface of the indoor scene geometry and construct a physical proxy mesh that wraps the hair growth point. The real-time graphics rendering pipeline and lighting and material calculation module, located in the computational shading channel of the graphics processor, is used to calculate the normalized hybrid dual quaternion using the skin weight matrix and the rotation quaternion time tensor, and apply it to the mesh vertex coordinates to calculate the deformed vertex coordinates. When the sampled 3D directed distance field scalar value is less than the preset hair collision cross-section radius, the gradient vector at the current sampling point is calculated based on the central difference. The difference between the hair collision cross-section radius and the 3D directed distance field scalar value is used as the intrusion depth. The position correction amount is calculated by multiplying the intrusion depth, the preset damping coefficient and the unit vector of the gradient vector. The position correction amount is superimposed on the deformed vertex coordinates to update the dynamic vertex buffer.
2. The virtual pet 3D scene rendering system according to claim 1, characterized in that, In the real pet data acquisition and mesh reconstruction module, each mesh vertex is bound to 4 dominant bones, and the sum of the skin weights corresponding to each dominant bone is 1. In the skin weight matrix, the transformation of the dominant bone at the current moment is represented by a dual quaternion containing rotation quaternions and translation quaternions, and the dual unit square of the dual quaternion is 0. The real-time graphics rendering pipeline and the lighting and material calculation module are used to calculate the mixed dual quaternion acting on the mesh vertex by weighted summation and normalization of the product of the dual quaternion of each dominant bone and its corresponding skin weight.
3. The virtual pet 3D scene rendering system according to claim 1, characterized in that, In the cross-modal interaction command parsing and pose temporal synthesis module, there is a temporal attention solver and a temporal decoding network. The temporal attention solver is used to map the received interaction commands containing text, speech or pose markers into latent space feature vectors. The temporal decoding network is used to calculate and output the target rotation quaternion temporal tensor based on the latent space feature vectors. When the pose changes, the rotation quaternion is smoothly transitioned with the frame time interval by performing quaternion spherical linear interpolation operator on the temporal tensor of the current skeleton pose quaternion and the target rotation quaternion. The frame time interval is from 0.016s to 0.033s, and the pose transition smoothing attenuation coefficient is 12.
0.
4. The virtual pet 3D scene rendering system according to claim 1, characterized in that, The indoor scene spatial topology and dynamic collision proxy construction module extracts a 3D directed distance field as a 3D texture scalar field covering the indoor physical space. The numerical values in the scalar field represent the signed Euclidean distance from the spatial sampling point to the nearest geometric surface in the room. The scalar value is positive when the sampling point is outside the geometry and negative when it is inside the geometry. The physical proxy mesh is a low-polygon mesh structure that wraps around the hair growth point. The world space coordinates of each hair strand node extending from the hair growth point are driven by the deformable vertex coordinates derived from the hybrid dual quaternion calculation.
5. A virtual pet 3D scene rendering system according to claim 4, characterized in that, The real-time graphics rendering pipeline and lighting and material calculation module include a collision deflection calculation unit. During the graphics processor's shading calculation phase, the collision deflection calculation unit is used to sample the three-dimensional oriented distance field and obtain hair nodes. Distance scalar value at And when the distance scalar value is less than the preset hair collision cross-section radius. At that time, the gradient vector of the three-dimensional directed range field at the hair node is calculated by sampling neighboring voxels using the central difference method. And calculate the position correction amount of the hair node. The position correction amount is equal to the preset overshoot damping coefficient. Depth of Intrusion , and the normalized direction vector of the gradient vector The product of the three, where the radius of the hair collision cross section is... The value ranges from 0.002m to 0.005m, and the overshoot damping coefficient is... The value ranges from 0.85 to 0.95; The collision deflection calculation unit is used to superimpose the position correction amount onto the predicted coordinates of the hair node, and write the corrected hair node coordinates and updated normals into the dynamic vertex buffer of the graphics processor.
6. A virtual pet 3D scene rendering system according to claim 5, characterized in that, The real-time graphics rendering pipeline and lighting and material calculation module also include a temporal momentum smoothing unit. The temporal momentum smoothing unit is used to calculate the instantaneous velocity vector of the hair node based on the corrected coordinates of the previous frame and the predicted coordinates of the current frame. When the hair node crosses the collision boundary isosurface of the three-dimensional directional distance field, the temporal momentum smoothing unit extracts the spatial rate of change of the gradient vector between adjacent frames and generates a weighted momentum smoothing factor. The collision deflection calculation unit is used to superimpose the weighted momentum smoothing factor on the position correction amount to smooth the acceleration step of the position correction vector, so that the hair strands maintain continuous acceleration when sliding along the surface of the scene geometry, thereby eliminating hair jitter and flicker artifacts.
7. A virtual pet 3D scene rendering system according to claim 1, characterized in that, The real-time graphics rendering pipeline and lighting and material calculation module adopt a multi-channel rendering architecture, including the view frustum culling channel, GPU computational shading channel, subsurface scattering shading channel, and shadow blending channel. The frustum culling channel is used to filter scene primitives and mesh primitives that fall within the field of view based on the hierarchical bounding box tree; The GPU compute shader channel is used to calculate per-vertex dual quaternion skin deformation and hair collision offset, updating the vertex buffer and normal buffer; the subsurface scattering shader channel is used for cached screen space depth map and normal map, using a Gaussian blur kernel to perform weighted sampling on hair and skin meshes to generate a shallow scattering effect of light penetrating a semi-transparent material; the shadow blending channel is used to alpha blend the output layer of the subsurface scattering shader channel with the scene shadow map and write it to the frame buffer.
8. A virtual pet 3D scene rendering system according to claim 7, characterized in that, In the real-time graphics rendering pipeline and the lighting and material calculation module, the GPU compute shading channel allocates an equal number of computing threads as the number of hair growth points on the main hair shaft to perform parallel collision resolution when rendering the current frame; in the subsurface scattering shading channel, the image space diffusion radius of the Gaussian blur kernel is set to 0.015m; the shadow map combined by the shadow blending channel is generated by drawing the scene depth map and has a resolution of 2048×2048; the system's rendering output frame rate on the display device is higher than 60FPS.
9. A virtual pet 3D scene rendering system according to claim 1, characterized in that, The mesh vertex topology matrix and skinning weight matrix constructed by the real pet data acquisition and mesh reconstruction module are stored in the shader storage buffer of the graphics processor. The real-time graphics rendering pipeline and lighting and material calculation module update the dynamic vertex buffer through the calculation shader channel within a single rendering frame cycle, without rewriting vertex data or repeatedly allocating video memory between different calculation channels, so as to maintain stable video memory bandwidth overhead.
10. A virtual pet 3D scene rendering system according to claim 1, characterized in that, The real-time graphics rendering pipeline and the lighting and material calculation module input the motion trajectory generated by the pose temporal synthesis module, the 3D directional distance field scalar data extracted by the indoor scene spatial topology and the dynamic collision proxy construction module into the same computational shading pipeline of the graphics processor. At the video memory data level, it completes the closed-loop calculation of pose interpolation, skin deformation, hair collision correction and subsurface scattering lighting and shading, and outputs the rendered image.
Citation Information
Patent Citations
A method and apparatus for real-time simulation of collision deformation of deformable objects
CN111192367B
