Object rendering method and apparatus, electronic device, and computer-readable storage medium

Abstract

This application discloses an object rendering method, apparatus, electronic device, and computer-readable storage medium. The method involves obtaining a skeleton binding file of a virtual object, performing generation processing based on a 3D mesh to obtain multiple Gaussian spheres for generating the virtual object, each Gaussian sphere including initial position points and initial shape indication information; performing weight transfer processing based on the skin weights of the initial position points' neighboring points in the 3D mesh to obtain the skin weights of the Gaussian spheres; obtaining the target transformation matrix of the skeleton at a target time; updating the initial position points and initial shape indication information based on the Gaussian spheres' skin weights and the target transformation matrix of the skeleton at the target time to obtain the target position and target shape indication information of the Gaussian spheres at the target time; and performing rendering processing based on the target position and target shape indication information of the Gaussian spheres at the target time to obtain the virtual object displayed at the target time.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to an object rendering method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] 3D Gaussian Splatting (3DGS) is a neural rendering technology based on explicit point clouds. It represents virtual objects in a scene using a 3D Gaussian sphere. The rendering results of this technology are highly realistic and can achieve real-time frame rate. It has broad application prospects in fields such as games, film and television production, and virtual reality.

[0003] However, current 3D Gaussian splashing relies on fixed human skeletons such as Skinned Multi-Person Linear Model (SMPL) and Face Linear Appreciation Model (FLAME), making it difficult to adapt to the large number of virtual objects without human skeletons in real-world applications. This prevents such virtual objects from being directly rendered using this technology. Furthermore, the conventional workflow for 3D Gaussian splashing requires first acquiring real video data and completing 3D reconstruction, which is inefficient. Additionally, since most virtual objects are original designs by artists and lack corresponding real-world video footage, this further limits the application of this technology in the field of virtual content creation. Summary of the Invention

[0004] This application provides an object rendering method, apparatus, electronic device, and computer-readable storage medium, which can be applied to rendering virtual objects with various skeletons and without the need for 3D reconstruction, thereby improving the applicability and rendering efficiency of 3D Gaussian splashing technology.

[0005] In a first aspect, embodiments of this application provide an object rendering method, including: Obtain the skeleton binding file of the virtual object, which includes the 3D mesh of the virtual object, the bones bound to the 3D mesh, and the skinning weights of the vertices on the 3D mesh; Based on the above three-dimensional mesh, multiple Gaussian spheres are generated to generate the above virtual object. The Gaussian spheres include initial position points and initial shape indication information. The skin weights of the Gaussian sphere are obtained by performing weight transfer processing based on the skin weights of the neighborhood points of the initial position point in the three-dimensional mesh. Obtain the target transformation matrix of the above skeleton at the target time; Based on the skin weights of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, update the initial position point and the initial shape indication information to obtain the target position and target shape indication information of the Gaussian sphere at the target time. Based on the target position and target shape indication information of the Gaussian sphere at the target time, the above-mentioned virtual object is obtained by rendering.

[0006] Secondly, embodiments of this application provide an object rendering method, including: Obtain the Gaussian sphere file corresponding to the virtual object. The Gaussian sphere file includes the initial position points, initial shape indication information, and skinning weights used to generate the Gaussian sphere of the virtual object. The initial shape indication information includes the initial rotation matrix and the initial scaling matrix. Obtain the target transformation matrix of the skeleton of the virtual object at the target time; Based on the target transformation matrix of the skeleton at the target time and the skinning weight of the Gaussian sphere, the rotation update information and scaling update information of the Gaussian sphere at the target time are determined, and the initial position point is updated to obtain the target position of the Gaussian sphere at the target time. The initial rotation matrix is ​​updated based on the rotation update information to obtain the target rotation matrix, and the initial scaling matrix is ​​updated based on the scaling update information to obtain the candidate scaling matrix of the Gaussian sphere at the target time. Determine the volume conservation constraint coefficients, and based on the volume conservation constraint coefficients and the candidate scaling matrix, determine the target scaling matrix of the Gaussian sphere at the target time; Based on the target rotation matrix and the target scaling matrix, target shape indication information of the Gaussian sphere at the target time is generated, and rendering processing is performed based on the target position of the Gaussian sphere at the target time and the target shape indication information to obtain the virtual object displayed at the target time.

[0007] Thirdly, embodiments of this application provide an object rendering apparatus, including: The file acquisition module is used to acquire the skeleton binding file of the virtual object. The skeleton binding file includes the 3D mesh of the virtual object, the bones bound to the 3D mesh, and the skinning weights of the vertices on the 3D mesh. The generation module is used to perform generation processing based on the above-mentioned three-dimensional mesh to obtain multiple Gaussian spheres for generating the above-mentioned virtual object. The Gaussian spheres include initial position points and initial shape indication information. The transfer module is used to perform weight transfer processing based on the skin weights of the neighboring points of the initial position point in the three-dimensional mesh to obtain the skin weights of the Gaussian sphere. The matrix acquisition module is used to acquire the target transformation matrix of the above skeleton at the target time. The update module is used to update the initial position point and the initial shape indication information based on the skin weight of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, so as to obtain the target position and target shape indication information of the Gaussian sphere at the target time. The rendering module is used to perform rendering processing based on the target position and target shape indication information of the Gaussian sphere at the target time, so as to obtain the virtual object displayed at the target time.

[0008] Fourthly, embodiments of this application provide an object rendering apparatus, including: The first acquisition module is used to acquire the Gaussian sphere file corresponding to the virtual object. The Gaussian sphere file includes the initial position points, initial shape indication information and skinning weights used to generate the Gaussian sphere of the virtual object. The initial shape indication information includes the initial rotation matrix and the initial scaling matrix. The second acquisition module is used to acquire the target transformation matrix of the skeleton of the aforementioned virtual object at the target time. The information update module is used to determine the rotation update information and scaling update information of the Gaussian sphere at the target time, and update the initial position point, based on the target transformation matrix of the skeleton at the target time and the skin weight of the Gaussian sphere, to obtain the target position of the Gaussian sphere at the target time; update the initial rotation matrix based on the rotation update information to obtain the target rotation matrix, and update the initial scaling matrix based on the scaling update information to obtain the candidate scaling matrix of the Gaussian sphere at the target time; determine the volume conservation constraint coefficient, and determine the target scaling matrix of the Gaussian sphere at the target time based on the volume conservation constraint coefficient and the candidate scaling matrix; and generate the target shape indication information of the Gaussian sphere at the target time based on the target rotation matrix and the target scaling matrix. The object rendering module is used to perform rendering processing based on the target position and target shape indication information of the Gaussian sphere at the target time to obtain the virtual object displayed at the target time.

[0009] Fifthly, embodiments of this application also provide an electronic device, including a memory storing multiple instructions; a processor loads instructions from the memory to execute the steps of any of the object rendering methods provided in embodiments of this application.

[0010] Sixthly, embodiments of this application also provide a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute the steps of any of the object rendering methods provided in embodiments of this application.

[0011] In a seventh aspect, embodiments of this application also provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps in any of the object rendering methods provided in embodiments of this application.

[0012] In this embodiment, a skeletal binding file for a virtual object is obtained. This file includes a 3D mesh of the virtual object, bones bound to the 3D mesh, and skinning weights of vertices on the 3D mesh. Based on the 3D mesh, a generation process is performed to obtain multiple Gaussian spheres for generating the virtual object. Each Gaussian sphere includes initial position points and initial shape indication information. Weight transfer processing is performed based on the skinning weights of the initial position points' neighboring points in the 3D mesh to obtain the skinning weights of the Gaussian spheres. The target transformation matrix of the bones at the target time is obtained. Based on the skinning weights of the Gaussian spheres and the target transformation matrix of the bones at the target time, the initial position points and initial shape indication information are updated to obtain the target position and target shape indication information of the Gaussian spheres at the target time. Rendering processing is performed based on the target position and target shape indication information of the Gaussian spheres at the target time to obtain the virtual object displayed at the target time. This method combines the use of the skeletal binding file and weight transfer processing to obtain the skinning weights of the Gaussian spheres, eliminating the need to rely on SMPL and FLAME for obtaining skinning weights. This allows the method to be applied to virtual objects with arbitrary skeletons without requiring 3D reconstruction, improving the applicability and rendering efficiency of the 3D Gaussian splashing technology.

[0013] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram illustrating an application scenario of the object rendering method provided by an exemplary embodiment of this disclosure; Figure 2 This is a flowchart illustrating an exemplary embodiment of the object rendering method provided in this disclosure; Figure 3 This is a schematic diagram of the system architecture provided by an exemplary embodiment of this disclosure; Figure 4 This is a flowchart illustrating the process of determining skin weights provided by an exemplary embodiment of this disclosure; Figure 5 This is a flowchart illustrating the process of determining target shape indication information provided in an exemplary embodiment of this disclosure; Figure 6 This is a flowchart illustrating another object rendering method provided by an exemplary embodiment of this disclosure; Figure 7 This is a schematic diagram of an object rendering system provided by an exemplary embodiment of this disclosure; Figure 8 This is a schematic diagram of another object rendering system provided by an exemplary embodiment of this disclosure; Figure 9 This is a schematic diagram of an object rendering apparatus provided in an exemplary embodiment of this disclosure; Figure 10 This is a schematic diagram of another object rendering apparatus provided by an exemplary embodiment of the present disclosure; Figure 11 This is a schematic diagram of an electronic device provided by an exemplary embodiment of this disclosure. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. At the same time, in the description of the embodiments of this application, the terms "first," "second," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0017] This application provides an object rendering method, apparatus, electronic device, and computer-readable storage medium. Specifically, this embodiment will be described from the perspective of an object rendering apparatus, which can be integrated into an electronic device, meaning that the object rendering method of this application embodiment can be executed by an electronic device. Optionally, the electronic device may include a terminal device or a server. Optionally, the terminal device may be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, game console, or personal computer (PC), etc.

[0018] Optionally, the server can be a standalone server, or a server network or cluster, including but not limited to computers, network hosts, single network servers, multiple network server sets, or cloud servers composed of multiple servers. Cloud servers consist of a large number of computers or network servers based on cloud computing. Terminal devices and servers can communicate bidirectionally via the network.

[0019] For example, such as Figure 1 As shown, the server obtains the skeleton binding file of the virtual object. The skeleton binding file includes the 3D mesh of the virtual object, the bones bound to the 3D mesh, and the skinning weights of the vertices on the 3D mesh. Based on the 3D mesh, generation processing is performed to obtain multiple Gaussian spheres used to generate the virtual object. Each Gaussian sphere includes initial position points and initial shape indication information. Weight transfer processing is performed based on the skinning weights of the initial position points' neighboring points in the 3D mesh to obtain the skinning weights of the Gaussian spheres. The terminal device obtains the target transformation matrix of the skeleton at the target time and the skinning weights of the Gaussian spheres from the server. Based on the skinning weights of the Gaussian spheres and the target transformation matrix of the skeleton at the target time, the initial position points and initial shape indication information are updated to obtain the target position and target shape indication information of the Gaussian spheres at the target time. Rendering processing is performed based on the target position and target shape indication information of the Gaussian spheres at the target time to obtain the virtual object displayed at the target time.

[0020] The following detailed description is provided in conjunction with the accompanying drawings. In this embodiment, the execution subject is a terminal device as an example. It should be noted that the order of description in the following embodiments is not intended to limit the preferred order of the embodiments. Although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than that shown in the accompanying drawings.

[0021] Please refer to Figure 2 The specific process of this object rendering method can be summarized in steps 201 to 206, where: Step 201: Obtain the skeleton binding file of the virtual object. The skeleton binding file includes the 3D mesh of the virtual object, the bones bound to the 3D mesh, and the skinning weights of the vertices on the 3D mesh.

[0022] In this context, a virtual object refers to a digital entity with a specific form and structure that exists in a virtual environment. It can be configured according to actual circumstances; for example, a virtual object can be a game character, game pet, game mount, virtual vehicle, or virtual monster, etc., though this embodiment does not impose such limitations. Optionally, a virtual object can be a game character in a game scene. For example, a virtual object can be a non-human character in a massively multiplayer online game, such as a quadruped, pterosaur, or mechanical guardian. Alternatively, a virtual object can be a high-fidelity fighting character in an action game, again without limitation in this embodiment.

[0023] A bone rigging file refers to data used to store the influence of bones on vertices in a 3D mesh. Optionally, the format of the bone rigging file can be set according to the actual situation. For example, the format of the bone rigging file can be FBX or GLB, which is not limited in this embodiment.

[0024] A 3D mesh refers to a digital 3D model used to represent virtual objects. Bones bound to a 3D mesh refer to the skeletons that drive the 3D mesh's movement. Optionally, the bones bound to the 3D mesh in the bone binding file can exist in the form of a bone tree. Vertex skinning weights are used to indicate the degree of influence of the bones on the vertices.

[0025] Optionally, when the bones bound to the 3D mesh exist in the form of a bone tree, the bones bound to the 3D mesh can be... Where B represents the skeletal tree, b j Indicates the first j Root bones, p j Indicates the first j The parent bone of the root bone, Indicates the first j The global transformation matrix of the root skeleton under the bound pose. Optionally, the set of vertices of the 3D mesh is... ,in, v k Representing the vertices of a 3D mesh k , As vertex k coordinates Normalized skin weights for vertices ( This indicates that each bone is related to the first... k The sum of the skinning weights of each vertex is 1.

[0026] Optionally, the shape of the surface region where the vertex is located can be set according to actual conditions. For example, the shape of the surface region where the vertex is located can be a triangle or a rectangle; this embodiment does not limit this. When the shape of the surface region is a triangle, the three vertices of the surface region are... .

[0027] Step 202: Generate multiple Gaussian spheres based on the 3D mesh to generate virtual objects. The Gaussian spheres include initial position points and initial shape indication information.

[0028] Here, the Gaussian sphere refers to the unit constituting the virtual object. The initial position point can refer to the location of the geometric center of the Gaussian sphere. Optionally, the initial position point can be a vertex on the 3D mesh, a point in the surface region of the 3D mesh, and / or a point in the internal space of the 3D mesh; this embodiment does not impose such limitations. Initial shape indication information is used to indicate the shape and orientation of the Gaussian sphere. Optionally, the initial shape indication information can include an initial rotation matrix and an initial scaling matrix. Optionally, the initial shape indication information can exist in the form of an initial covariance, in which case the initial covariance can be expressed as: ,in, Let R represent the initial covariance matrix, S represent the initial rotation matrix, and T represent the initial scaling matrix. Optionally, the Gaussian sphere may also include color parameters and / or opacity parameters, etc.

[0029] Optionally, the surface region of the 3D mesh can be sampled to obtain initial position points, and initialization processing can be performed to obtain initial shape indication information. A Gaussian sphere is then obtained based on the initial position points and initial shape indication information. Optionally, the initial shape indication information can be set based on the local curvature of the 3D mesh or based on an isotropic sphere. When the initial shape indication information is set based on an isotropic sphere, the Gaussian sphere is initially a sphere, which is subsequently transformed into an ellipsoid. Optionally, when the Gaussian sphere also includes color parameters and / or opacity parameters, the initialization processing can also obtain color parameters and / or opacity parameters. Optionally, the opacity parameter can be a default parameter value, such as 1 or 1.5, which is not limited in this embodiment.

[0030] Optionally, the coordinates of the initial position points obtained by sampling the surface region of the 3D mesh can be obtained by interpolation based on the coordinates of the target vertices of the surface region. Specifically, the coordinates of the initial position points can be obtained using formula (1): (1) in, Let represent the initial position of the i-th Gaussian sphere. This represents the coordinates of the target vertex k1. This represents the coordinates of the target vertex k2. This represents the coordinates of the target vertex k3. , and The corresponding weights can be obtained through a random algorithm and their sum is 1.

[0031] Optionally, the surface region of the 3D mesh can be sampled according to a preset sampling algorithm to obtain initial position points. The preset sampling algorithm can be set according to the actual situation, for example, it can be a preset density strategy sampling algorithm or a neural network model, which is not limited in this embodiment. Optionally, when the preset sampling algorithm is a preset density strategy sampling algorithm, the sampling density of the joint region can be set to M times that of the non-joint region. M can be set according to the actual situation, for example, M is a value between 1.5 and 4, which is not limited in this embodiment. When the preset sampling algorithm is a preset density strategy sampling algorithm, there can be more initial position points in the joint region (elbows, knees, and other easily bent areas), thereby enabling better fitting of surface details.

[0032] In some embodiments, the generation process is based on a 3D mesh to obtain multiple Gaussian spheres for generating virtual objects, including: The surface region and internal space of the 3D mesh are sampled to obtain multiple initial position points; Determine the initial shape indication information corresponding to the initial position point; Based on the initial position point and initial shape indication information, multiple Gaussian spheres are obtained for generating virtual objects.

[0033] The sampling process performed on the surface region of the 3D mesh can be called surface sampling processing, and the sampling process performed on the internal space of the 3D mesh can be called volume sampling processing. Optionally, the initial shape indication information can be set based on the curvature of the surface region where the initial position point is located, or the initial shape indication information can be set based on an isotropic sphere; this embodiment does not limit this. Optionally, volume sampling processing can also be performed according to a preset sampling algorithm.

[0034] In this embodiment, the surface area and internal space of the 3D mesh are sampled to obtain multiple initial position points; the initial shape indication information corresponding to the initial position points is determined; based on the initial position points and the initial shape indication information, multiple Gaussian spheres for generating virtual objects are obtained. This achieves the acquisition of Gaussian spheres through surface sampling and volume sampling, thereby increasing the diversity of the obtained Gaussian spheres and improving their accuracy. This ensures the realism of the virtual objects even when applied to virtual objects with complex internal structures.

[0035] In some embodiments, sampling the internal space of the three-dimensional mesh to obtain initial position points includes: Based on the semantic labels and / or geometric features of the surface regions in the 3D mesh, the target surface regions that need to undergo volume sampling are determined. Volume sampling is performed on the target surface area within a preset distance from the interior of the 3D mesh to obtain the initial position point.

[0036] In this embodiment, semantic tags are used to indicate the location of a surface area; for example, semantic tags can be hair, cape, skirt, wing membrane, thick fabric, etc. Geometric features are used to describe the geometric characteristics of the surface area; for example, geometric features can be the thickness and / or curvature of the surface area. Optionally, surface areas whose geometric features meet preset feature conditions and / or surface areas with target semantic tags can be determined as target surface areas. Preset feature conditions are used to indicate that the surface area has a strong sense of volume; for example, preset feature conditions can be that the thickness of the surface area exceeds a preset thickness threshold, which is not limited in this embodiment. Target semantic tags are semantic tags that indicate that the surface area has a strong sense of volume; for example, target semantic tags are semantic tags for surface areas containing hair or wing membranes, which are not limited in this embodiment.

[0037] In this embodiment, based on the semantic labels and / or geometric features of the surface region of the three-dimensional mesh, the target surface region that needs to be volume sampled is determined; volume sampling is performed on the target surface region within a preset distance range into the interior of the three-dimensional mesh to obtain the initial position point. This enables the spatial range that needs to be volume sampled to be obtained through semantic labels and / or geometric features, so that volume sampling can be performed on surface regions with a strong sense of volume, thereby improving the accuracy of the initial position point obtained through volume sampling.

[0038] Step 203: Perform weight transfer processing based on the skin weights of the initial position point's neighboring points in the 3D mesh to obtain the skin weights of the Gaussian sphere.

[0039] In this context, a neighboring point refers to a point within a certain region centered on the initial position point. For example, a neighboring point can be a vertex of the surface region where the initial position point is located, or it can be a point in the 3D mesh whose distance from the initial position point meets a preset distance condition. When a neighboring point is a vertex of the surface region where the initial position point is located, for example, when the surface region is triangular, the neighboring points are the three vertices of the triangular facet containing the initial position point. When a neighboring point is a point in the 3D mesh whose distance from the initial position point meets a preset distance condition, this point can be at least one of the following: a vertex of the 3D mesh, a point in the upper surface region of the 3D mesh, or a point in the internal space of the 3D mesh; this embodiment does not impose any limitations on this.

[0040] Optionally, the skin weights of the neighboring points can be known or unknown. When the skin weights of the neighboring points are unknown, the skin weights of the Gaussian sphere can be obtained through iteration based on the skin weights of the neighboring points. Optionally, the skin weights of the initial position points located in the surface region and / or the skin weights of the initial position points that will be vertices can be set as Dirichlet boundary conditions before iteration.

[0041] It is understandable that when the initial position point is a vertex on a 3D mesh, the skin weight of the Gaussian sphere where the initial position point is located is the skin weight of the vertex. In this case, it is not necessary to determine the skin weight of the Gaussian sphere based on the skin weight of the neighboring points.

[0042] In some embodiments, the neighborhood points include target vertices of the surface region where the nearest neighbor points and / or the initial position point are located on the 3D mesh. The nearest neighbor points are position points in the 3D mesh whose distance from the initial position point satisfies a preset distance condition. Weight transfer processing is performed based on the skin weights of the neighborhood points of the initial position point in the 3D mesh to obtain the skin weights of the Gaussian sphere, including: The skin weights of the Gaussian sphere are obtained by interpolation based on the skin weights of the target vertices. And / or, perform diffusion processing based on the skin weights of nearest neighbor locations to obtain the skin weights of the Gaussian sphere.

[0043] Specifically, when the surface region is triangular, the target vertices are the three vertices of the triangular facet; when the surface region is rectangular, the target vertices are the four vertices of the rectangle. The number of nearest neighbor points can be set according to the actual situation. For example, if the number of nearest neighbor points is 5, then the nearest neighbor points can be the 5 points closest to the initial position point. This embodiment does not impose any limitation on this.

[0044] The principle of diffusion treatment can be referenced from the principle of thermal expansion, which uses the location of the known skin weight as the "heat source" and diffuses to the surrounding area according to physical laws.

[0045] Understandably, since the skin weights of the target vertex are known, interpolation can be performed based on these skin weights to obtain the skin weights of the Gaussian sphere. However, since the skin weights of the nearest neighbor points may or may not be known, the skin weights of the Gaussian sphere cannot be obtained through interpolation; instead, they are obtained through diffusion processing.

[0046] Optionally, the skinning weights of each target vertex can be weighted and summed based on the weights corresponding to each target vertex to obtain the skinning weights of the Gaussian sphere, thereby achieving interpolation processing. Specifically, the skinning weights of the Gaussian sphere can be obtained through formula (2): (2) in, w i Indicates the first i Skin weights of a Gaussian sphere Target vertex k Skin weight of 1 Target vertex k Skin weight of 2 Target vertex k Skin weight of 3 , Target vertex k The weight corresponding to 1 Target vertex k The weight corresponding to 2, Target vertex k The weight corresponding to 3.

[0047] Optionally, the weights corresponding to each target vertex can be pre-set weights, or the weights corresponding to each target vertex can be the relative positions between the target vertex and the initial position point. This embodiment does not impose any limitations on this.

[0048] In this embodiment, the neighborhood points include the target vertices of the surface regions where the nearest neighbor points and / or the initial position point are located on the 3D mesh. The nearest neighbor points are the position points in the 3D mesh whose distance from the initial position point meets a preset distance condition. The skin weight of the Gaussian sphere is obtained by interpolation based on the skin weight of the target vertex; and / or, the skin weight of the Gaussian sphere is obtained by diffusion processing based on the skin weight of the nearest neighbor points. This achieves the acquisition of the skin weight of the Gaussian sphere through multiple methods, thereby ensuring that the skin weight of the Gaussian sphere can be obtained. Furthermore, the spatial coherence of the weight distribution can be guaranteed by utilizing the accuracy of the coordinates and the continuity of thermal diffusion.

[0049] In some embodiments, interpolation is performed based on the skinning weights of the target vertex to obtain the skinning weights of the Gaussian sphere, including: When the initial position point is located in the surface region of the 3D mesh, interpolation is performed based on the skin weight of the target vertex to obtain the skin weight of the Gaussian sphere. And / or, perform diffusion processing based on the skin weights of nearest neighbor locations to obtain the skin weights of the Gaussian sphere, including: When the initial position point is not located on the surface region of the 3D mesh, the skin weight of the Gaussian sphere is obtained by diffusion processing based on the skin weight of the nearest position point.

[0050] Since the skin weights of the target vertex are known, interpolation can be performed based on these skin weights to obtain the skin weights of the Gaussian sphere. However, because the skin weights of the nearest neighbor points may be known or unknown, interpolation cannot be used to obtain the skin weights of the Gaussian sphere; instead, a diffusion process is used to obtain them.

[0051] In this embodiment, when the initial position point is located on the surface region of the 3D mesh, interpolation is performed based on the skin weight of the target vertex to obtain the skin weight of the Gaussian sphere; and / or, when the initial position point is not located on the surface region of the 3D mesh, diffusion is performed based on the skin weight of the nearest neighbor position point to obtain the skin weight of the Gaussian sphere. This realizes the selection of different skin weight determination methods based on different initial position point conditions, further ensuring that the skin weight of the Gaussian sphere can be obtained.

[0052] In some embodiments, interpolation is performed based on the skinning weights of the target vertex to obtain the skinning weights of the Gaussian sphere, including: Determine the relative position between the initial position point and the target vertex; Based on the relative position, the skin weights of the target vertex are interpolated to obtain the skin weights of the Gaussian sphere.

[0053] The relative position can be obtained by subtracting the coordinates of the target vertex from the coordinates of the initial position point, or by subtracting the coordinates of the initial position point from the coordinates of the target vertex. This embodiment does not limit the specific method used. After obtaining the relative position, it can be used as the weight corresponding to the target vertex. The skinning weights of the target vertex are then weighted and summed to obtain the skinning weights of the Gaussian sphere.

[0054] It is understandable that the skin weights of a Gaussian sphere are a vector, comprising multiple weight values, and the sum of these weight values ​​for a Gaussian sphere is 1. Optionally, the relative positions can be processed so that the sum between the relative positions is 1.

[0055] Since there may be multiple initial position points on a surface region, if the weight corresponding to the target vertex is a fixed weight, the skinning weights of the initial position points on the same surface region may be the same, resulting in a decrease in the realism of the virtual object rendered based on the Gaussian sphere. Therefore, in this embodiment, the relative position between the initial position point and the target vertex is determined; based on the relative position, the skinning weight of the target vertex is interpolated to obtain the skinning weight of the Gaussian sphere. This ensures that not only are more Gaussian spheres obtained, but also that the skinning weights of different Gaussian spheres are different, improving the realism of the virtual object. Furthermore, the universality of the skeleton is carried by the dimension of the skinning weight (the skinning weight is a vector), so the above interpolation method can be applied without modification regardless of the number of bones.

[0056] In some embodiments, diffusion processing is performed based on the skin weights of nearest neighbor locations to obtain the skin weights of the Gaussian sphere, including: Determine the first distance between the initial position point and each of its nearest neighbor positions, as well as the total distance corresponding to the first distance; Based on the first distance and the total distance, determine the normalized distance of the initial location point to its nearest neighbor locations; The skin weights of the Gaussian sphere are obtained by fusing the skin weights of the nearest neighbor points based on the normalized distance.

[0057] The calculation method of the first distance can be set according to the actual situation. For example, the first distance can be Euclidean distance or Mahalanobis distance. This embodiment does not limit it here. Optionally, the total distance can be the sum of each first distance, or it can be based on the exponential operation of each first distance, and then the results of each exponential operation are added together to get the total distance. For example, the first exponential operation can be based on the first distance to get the first exponential value. The first exponential values ​​are added together to get the second exponential value, which is the total distance. The first exponential value is divided by the second exponential value to get the normalized distance. Specifically, the coordinates of the initial position point and the coordinates of the nearest position points can be substituted into formula (3) for calculation to get the normalized distance: (3) in, p ij Indicates the first i The Gaussian sphere is aimed at the first j Normalized distance of the nearest neighbor points exp Indicates exponentiation. N i Indicates the first i The set of nearest neighbor points of a Gaussian sphere. This represents the first distance between the i-th Gaussian sphere and the j-th nearest neighbor point. Let represent the coordinates of the initial position of the i-th Gaussian sphere. This represents the coordinates of the j-th nearest neighbor point. This indicates the initial shape information of the i-th Gaussian sphere. Indicates the first index value. This represents the first distance between the i-th Gaussian sphere and its k-th nearest neighbor. This represents the coordinates of the k-th nearest neighbor point. This represents the second exponent value.

[0058] After obtaining the normalized distances of the initial position point to each nearest neighbor position point, the normalized distances are used as weights to perform a weighted sum of the skin weights of each nearest neighbor position point, thereby obtaining the skin weights of the Gaussian sphere. Specifically, the normalized distances and the skin weights of each nearest neighbor position point can be substituted into formula (4) for calculation to obtain the skin weights of the Gaussian sphere: (4) in, w i This represents the skin weight of the i-th Gaussian sphere. w j This represents the skin weight of the j-th nearest neighbor point. N i Indicates the first i The set of nearest neighbor points of a Gaussian sphere. p ij Indicates the first i The Gaussian sphere is aimed at the first j Normalized distances between the nearest neighbor locations.

[0059] It is understandable that, since the skin weights of some neighboring points are unknown, equation (4) can be iteratively solved to obtain the skin weights of the Gaussian sphere. Optionally, during the iterative solution, the known skin weights of the initial points on the surface region of the 3D mesh can be used as boundary conditions.

[0060] In this embodiment, a first distance and a total distance corresponding to the first distance between the initial position point and each of the nearest neighbor position points are determined; based on the first distance and the total distance, a normalized distance of the initial position point relative to the nearest neighbor position points is determined; the skin weights of the nearest neighbor position points are fused based on the normalized distance to obtain the skin weights of the Gaussian sphere, thereby improving the accuracy of the obtained skin weights of the Gaussian sphere by expanding the first distance between the initial position point and the nearest neighbor position points.

[0061] In some embodiments, the first distance is the Mahalanobis distance, and determining the first distance between the initial location point and each of its nearest neighbor locations includes: Based on the initial shape indication information included in the Gaussian sphere where the initial position point is located, the first distance between the initial position point and each of the nearest neighbor position points is determined.

[0062] The coordinates of the initial position point, the coordinates of the nearest position points, and the initial shape indication information can be substituted into formula (5) for calculation to obtain the first distance: (5) Where T represents the transpose, and the initial shape indication information is represented by the initial covariance matrix. Let represent the inverse of the initial covariance matrix of the i-th Gaussian sphere.

[0063] In this embodiment, the first distance is the Mahalanobis distance. Based on the initial shape indication information included in the Gaussian sphere where the initial position point is located, the first distance between the initial position point and each of the nearest neighbor position points is determined so that the geometry of the Gaussian sphere is taken into account when determining the skin weight of the Gaussian sphere. This makes heat conduction faster in the wider direction of the Gaussian sphere (the skin weight propagates further) and heat conduction slower in the narrower direction of the Gaussian sphere (the skin weight propagates closer). As a result, the skin weight propagation will flow naturally along the geometry rather than simply diffuse in a spherical shape, thereby making the directionality of weight propagation consistent with the local geometry.

[0064] Step 204: Obtain the target transformation matrix of the skeleton at the target time.

[0065] The target transformation matrix, also known as the target global transformation matrix, indicates the transformation relationship of the skeleton from local space to world space at the target time. Optionally, there can be multiple target times, including the current time. Different target times can correspond to different image frames, and the target transformation matrix at the target time is the target transformation matrix corresponding to the image frame displayed at the target time.

[0066] Optionally, the target transformation matrix of a bone at the target time can be determined through a bone rigging file. Specifically, the bone rigging file may also include the local transformation matrix of the bone at keyframes. The local transformation matrix at the target time can be obtained based on the local transformation matrix of the bone at the keyframes, and then the target transformation matrix of the bone at the target time can be obtained based on the target transformation matrix of the parent bone and the local transformation matrix of the bone. Optionally, the target transformation matrix of the bone at the target time can be obtained through a game engine animation system. Optionally, the target transformation matrices of all bones at the target time can be represented by a set, which can be: Each target transformation matrix can be a 4x4 homogeneous transformation matrix. This represents the set of all bones containing their target transformation matrices at target time t. Let represent the target transformation matrix of the j-th bone at target time t.

[0067] Step 205: Update the initial position point and initial shape indication information based on the skin weight of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, so as to obtain the target position and target shape indication information of the Gaussian sphere at the target time.

[0068] Among them, the initial position point and initial shape indication information can be understood as the position and shape indication information under the bound posture, while the target position and target shape indication information are the position and shape indication information of the posture at the target time.

[0069] In some embodiments, the skeleton binding file further includes an initial transformation matrix for the skeleton, updates the initial position points based on the skinning weights of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, and obtains the target position of the Gaussian sphere at the target time, including: Based on the target transformation matrix of the skeleton at the target time and the initial transformation matrix of the skeleton, the relative transformation matrix of the skeleton is determined. The skin weights of the Gaussian sphere, the relative transformation matrix affecting the skeleton of the Gaussian sphere, and the initial position point are fused to obtain the target position of the Gaussian sphere at the target time.

[0070] The initial transformation matrix of the skeleton, also known as the initial global transformation matrix, is the transformation matrix of the skeleton under the bound pose, used to indicate the transformation of the skeleton from local space to world space. The relative transformation matrix, also known as the skinning matrix, can be obtained by multiplying the target transformation matrix by the inverse of the initial transformation matrix.

[0071] There is at least one bone that affects the Gaussian sphere. The degree of influence of each bone on the Gaussian sphere is called the skin weight of that bone relative to the Gaussian sphere. The skin weight of the Gaussian sphere includes the skin weight of each bone relative to the Gaussian sphere. The skin weight of the bone relative to the Gaussian sphere can be multiplied by the relative transformation matrix of the bone, and the results of the multiplication of each bone can be added together and multiplied by the initial position point to obtain the target position. Specifically, the skin weight of the Gaussian sphere, the relative transformation matrix of the bones and the initial position point can be substituted into formula (7) for calculation to obtain the target position of the Gaussian sphere at the target time: (7) in, Let represent the target position of the i-th Gaussian sphere at the target time t, j represent the j-th bone affecting the Gaussian sphere, and J represent the number of bones affecting the Gaussian sphere. This represents the target transformation matrix of the j-th bone at target time t. This represents the initial transformation matrix for the j-th bone. The matrix representing the inverse of the initial transformation matrix of the j-th bone is... Let represent the initial position of the i-th Gaussian sphere.

[0072] In this embodiment, the skeleton binding file also includes an initial transformation matrix of the skeleton. Based on the target transformation matrix of the skeleton at the target time and the initial transformation matrix of the skeleton, the relative transformation matrix of the skeleton is determined. The skin weights of the Gaussian sphere, the relative transformation matrices of the skeletons affecting the Gaussian sphere, and the initial position points are fused to obtain the target position of the Gaussian sphere at the target time. This realizes the updating through the relative transformation matrix and skin weights, reducing the error accumulation caused by spatial transformation and improving the accuracy of the target position of the Gaussian sphere at the target time.

[0073] In some embodiments, the initial shape indication information is updated based on the skin weights of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, to obtain the target shape indication information of the Gaussian sphere at the target time, including: Based on the target transformation matrix of the skeleton at the target time, the initial transformation matrix of the skeleton, and the skinning weights of the Gaussian sphere, the rotation update information and scaling update information at the target time are determined. Based on the rotation update information and scaling update information, the initial shape indication information is updated to obtain the target shape indication information of the Gaussian sphere at the target time.

[0074] The rotation update information describes the rotational changes from the bound attitude to the target time. The scaling update information describes the scaling changes from the bound attitude to the target time. Optionally, the initial shape indication information may include an initial rotation matrix and an initial scaling matrix. The rotation update information and the initial rotation matrix can be multiplied to obtain the target rotation matrix, and the scaling update information and the initial scaling matrix can be multiplied to obtain the target scaling matrix. The target shape indication information is then generated based on the target rotation matrix and the target scaling matrix.

[0075] In some embodiments, rotation update information at the target time is determined based on the target transformation matrix of the skeleton at the target time, the initial transformation matrix of the skeleton, and the skinning weights of the Gaussian sphere, including: Based on the target transformation matrix of the skeleton at the target time and the initial transformation matrix of the skeleton, the relative transformation matrix of the skeleton is determined. Based on the relative transformation matrix of the skeleton, the first relative rotation information of the skeleton is determined; The rotation update information at the target time is obtained by fusing the quaternion based on the first relative rotation information of the skeleton affecting the Gaussian sphere and the skin weight of the Gaussian sphere.

[0076] The relative transformation matrix may include first relative rotation information and first relative scaling information. The initial shape indication information may include an initial rotation matrix and an initial scaling matrix. Rotation update information is obtained by fusing the quaternion of the first relative rotation information affecting the skeleton of the Gaussian sphere with the skin weights of the Gaussian sphere. The initial rotation matrix is ​​then updated based on the rotation update information to obtain the target rotation matrix at the target time. Similarly, scaling update information is obtained by fusing the first relative scaling information affecting the skeleton of the Gaussian sphere with the skin weights of the Gaussian sphere. The initial scaling matrix is ​​then updated based on the scaling update information to obtain the target scaling matrix at the target time. Optionally, the target shape indication information may include a target rotation matrix and a target scaling matrix, or a target covariance matrix at the target time can be generated based on the target rotation matrix and the target scaling matrix; the target covariance matrix is ​​the target shape indication information.

[0077] Optionally, the unit quaternion of the first relative rotation information and the skinning weight of the bone relative to the Gaussian sphere can be fused to obtain the rotation update information. Specifically, the unit quaternion of the first relative rotation information and the skinning weight of the bone relative to the Gaussian sphere can be substituted into formula (8) for calculation to obtain the rotation update information: (8) in, A i Indicates that for the first i The rotation update information for each Gaussian sphere, where j represents the j-th bone affecting the i-th Gaussian sphere. This represents the skinning weight of the j-th bone with respect to the i-th Gaussian sphere. q j The unit quaternion represents the first relative rotation information, and T represents the transpose. express q j It is a vector consisting of 4 real numbers.

[0078] Optionally, the rotation update information can be multiplied with the initial rotation matrix to update the initial rotation matrix and obtain the target rotation matrix, and the scaling update information can be multiplied with the initial scaling matrix to obtain the target rotation matrix at the target time.

[0079] Since directly fusing the first relative rotation information and the skin weights of the Gaussian sphere would result in erroneous rotations, this embodiment determines the relative transformation matrix of the skeleton based on the target transformation matrix and the initial transformation matrix of the skeleton at the target time. Based on the relative transformation matrix of the skeleton, the first relative rotation information of the skeleton is determined. The rotation update information is obtained by fusing the quaternion of the first relative rotation information of the skeleton that affects the Gaussian sphere with the skin weights of the Gaussian sphere. This achieves the fusion of the quaternion of the first relative rotation information and the skin weights, avoiding erroneous rotations and improving the accuracy of the obtained target shape indication information.

[0080] In some embodiments, the initial shape indication information includes an initial rotation matrix and an initial scaling matrix. Based on the rotation update information and the scaling update information, the initial shape indication information is updated to obtain the target shape indication information of the Gaussian sphere at the target time, including: Determine the largest eigenvalue of the rotation update information; Based on the maximum eigenvalue, update the initial rotation matrix to obtain the target rotation matrix of the Gaussian sphere at the target time; The target scaling matrix is ​​obtained by updating the initial scaling matrix based on the scaling update information, and the target shape indication information of the Gaussian sphere at the target time is determined based on the target rotation matrix and the target scaling matrix.

[0081] The rotation update information exists in the form of a matrix, which is obtained by solving formula (8). A i The maximum eigenvalue is obtained, and then the maximum eigenvalue is multiplied by the initial rotation matrix to obtain the target rotation matrix of the Gaussian sphere at the target time. Based on the target rotation matrix and the target scaling matrix, the target shape indication information is obtained.

[0082] Alternatively, the maximum eigenvalue can be efficiently solved on a graphics processor in a fixed number of steps using a power iteration method.

[0083] In this embodiment, the initial shape indication information includes an initial rotation matrix and an initial scaling matrix. The maximum eigenvalue of the rotation update information is determined. Based on the maximum eigenvalue, the initial rotation matrix is ​​updated to obtain the target rotation matrix of the Gaussian sphere at the target time. Based on the scaling update information, the initial scaling matrix is ​​updated to obtain the target scaling matrix. Based on the target rotation matrix and the target scaling matrix, the target shape indication information of the Gaussian sphere at the target time is determined. This achieves the goal of updating the initial rotation matrix through the maximum eigenvalue, rather than directly updating the initial rotation matrix through the rotation update information, thereby further improving the accuracy of the rotation.

[0084] In some embodiments, rotation update information and scaling update information at the target time are determined based on the target transformation matrix of the skeleton at the target time, the initial transformation matrix of the skeleton, and the skinning weights of the Gaussian sphere, including: Based on the target transformation matrix of the skeleton at the target time and the initial transformation matrix of the skeleton, the relative transformation matrix of the skeleton is determined. The relative transformation matrix of the skeleton and the skinning weights of the Gaussian sphere are fused to obtain the hybrid transformation matrix; Perform polar decomposition on the hybrid transformation matrix to update rotation and scaling information at the target time.

[0085] In this embodiment, the initial rotation matrix can be updated based on the quaternion of the rotation update information of the bones affecting the Gaussian sphere. It is understood that the implementation method in this embodiment and the above embodiment, which "determines the relative transformation matrix of the bones based on the target transformation matrix and the initial transformation matrix of the bones at the target time; determines the first relative rotation information of the bones based on the relative transformation matrix of the bones; and performs fusion processing based on the quaternion of the first relative rotation information of the bones affecting the Gaussian sphere and the skinning weights of the Gaussian sphere to obtain the rotation update information at the target time," are parallel schemes, and either method can be chosen to obtain the rotation update information.

[0086] Alternatively, the hybrid transformation matrix can be expressed as shown in equation (9): (9) in, F i Represents the mixed transformation matrix, This represents the skinning weight of the j-th bone with respect to the i-th Gaussian sphere. This represents the target transformation matrix of the j-th bone at target time t. This represents the initial transformation matrix for the j-th bone. Let represent the inverse of the initial transformation matrix of the j-th bone.

[0087] Alternatively, polar decomposition can be performed using formula (10): (10) in, This represents the rotation update information for the i-th Gaussian sphere. express It is a valid 3x3 matrix. This represents the scaling update information for the i-th Gaussian sphere.

[0088] In this embodiment, the relative transformation matrix of the skeleton is determined based on the target transformation matrix of the skeleton at the target time and the initial transformation matrix of the skeleton; the relative transformation matrix of the skeleton and the skin weight of the Gaussian sphere are fused to obtain the hybrid transformation matrix; extreme decomposition is performed on the hybrid transformation matrix to update the rotation and scaling information at the target time, thereby achieving updates through extreme decomposition, avoiding erroneous rotations, and improving the accuracy of the obtained target shape indication information.

[0089] In some embodiments, the initial shape indication information includes an initial scaling matrix and an initial rotation matrix. Based on the rotation update information and the scaling update information, the initial shape indication information is updated to obtain the target shape indication information of the Gaussian sphere at the target time, including: The initial rotation matrix is ​​updated based on the rotation update information to obtain the target rotation matrix of the Gaussian sphere at the target time. The initial scaling matrix is ​​updated based on the scaling update information to obtain the candidate scaling matrix of the Gaussian sphere at the target time; Determine the volume conservation constraint coefficients; Based on the volume conservation constraint coefficients and candidate scaling matrices, determine the target scaling matrix of the Gaussian sphere at the target time. Based on the target rotation matrix and the target scaling matrix, the target shape indication information of the Gaussian sphere at the target time is determined.

[0090] The volume conservation constraint coefficient is used to ensure that the change in the "volume" of the Gaussian sphere before and after scaling does not exceed a certain proportion. This coefficient can be a fixed threshold or a dynamically determined value. When the volume conservation constraint coefficient is dynamically determined, different Gaussian spheres can have different volume conservation constraint coefficients.

[0091] Optionally, when obtaining the target scaling matrix through the volume conservation constraint coefficient, the rotation update information can be obtained by fusing the quaternion of the first relative rotation information and the skin weight of the Gaussian sphere, or it can be obtained through polar decomposition. This embodiment does not limit this.

[0092] Because the scaling matrix may undergo uncontrolled volume expansion or compression after transformation, visual stretching or volume collapse artifacts may appear in the joint area, leading to shape distortion of the Gaussian sphere under large deformation. Therefore, in this embodiment, the initial shape indication information includes an initial scaling matrix and an initial rotation matrix. The initial rotation matrix is ​​updated based on the rotation update information to obtain the target rotation matrix of the Gaussian sphere at the target time. The initial scaling matrix is ​​updated based on the scaling update information to obtain the candidate scaling matrix of the Gaussian sphere at the target time. A volume conservation constraint coefficient is determined. Based on the volume conservation constraint coefficient and the candidate scaling matrix, the target scaling matrix of the Gaussian sphere at the target time is determined. Based on the target rotation matrix and the target scaling matrix, the target shape indication information of the Gaussian sphere at the target time is determined, ensuring that the change in the "volume" of the Gaussian sphere before and after scaling does not exceed a certain proportion. This ensures that even with large deformation, the shape of the Gaussian sphere will not be distorted. Furthermore, the volume conservation constraint coefficient in this embodiment is a purely runtime, training-independent constraint mechanism, which can improve the efficiency of determining the target scaling matrix.

[0093] For example, action games demand extremely high visual quality from high-fidelity fighting characters. Close-up shots (execution animations, victory poses) reveal muscle details, clothing edges, and flowing hair that directly impact the player experience. This application's volume conservation constraint ensures that the Gaussian spheres in the joint areas do not exhibit visual tearing during extreme actions (severe arm twisting, deep waist bending), providing quality assurance for high-intensity action scenes.

[0094] In some embodiments, determining the volume conservation constraint coefficient includes: Determine the distance between the initial position point and the specified bone, wherein the distance between the specified bone and the initial position point satisfies the first distance condition; The constraint strength coefficient is determined based on the distance and preset coefficients; Based on the constraint strength coefficient, the volume conservation constraint coefficient is determined.

[0095] The designated bone can be the joint center of the bone closest to the initial position point. The preset coefficient can be a preset attenuation coefficient. The distance and volume conservation constraint coefficient are directly proportional; the greater the distance, the larger the volume conservation constraint coefficient, and vice versa.

[0096] Optionally, the constraint strength coefficient can be determined based on the distance and a preset attenuation coefficient, and then the volume conservation constraint coefficient can be determined based on the constraint strength coefficient and a fixed threshold. The larger the distance, the smaller the constraint strength coefficient; conversely, the smaller the distance, the larger the constraint strength coefficient. Optionally, the distance can be substituted into formula (11) for calculation to obtain the constraint strength coefficient: (11) in,a i The constraint strength coefficient corresponding to the i-th Gaussian sphere is represented by exp, where exp represents the exponential operation. Indicates the preset attenuation coefficient. d i Indicates the first i The distance between the initial position of a Gaussian sphere and the specified bone.

[0097] After obtaining the constraint strength coefficient, the fixed threshold is divided by the constraint strength coefficient to obtain the volume conservation constraint coefficient corresponding to the Gaussian sphere. Specifically, the constraint strength coefficient can be substituted into formula (12) for calculation to obtain the volume conservation constraint coefficient corresponding to the Gaussian sphere: (12) in, This represents the volume conservation constraint coefficient corresponding to the i-th Gaussian sphere. Indicates a fixed threshold. a i This represents the constraint strength coefficient corresponding to the i-th Gaussian sphere.

[0098] Because Gaussian spheres at joints are prone to extreme deformation, while those far from joints do not undergo such drastic deformation, the ellipsoidal constraints on Gaussian spheres at joints are more stringent, while those far from joints are less restrictive. Therefore, in this embodiment, the distance between the initial position point and the designated bone is determined, wherein the distance between the designated bone and the initial position point satisfies the first distance condition. Based on the distance and a preset coefficient, a constraint strength coefficient is determined. Based on the constraint strength coefficient, a volume conservation constraint coefficient is determined, such that the volume conservation constraint coefficient corresponding to the Gaussian sphere closer to the joint is smaller, thereby making the volume constraint on the Gaussian sphere closer to the joint more stringent, and the volume conservation constraint coefficient corresponding to the Gaussian sphere farther from the joint is larger, thereby making the volume constraint on the Gaussian sphere farther from the joint more relaxed. This balances physical rationality and animation expressiveness, ensuring that when virtual objects perform large-scale movements (such as fighting rolls, extreme stretches, and jumping landings), the Gaussian spheres at joints do not exhibit visual distortions such as being "stretched into thin strips" or "compressed into thin sheets," thus guaranteeing the appearance fidelity of the Gaussian spheres and improving the rendering quality of high-dynamic action scenes.

[0099] In some embodiments, the target scaling matrix of the Gaussian sphere at the target time is determined based on the volume conservation constraint coefficients and the candidate scaling matrix, including: Based on the candidate scaling matrix and the initial scaling matrix, the difference information is determined; If the difference information is greater than the volume conservation constraint coefficient, the candidate scaling matrix is ​​compressed based on the initial scaling matrix to obtain the target scaling matrix of the Gaussian sphere at the target time. If the difference information is less than or equal to the volume conservation constraint coefficient, then the candidate scaling matrix is ​​determined as the target scaling matrix of the Gaussian sphere at the target time.

[0100] Specifically, determinant operations can be performed on the candidate scaling matrix and the initial scaling matrix respectively to obtain the first determinant result and the second determinant result. The difference information can be obtained by subtracting the second determinant result from the first determinant result, or by dividing the first determinant result by the second determinant result. When the difference information is obtained by subtracting the second determinant result from the first determinant result, the candidate scaling matrix and the initial scaling matrix can be substituted into formula (13) for calculation to obtain the difference information: (13) Among them, z i To represent the difference information, `det()` represents the determinant operation. This represents the result of the first determinant. Let represent the candidate scaling matrix of the i-th Gaussian sphere at the target time. This represents the result of the second determinant. Represents the initial scaling matrix. It represents the absolute value.

[0101] It is understandable that when the difference in information is greater than the volume conservation constraint coefficient, that is... When the difference is less than or equal to the volume conservation constraint coefficient, it indicates that the Gaussian sphere has changed too much before and after scaling and needs to be compressed to bring the change back to the allowable range. Therefore, the candidate scaling matrix is ​​compressed based on the initial scaling matrix. When the difference information is less than or equal to the volume conservation constraint coefficient, it indicates that the Gaussian sphere has changed little before and after scaling and does not need to be compressed. Therefore, the candidate scaling matrix is ​​used as the target scaling matrix.

[0102] Optionally, a compression ratio can be determined based on the initial scaling matrix and the candidate scaling matrix, and then the compression ratio is multiplied by the candidate scaling matrix to obtain the target scaling matrix. Optionally, a first ratio can be determined based on the first determinant result and the second determinant result, and then a certain proportion of the first ratio is taken as the compression ratio. For example, when one-third of the first ratio is taken as the compression ratio, the initial scaling matrix and the candidate scaling matrix can be substituted into formula (14) for compression processing to obtain the target scaling matrix. (14) in, Let represent the target scaling matrix of the i-th Gaussian sphere at the target time.

[0103] In this embodiment, difference information is determined based on the candidate scaling matrix and the initial scaling matrix. If the difference information is greater than the volume conservation constraint coefficient, the candidate scaling matrix is ​​compressed based on the initial scaling matrix to obtain the target scaling matrix of the Gaussian sphere at the target time. If the difference information is less than or equal to the volume conservation constraint coefficient, the candidate scaling matrix is ​​determined as the target scaling matrix of the Gaussian sphere at the target time, so that compression is performed when the change of the Gaussian sphere before and after scaling is large, thus ensuring the rationality of the appearance of the Gaussian sphere.

[0104] In some embodiments, the initial position point and initial shape indication information are updated based on the skin weights of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, to obtain the target position and target shape indication information of the Gaussian sphere at the target time, including: By using the computational shader in the graphics processor, the initial position point and initial shape indication information of the Gaussian sphere are updated based on the skin weights of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, thereby obtaining the target position and target shape indication information of the Gaussian sphere at the target time.

[0105] Specifically, the target transformation matrix can be stored in the structured buffer of the graphics processor so that the compute shader in the graphics processor can update the initial position point and initial shape indication information based on the skin weight of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, thereby obtaining the target position and target shape indication information of the Gaussian sphere at the target time.

[0106] Optionally, a Gaussian sphere can be processed by a single thread. That is, a single thread updates the initial position point and initial shape indication information based on the skin weights of a Gaussian sphere and the target transformation matrix of the skeleton at the target time, thereby obtaining the target position and target shape indication information of a Gaussian sphere at the target time. The number of thread groups is equal to the total number of Gaussian spheres divided by the thread group size.

[0107] Optionally, the specific implementation process of updating the initial position point and initial shape indication information based on the skin weight of the Gaussian sphere and the target transformation matrix of the skeleton at the target time to obtain the target position and target shape indication information of the Gaussian sphere at the target time can be referred to the specific implementation process in other embodiments, and will not be repeated here.

[0108] In 3D Gaussian splashing technology, the reliance on MLP networks for online inference of skeletal deformation introduces uncontrollable GPU computational overhead within the game engine and completely disconnects it from the engine's animation state machine (AnimGraph) workflow. In this embodiment, the initial position points and initial shape indication information are updated using the computational shader in the graphics processor based on the skinning weights of the Gaussian sphere and the target transformation matrix of the skeleton at the target time. This yields the target position and target shape indication information of the Gaussian sphere at the target time, establishing an interface between the game engine and Gaussian rendering. This enables direct connection between the game engine and the 3D Gaussian point cloud, allowing virtual objects to be driven in real-time by the game engine's animation state machine, just like traditional mesh objects (the game engine's animation system calculates the target transformation matrix for each target time and transmits it to the GPU memory). This eliminates the need for online neural network inference, achieving a better rendered appearance while maintaining the existing production workflow. Furthermore, the computation shader is implemented using pure linear algebra operations, eliminating the need for online inference using MLP neural networks. This demonstrates that runtime computation time increases linearly with the number of bones, rather than nonlinearly with network complexity. Computational overhead is predictable and budgetable, meeting the game engine's requirement for deterministic frame budget.

[0109] Step 206: Render the object based on the target position and shape indication information of the Gaussian sphere at the target time to obtain the virtual object displayed at the target time.

[0110] The Gaussian sphere, which includes target position and target shape indication information, can be called the transformed Gaussian sphere. The transformed Gaussian sphere is input into the 3D Gaussian block rasterization pipeline. After being sorted by depth by the 3D Gaussian block rasterization pipeline, it is alpha blended and rendered to output the image frame displayed at the target time. The image frame displayed at the target time includes virtual objects.

[0111] Understandably, this embodiment can be divided into an offline preparation phase and a real-time execution phase. In the offline preparation phase (which is done only once): the skeletal rigging file delivered by the art team is read, multiple Gaussian spheres are generated based on the 3D mesh, and the skinning weights of the vertices are propagated to the Gaussian spheres through weight transfer processing. In the real-time execution phase (executed every frame): the game engine's animation system calculates the target transformation matrix every frame, and drives the update of the position, rotation, and scaling of all Gaussian spheres through the graphics processor's computational shaders.

[0112] For example, the system architecture diagram of this application can be as follows: Figure 3As shown, the system architecture of this application includes a parsing module, a weight propagation engine, an animation system of a game engine, a computational shader of a graphics processor, and a Gaussian rasterization module. The parsing module inputs a skeleton binding file, which generates multiple Gaussian spheres based on the 3D mesh in the skeleton binding file. The weight propagation engine then performs weight transfer based on the skinning weights of the initial position point's neighboring points in the 3D mesh, obtaining the skinning weights of the Gaussian spheres. The game engine's animation system obtains the target transformation matrix of the skeleton at the target time. The graphics processor's computational shader updates the initial position point and initial shape indication information based on the Gaussian sphere's skinning weights and the target transformation matrix of the skeleton at the target time, obtaining the target position and target shape indication information of the Gaussian sphere at the target time. The Gaussian rasterization module performs rendering based on the target position and target shape indication information of the Gaussian sphere at the target time, obtaining a rendering frame. The rendering frame includes a virtual object displayed at the target time.

[0113] Specifically, the process of obtaining the skin weights of the Gaussian sphere by transferring skin weights based on the skin weights of the initial position point's neighboring points in the 3D mesh through a weight propagation engine can be described as follows: Figure 4 As shown, it is determined whether the initial position point of the Gaussian sphere is located on the surface region of the 3D mesh. If the initial position point is located on the surface region of the 3D mesh, the relative position between the initial position point and the target vertex is determined. Based on the relative position, the skin weight of the target vertex is interpolated to obtain the skin weight of the Gaussian sphere. If the initial position point is not located on the surface region of the 3D mesh, the skin weight of the Gaussian sphere is obtained through diffusion processing. In the diffusion processing, the known skin weight of the Gaussian sphere is used as the boundary condition, and the Euclidean distance and the geometry of the Gaussian sphere are considered, that is, the geometry of the Gaussian sphere is considered through Mahalanobis distance.

[0114] The process involves using the graphics processor's computational shader to update the initial position points and initial shape indication information based on the Gaussian sphere's skin weights and the bone's target transformation matrix at the target time. This yields the target position and shape indication information of the Gaussian sphere at the target time. Then, through the Gaussian rasterization module, rendering is performed based on the target position and shape indication information of the Gaussian sphere at the target time, resulting in a rendered frame. The process can be described as follows: Figure 5As shown, the input data for the computational shader of the graphics processor are the Gaussian sphere and the target transformation matrix output by the animation system of the game engine. The target transformation matrix is ​​stored in the buffer of the graphics processor. Based on the target transformation matrix of the bone at the target time and the initial transformation matrix of the bone, the computational shader of the graphics processor determines the relative transformation matrix of the bone. The relative transformation matrix of the bone and the skinning weight of the Gaussian sphere are fused to obtain the hybrid transformation matrix. Extreme decomposition is performed on the hybrid transformation matrix to obtain the second relative rotation information and the first scaling component of the bone. Based on the quaternion of the second relative rotation information of the bone affecting the Gaussian sphere, the initial rotation matrix is ​​updated to obtain the target rotation matrix of the Gaussian sphere at the target time. Based on the first scaling component, the initial scaling matrix is ​​updated to obtain the candidate scaling matrix of the Gaussian sphere at the target time. The volume conservation constraint coefficient is determined. Based on the volume conservation constraint coefficient and the candidate scaling matrix, the target scaling matrix of the Gaussian sphere at the target time is determined. Through the Gaussian rasterization module, the rendering process is performed based on the target position, target scaling matrix and target rotation matrix of the Gaussian sphere to obtain the rendering frame.

[0115] In 3D Gaussian splashing, human skeletons such as SMPL or FLAME are relied upon. These skeletons have fixed joint topologies (e.g., SMPL's 24 joints). However, many non-human characters in games, such as quadrupedal mounts, pterosaurs, mechs, and multi-legged monsters, use custom skeletal topologies created by the art team. These topologies differ fundamentally from SMPL / FLAME and cannot be directly applied to 3D Gaussian splashing. A complete redesign of the rigging scheme is required, limiting its use to a very narrow scenario with only standard human figures available in real-world video. Furthermore, existing 3D Gaussian splashing techniques heavily rely on video reconstruction and cannot be integrated into game art asset pipelines. Current techniques require starting from real-shot video sequences and going through a series of complex processes, including SMPL pose estimation, background separation, and multi-view reconstruction, to generate a drivable Gaussian character. Game industry character assets are typically modeled from scratch by artists and delivered as skeletal rigging files, without corresponding real-world video. This means that existing 3D Gaussian splashing techniques lack a usable input source in game development scenarios.

[0116] In this embodiment, starting from the skeletal rigging file, the skinning weights are directly propagated from the vertices of the 3D mesh to the Gaussian sphere using a weight transfer processing method. The entire process requires no learning or reconstruction steps. For game projects, all custom skeleton characters, including quadrupeds, multi-winged flying units, mechanical limbs, and alien creatures, can directly use the technical solution of this application. This extends the 3D Gaussian splashing technology to any virtual object of any shape with a skeletal rigging file, covering all types of virtual objects in the game industry. This significantly expands the applicable character coverage of Gaussian rendering technology, breaks through the limitations of skeletal universality, and improves its applicability. Furthermore, this application directly uses the skeletal rigging file already delivered by the art team as input, making it suitable for characters modeled purely by art. It skips all reconstruction processes and directly interfaces with the standard art production workflow in the game industry, effectively shortening the preparation cycle from asset creation to a renderable state and improving rendering efficiency.

[0117] The object rendering method provided in this application is illustrated below. In this embodiment, the virtual object is a mythical beast mount ridden by a player character in an open-world role-playing game. The types of mythical beast mounts can include quadrupedal beasts, pterosaurs, and mechanical beasts. Each mythical beast mount is modeled from scratch by artists, using a completely custom skeletal structure (the number of bones varies from 40 to 120, and the topology differs significantly from the human skeleton). Input conditions: Taking the pterosaur mount as an example, the art team delivered a skeletal binding file in FBX format, containing a 3D mesh of approximately 80,000 triangular faces, 89 custom bones (including the main spine chain, wing bone chains, limb bone chains, cervical spine chain, and tail bone chain), and the corresponding vertex skinning weights. The animation team has completed the editing of the animation state machine for approximately 200 actions. The game development team hopes to upgrade the appearance of these mythical beast mounts to Gaussian rendering to improve the visual quality of close-up shots (player's riding perspective) while maintaining the existing animation system.

[0118] The FBX format skeletal rigging file of the pterosaur mount was analyzed, and the hierarchical tree of 89 bones and their initial transformation matrices under the rigging posture were read. The coordinates of the vertices and skinning weights of approximately 80,000 triangular facets in the 3D mesh were also read (each vertex affects the sparse weights of a maximum of 4 bones). Initial position points were sampled on the surface region of the 3D mesh using an adaptive density strategy. High-density sampling was used in the wing membrane region (large facets requiring detailed lighting and shadow representation), and additional density was applied at joints in bone detail regions (scale textures, etc.), generating a total of approximately 400,000 initial position points. Each initial position point was initialized as an isotropic sphere, resulting in a Gaussian sphere with an opacity parameter set to 1.

[0119] For approximately 380,000 Gaussian spheres located on a triangular facet, their relative positions to the three vertices of their respective facets are calculated. Based on these relative positions and the coordinates of the three vertices, interpolation is performed to obtain the skinning weights for the 89 bones corresponding to each Gaussian sphere. These skinning weights are naturally normalized and require no post-processing. For approximately 20,000 Gaussian spheres within the interior space of the 3D mesh (used to fill the volume of the pterosaur's neck and torso), the skinning weights of the Gaussian spheres in the surface region are used as boundary conditions. The skinning weights are propagated iteratively through diffusion, taking into account both Euclidean distance and the geometry of the Gaussian spheres. The weight propagation calculation for all Gaussian spheres is completed in a single offline operation, and the results are saved to the GPU memory.

[0120] In the rendering frame at the target moment, i.e., in the current frame, the pterosaur mount performs a "spreading wings and flying" action. The animation state machine outputs a 4×4 target transformation matrix for 89 bones and writes it to the GPU buffer. The computation shader processes all approximately 400,000 Gaussian points in parallel using 6250 threads (64 threads per group): each thread reads the skin weights of the Gaussian sphere and the target transformation matrices of the four main influencing bones. Based on the skin weights of the Gaussian sphere and the target transformation matrices of the bones at the target moment, it updates the initial position points and initial shape indication information, obtaining the target position and target shape indication information of the Gaussian sphere at the target moment. Among these, the Gaussian sphere at the root joint of the wing (where the deformation is most severe at the connection between the wing membrane and the trunk) has a higher constraint strength during polar decomposition, effectively preventing stretching and tearing artifacts that occur in the root region of the wing membrane during large wing-spreading movements.

[0121] After transformation, the 400,000 Gaussian spheres are sorted according to their Z-values ​​in view space. Alpha blending rendering is then performed using a 3D Gaussian block rasterization model to output the pterosaur image for the current frame. The runtime process is completed within a single frame loop, meeting the game engine's frame budget requirements.

[0122] As can be seen from the above, in this embodiment, a skeleton binding file for a virtual object is obtained. The skeleton binding file includes a 3D mesh of the virtual object, bones bound to the 3D mesh, and skinning weights of vertices on the 3D mesh. Generation processing is performed based on the 3D mesh to obtain multiple Gaussian spheres for generating the virtual object. Each Gaussian sphere includes initial position points and initial shape indication information. Weight transfer processing is performed based on the skinning weights of the initial position points' neighboring points in the 3D mesh to obtain the skinning weights of the Gaussian spheres. The target transformation matrix of the bones at the target time is obtained. The initial position points and initial shape indication information are updated based on the skinning weights of the Gaussian spheres and the target transformation matrix of the bones at the target time to obtain the target position and target shape indication information of the Gaussian spheres at the target time. Rendering processing is performed based on the target position and target shape indication information of the Gaussian spheres at the target time to obtain the virtual object displayed at the target time. This achieves the combined use of the skeleton binding file and weight transfer processing to obtain the skinning weights of the Gaussian spheres, eliminating the need to rely on SMPL and FLAME to obtain the skinning weights. This allows the technology to be applied to virtual objects with any skeleton and eliminates the need for 3D reconstruction, improving the applicability and rendering efficiency of the 3D Gaussian splashing technology.

[0123] The following provides a further description of another object rendering method provided in this application. Detailed descriptions are provided below with reference to the accompanying drawings. In this embodiment, the execution subject is a terminal device. It should be noted that the order of description in the following embodiments is not intended to limit the preferred order of embodiments. Although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than that shown in the accompanying drawings.

[0124] Please refer to Figure 6 The specific process of this object rendering method can be summarized in steps 601 to 606, where: Step 601: Obtain the Gaussian sphere file corresponding to the virtual object. The Gaussian sphere file includes the initial position points, initial shape indication information and skinning weights of the Gaussian sphere used to generate the virtual object. The initial shape indication information includes the initial rotation matrix and the initial scaling matrix.

[0125] In some embodiments, obtaining the Gaussian sphere file corresponding to the virtual object includes: Obtain the skeleton binding file of the virtual object. The skeleton binding file includes the 3D mesh of the virtual object, the bones bound to the 3D mesh, and the skinning weights of the vertices on the 3D mesh. Based on the 3D mesh, multiple Gaussian spheres are generated to generate virtual objects. Each Gaussian sphere includes initial position points and initial shape indication information. The skin weights of the Gaussian sphere are obtained by weight transfer processing based on the skin weights of the neighborhood points of the initial position point in the 3D mesh. Based on the skin weights, initial position points, and initial shape indication information of the Gaussian sphere, a Gaussian sphere file corresponding to the virtual object is generated.

[0126] The specific implementation of generating the Gaussian sphere file in this embodiment can be referred to the above-described object rendering method embodiment, and will not be repeated here.

[0127] It is understood that the method of generating the Gaussian sphere file corresponding to the virtual object is not limited to the specific form disclosed in the embodiments of this application. Existing disclosed implementations in the field, as well as new alternatives that have emerged with the development of technology, can all be adapted and used in conjunction with the technical solution of this application.

[0128] Step 602: Obtain the target transformation matrix of the virtual object's skeleton at the target time.

[0129] Step 603: Based on the target transformation matrix of the skeleton at the target time and the skin weight of the Gaussian sphere, determine the rotation update information and scaling update information of the Gaussian sphere at the target time, as well as update the initial position point, to obtain the target position of the Gaussian sphere at the target time.

[0130] The rotation update information describes the rotational changes from the bound attitude to the target time. The scaling update information describes the scaling changes from the bound attitude to the target time.

[0131] Optionally, the process of updating the initial position point based on the target transformation matrix of the skeleton at the target time and the skin weight of the Gaussian sphere to obtain the target position of the Gaussian sphere at the target time can be as follows: determine the relative transformation matrix of the skeleton based on the target transformation matrix of the skeleton at the target time and the initial transformation matrix of the skeleton; fuse the skin weight of the Gaussian sphere, the relative transformation matrix of the skeleton affecting the Gaussian sphere, and the initial position point to obtain the target position of the Gaussian sphere at the target time.

[0132] In some embodiments, the rotation update information and scaling update information of the Gaussian sphere at the target time are determined based on the target transformation matrix of the skeleton at the target time and the skin weights of the Gaussian sphere. This includes: determining the relative transformation matrix of the skeleton based on the target transformation matrix of the skeleton at the target time and the initial transformation matrix of the skeleton; fusing the relative transformation matrix of the skeleton and the skin weights of the Gaussian sphere to obtain a hybrid transformation matrix; and performing polar decomposition on the hybrid transformation matrix to obtain the rotation update information and scaling update information at the target time.

[0133] In some embodiments, the rotation update information of the Gaussian sphere at the target time is determined based on the target transformation matrix of the skeleton at the target time and the skin weights of the Gaussian sphere, including: determining the relative transformation matrix of the skeleton based on the target transformation matrix of the skeleton at the target time and the initial transformation matrix of the skeleton; determining the first relative rotation information of the skeleton based on the relative transformation matrix of the skeleton; and performing fusion processing based on the quaternion of the first relative rotation information of the skeleton affecting the Gaussian sphere and the skin weights of the Gaussian sphere to obtain the rotation update information at the target time.

[0134] Step 604: Update the initial rotation matrix based on the rotation update information to obtain the target rotation matrix, and update the initial scaling matrix based on the scaling update information to obtain the candidate scaling matrix of the Gaussian sphere at the target time.

[0135] The rotation update information can be multiplied by the initial rotation matrix to obtain the target rotation matrix, and the scaling update information can be multiplied by the initial scaling matrix to obtain the candidate scaling matrix.

[0136] In some embodiments, updating the initial rotation matrix based on the rotation update information to obtain the target rotation matrix includes: determining the maximum eigenvalue corresponding to the rotation update information; and updating the initial rotation matrix based on the maximum eigenvalue to obtain the target rotation matrix of the Gaussian sphere at the target time.

[0137] The target rotation matrix can be obtained by multiplying the largest eigenvalue and the initial rotation matrix.

[0138] Step 605: Determine the volume conservation constraint coefficients, and based on the volume conservation constraint coefficients and candidate scaling matrices, determine the target scaling matrix of the Gaussian sphere at the target time.

[0139] The volume conservation constraint coefficient can be a pre-set fixed coefficient, or it can be a dynamically determined value. When it is a dynamically determined value, determining the volume conservation constraint coefficient includes: determining the distance between the initial position point and the specified bone, wherein the distance between the specified bone and the initial position point satisfies the first distance condition; determining the constraint strength coefficient based on the distance and the preset coefficient; and determining the volume conservation constraint coefficient based on the constraint strength coefficient.

[0140] In some embodiments, determining the target scaling matrix of the Gaussian sphere at the target time based on the volume conservation constraint coefficient and the candidate scaling matrix includes: determining the difference information based on the candidate scaling matrix and the initial scaling matrix; if the difference information is greater than the volume conservation constraint coefficient, then compressing the candidate scaling matrix based on the initial scaling matrix to obtain the target scaling matrix of the Gaussian sphere at the target time; if the difference information is less than or equal to the volume conservation constraint coefficient, then determining the candidate scaling matrix as the target scaling matrix of the Gaussian sphere at the target time.

[0141] Step 606: Based on the target rotation matrix and the target scaling matrix, generate the target shape indication information of the Gaussian sphere at the target time, and perform rendering processing based on the target position and target shape indication information of the Gaussian sphere at the target time to obtain the virtual object displayed at the target time.

[0142] In related technologies, due to the possibility of uncontrolled volume expansion or compression of the scaling matrix after transformation, visual stretching, tearing, or volume collapse artifacts may appear in the joint areas of the virtual object, resulting in distortion of the Gaussian sphere shape under large deformation. Therefore, in this embodiment, the Gaussian sphere file corresponding to the virtual object is obtained. The Gaussian sphere file includes the initial position points, initial shape indication information, and skinning weights used to generate the Gaussian sphere of the virtual object. The initial shape indication information includes the initial rotation matrix and the initial scaling matrix. The target transformation matrix of the skeleton of the virtual object at the target time is obtained. Based on the target transformation matrix of the skeleton at the target time and the skinning weights of the Gaussian sphere, the rotation update information and scaling update information of the Gaussian sphere at the target time are determined, and the initial position points are updated to obtain the target position of the Gaussian sphere at the target time. The initial rotation matrix is ​​updated based on the rotation update information to obtain the target rotation matrix, and the initial scaling matrix is ​​updated based on the scaling update information to obtain the candidate scaling matrix of the Gaussian sphere at the target time. The volume is then determined. The system uses volume conservation constraint coefficients and candidate scaling matrices to determine the target scaling matrix of the Gaussian sphere at the target time. Based on the target rotation matrix and target scaling matrix, it generates target shape indication information for the Gaussian sphere at the target time. Rendering is then performed based on the target position and shape indication information of the Gaussian sphere at the target time to obtain the virtual object displayed at the target time. This achieves the constraint of the scaling matrix change through volume conservation constraint coefficients, ensuring that the change in the "volume" of the Gaussian sphere before and after scaling does not exceed a certain proportion. This prevents the shape of the Gaussian sphere from becoming distorted even with significant deformation. Furthermore, the volume conservation constraint coefficients in this embodiment are a purely runtime, training-independent constraint mechanism, which can improve the efficiency of determining the target scaling matrix.

[0143] For example, action games demand extremely high visual quality from high-fidelity fighting characters. Close-up shots (execution animations, victory poses) reveal muscle details, clothing edges, and flowing hair that directly impact the player experience. This application's volume conservation constraint ensures that the Gaussian spheres in the joint areas do not exhibit visual tearing during extreme actions (severe arm twisting, deep waist bending), providing quality assurance for high-intensity action scenes.

[0144] For the explanation of terms, specific implementation methods, and corresponding beneficial effects of this embodiment, please refer to the above-described object rendering method embodiment. This embodiment will not repeat them here.

[0145] This embodiment also provides an object rendering system, such as... Figure 7 As shown, it may include: The binding module 701 is used to obtain the skeleton binding file of the virtual object. The skeleton binding file includes the 3D mesh of the virtual object, the bones bound to the 3D mesh, and the skinning weights of the vertices on the 3D mesh. Based on the 3D mesh, a generation process is performed to obtain multiple Gaussian spheres for generating the virtual object. The Gaussian spheres include initial position points and initial shape indication information. Based on the skinning weights of the neighboring points of the initial position points in the 3D mesh, a weight transfer process is performed to obtain the skinning weights of the Gaussian spheres.

[0146] The first skinning driving module is used to obtain the target transformation matrix of the skeleton at the target time; based on the skinning weight of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, the initial position point and initial shape indication information are updated to obtain the target position and target shape indication information of the Gaussian sphere at the target time.

[0147] The first rendering module is used to perform rendering processing based on the target position and target shape indication information of the Gaussian sphere at the target time, so as to obtain the virtual object displayed at the target time.

[0148] In practice, each of the above modules can be implemented as an independent entity or can be combined arbitrarily to be implemented as the same or several entities. For the specific implementation methods and corresponding beneficial effects of each of the above modules, please refer to the previous method embodiments, which will not be repeated here.

[0149] This embodiment also provides an object rendering system, such as... Figure 8 As shown, it may include: The acquisition module 801 is used to acquire the Gaussian sphere file corresponding to the virtual object. The Gaussian sphere file includes the initial position points, initial shape indication information and skinning weights of the Gaussian sphere used to generate the virtual object. The initial shape indication information includes the initial rotation matrix and the initial scaling matrix.

[0150] The second skinning driving module 802 is used to obtain the target transformation matrix of the skeleton of the virtual object at the target time; based on the target transformation matrix of the skeleton at the target time and the skinning weight of the Gaussian sphere, determine the rotation update information and scaling update information of the Gaussian sphere at the target time and update the initial position point to obtain the target position of the Gaussian sphere at the target time; update the initial rotation matrix based on the rotation update information to obtain the target rotation matrix, and update the initial scaling matrix based on the scaling update information to obtain the candidate scaling matrix of the Gaussian sphere at the target time; determine the volume conservation constraint coefficient, and based on the volume conservation constraint coefficient and the candidate scaling matrix, determine the target scaling matrix of the Gaussian sphere at the target time; and generate the target shape indication information of the Gaussian sphere at the target time based on the target rotation matrix and the target scaling matrix.

[0151] The second rendering module 803 is used to perform rendering processing based on the target position and target shape indication information of the Gaussian sphere at the target time, so as to obtain the virtual object displayed at the target time.

[0152] In practice, each of the above modules can be implemented as an independent entity or can be combined arbitrarily to be implemented as the same or several entities. For the specific implementation methods and corresponding beneficial effects of each of the above modules, please refer to the previous method embodiments, which will not be repeated here.

[0153] This embodiment also provides an object rendering apparatus, which can be integrated into a terminal device. For example, such as Figure 9 As shown, the object rendering apparatus may include: The file acquisition module 901 is used to acquire the skeleton binding file of the virtual object. The skeleton binding file includes the 3D mesh of the virtual object, the bones bound to the 3D mesh, and the skinning weights of the vertices on the 3D mesh.

[0154] The generation module 902 is used to perform generation processing based on a 3D mesh to obtain multiple Gaussian spheres for generating virtual objects. The Gaussian spheres include initial position points and initial shape indication information.

[0155] The transfer module 903 is used to perform weight transfer processing based on the skin weights of the initial position point's neighboring points in the 3D mesh to obtain the skin weights of the Gaussian sphere.

[0156] The matrix acquisition module 904 is used to acquire the target transformation matrix of the skeleton at the target time.

[0157] The update module 905 is used to update the initial position point and initial shape indication information based on the skin weight of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, so as to obtain the target position and target shape indication information of the Gaussian sphere at the target time.

[0158] The rendering module 906 is used to perform rendering processing based on the target position and target shape indication information of the Gaussian sphere at the target time, so as to obtain the virtual object displayed at the target time.

[0159] In some embodiments, the neighborhood points include target vertices of the surface region where the nearest neighbor points and / or the initial position point are located on the 3D mesh, and the nearest neighbor points are position points in the 3D mesh whose distance from the initial position point satisfies a preset distance condition; the transmission module 903 is specifically used to perform: The skin weights of the Gaussian sphere are obtained by interpolation based on the skin weights of the target vertices. And / or, perform diffusion processing based on the skin weights of nearest neighbor locations to obtain the skin weights of the Gaussian sphere.

[0160] In some embodiments, the delivery module 903 is specifically used to perform: When the initial position point is located in the surface region of the 3D mesh, interpolation is performed based on the skin weight of the target vertex to obtain the skin weight of the Gaussian sphere. And / or, if the initial position point is not located on the surface region of the 3D mesh, the skin weight of the Gaussian sphere is obtained by diffusion processing based on the skin weight of the nearest position point.

[0161] In some embodiments, the delivery module 903 is specifically used to perform: Determine the first distance between the initial position point and each of its nearest neighbor positions, as well as the total distance corresponding to the first distance; Based on the first distance and the total distance, determine the normalized distance of the initial location point to its nearest neighbor locations; The skin weights of the Gaussian sphere are obtained by fusing the skin weights of the nearest neighbor points based on the normalized distance.

[0162] In some embodiments, the first distance is a Mahalanobis distance, and the transmission module 903 is specifically used to perform: Based on the initial shape indication information included in the Gaussian sphere where the initial position point is located, the first distance between the initial position point and each of the nearest neighbor position points is determined.

[0163] In some embodiments, the delivery module 903 is specifically used to perform: Determine the relative position between the initial position point and the target vertex; Based on the relative position, the skin weights of the target vertex are interpolated to obtain the skin weights of the Gaussian sphere.

[0164] In some embodiments, the bone binding file also includes an initial transformation matrix for the bones; the update module 905 is specifically used to perform: Based on the target transformation matrix of the skeleton at the target time and the initial transformation matrix of the skeleton, the relative transformation matrix of the skeleton is determined. By fusing the skin weights of the Gaussian sphere, the relative transformation matrix affecting the skeleton of the Gaussian sphere, and the initial position point, the target position of the Gaussian sphere at the target time is obtained.

[0165] In some embodiments, the bone binding file also includes an initial transformation matrix for the bones; the update module 905 is specifically used to perform: Based on the target transformation matrix of the skeleton at the target time, the initial transformation matrix of the skeleton, and the skinning weights of the Gaussian sphere, the rotation update information and scaling update information at the target time are determined. Based on the rotation update information and scaling update information, the initial shape indication information is updated to obtain the target shape indication information of the Gaussian sphere at the target time.

[0166] In some embodiments, the update module 905 is specifically used to perform: Based on the target transformation matrix of the skeleton at the target time and the initial transformation matrix of the skeleton, the relative transformation matrix of the skeleton is determined. Based on the relative transformation matrix of the skeleton, the first relative rotation information of the skeleton is determined; The rotation update information at the target time is obtained by fusing the quaternion based on the first relative rotation information of the skeleton affecting the Gaussian sphere and the skin weight of the Gaussian sphere.

[0167] In some embodiments, the initial shape indication information includes an initial rotation matrix and an initial scaling matrix, and the update module 905 is specifically used to perform: Determine the maximum eigenvalue corresponding to the rotation update information; Based on the maximum eigenvalue, update the initial rotation matrix to obtain the target rotation matrix of the Gaussian sphere at the target time; The target scaling matrix is ​​obtained by updating the initial scaling matrix based on the scaling update information, and the target shape indication information of the Gaussian sphere at the target time is determined based on the target rotation matrix and the target scaling matrix.

[0168] In some embodiments, the update module 905 is specifically used to perform: Based on the target transformation matrix of the skeleton at the target time and the initial transformation matrix of the skeleton, the relative transformation matrix of the skeleton is determined. The relative transformation matrix of the skeleton and the skinning weights of the Gaussian sphere are fused to obtain the hybrid transformation matrix; Perform polar decomposition on the hybrid transformation matrix to obtain rotation update information and scaling update information at the target time.

[0169] In some embodiments, the initial shape indication information includes an initial scaling matrix and an initial rotation matrix, and the update module 905 is specifically used to perform: The initial rotation matrix is ​​updated based on the rotation update information to obtain the target rotation matrix of the Gaussian sphere at the target time. The initial scaling matrix is ​​updated based on the scaling update information to obtain the candidate scaling matrix of the Gaussian sphere at the target time. Determine the volume conservation constraint coefficients; Based on the volume conservation constraint coefficients and candidate scaling matrices, determine the target scaling matrix of the Gaussian sphere at the target time. Based on the target rotation matrix and the target scaling matrix, the target shape indication information of the Gaussian sphere at the target time is determined.

[0170] In some embodiments, the update module 905 is further configured to perform: Determine the distance between the initial position point and the specified bone, wherein the distance between the specified bone and the initial position point satisfies the first distance condition; The constraint strength coefficient is determined based on the distance and preset coefficients; Based on the constraint strength coefficient, the volume conservation constraint coefficient is determined.

[0171] In some embodiments, the update module 905 is specifically used to perform: Based on the candidate scaling matrix and the initial scaling matrix, the difference information is determined; If the difference information is greater than the volume conservation constraint coefficient, the candidate scaling matrix is ​​compressed based on the initial scaling matrix to obtain the target scaling matrix of the Gaussian sphere at the target time. If the difference information is less than or equal to the volume conservation constraint coefficient, then the candidate scaling matrix is ​​determined as the target scaling matrix of the Gaussian sphere at the target time.

[0172] In some embodiments, the update module 905 is specifically used to perform: By using the computational shader in the graphics processor, the initial position point and initial shape indication information of the Gaussian sphere are updated based on the skin weights of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, thereby obtaining the target position and target shape indication information of the Gaussian sphere at the target time.

[0173] In some embodiments, the generation module 902 is specifically used to perform: The surface region and internal space of the 3D mesh are sampled to obtain multiple initial position points; Determine the initial shape indication information corresponding to the initial position point; Based on the initial position point and initial shape indication information, multiple Gaussian spheres are obtained for generating virtual objects.

[0174] In some embodiments, the generation module 902 is specifically used to perform: Based on the semantic labels and / or geometric features of the surface regions in the 3D mesh, the target surface regions that need to undergo volume sampling are determined. Volume sampling is performed on the target surface area within a preset distance from the interior of the 3D mesh to obtain the initial position point.

[0175] In practice, each of the above modules can be implemented as an independent entity or can be combined arbitrarily to be implemented as the same or several entities. For the specific implementation methods and corresponding beneficial effects of each of the above modules, please refer to the previous method embodiments, which will not be repeated here.

[0176] This embodiment also provides an object rendering apparatus, which can be integrated into a terminal device. For example, such as Figure 10As shown, the object rendering apparatus may include: The first acquisition module 1001 is used to acquire the Gaussian sphere file corresponding to the virtual object. The Gaussian sphere file includes the initial position points, initial shape indication information and skinning weights of the Gaussian sphere used to generate the virtual object. The initial shape indication information includes the initial rotation matrix and the initial scaling matrix.

[0177] The second acquisition module 1002 is used to acquire the target transformation matrix of the skeleton of the virtual object at the target time.

[0178] The information update module 1003 is used to determine the rotation update information and scaling update information of the Gaussian sphere at the target time, as well as update the initial position point, based on the target transformation matrix of the skeleton at the target time and the skin weight of the Gaussian sphere, to obtain the target position of the Gaussian sphere at the target time; update the initial rotation matrix based on the rotation update information to obtain the target rotation matrix, and update the initial scaling matrix based on the scaling update information to obtain the candidate scaling matrix of the Gaussian sphere at the target time; determine the volume conservation constraint coefficient, and determine the target scaling matrix of the Gaussian sphere at the target time based on the volume conservation constraint coefficient and the candidate scaling matrix; and generate the target shape indication information of the Gaussian sphere at the target time based on the target rotation matrix and the target scaling matrix.

[0179] The object rendering module 1004 is used to perform rendering processing based on the target position and target shape indication information of the Gaussian sphere at the target time, so as to obtain the virtual object displayed at the target time.

[0180] In some embodiments, the information update module 1003 is specifically used to perform: Determine the distance between the initial position point and the specified bone, wherein the distance between the specified bone and the initial position point satisfies the first distance condition; The constraint strength coefficient is determined based on the distance and preset coefficients; Based on the constraint strength coefficient, the volume conservation constraint coefficient is determined.

[0181] In some embodiments, the information update module 1003 is specifically used to perform: Based on the candidate scaling matrix and the initial scaling matrix, the difference information is determined; If the difference information is greater than the volume conservation constraint coefficient, the candidate scaling matrix is ​​compressed based on the initial scaling matrix to obtain the target scaling matrix of the Gaussian sphere at the target time. If the difference information is less than or equal to the volume conservation constraint coefficient, then the candidate scaling matrix is ​​determined as the target scaling matrix of the Gaussian sphere at the target time.

[0182] In some embodiments, the information update module 1003 is specifically used to perform: Based on the target transformation matrix of the skeleton at the target time and the initial transformation matrix of the skeleton, the relative transformation matrix of the skeleton is determined. The relative transformation matrix of the skeleton and the skinning weights of the Gaussian sphere are fused to obtain the hybrid transformation matrix; Perform polar decomposition on the hybrid transformation matrix to obtain rotation update information and scaling update information at the target time.

[0183] In some embodiments, the information update module 1003 is specifically used to perform: Based on the target transformation matrix of the skeleton at the target time and the initial transformation matrix of the skeleton, the relative transformation matrix of the skeleton is determined. Based on the relative transformation matrix of the skeleton, the first relative rotation information of the skeleton is determined; The rotation update information at the target time is obtained by fusing the quaternion based on the first relative rotation information of the skeleton affecting the Gaussian sphere and the skin weight of the Gaussian sphere.

[0184] In some embodiments, the information update module 1003 is specifically used to perform: Determine the maximum eigenvalue corresponding to the rotation update information; Based on the largest eigenvalue, the initial rotation matrix is ​​updated to obtain the target rotation matrix of the Gaussian sphere at the target time.

[0185] In some embodiments, the first acquisition module 1001 is specifically used to perform: Obtain the skeleton binding file of the virtual object. The skeleton binding file includes the 3D mesh of the virtual object, the bones bound to the 3D mesh, and the skinning weights of the vertices on the 3D mesh. Based on the 3D mesh, multiple Gaussian spheres are generated to generate virtual objects. Each Gaussian sphere includes initial position points and initial shape indication information. The skin weights of the Gaussian sphere are obtained by weight transfer processing based on the skin weights of the neighborhood points of the initial position point in the 3D mesh. Based on the skin weights, initial position points, and initial shape indication information of the Gaussian sphere, a Gaussian sphere file corresponding to the virtual object is generated.

[0186] In practice, each of the above modules can be implemented as an independent entity or can be combined arbitrarily to be implemented as the same or several entities. For the specific implementation methods and corresponding beneficial effects of each of the above modules, please refer to the previous method embodiments, which will not be repeated here.

[0187] Accordingly, this application also provides an electronic device, which can be a terminal, such as a smartphone, tablet computer, laptop computer, touch screen, game console, personal computer (PC), personal digital assistant (PDA), or other terminal device. Alternatively, the electronic device can be a server.

[0188] like Figure 11 As shown, Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 1100 includes a processor 1101 with one or more processing cores, a memory 1102 with one or more computer-readable storage media, and a computer program stored on the memory 1102 and executable on the processor. The processor 1101 and the memory 1102 are electrically connected. Those skilled in the art will understand that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0189] The processor 1101 is the control center of the electronic device 1100. It connects various parts of the electronic device 1100 via various interfaces and lines. By running or loading software programs and / or units stored in the memory 1102, and by calling data stored in the memory 1102, it executes various functions of the electronic device 1100 and processes data, thereby providing overall monitoring of the electronic device 1100. The processor 1101 can be a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc., and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0190] In this embodiment, the processor 1101 in the electronic device 1100 loads the instructions corresponding to the processes of one or more applications into the memory 1102 according to the following steps, and the processor 1101 runs the applications stored in the memory 1102 to realize various functions, such as: Obtain the skeleton binding file of the virtual object. The skeleton binding file includes the 3D mesh of the virtual object, the bones bound to the 3D mesh, and the skinning weights of the vertices on the 3D mesh. Based on the 3D mesh, multiple Gaussian spheres are generated to generate virtual objects. Each Gaussian sphere includes initial position points and initial shape indication information. The skin weights of the Gaussian sphere are obtained by weight transfer processing based on the skin weights of the neighborhood points of the initial position point in the 3D mesh. Obtain the target transformation matrix of the skeleton at the target time; Based on the skin weights of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, the initial position point and initial shape indication information are updated to obtain the target position and target shape indication information of the Gaussian sphere at the target time. The virtual object displayed at the target time is obtained by rendering based on the target position and target shape indication information of the Gaussian sphere at the target time.

[0191] The specific implementation of each of the above operations and their corresponding beneficial effects can be found in the previous embodiments, and will not be repeated here.

[0192] Optional, such as Figure 11 As shown, the electronic device 1100 also includes: a touch display screen 1103, a radio frequency circuit 1104, an audio circuit 1105, an input unit 1106, and a power supply 1107. The processor 1101 is electrically connected to the touch display screen 1103, the radio frequency circuit 1104, the audio circuit 1105, the input unit 1106, and the power supply 1107. Those skilled in the art will understand that... Figure 11 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0193] The touch display screen 1103 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. The touch display screen 1103 may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Optionally, the display panel can be configured using a liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar technologies. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program according to the operation commands. Optionally, the touch panel may include a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1101. It can also receive and execute commands from the processor 1101. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor 1101 to determine the type of touch event. Subsequently, the processor 1101 provides corresponding visual output on the display panel based on the type of touch event. In this embodiment, the touch panel and the display panel can be integrated into the touch display screen 1103 to achieve input and output functions. However, in some embodiments, the touch panel and the touch display screen 1103 can be implemented as two independent components to achieve input and output functions. That is, the touch display screen 1103 can also be used as part of the input unit 1106 to achieve input functions.

[0194] The radio frequency circuit 1104 can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other electronic devices, and to transmit and receive signals with network devices or other electronic devices.

[0195] Audio circuit 1105 can be used to provide an audio interface between a user and an electronic device via a speaker and a microphone. Audio circuit 1105 can convert received audio data into electrical signals and transmit them to the speaker, where the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuit 1105, converted back into audio data, and then processed by processor 1101 before being transmitted via radio frequency circuit 1104 to, for example, another electronic device, or output to memory 1102 for further processing. Audio circuit 1105 may also include an earphone jack to provide communication between peripheral headphones and electronic devices.

[0196] The input unit 1106 can be used to receive input numbers, characters, or user characteristic information (such as fingerprints, iris, facial information, etc.), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0197] Power supply 1107 is used to supply power to various components of electronic device 1100. Optionally, power supply 1107 can be logically connected to processor 1101 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Power supply 1107 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0198] although Figure 11 As not shown in the diagram, the electronic device 1100 may also include a camera, sensor, wireless fidelity module, Bluetooth module, etc., which will not be described in detail here.

[0199] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0200] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0201] Therefore, embodiments of this application provide a computer-readable storage medium storing multiple computer programs that can be loaded by a processor to execute any of the object rendering methods provided in embodiments of this application. The computer program can execute the following steps of the object rendering method: Obtain the skeleton binding file of the virtual object. The skeleton binding file includes the 3D mesh of the virtual object, the bones bound to the 3D mesh, and the skinning weights of the vertices on the 3D mesh. Based on the 3D mesh, multiple Gaussian spheres are generated to generate virtual objects. Each Gaussian sphere includes initial position points and initial shape indication information. The skin weights of the Gaussian sphere are obtained by weight transfer processing based on the skin weights of the neighborhood points of the initial position point in the 3D mesh. Obtain the target transformation matrix of the skeleton at the target time; Based on the skin weights of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, the initial position point and initial shape indication information are updated to obtain the target position and target shape indication information of the Gaussian sphere at the target time. The virtual object displayed at the target time is obtained by rendering based on the target position and target shape indication information of the Gaussian sphere at the target time.

[0202] The specific implementation of each of the above operations and their corresponding beneficial effects can be found in the previous embodiments, and will not be repeated here.

[0203] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0204] Since the computer program stored in the computer-readable storage medium can execute any of the object rendering methods provided in the embodiments of this application, it can achieve the beneficial effects that any of the object rendering methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0205] According to one aspect of this application, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in the various optional implementations of the above embodiments.

[0206] It should be noted that, in the data processing stage, the technical solution of this application has strictly limited the scope of data collection to the minimum necessary to achieve the technical objectives, preventing the acquisition of irrelevant information. For any user information to be collected, the data subject will be clearly informed and their consent obtained. Furthermore, technologies such as encrypted storage and access control are employed to strengthen data security and ensure the security and compliance of the entire data processing process. The technical model and decision-making mechanism are based on objective technical parameters and do not introduce unnecessary parameters such as gender or age that may lead to discrimination, resolutely eliminating algorithmic discrimination and upholding public order and good morals. In addition, the specification fully describes the technical implementation methods, application scenarios, and compliance protection details. The claims are consistent with the content of the specification, key compliance designs are clear and verifiable, and the overall technical design is guided by the protection of public interests and adherence to social ethics, without any circumstances that harm public interests or violate public order and good morals.

[0207] In the above embodiments of the object rendering apparatus, computer-readable storage medium, electronic device, and computer program product, the descriptions of each embodiment have different focuses. Parts not described in detail in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes and beneficial effects of the object rendering apparatus, computer-readable storage medium, computer program product, electronic device, and their corresponding units described above can be referred to the description of the object rendering method in the above embodiments, and will not be repeated here.

[0208] The foregoing has provided a detailed description of an object rendering method, apparatus, electronic device, computer-readable storage medium, and computer program product provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An object rendering method, characterized in that, include: Obtain the skeleton binding file of the virtual object, the skeleton binding file including the 3D mesh of the virtual object, the bones bound to the 3D mesh, and the skinning weights of the vertices on the 3D mesh; Based on the three-dimensional mesh, a generation process is performed to obtain multiple Gaussian spheres for generating the virtual object. Each Gaussian sphere includes an initial position point and initial shape indication information. The skin weights of the Gaussian sphere are obtained by performing weight transfer processing based on the skin weights of the neighboring points of the initial position point in the three-dimensional mesh. Obtain the target transformation matrix of the skeleton at the target time; Based on the skin weights of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, the initial position point and the initial shape indication information are updated to obtain the target position and target shape indication information of the Gaussian sphere at the target time. The virtual object displayed at the target time is obtained by rendering based on the target position and target shape indication information of the Gaussian sphere at the target time.

2. The method according to claim 1, characterized in that, The neighborhood points include the nearest neighbor points and / or the target vertices of the surface region where the initial position point is located on the three-dimensional mesh. The nearest neighbor points are the points in the three-dimensional mesh whose distance from the initial position point satisfies a preset distance condition. The process of transferring skin weights based on the neighborhood points of the initial position point in the 3D mesh to obtain the skin weights of the Gaussian sphere includes: The skin weights of the Gaussian sphere are obtained by interpolation based on the skin weights of the target vertex. And / or, based on the skin weights of the nearest neighbor points, a diffusion process is performed to obtain the skin weights of the Gaussian sphere.

3. The method according to claim 2, characterized in that, The interpolation process based on the skinning weights of the target vertex to obtain the skinning weights of the Gaussian sphere includes: When the initial position point is located in the surface region of the three-dimensional mesh, interpolation is performed based on the skin weight of the target vertex to obtain the skin weight of the Gaussian sphere; And / or, the diffusion processing based on the skin weights of the nearest neighbor points to obtain the skin weights of the Gaussian sphere includes: If the initial position point is not located on the surface region of the three-dimensional mesh, the skin weight of the Gaussian sphere is obtained by diffusion processing based on the skin weight of the nearest position point.

4. The method according to claim 2, characterized in that, The diffusion process based on the skin weights of the nearest neighbor points to obtain the skin weights of the Gaussian sphere includes: Determine the first distance between the initial position point and each of its nearest neighbor positions, as well as the total distance corresponding to the first distance, wherein the first distance includes Mahalanobis distance; Based on the first distance and the total distance, determine the normalized distance of the initial position point relative to the nearest neighbor position point; The skin weights of the nearest neighbor points are fused based on the normalized distance to obtain the skin weights of the Gaussian sphere.

5. The method according to claim 2, characterized in that, The interpolation process based on the skinning weights of the target vertex to obtain the skinning weights of the Gaussian sphere includes: Determine the relative position between the initial position point and the target vertex; Based on the relative position, the skin weights of the target vertex are interpolated to obtain the skin weights of the Gaussian sphere.

6. The method according to claim 1, characterized in that, The skeleton binding file also includes the initial transformation matrix of the skeleton; the initial shape indication information is updated based on the skinning weights of the Gaussian sphere and the target transformation matrix of the skeleton at the target time to obtain the target shape indication information of the Gaussian sphere at the target time, including: Based on the target transformation matrix of the skeleton at the target time, the initial transformation matrix of the skeleton, and the skinning weights of the Gaussian sphere, the rotation update information and scaling update information at the target time are determined. Based on the rotation update information and the scaling update information, the initial shape indication information is updated to obtain the target shape indication information of the Gaussian sphere at the target time.

7. The method according to claim 6, characterized in that, Based on the target transformation matrix of the skeleton at the target time, the initial transformation matrix of the skeleton, and the skinning weights of the Gaussian sphere, the rotation update information at the target time is determined, including: Based on the target transformation matrix of the skeleton at the target time and the initial transformation matrix of the skeleton, the relative transformation matrix of the skeleton is determined; Based on the relative transformation matrix of the bone, the first relative rotation information of the bone is determined; The rotation update information at the target time is obtained by fusing the quaternion of the first relative rotation information affecting the skeleton of the Gaussian sphere with the skin weight of the Gaussian sphere.

8. The method according to claim 7, characterized in that, The initial shape indication information includes an initial rotation matrix and an initial scaling matrix. The step of updating the initial shape indication information based on the rotation update information and the scaling update information to obtain the target shape indication information of the Gaussian sphere at the target time includes: Determine the maximum eigenvalue corresponding to the rotation update information; Based on the maximum eigenvalue, the initial rotation matrix is ​​updated to obtain the target rotation matrix of the Gaussian sphere at the target time; The initial scaling matrix is ​​updated based on the scaling update information to obtain the target scaling matrix, and the target shape indication information of the Gaussian sphere at the target time is determined based on the target rotation matrix and the target scaling matrix.

9. The method according to claim 6, characterized in that, The initial shape indication information includes an initial scaling matrix and an initial rotation matrix. The step of updating the initial shape indication information based on the rotation update information and the scaling update information to obtain the target shape indication information of the Gaussian sphere at the target time includes: The initial rotation matrix is ​​updated based on the rotation update information to obtain the target rotation matrix of the Gaussian sphere at the target time; The initial scaling matrix is ​​updated based on the scaling update information to obtain the candidate scaling matrix of the Gaussian sphere at the target time; Determine the volume conservation constraint coefficients; Based on the volume conservation constraint coefficient and the candidate scaling matrix, the target scaling matrix of the Gaussian sphere at the target time is determined; Based on the target rotation matrix and the target scaling matrix, the target shape indication information of the Gaussian sphere at the target time is determined.

10. The method according to claim 9, characterized in that, The determination of the volume conservation constraint coefficient includes: Determine the distance between the initial position point and the specified bone, wherein the distance between the specified bone and the initial position point satisfies a first distance condition; Based on the distance and the preset coefficient, the constraint strength coefficient is determined; Based on the aforementioned constraint strength coefficient, the volume conservation constraint coefficient is determined.

11. The method according to claim 9, characterized in that, The step of determining the target scaling matrix of the Gaussian sphere at the target time based on the volume conservation constraint coefficient and the candidate scaling matrix includes: Based on the candidate scaling matrix and the initial scaling matrix, the difference information is determined; If the difference information is greater than the volume conservation constraint coefficient, then the candidate scaling matrix is ​​compressed based on the initial scaling matrix to obtain the target scaling matrix of the Gaussian sphere at the target time. If the difference information is less than or equal to the volume conservation constraint coefficient, then the candidate scaling matrix is ​​determined as the target scaling matrix of the Gaussian sphere at the target time.

12. The method according to claim 1, characterized in that, The process of updating the initial position point and the initial shape indication information based on the skin weights of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, to obtain the target position and target shape indication information of the Gaussian sphere at the target time, includes: By using the computational shader in the graphics processor, the initial position point and the initial shape indication information are updated based on the skinning weight of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, so as to obtain the target position and target shape indication information of the Gaussian sphere at the target time.

13. The method according to any one of claims 1-12, characterized in that, The generation process based on the three-dimensional mesh, resulting in multiple Gaussian spheres used to generate the virtual object, includes: The surface region and the internal space of the three-dimensional mesh are sampled to obtain multiple initial position points; Determine the initial shape indication information corresponding to the initial position point; Based on the initial position point and the initial shape indication information, multiple Gaussian spheres are obtained for generating the virtual object.

14. An object rendering method, characterized in that, include: Obtain the Gaussian sphere file corresponding to the virtual object. The Gaussian sphere file includes the initial position points, initial shape indication information, and skinning weights used to generate the Gaussian sphere of the virtual object. The initial shape indication information includes the initial rotation matrix and the initial scaling matrix. Obtain the target transformation matrix of the skeleton of the virtual object at the target time; Based on the target transformation matrix of the skeleton at the target time and the skinning weight of the Gaussian sphere, the rotation update information and scaling update information of the Gaussian sphere at the target time are determined, and the initial position point is updated to obtain the target position of the Gaussian sphere at the target time. The initial rotation matrix is ​​updated based on the rotation update information to obtain the target rotation matrix, and the initial scaling matrix is ​​updated based on the scaling update information to obtain the candidate scaling matrix of the Gaussian sphere at the target time. Determine the volume conservation constraint coefficients, and based on the volume conservation constraint coefficients and the candidate scaling matrix, determine the target scaling matrix of the Gaussian sphere at the target time; Based on the target rotation matrix and the target scaling matrix, target shape indication information of the Gaussian sphere at the target time is generated, and rendering processing is performed based on the target position of the Gaussian sphere at the target time and the target shape indication information to obtain the virtual object displayed at the target time.

15. The method according to claim 14, characterized in that, The determination of the volume conservation constraint coefficient includes: Determine the distance between the initial position point and the specified bone, wherein the distance between the specified bone and the initial position point satisfies a first distance condition; Based on the distance and the preset coefficient, the constraint strength coefficient is determined; Based on the aforementioned constraint strength coefficient, the volume conservation constraint coefficient is determined.

16. The method according to claim 14, characterized in that, The step of determining the target scaling matrix of the Gaussian sphere at the target time based on the volume conservation constraint coefficient and the candidate scaling matrix includes: Based on the candidate scaling matrix and the initial scaling matrix, the difference information is determined; If the difference information is greater than the volume conservation constraint coefficient, then the candidate scaling matrix is ​​compressed based on the initial scaling matrix to obtain the target scaling matrix of the Gaussian sphere at the target time. If the difference information is less than or equal to the volume conservation constraint coefficient, then the candidate scaling matrix is ​​determined as the target scaling matrix of the Gaussian sphere at the target time.

17. The method according to any one of claims 14-16, characterized in that, The step of obtaining the Gaussian sphere file corresponding to the virtual object includes: Obtain the skeleton binding file of the virtual object, the skeleton binding file including the 3D mesh of the virtual object, the bones bound to the 3D mesh, and the skinning weights of the vertices on the 3D mesh; Based on the three-dimensional mesh, a generation process is performed to obtain multiple Gaussian spheres for generating the virtual object. Each Gaussian sphere includes an initial position point and initial shape indication information. The skin weights of the Gaussian sphere are obtained by performing weight transfer processing based on the skin weights of the neighboring points of the initial position point in the three-dimensional mesh. Based on the skinning weights of the Gaussian sphere, the initial position point, and the initial shape indication information, a Gaussian sphere file corresponding to the virtual object is generated.

18. An object rendering apparatus, characterized in that, The device includes: The file acquisition module is used to acquire the skeleton binding file of the virtual object. The skeleton binding file includes the three-dimensional mesh of the virtual object, the bones bound to the three-dimensional mesh, and the skinning weights of the vertices on the three-dimensional mesh. A generation module is used to perform generation processing based on the three-dimensional mesh to obtain multiple Gaussian spheres for generating the virtual object. The Gaussian spheres include initial position points and initial shape indication information. The transfer module is used to perform weight transfer processing based on the skin weights of the neighboring points of the initial position point in the three-dimensional mesh to obtain the skin weights of the Gaussian sphere. A matrix acquisition module is used to acquire the target transformation matrix of the skeleton at the target time. The update module is used to update the initial position point and the initial shape indication information based on the skin weight of the Gaussian sphere and the target transformation matrix of the skeleton at the target time, so as to obtain the target position and target shape indication information of the Gaussian sphere at the target time. The rendering module is used to perform rendering processing based on the target position and target shape indication information of the Gaussian sphere at the target time, so as to obtain the virtual object displayed at the target time.

19. An electronic device, characterized in that, The device includes a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to perform the steps of the object rendering method as described in any one of claims 1 to 17.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the object rendering method as described in any one of claims 1 to 17.

Images