Rendering method and device of 3D digital human, electronic equipment and medium

Abstract

The invention provides a 3D digital human rendering method and device, electronic equipment, a medium and a program product, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models, augmented reality and the like, and can be applied to scenes such as digital humans and the like. According to the specific implementation scheme, in response to a Gaussian rendering command, dynamic texture data corresponding to a to-be-rendered target image is calculated and stored in a GPU; calling a target neural network model realized based on a GPU, calculating dynamic Gaussian attributes, and storing the dynamic Gaussian attributes in the GPU; and rendering and displaying the target image according to the dynamic Gaussian attribute. The dynamic Gaussian attribute matched with the dynamic texture data is calculated through the target neural network model, and more detailed feature information of the 3D digital human can be obtained. And texture calculation, dynamic Gaussian attribute calculation and image rendering are executed by the GPU, so that the execution efficiency of rendering is improved, detail description of each frame of the 3D digital human is realized, and the rendering effect is improved.

Description

Technical Field

[0001] It relates to the field of artificial intelligence technology, especially to the fields of computer vision, deep learning, large models, and augmented reality, etc., and can be applied to scenarios such as digital humans. Specifically, it relates to a rendering method for 3D digital humans, a rendering device for 3D digital humans, an electronic device, a non-transitory computer-readable storage medium, and a computer program product. Background Art

[0002] With the rapid development of artificial intelligence technology, digital humans, as a new type of human-computer interaction method, are receiving more and more attention. Especially for 3D (3Dimensional) digital humans, their applications have shown great potential in many fields. With the continuous improvement of users' requirements for the limb movement expression and rendering effect of dynamic digital humans, higher requirements are put forward for 3D digital human rendering technology. Summary of the Invention

[0003] The present disclosure provides a rendering method for 3D digital humans, a rendering device for 3D digital humans, an electronic device, a non-transitory computer-readable storage medium, and a computer program product.

[0004] According to one aspect of the present disclosure, there is provided a rendering method for 3D digital humans, which is executed by a hardware rendering component configured in a GPU (Graphics Processing Unit), and includes:

[0005] In response to a Gaussian rendering command sent by a software engine, calculate dynamic texture data corresponding to a target image to be rendered, and store the dynamic texture data in the GPU, where the target image includes a 3D digital human;

[0006] By calling a target neural network model implemented based on the GPU, calculate a dynamic Gaussian attribute that matches the dynamic texture data, and store the dynamic Gaussian attribute in the GPU;

[0007] Render the target image according to the dynamic Gaussian attribute calculated by the target neural network model, and display the target image.

[0008] According to another aspect of the present disclosure, there is also provided a rendering method for 3D digital humans, which is executed by a software engine configured in a CPU, and includes:

[0009] When the dynamic Gaussian rendering condition for the content to be displayed is met, construct a Gaussian rendering command corresponding to each frame of the image to be rendered;

[0010] where the image to be rendered includes a 3D digital human;

[0011] Send each of the Gaussian rendering commands to the hardware rendering component in the GPU in sequence, so that the hardware rendering component can generate dynamic Gaussian attributes corresponding to the dynamic texture data of each frame of the image to be rendered by calling the target neural network model implemented based on the GPU, and perform real-time rendering and display of the content to be displayed according to each of the dynamic Gaussian attributes.

[0012] According to another aspect of the present disclosure, there is also provided an electronic device, including at least one GPU; and a memory communicatively connected to the at least one GPU;

[0013] The GPU is configured to execute the 3D digital human rendering method performed by the hardware rendering component configured in the GPU as described in any one of the embodiments of the present disclosure.

[0014] According to another aspect of the present disclosure, there is also provided another electronic device, including at least one CPU; and a memory communicatively connected to the at least one CPU;

[0015] The CPU is configured to execute the 3D digital human rendering method performed by the software engine configured in the CPU as described in any one of the embodiments of the present disclosure.

[0016] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause an electronic device to execute the 3D digital human rendering method as described in any one of the embodiments of the present disclosure.

[0017] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0019] Figure 1 is a schematic diagram of a 3D digital human rendering method provided according to an embodiment of the present disclosure;

[0020] Figure 2 is a schematic diagram of another 3D digital human rendering method provided according to an embodiment of the present disclosure;

[0021] Figure 3 is a schematic diagram of yet another 3D digital human rendering method provided according to an embodiment of the present disclosure;

[0022] Figure 4 is a schematic diagram of still another 3D digital human rendering method provided according to an embodiment of the present disclosure;

[0023] Figure 5 is a schematic diagram of a 3D digital human rendering device provided according to an embodiment of the present disclosure;

[0024] Figure 6 is a schematic diagram of another 3D digital human rendering device provided according to an embodiment of the present disclosure;

[0025] Figure 7 is a block diagram of an electronic device for implementing the 3D digital human rendering method according to an embodiment of the present disclosure;

[0026] Figure 8 is a block diagram of another electronic device for implementing the 3D digital human rendering method according to an embodiment of the present disclosure. Detailed implementation manners

[0027] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to assist in understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following.

[0028] In the related art, the 3D digital human rendering method based on a fixed Gaussian attribute value needs to calculate a fixed Gaussian attribute value offline, and then dynamically render each frame of animation in combination with this value. It cannot effectively adapt to complex 3D digital human motion change scenarios, resulting in the rendering result being unable to accurately depict various dynamic deformations and character details of the 3D digital human. In addition, the neural network dynamic 3D Gaussian technology has low efficiency in processing complex graphics rendering tasks, cannot meet the requirements of real-time rendering, and reduces the visual authenticity of the 3D rendering result and the user's real-time interaction experience.

[0029] Figure 1 is a schematic diagram of a 3D digital human rendering method provided according to an embodiment of the present disclosure. The embodiments of the present disclosure are applicable to the situation of rendering 3D digital humans in a GPU. This method can be executed by a 3D digital human rendering device, which can be implemented in a hardware and / or software manner and is generally configured in a hardware rendering component in the GPU. Among them, the hardware rendering component is used to calculate and process a large amount of graphic data, execute complex graphics rendering tasks, and achieve smooth animation effects and high-resolution image output.

[0030] Correspondingly, as Figure 1 shown, the method may specifically include:

[0031] S110. In response to a Gaussian rendering command sent by a software engine, calculate dynamic texture data corresponding to a target image to be rendered, and store the dynamic texture data in the GPU.

[0032] Among them, the target image contains a 3D digital human.

[0033] Generally speaking, the software engine can be specifically understood as: a program or module running on a CPU (Central Processing Unit), such as a software thread at the CPU layer, which is used to prepare and organize the resources and instructions required for rendering and send them to the GPU. The target image can be specifically understood as: an image that needs to be rendered into an image, which contains 3D digital humans to be rendered, and these 3D digital humans can be composed of elements such as textures.

[0034] In image rendering, a texture is a data structure used to describe the surface details of an object. Dynamic texture data can be specifically understood as: texture data that changes according to rendering conditions (such as time, viewing angle, or actions, etc.) during the rendering process. For example, the dynamic texture data can include the skin details, action details, expression details, and clothing textures of the digital human, etc., and these textures need to be updated in real time according to the action process of the digital human and the viewing angle of the digital human. These data need to be calculated and prepared before rendering.

[0035] Generally speaking, the hardware rendering component can be specifically understood as: a cross-platform graphics application programming interface, such as OpenGL (Open Graphics Library). Correspondingly, developers can call the hardware rendering functions of the GPU through these hardware rendering components. The hardware rendering component instructions can include: calculation instructions and rendering instructions. Among them, the calculation instructions can be specifically understood as: instructions for performing general computing tasks on the GPU, which can be used to generate, transform, and update texture data. The rendering instructions can be specifically understood as: instructions specifically used for graphics rendering tasks executed on the GPU, which are used to apply the calculated texture data to the 3D digital human for final image rendering. For example, through the rendering instructions, the dynamic texture is mapped onto the surface of the 3D digital human to achieve rendering into an image. The Gaussian rendering command can be specifically understood as: a rendering command used to trigger the calculation of dynamic Gaussian attributes, including calculation instructions, instructions for writing and calling a target neural network model based on the GPU, and rendering instructions.

[0036] Specifically, in response to the calculation instructions in the Gaussian rendering command sent by the software engine, according to the content and rendering requirements of the target image, the dynamic texture data corresponding to the current frame of the target image is calculated, which can include operations such as texture generation, transformation, and update, and is stored in the memory of the GPU (such as video memory), ensuring that the GPU can directly access this data during rendering to improve rendering efficiency.

[0037] S120. By invoking the target neural network model implemented based on the GPU, calculate the dynamic Gaussian attributes that match the dynamic texture data, and store the dynamic Gaussian attributes in the GPU.

[0038] Generally speaking, the target neural network model can be specifically understood as: a neural network model used to process dynamic texture data and generate dynamic Gaussian attributes. This model has usually been pre-trained and deployed on the GPU, and can utilize the parallel computing power of the GPU for efficient calculation.

[0039] The dynamic Gaussian attributes can be specifically understood as: in the Gaussian splatter technology, a set of parameters (which can be represented as a multi-dimensional array) used to describe and control the dynamic changes of the Gaussian function, and can be used to describe: position, rotation, scale, color, and opacity, etc. The dynamic Gaussian attributes can be adjusted in real time according to the dynamic changes of the scene or object. For example, the 3D digital human model adjusts its appearance and details in real time according to different actions and expressions.

[0040] Gaussian splatter is a real-time rendering technology based on dynamic point clouds. By simulating complex surfaces with tens of thousands to millions of point clouds with Gaussian distribution attributes (i.e., Gaussian points), it can replace traditional triangle meshes and improve the effect of simulating the movement and deformation of 3D digital humans.

[0041] Specifically, in response to the instruction in the Gaussian rendering command sent by the software engine to call the target neural network model based on the GPU, through the target neural network model based on the GPU, the dynamic Gaussian attributes that match the dynamic texture data can be calculated in real time, that is, the dynamic texture data calculated in the GPU memory is transformed into a multi-dimensional array, and these attributes are stored in the GPU memory for quick access and use during rendering.

[0042] Typically, the target neural network model can be an ONNX (Open Neural Network Exchange) neural network. In ONNX, the input and output of the model can be defined according to the requirements of the task.

[0043] Typically, it can be set that the ONNX model requires an input texture and outputs three output textures based on this one input texture. The ONNX neural network can generate more complex textures based on the input texture data, providing more details and levels to better meet complex rendering requirements and enhance the visual effect.

[0044] S130. Render the target image according to the dynamic Gaussian attribute calculated by the target neural network model, and display the target image.

[0045] Specifically, in response to the rendering instruction in the Gaussian rendering command sent by the software engine, convert the calculated dynamic Gaussian attribute in the memory of the GPU into a texture, map it to each part of the 3D model, generate the final image, and display it.

[0046] The technical solution of the embodiment of the present disclosure calculates the dynamic texture data corresponding to the target image to be rendered in response to the Gaussian rendering command sent by the software engine, and stores the dynamic texture data in the GPU; calls the target neural network model implemented based on the GPU to calculate the dynamic Gaussian attribute matching the dynamic texture data, and stores the dynamic Gaussian attribute in the GPU; renders the target image according to the dynamic Gaussian attribute calculated by the target neural network model, and displays the target image. By calling the target neural network model implemented based on the GPU to calculate the dynamic Gaussian attribute matching the dynamic texture data, more detailed dynamic features and detail information of the 3D digital human can be obtained. By reasonably allocating various GPU hardware resources and the implementation method of synchronously generating texture data and Gaussian attributes on the GPU, the Gaussian attribute adapted to each frame of image data can be generated in real time. The entire process of texture calculation, dynamic Gaussian attribute calculation, and image rendering is executed by the GPU, which can make full use of the parallel computing ability and high data throughput ability of the GPU, avoid the efficiency loss and quality degradation caused by the segmentation of links, improve the execution efficiency of the 3D digital human rendering process, realize the dynamic deformation and character details of each frame of the 3D digital human, and the real-time dynamic rendering of the 3D digital human, and improve the rendering effect and user experience of the 3D digital human.

[0047] Figure 2 It is a schematic diagram of another rendering method of a 3D digital human provided according to an embodiment of the present disclosure. This embodiment is refined based on the above-mentioned embodiments. In this embodiment, the operation of "calculating the dynamic texture data corresponding to the target image to be rendered and storing it in the GPU" is specifically implemented as: "calculating the dynamic texture data corresponding to the target image, and storing the dynamic texture data in the first memory area pre-applied by the hardware rendering component; by calling the standard computing component in the GPU, mapping the first memory area to the standard memory area adapted to the standard computing component."

[0048] Correspondingly, as Figure 2 shown, the method may specifically include:

[0049] S210. Respond to the Gaussian rendering command sent by the software engine.

[0050] S220. Calculate the dynamic texture data corresponding to the target image, and store the dynamic texture data in the first memory area pre-applied by the hardware rendering component.

[0051] Among them, the target image contains a 3D digital human.

[0052] Generally speaking, the first memory area can be specifically understood as: a specific storage area pre-allocated for the hardware rendering component in the GPU memory, used to store the calculated dynamic texture data corresponding to the target image to be rendered. Among them, the size of the first memory area can be based on the rendering requirements and the input and output data sizes of the target neural network model implemented based on the GPU, and a sufficiently large memory area can be pre-applied in the GPU memory as the first memory area.

[0053] Typically, for the calculation of the dynamic texture data of the current frame of the target image, multiple dynamic texture calculations may be involved. Taking a single calculation as an example, when the target neural network model is set to require one input texture and output three output texture data volumes of dynamic Gaussian attributes based on this one input texture, for this single calculation, at least the same number and the same size of textures (width and height) corresponding data volume storage space can be applied, for example, apply a first memory area with a total of 4 texture data volume storage spaces, where 1 texture data volume storage space is used to store the calculated dynamic texture data, and 3 texture data volume storage spaces are used to store the dynamic Gaussian attributes. Or, at least the same number and the same size of textures (width and height) corresponding data volume storage space can be applied, for example, apply a first memory area with 3 texture data volume storage spaces, and when storing the dynamic Gaussian attributes, overwrite the 1 texture data volume of dynamic texture data calculated previously.

[0054] S230. Map the first memory area to a standard memory area adapted to the standard computing component by calling the standard computing component in the GPU.

[0055] Generally speaking, the standard computing components can be specifically understood as: modules or interfaces in the GPU used to execute general computing tasks. These components provide different application interfaces, allowing developers to utilize the parallel computing capabilities of the GPU to process complex computing tasks, such as image rendering and other tasks, or for data transfer. The standard memory area can be specifically understood as: the memory area in the GPU adapted to the standard computing components in the GPU. Among them, the size of the standard memory area is at least equal to the storage space required to establish a mapping with all memory addresses of the first memory area.

[0056] Specifically, convert the address space of the first memory area (for storing dynamic texture data, etc.) previously allocated in the GPU into a form that can be directly accessed by the standard computing components, and construct a mapping between the standard memory area and each address of the first memory area, so that the standard computing components can directly read and write the data in the first memory area without data copying, reducing the overhead of data transmission and improving the computing and rendering performance.

[0057] S240. Calculate the dynamic Gaussian attributes matching the dynamic texture data by calling the target neural network model implemented based on the GPU, and store the dynamic Gaussian attributes in the GPU.

[0058] In an optional implementation manner of this embodiment, calculating the dynamic Gaussian attributes matching the dynamic texture data by calling the target neural network model implemented based on the GPU and storing the dynamic Gaussian attributes in the GPU may include:

[0059] Copy the dynamic texture data from the first memory area to the second memory area previously allocated for the target neural network model through the standard computing components with the standard memory area as a relay;

[0060] Calculate the dynamic Gaussian attributes matching the dynamic texture data according to the dynamic texture data obtained from the second memory area by calling the target neural network model implemented based on the GPU and store them in the GPU.

[0061] Generally speaking, the second memory area can be specifically understood as: the memory area in the GPU previously allocated for the target neural network model implemented based on the GPU, used to store the processed dynamic texture data so that the neural network model can directly access and use these data, and used to store the calculated dynamic Gaussian attributes matching the dynamic texture data. Among them, the size of the second memory area needs to be determined according to the input and output data sizes of the target neural network model implemented based on the GPU.

[0062] Typically, the calculation of the dynamic Gaussian attributes for the dynamic texture data of the current frame of the target image may involve multiple calculations of dynamic Gaussian attributes. Taking a single calculation as an example, when the target neural network model is set to require one input texture and output the dynamic Gaussian attributes of three output texture data volumes based on this one input texture, for this single calculation, it is possible to apply for at least the storage space corresponding to the data volume of the textures (width and height) with the same number and the same size as the input and output. For example, apply for a second memory area with a total of 4 texture data volume storage spaces, where 1 texture data volume storage space is used to store the dynamic texture data, and 3 texture data volume storage spaces are used to store the dynamic Gaussian attributes. Alternatively, it is possible to apply for at least the storage space corresponding to the data volume of the textures (width and height) with the same number and the same size as the input or output. For example, apply for a second memory area with 3 texture data volume storage spaces, and when storing the dynamic Gaussian attributes, overwrite the previous 1 texture data volume of dynamic texture data.

[0063] Specifically, according to the address mapping relationship between the first memory area and the standard memory area, copy the dynamic texture data to the second memory area with the standard memory area as a relay. Call the target neural network model implemented based on the GPU on the GPU, obtain the dynamic texture data from the second memory area, use the target neural network model to process the obtained dynamic texture data, calculate the matching dynamic Gaussian attributes, and store the calculated dynamic Gaussian attributes in the second memory area.

[0064] By mapping the first memory area to the standard memory area adapted to the standard computing component, even when the second memory area and the first memory area of the target neural network are independent of each other and cannot directly interact, data interaction can also be carried out through the standard computing component as a relay, improving the efficiency and flexibility of data processing, so that when calculating the dynamic Gaussian attributes through the target neural network implemented based on the GPU subsequently, complex computing tasks can be completed more efficiently, and more detailed dynamic features and detailed information of the 3D digital human can be obtained.

[0065] S250. Render the target image according to the dynamic Gaussian attributes calculated by the target neural network model, and display the target image.

[0066] The technical solution of the embodiment of the present disclosure calculates the dynamic texture data corresponding to the target image and stores the dynamic texture data in the first memory area pre-applied by the hardware rendering component; by calling the standard computing component in the GPU, the first memory area is mapped to the standard memory area adapted to the standard computing component. By mapping the first memory area to the standard memory area adapted to the standard computing component, the standard computing component can directly read and write the data in the first memory area without data copying, reducing the overhead of data transmission, improving the computing and rendering performance, and enabling more efficient completion of complex computing tasks and obtaining more detailed dynamic features and detail information of the 3D digital human when calculating the dynamic Gaussian attributes through the target neural network implemented based on the GPU in the subsequent process. By reasonably allocating various GPU hardware resources and the implementation method of synchronously generating texture data and Gaussian attributes on the GPU, it is possible to generate adaptively matched Gaussian attributes for each frame of image data in real time. The GPU executes the entire process of texture calculation, dynamic Gaussian attribute calculation, and image rendering, which can make full use of the parallel computing ability and high data throughput ability of the GPU, avoid the efficiency loss and quality degradation caused by the fragmentation of links, improve the execution efficiency of the 3D digital human rendering process, realize the dynamic deformation and detail characterization of each frame of the 3D digital human and the real-time dynamic rendering of the 3D digital human, and improve the rendering effect and user experience of the 3D digital human.

[0067] In another alternative implementation manner of this embodiment, by calling the target neural network model implemented based on the GPU, according to the dynamic texture data obtained from the second memory area, calculating the dynamic Gaussian attributes matching the dynamic texture data and storing them in the GPU, further includes:

[0068] By calling and executing the target neural network model, updating and storing the calculated dynamic Gaussian attributes matching the dynamic texture data in the second memory area;

[0069] By calling the standard computing component and using the standard memory area as a relay, copying the dynamic Gaussian attributes from the second memory area to the first memory area.

[0070] Specifically, call the target neural network model implemented based on the GPU on the GPU, calculate the dynamic Gaussian attributes matching the dynamic texture data and store them in the second memory area. According to the address mapping relationship between the first memory area and the standard memory area, copy the dynamic Gaussian attributes from the second memory area to the first memory area with the standard memory area as a relay. By mapping the first memory area to the standard memory area adapted to the standard computing component, even when the second memory area and the first memory area of the target neural network are independent of each other and cannot directly interact, data interaction can also be carried out through the standard computing component as a relay, improving the efficiency and flexibility of data processing, and improving the efficiency of subsequent hardware rendering components to perform image rendering and display through the dynamic Gaussian attributes in the first memory area.

[0071] In another optional implementation manner of this embodiment, after the target neural network model is triggered to be called, by making at least one call to the standard computing component, according to the dynamic texture data obtained from the input data area of the second memory area, calculate the dynamic Gaussian attributes and store them in the output data area of the second memory area.

[0072] Generally speaking, since a frame of image may contain multiple textures and the input data of the target neural network model is usually data of 1 texture data volume, therefore, for the calculation of the dynamic Gaussian attributes of the dynamic texture data of the current frame of the target image, it may include multiple calculations of the dynamic Gaussian attributes. After the target neural network model is triggered to be called, at least one call to the standard computing component is required to calculate all the dynamic texture data of the current frame.

[0073] Specifically, when it is necessary to process the dynamic texture data of the current frame of the target image, the target neural network model is triggered. Each time the standard computing component is called, the dynamic texture data in the first memory area is copied to the second memory area, the dynamic texture data is obtained from the input data area of the second memory area, and the target neural network model calculates the matching dynamic Gaussian attributes and stores them in the second memory area (such as the output data area of the second memory area).

[0074] Typically, taking a single calculation as an example, when the target neural network model is set to require one input texture and output dynamic Gaussian attributes of three output texture data volumes according to this one input texture, for this single calculation, a second memory area with a total storage space of 4 texture data volumes can be applied. Among them, the storage space of 1 texture data volume is used to store the dynamic texture data as the input data area of the second memory area, and the storage space of 3 texture data volumes is used to store the dynamic Gaussian attributes as the output data area of the second memory area.

[0075] By mapping the first memory area to a standard memory area adapted to the standard computing component, even when the second memory area and the first memory area of the target neural network are independent of each other and cannot directly interact, data interaction can also be performed through the standard computing component as a relay, improving the efficiency and flexibility of data processing. Through at least one call to the standard computing component, independent dynamic Gaussian attribute calculations for each dynamic texture data are realized, precisely processing the details of each texture to support the implementation of more complex rendering effects and obtaining more detailed dynamic features and detail information of the 3D digital human. Executing the dynamic Gaussian attribute calculation by the GPU can make full use of the parallel computing power and high data throughput capacity of the GPU, improving the execution efficiency of the 3D digital human rendering process, realizing the dynamic deformation and character detail rendering of each frame of the 3D digital human, as well as the real-time dynamic rendering of the 3D digital human, and improving the rendering effect and user experience of the 3D digital human.

[0076] In another optional implementation manner of this embodiment, rendering the target image according to the dynamic Gaussian attribute calculated by the target neural network model may include:

[0077] Obtain the dynamic Gaussian attribute calculated by the target neural network model from the first memory area, and render the target image according to the dynamic texture data.

[0078] Specifically, obtain the dynamic Gaussian attribute calculated by the target neural network model in the output data area of the second memory area copied through the standard memory area as a relay from the first memory area, convert the dynamic Gaussian attribute into dynamic texture data according to the rendering instruction, and generate a final image for display. By obtaining the dynamic Gaussian attribute and dynamic texture data from the first memory area, real-time dynamic rendering can be achieved, enabling the 3D digital human to adjust its appearance and details in real time, improving the realism and smoothness of the rendering. Executing the processing of the dynamic Gaussian attribute and the rendering task of the target image by the GPU can utilize the parallel computing power of the GPU to accelerate the rendering speed, ensure the real-time nature of the rendering, and improve the user experience.

[0079] In another optional implementation manner of this embodiment, the Gaussian rendering command includes: an extended calculation instruction, a Gaussian attribute update instruction, and a standard rendering instruction. The execution order of the extended calculation instruction is prior to that of the Gaussian attribute update instruction, and the execution order of the Gaussian attribute update instruction is prior to that of the standard rendering instruction. The rendering method of the 3D digital human specifically includes:

[0080] In response to the Gaussian rendering command for the target image to be rendered, execute each instruction encapsulated in the Gaussian rendering command in sequence according to the preset instruction execution order;

[0081] When executing the extended calculation instruction, calculate the dynamic texture data corresponding to the target image and store the dynamic texture data in the GPU;

[0082] When executing the Gaussian attribute update instruction, calculate the dynamic Gaussian attribute matching the dynamic texture data by calling the target neural network model implemented based on the GPU, and store the dynamic Gaussian attribute in the GPU;

[0083] When executing the standard rendering instruction, render the target image according to the dynamic Gaussian attribute calculated by the target neural network model, and display the target image.

[0084] Generally speaking, the extended calculation instruction can be specifically understood as: the instruction included in the Gaussian rendering command that triggers the calculation of the dynamic Gaussian attribute and is used to execute the pre-dynamic texture data calculation task, so as to provide input data for calculating the dynamic Gaussian attribute of the target neural network model based on the GPU. The Gaussian attribute update instruction can be specifically understood as: the instruction included in the Gaussian rendering command that triggers the calculation of the dynamic Gaussian attribute and is used to call the target neural network model based on the GPU to calculate the dynamic Gaussian attribute. The standard rendering instruction can be specifically understood as: the instruction used to execute the actual rendering operation, convert it into a dynamic texture, and apply it to the 3D digital human model, render the target image and display it. The extended calculation instruction, the Gaussian attribute update instruction, and the standard rendering instruction together constitute the Gaussian rendering command. To ensure the correctness of the execution of the Gaussian rendering command and the stability of the rendering, it is necessary to ensure the correctness of the execution order of the extended calculation instruction, the Gaussian attribute update instruction, and the standard rendering instruction during the instruction writing process.

[0085] Specifically, during the rendering process, when the target image needs to be rendered and the system receives a Gaussian rendering command, it first needs to complete general computing tasks through extended computing instructions, calculate the dynamic texture data corresponding to the target image, and store this data in the GPU's memory to provide the necessary data and environment for the update of Gaussian attributes. After completing the extended computing, the Gaussian attribute update instruction is executed to calculate the dynamic Gaussian attributes, that is, the updated Gaussian attributes, based on the target neural network model of the GPU according to the extended computing results, and store these attributes in the GPU's memory. Then, the standard rendering instruction is executed, and the updated Gaussian attributes are used for the execution of the standard rendering instruction to ensure that the rendered image can accurately reflect the dynamic features and details of the 3D digital human. The final target image is generated through graphics rendering and displayed on the screen. Through a reasonable instruction sequence, the dynamic features of the 3D digital human can be processed and updated in real time, achieving smooth animation effects and real-time interaction, ensuring that each rendering step has accurate data support, improving the stability of Gaussian rendering of the 3D digital human, increasing the rendering efficiency, reducing the rendering latency, and enhancing the user experience. Structuring the Gaussian rendering command into an instruction set of extended computing instructions, Gaussian attribute update instructions, and standard rendering instructions makes the rendering process clearer and easier to adjust and expand according to requirements, such as adding new extended computing modules, Gaussian attribute update modules, or rendering modules.

[0086] In another alternative implementation of this embodiment, the Gaussian rendering command is an extension of the standard rendering command. The standard rendering command includes a standard computing instruction and a standard rendering instruction. The extended computing instruction is constructed by adding a target flag bit to the standard computing instruction; the Gaussian attribute update instruction is a newly added extended instruction.

[0087] Among them, the target flag bit is used to instruct the hardware rendering component to establish a memory mapping relationship between the hardware rendering component and the standard computing component after completing the calculation and storage of the dynamic texture data.

[0088] Generally speaking, the standard rendering command can be specifically understood as: hardware rendering component instructions, including a standard computing instruction and a standard rendering instruction. The standard computing instruction can be specifically understood as: an instruction for performing general computing tasks, such as generating, transforming, and updating texture data. The standard rendering instruction can be specifically understood as: an instruction for performing actual rendering operations, such as applying the calculated texture data to the 3D digital human for final image rendering.

[0089] The extended computing instruction can be specifically understood as an instruction constructed by adding a target flag bit to a standard computing instruction. The target flag bit can be specifically understood as a binary bit used to instruct the hardware rendering component to trigger the establishment of a memory mapping relationship operation between the hardware rendering component and the standard computing component after completing the calculation and storage of dynamic texture data.

[0090] Typically, the target flag can be set to 0 or 1, which respectively indicates that there is no need to establish a memory mapping relationship between the hardware rendering component and the standard computing component (indicating that the rendering of the target image does not require the use of Gaussian attribute update instructions to calculate dynamic Gaussian attributes), or that a memory mapping relationship between the hardware rendering component and the standard computing component needs to be established (indicating that the rendering of the target image does not require the use of Gaussian attribute update instructions to calculate dynamic Gaussian attributes, that is, the rendering method of the target image that needs to execute the Gaussian attribute update instruction needs to set the target flag of the extended calculation instruction to 1). The Gaussian attribute update instruction can be specifically understood as: a newly added extended instruction for calling the target neural network model implemented based on the GPU to calculate dynamic Gaussian attributes, that is, updating the Gaussian attributes and storing them in the GPU memory pre-applied for the target neural network model.

[0091] The Gaussian rendering command introduces extended calculation instructions and Gaussian attribute update instructions by extending the standard rendering command, ensuring efficient processing of dynamic texture data and real-time update of Gaussian attributes. The use of the target flag further ensures the stability of data transmission and processing flow execution between the GPU memory of the hardware rendering component and the GPU memory of the target neural network model implemented on the GPU, with the standard computing component as the relay, improving the efficiency and flexibility of data processing, so as to achieve the subsequent dynamic deformation of each frame of the 3D digital human and the portrayal of the character details, as well as the real-time dynamic rendering of the 3D digital human, and improving the rendering effect and user experience of the 3D digital human.

[0092] In another optional implementation of the present embodiment, a target execution code is pre-imported into the Gaussian attribute update instruction, and when the target execution code in the Gaussian attribute update instruction is executed, the target neural network model implemented based on the GPU is triggered to be called.

[0093] Generally speaking, the target execution code can be understood as: the code that defines how to call the target neural network model, how to obtain and process input data, and how to output and store output data. By inserting Gaussian attribute update instructions in a modular form into the appropriate position of the Gaussian rendering command, it is ensured that the execution order of the extended calculation instructions precedes the Gaussian attribute update instructions, and the execution order of the Gaussian attribute update instructions precedes the standard rendering instructions, so as to ensure the accuracy of calling the target neural network model to calculate the dynamic Gaussian attributes.

[0094] Specifically, when the target execution code in the Gaussian attribute update instruction is executed, it triggers the invocation of the target neural network model implemented based on the GPU, and copies the dynamic texture data from the GPU memory of the hardware rendering component to the GPU memory pre-applied for the target neural network model with the standard memory area as the relay. The dynamic texture data is obtained from the GPU memory corresponding to the target neural network model as the input of the model. The target neural network model calculates the matching dynamic Gaussian attributes according to the input dynamic texture data and stores them in the GPU memory corresponding to the target neural network model. Then, with the standard memory area as the relay, the data in the GPU memory of the hardware rendering component is updated to the matching dynamic Gaussian attributes.

[0095] By triggering the invocation of the target neural network model implemented based on the GPU during the execution of the target execution code pre-imported in the Gaussian attribute update instruction, calculating the dynamic Gaussian attributes and storing them in the GPU, it ensures the real-time update and efficient calculation of the dynamic Gaussian attributes, improves the rendering quality and user experience. By controlling a reasonable instruction sequence, it can process and update the dynamic features of the 3D digital human in real time, improves the fluency and real-time performance of the rendering, ensures that there is accurate data support for calling the target neural network model to calculate the dynamic Gaussian attributes, improves the stability of the Gaussian rendering of the 3D digital human, improves the rendering efficiency, and reduces the rendering latency. Structuring the Gaussian rendering command and constructing the Gaussian attribute update instruction in it in the form of importing the target execution code makes the Gaussian attribute update process clearer and easier to adjust according to requirements, such as modifying the input data and the form of the input data of the target neural network model, changing the Gaussian attribute update strategy, etc.

[0096] In another optional implementation manner of this embodiment, each instruction included in the Gaussian rendering command has a matching classification identification sequence;

[0097] The binary size of the classification identification sequence is used to describe the execution order of the instruction to which the classification identification sequence belongs; the last bit in the classification identification sequence is used to describe whether the instruction to which the classification identification sequence belongs is the Gaussian attribute update instruction.

[0098] Generally speaking, the classification identification sequence can be specifically understood as: a binary sequence used to identify the execution order and instruction type of instructions, which can be used as the unique identifier of instructions. The binary size of the classification identification sequence is used to describe the execution order of instructions. Typically, in binary representation, a sequence with a smaller value can represent a higher priority. Correspondingly, in the Gaussian rendering command, the instruction that needs to be executed first has a smaller binary size of the classification identification sequence, that is, the binary size of the extended calculation instruction is smaller than that of the Gaussian attribute update instruction, and the binary size of the Gaussian attribute update instruction is smaller than that of the standard rendering instruction. In this way, the system can execute instructions in a preset order to ensure the correctness and efficiency of the rendering process.

[0099] In addition, since the Gaussian attribute update instruction needs to be executed at a specific stage (after the extended calculation instruction is executed) and needs to call the target neural network model, it is necessary to ensure the correct calculation and application of the dynamic Gaussian attributes. To distinguish the Gaussian attribute update instruction from the rest of the instructions, the last bit (i.e., the least significant bit) is selected as the identification bit of the Gaussian attribute update instruction, which has the least impact on the overall sorting. Because the weight of the least significant bit is the lowest, changing it will not significantly change the overall binary size of the classification identification sequence, thus not significantly affecting the execution order of instructions.

[0100] Specifically, according to the order of the classification identification sequence, the extended calculation instruction is executed first to calculate the dynamic texture data corresponding to the target image and store this data in the memory of the GPU. When the last bit of the classification identification sequence is identified as the Gaussian attribute update instruction, it triggers the call of the target neural network model implemented based on the GPU to calculate the dynamic Gaussian attributes matching the dynamic texture data and store these attributes in the memory of the GPU. Finally, the standard rendering instruction is executed to generate the final target image through graphics rendering using the dynamic Gaussian attributes and display the image on the screen. The Gaussian rendering command determines the execution order of instructions through the binary size of the classification identification sequence, ensuring the correct execution order and type recognition of instructions. By using the last bit of the classification identification sequence to identify whether it is the Gaussian attribute update instruction, it ensures the stability of calling the target neural network model to calculate the dynamic Gaussian attributes. The system can quickly distinguish that this instruction is the Gaussian attribute update instruction, thereby calling the corresponding processing logic, improving the execution efficiency of the 3D digital human rendering method, reducing the rendering latency, realizing real-time processing and updating of the dynamic features of the 3D digital human, improving the fluency and real-time performance of the rendering, and thus improving the stability of the 3D digital human Gaussian rendering and enhancing the user experience.

[0101] Figure 3It is a schematic diagram of another 3D digital human rendering method provided according to an embodiment of the present disclosure. The embodiments of the present disclosure are applicable to the situation of triggering GPU to render 3D digital humans. This method can be executed by a 3D digital human rendering device, which can be implemented in a hardware and / or software manner and is generally configured in a software engine in the CPU. Among them, the software engine, such as a software thread at the CPU layer, is used to organize and prepare the resources and instructions required for rendering on the CPU and send them to a hardware rendering component, such as a Render Hardware Interface (RHI), to trigger the GPU to perform 3D digital human rendering.

[0102] Correspondingly, as Figure 3 shown, the method may specifically include:

[0103] S310. When the dynamic Gaussian rendering condition for the content to be displayed is met, construct Gaussian rendering commands respectively corresponding to each frame of the image to be rendered.

[0104] Among them, the image to be rendered contains a 3D digital human.

[0105] Generally speaking, the dynamic Gaussian rendering condition can be specifically understood as: the need for more complex rendering processing of the 3D digital human contained in the content to be displayed. Typically, when there are multiple dynamic objects in the scene and there are complex interactions and motion patterns between these objects (for example, when the number of 3D digital humans contained in the content to be displayed exceeds the preset digital human quantity threshold and the motion pattern exceeds the preset pattern threshold), more complex rendering processing is required to accurately depict the changes in the scene, which is regarded as meeting the dynamic Gaussian rendering condition for the content to be displayed.

[0106] Specifically, when the software engine in the CPU checks that the content to be displayed contains a 3D digital human and meets the dynamic Gaussian rendering condition, construct extended calculation instructions, Gaussian attribute update instructions, and standard rendering instructions for each frame of the image to be rendered, and combine them into Gaussian rendering commands.

[0107] S320. Send each of the Gaussian rendering commands to the hardware rendering component in the GPU in sequence, so that the hardware rendering component can generate dynamic Gaussian attributes respectively corresponding to the dynamic texture data of each frame of the image to be rendered by calling a target neural network model implemented based on the GPU, and perform real-time rendering and display of the content to be displayed according to each of the dynamic Gaussian attributes.

[0108] In another optional implementation manner of this embodiment, the Gaussian rendering command contains an extended calculation instruction, a Gaussian attribute update instruction, and a standard rendering instruction; the execution order of the extended calculation instruction is prior to that of the Gaussian attribute update instruction, and the execution order of the Gaussian attribute update instruction is prior to that of the standard rendering instruction.

[0109] Through a reasonable instruction sequence, the dynamic features of the 3D digital human can be processed and updated in real time, achieving smooth animation effects and real-time interaction, ensuring that each rendering step has accurate data support, improving the stability of Gaussian rendering of the 3D digital human, enhancing the rendering efficiency, reducing the rendering latency, and improving the user experience. Structuring the Gaussian rendering commands into an instruction set of extended calculation instructions, Gaussian attribute update instructions, and standard rendering instructions makes the rendering process clearer and easier to adjust and expand according to requirements, such as adding new extended calculation modules, Gaussian attribute update modules, or rendering modules.

[0110] In the technical solution of the embodiments of the present disclosure, the software engine in the CPU constructs Gaussian rendering commands corresponding to each frame of the image to be rendered, including extended calculation instructions, Gaussian attribute update instructions, and standard rendering instructions, and sends them to the hardware rendering component in the GPU in sequence. By invoking a target neural network model implemented based on the GPU to calculate the dynamic Gaussian attributes matching the dynamic texture data, more detailed dynamic features and detailed information of the 3D digital human can be obtained. Through the reasonable allocation of various GPU hardware resources and the implementation method of synchronously generating texture data and Gaussian attributes on the GPU, Gaussian attributes adapted to each frame of image data can be generated in real time. The GPU executes the entire process of texture calculation, dynamic Gaussian attribute calculation, and image rendering, which can make full use of the parallel computing ability and high data throughput ability of the GPU, avoid the efficiency loss and quality degradation caused by link fragmentation, improve the execution efficiency of the 3D digital human rendering process, realize the dynamic deformation and character detail rendering of each frame of the 3D digital human, and improve the rendering effect and user experience of the 3D digital human.

[0111] Figure 4 It is a schematic diagram of another rendering method of a 3D digital human provided according to the embodiments of the present disclosure. This embodiment is refined based on the above embodiments. In this embodiment, the operation of "meeting the dynamic Gaussian rendering condition for the content to be displayed" is specifically: "When the content to be displayed contains a 3D digital human and the dynamic index of the 3D digital human in the content to be displayed exceeds a preset threshold condition, it is determined that the dynamic Gaussian rendering condition for the content to be displayed is met."

[0112] Correspondingly, as Figure 4 shown, the method may specifically include:

[0113] S410. When the content to be displayed contains a 3D digital human and the dynamic index of the 3D digital human in the content to be displayed exceeds a preset threshold condition, it is determined that the dynamic Gaussian rendering condition for the content to be displayed is met.

[0114] Generally speaking, the dynamic index can be specifically understood as an indicator that measures the dynamic degree of a 3D digital human in the content to be displayed, which can include: movement amplitude, movement speed, and scene complexity, etc. For example, when the movement amplitude of the digital human is larger, the dynamic index is higher; when the execution speed of the movement is faster, the dynamic index is higher; the more dynamic objects are included in the scene, the higher the dynamic index. According to the rendering requirements, the indicators of the corresponding dynamic degree are quantified and summed to calculate the dynamic index. When the dynamic index exceeds the preset threshold condition, the system considers that the current rendering requirement exceeds the application scope of the ordinary rendering method, and it is necessary to use the dynamic Gaussian rendering method to ensure the rendering quality and real-time performance.

[0115] S420. Construct Gaussian rendering commands respectively corresponding to each frame of the image to be rendered.

[0116] Among them, the image to be rendered includes a 3D digital human; the Gaussian rendering command includes an extended calculation instruction, a Gaussian attribute update instruction, and a standard rendering instruction; the execution order of the extended calculation instruction is prior to that of the Gaussian attribute update instruction, and the execution order of the Gaussian attribute update instruction is prior to that of the standard rendering instruction.

[0117] S430. Send each of the Gaussian rendering commands to the hardware rendering component in the GPU in sequence, so that the hardware rendering component can generate dynamic Gaussian attributes respectively corresponding to the dynamic texture data of each frame of the image to be rendered by calling the target neural network model implemented based on the GPU, and perform real-time rendering and display of the content to be displayed according to each of the dynamic Gaussian attributes.

[0118] The technical solution of the embodiment of the present disclosure only starts the dynamic Gaussian rendering when the dynamic index of the 3D digital human exceeds the preset threshold condition, avoiding excessive calculation of static or slightly changing scenes and improving the resource utilization efficiency. In the case of a large movement amplitude, by calling the target neural network model implemented based on the GPU to calculate the dynamic Gaussian attributes matching the dynamic texture data, more detailed dynamic features and detail information of the 3D digital human can be obtained. By reasonably allocating various GPU hardware resources, the implementation method of synchronously generating texture data and Gaussian attributes on the GPU can generate adaptively matched Gaussian attributes for each frame of image data in real time. The GPU executes the whole process of texture calculation, dynamic Gaussian attribute calculation, and image rendering, which can make full use of the parallel computing ability and high data throughput ability of the GPU, avoid the efficiency loss and quality decline caused by the fragmentation of links, improve the execution efficiency of the 3D digital human rendering process, realize the dynamic deformation and character detail description of each frame of the 3D digital human, and the real-time dynamic rendering of the 3D digital human, and improve the rendering effect and user experience of the 3D digital human.

[0119] For ease of understanding, the specific application scenarios applicable to each of the disclosed embodiments will now be described. When implementing the real-time dynamic display of a 3D digital human in related technologies, it is first necessary to offline calculate a fixed Gaussian attribute value, and then combine this Gaussian attribute value to perform dynamic rendering and display on each frame of the 3D digital human's animation. This rendering method of the 3D digital human based on a fixed Gaussian attribute value cannot effectively adapt to complex 3D digital human motion change scenarios, and the rendering result cannot accurately depict various dynamic deformations and character details of the 3D digital human, reducing the visual authenticity of the 3D rendering result and thus affecting the real-time interaction experience.

[0120] To solve the above problems, the disclosed embodiments propose a rendering method for a 3D digital human, which may specifically include:

[0121] First, an engine thread (such as a software thread at the CPU layer) sends a target rendering instruction (equivalent to the Gaussian rendering command above) for rendering a new frame of the 3D Gaussian digital human image to a hardware rendering thread (RHI); wherein, the target rendering instruction includes a calculation instruction (equivalent to the extended calculation instruction above), a Gaussian attribute update instruction (CustomCode), and a drawing instruction (equivalent to the standard rendering instruction above), and the calculation instruction contains a flag bit.

[0122] Optionally, based on the above embodiments, when the content to be displayed contains a 3D digital human and the dynamic index of the 3D digital human in the content to be displayed exceeds a preset threshold condition, it is determined that the dynamic Gaussian rendering condition for the content to be displayed is satisfied, and the engine thread sends a target rendering instruction for rendering a new frame of the 3D Gaussian digital human image to the hardware rendering thread.

[0123] The hardware rendering thread executes the calculation instruction on the GPU, creates texture data and stores it in the first memory (equivalent to the GPU memory pre-applied by the hardware rendering component (such as Opengl) above), and establishes a memory mapping from the first memory to a second memory (equivalent to the standard memory area above) that matches the GPU calculation interface according to the identification bit contained in the calculation instruction.

[0124] The hardware rendering thread executes the Gaussian attribute update instruction on the GPU. After calling the GPU calculation interface to copy and obtain the texture data from the first memory as the model input data of the target neural network model and storing it in the third memory (equivalent to the second memory area pre-applied by the target neural network model above) corresponding to the target neural network model, the target code pre-injected into the Gaussian attribute update instruction is called to execute the target neural network model (such as the ONNX model) implemented based on the GPU.

[0125] The target neural network model calculates the dynamic Gaussian attributes matching the texture data according to the model input data through the GPU computing interface and stores them again in the third memory. Based on this memory mapping relationship, the hardware rendering thread calls the GPU computing interface to copy the dynamic Gaussian attributes from the third memory to the second memory again for updated storage. Since there is a memory mapping relationship between the first memory and the second memory matching the GPU computing interface, the dynamic Gaussian attributes in the third memory are also updated in the first memory. The hardware rendering thread executes the drawing instruction on the GPU. Based on the above memory mapping relationship, it obtains the dynamic Gaussian attributes from the first memory and dynamically draws a new frame of 3D Gaussian digital human image for display.

[0126] In addition, in the Gaussian attribute update instruction, the target execution code is pre-imported. When the execution reaches the target execution code in the Gaussian attribute update instruction, the target neural network model implemented based on the GPU is triggered to be called.

[0127] In addition, each instruction included in the target rendering instruction has a matching classification identification sequence;

[0128] The binary size of the classification identification sequence is used to describe the execution order of the instruction to which the classification identification sequence belongs; the last bit in the classification identification sequence is used to describe whether the instruction to which the classification identification sequence belongs is the Gaussian attribute update instruction.

[0129] In a specific example, a GPU chip can be used to implement the rendering method of the 3D digital human. Since the storage result of Opengl needs to be stored in the adapted GPU memory, that is, the first memory, and the calculation result of the ONNX neural network needs to be stored in another adapted GPU memory, that is, the third memory, these two memories are independent of each other and cannot directly perform data interaction. The standard computing component is an interface for GPU computing provided by the GPU chip and is a medium for data interaction between Opengl and ONNX. Both the input and output of ONNX are in the form of transforming the texture into a multi-dimensional array. The essence of this method is to input the texture used for Opengl rendering into ONNX to generate dynamic Gaussian attributes, and then convert the dynamic Gaussian attributes output by ONNX into Opengl textures for rendering.

[0130] First, apply for storage space corresponding to the data volume of textures (width and height) of the same size as the input and output on Opengl. Specifically, for a single texture calculation of the current frame, 1 texture can be applied as the ONNX input, and 3 textures can be applied as the ONNX output (that is, ONNX will generate dynamic Gaussian attributes with a data volume of 3 textures based on the input data with a data volume of 1 texture). When creating the texture, pass a flag bit to the RHI layer, and the flag bit is used to create a mapping between the standard computing component memory (the second memory) and the RHI layer for the texture. In a frame, the target rendering instruction of the 3D Gaussian digital human image first submits a Compute (computing) instruction for GPU computing, and Opengl updates the ONNX network input of the current frame through GPU computing. Then a section of external code is submitted, which actually calls the ONNX network. Since the GPU implementation version depends on the standard computing component, with the GPU implementation version of onnxruntime (ONNX runtime environment), both the memory and the computing process required by the rendering method of the 3D digital human are provided and executed by the GPU.

[0131] Through the standard computing component, map the memory of Opengl to the memory of the standard computing component. Here, the mapping does not generate actual data copying, but only address mapping. Before the ONNX neural network runs, call the instruction of the standard computing component to copy the texture content into the array of the ONNX input. After the neural network runs, copy the content of the output array to the corresponding standard computing component memory. At this time, because it is a memory mapping, the texture content of Opengl is also updated to the ONNX output. Finally, the target rendering instruction of the 3D Gaussian digital human image submits a Draw (drawing) instruction for drawing the Gaussian, and a dynamic 3D Gaussian digital human can be drawn on the screen in real time.

[0132] In the RHI architecture of a cross-platform graphics rendering library (such as bgfx), the ONNX running code is inserted into the OpenGL rendering code. All rendering instructions (computation instructions, Gaussian attribute update instructions, and drawing instructions) for one frame are included in the Frame (frame management) class. Drawing is performed at the end of one frame. Frame executes instructions in the order of the drawing instructions. The instructions were originally divided into two categories: the Draw instruction for drawing objects and the Compute instruction for GPU computation. By adding a new instruction type, CustomCode (Gaussian attribute update instruction), a section of external programming language-based (such as C++) code can be injected between multiple instructions in a Frame without disrupting the original order. That is, OpenGL first calculates the input of the neural network, then calls the Gaussian attribute update instruction code to execute the ONNX neural network, and finally OpenGL performs the rendering. This ensures that after inserting the external instructions for Gaussian rendering, the execution order of the original code remains unchanged, thus enabling the called code to execute the ONNX network to output dynamic Gaussian attributes and achieve real-time dynamic rendering of 3D digital humans.

[0133] Correspondingly, a Frame is composed of multiple instructions. To ensure the execution order of each piece of code during the rendering of 3D digital humans, a SortKey (sorting key) (equivalent to the classification identification sequence mentioned above) can be assigned to each instruction in the Frame to represent the type of the instruction. For example, whether the instruction is for drawing (Draw), computation (Compute), or Gaussian attribute update instruction (CustomCode), etc., and to record the execution order of the instruction in the Frame as the unique identifier of the instruction, ensuring that the instructions are executed in the correct order and avoiding chaos in the rendering order.

[0134] In the original SortKey, the order of the instructions can be determined by the higher-order Seq (sequence). The Seq of the instructions that are preferentially executed in the Frame is smaller. Therefore, when rendering, executing the instructions in a Frame in ascending order according to the SortKey can ensure the sequential execution of the instructions.

[0135] The main function of the SortKey is sorting, and the higher bits have a greater impact on sorting. To distinguish the Gaussian attribute update instruction from the other instructions, the last bit (i.e., the lowest bit) is selected as the identification bit for CustomCode, which has the least impact on the overall sorting. Because the weight of the lowest bit is the lowest, changing it will not significantly change the overall value of the SortKey, thus not affecting the execution order of the instructions. By setting one bit in the SortKey as the identification for CustomCode, the system can quickly distinguish that the instruction is a Gaussian attribute update instruction, thereby calling the corresponding processing logic and improving the execution efficiency of the rendering method for 3D digital humans.

[0136] The rendering method of the 3D digital human proposed in the embodiments of the present disclosure enables the 3D Gaussian digital human that requires a neural network to be rendered in real time on a computer equipped with a GPU and a CPU. By rendering each frame of the 3D digital human based on the dynamic Gaussian attributes, the dynamic deformation and character details of each frame of the 3D digital human can be accurately depicted, improving the rendering effect of the 3D digital human.

[0137] As an implementation of the above rendering method of the 3D digital human executed by the hardware rendering component configured in the GPU, the present disclosure also provides an optional embodiment of an execution device for implementing the above rendering methods of the 3D digital human.

[0138] Figure 5 It is a schematic diagram of a rendering device for a 3D digital human provided according to an embodiment of the present disclosure. As Figure 5 shown, the device includes: a command response module 510, an attribute calculation module 520, and an image rendering module 530, where:

[0139] The command response module 510 is configured to respond to a Gaussian rendering command sent by a software engine, calculate dynamic texture data corresponding to a target image to be rendered, and store the dynamic texture data in the GPU, where the target image includes a 3D digital human;

[0140] The attribute calculation module 520 is configured to calculate dynamic Gaussian attributes that match the dynamic texture data by invoking a target neural network model implemented based on the GPU, and store the dynamic Gaussian attributes in the GPU;

[0141] The image rendering module 530 is configured to render the target image according to the dynamic Gaussian attributes calculated by the target neural network model, and display the target image.

[0142] The technical solution of the embodiment of the present disclosure calculates dynamic texture data corresponding to a target image to be rendered in response to a Gaussian rendering command sent by a software engine, and stores the dynamic texture data in a GPU; calculates a dynamic Gaussian attribute matching the dynamic texture data by calling a target neural network model implemented based on the GPU, and stores the dynamic Gaussian attribute in the GPU; renders the target image according to the dynamic Gaussian attribute calculated by the target neural network model, and displays the target image. By calling the target neural network model implemented based on the GPU to calculate the dynamic Gaussian attribute matching the dynamic texture data, more detailed dynamic features and detail information of the 3D digital human can be obtained. By reasonably allocating various GPU hardware resources and implementing the generation of texture data and Gaussian attributes synchronously on the GPU, a Gaussian attribute adapted to each frame of image data can be generated in real time. The GPU executes the entire process of texture calculation, dynamic Gaussian attribute calculation, and image rendering, which can make full use of the parallel computing ability and high data throughput ability of the GPU, avoid efficiency loss and quality degradation caused by link fragmentation, improve the execution efficiency of the 3D digital human rendering process, realize the dynamic deformation and character detail rendering of each frame of the 3D digital human, and improve the rendering effect and user experience of the 3D digital human.

[0143] Based on the above embodiments, the command response module 510 is specifically configured to:

[0144] Calculate dynamic texture data corresponding to the target image, and store the dynamic texture data in a first memory area pre-applied by the hardware rendering component;

[0145] Map the first memory area to a standard memory area adapted to the standard computing component by calling the standard computing component in the GPU.

[0146] Based on the above embodiments, the attribute calculation module 520 is specifically configured to:

[0147] Copy the dynamic texture data from the first memory area to a second memory area pre-applied for the target neural network model through the standard computing component with the standard memory area as a relay;

[0148] Calculate a dynamic Gaussian attribute matching the dynamic texture data according to the dynamic texture data obtained from the second memory area by calling the target neural network model implemented based on the GPU, and store it in the GPU.

[0149] Based on the above embodiments, the attribute calculation module 520 is further configured to:

[0150] By invoking and executing the target neural network model, the calculated dynamic Gaussian attributes matching the dynamic texture data are updated and stored in the second memory area;

[0151] By invoking the standard computing component and using the standard memory area as a relay, the dynamic Gaussian attributes are copied from the second memory area to the first memory area.

[0152] Based on the above embodiments, after the target neural network model is triggered and invoked, through at least one invocation of the standard computing component, according to the dynamic texture data obtained from the input data area of the second memory area, the dynamic Gaussian attributes are calculated and stored in the output data area of the second memory area.

[0153] Based on the above embodiments, the image rendering module 530 is specifically configured to:

[0154] Obtain the dynamic Gaussian attributes calculated by the target neural network model from the first memory area, and render the target image according to the dynamic texture data.

[0155] Based on the above embodiments, the Gaussian rendering command may include: an extended calculation instruction, a Gaussian attribute update instruction, and a standard rendering instruction. The execution order of the extended calculation instruction is prior to that of the Gaussian attribute update instruction, and the execution order of the Gaussian attribute update instruction is prior to that of the standard rendering instruction. The 3D digital human rendering device may further include: an instruction execution unit, a texture calculation unit, an attribute update unit, and an image display unit, where:

[0156] The instruction execution unit is configured to, in response to the Gaussian rendering command for the target image to be rendered, sequentially execute the instructions encapsulated in the Gaussian rendering command according to the preset instruction execution order;

[0157] The texture calculation unit is configured to, when executing the extended calculation instruction, calculate the dynamic texture data corresponding to the target image and store the dynamic texture data in the GPU;

[0158] The attribute update unit is configured to, when executing the Gaussian attribute update instruction, calculate the dynamic Gaussian attributes matching the dynamic texture data by invoking the target neural network model implemented based on the GPU, and store the dynamic Gaussian attributes in the GPU;

[0159] The image display unit is configured to, when executing the standard rendering instruction, render the target image according to the dynamic Gaussian attributes calculated by the target neural network model and display the target image.

[0160] Based on the above embodiments, the Gaussian rendering command is an extension of the standard rendering command. The standard rendering command may include a standard calculation instruction and a standard rendering instruction. The extended calculation instruction is constructed by adding a target flag bit to the standard calculation instruction; the Gaussian attribute update instruction is a newly added extended instruction.

[0161] Wherein, the target flag bit is used to indicate that after the hardware rendering component completes the calculation and storage of the dynamic texture data, a memory mapping relationship is established between the hardware rendering component and the standard calculation component.

[0162] Based on the above embodiments, in the Gaussian attribute update instruction, target execution code is pre-imported. When the execution reaches the target execution code in the Gaussian attribute update instruction, the target neural network model implemented based on the GPU is triggered to be called.

[0163] Based on the above embodiments, each instruction included in the Gaussian rendering command has a matching classification identification sequence;

[0164] The binary size of the classification identification sequence is used to describe the execution order of the instruction to which the classification identification sequence belongs; the last bit in the classification identification sequence is used to describe whether the instruction to which the classification identification sequence belongs is the Gaussian attribute update instruction.

[0165] The above product can execute the 3D digital human rendering method executed by the hardware rendering component configured in the GPU of any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0166] As an implementation of the above 3D digital human rendering method executed by the software engine configured in the CPU, the present disclosure also provides an alternative embodiment of an execution device for implementing the above 3D digital human rendering method.

[0167] Figure 6 It is a schematic diagram of another 3D digital human rendering device provided according to an embodiment of the present disclosure. As Figure 6 shown, the device includes: a command construction module 610 and a command sending module 620, wherein:

[0168] The command construction module 610 is used to construct Gaussian rendering commands corresponding to each frame of the image to be rendered when the dynamic Gaussian rendering condition for the content to be displayed is met;

[0169] Wherein, the image to be rendered includes a 3D digital human;

[0170] A command sending module 620, configured to sequentially send each of the Gaussian rendering commands to a hardware rendering component in a GPU, so that the hardware rendering component generates dynamic Gaussian attributes respectively corresponding to the dynamic texture data of each frame of the image to be rendered by invoking a target neural network model implemented based on the GPU, and perform real-time rendering and display of the content to be displayed according to each of the dynamic Gaussian attributes.

[0171] In the technical solution of the embodiments of the present disclosure, a software engine in a CPU constructs Gaussian rendering commands corresponding to each frame of the image to be rendered, including extended calculation instructions, Gaussian attribute update instructions, and standard rendering instructions, and sends them to a hardware rendering component in a GPU in sequence. By invoking a target neural network model implemented based on the GPU to calculate dynamic Gaussian attributes matching the dynamic texture data, more detailed dynamic features and detail information of the 3D digital human can be obtained. By reasonably allocating various GPU hardware resources and implementing the generation of texture data and Gaussian attributes synchronously on the GPU, Gaussian attributes adapted to each frame of image data can be generated in real time. The GPU executes the entire process of texture calculation, dynamic Gaussian attribute calculation, and image rendering, which can make full use of the parallel computing ability and high data throughput ability of the GPU, avoid efficiency loss and quality degradation caused by link fragmentation, improve the execution efficiency of the 3D digital human rendering process, realize the dynamic deformation and character detail rendering of each frame of the 3D digital human, and improve the rendering effect and user experience of the 3D digital human.

[0172] Based on the above embodiments, the Gaussian rendering command includes an extended calculation instruction, a Gaussian attribute update instruction, and a standard rendering instruction; the execution order of the extended calculation instruction is prior to that of the Gaussian attribute update instruction, and the execution order of the Gaussian attribute update instruction is prior to that of the standard rendering instruction.

[0173] Based on the above embodiments, the command construction module 610 is specifically configured to:

[0174] When the content to be displayed includes a 3D digital human and the dynamic index of the 3D digital human in the content to be displayed exceeds a preset threshold condition, it is determined that the dynamic Gaussian rendering condition for the content to be displayed is satisfied.

[0175] The above product can execute the rendering method of the 3D digital human executed by the software engine configured in the CPU in any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects of the execution method.

[0176] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0177] According to an embodiment of the present disclosure, the present disclosure also provides two electronic devices, a readable storage medium, and a computer program product.

[0178] Figure 7 A schematic block diagram of an exemplary electronic device 700 that can be used to implement the embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0179] As Figure 7 shown, the device 700 includes a GPU 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit (memory) 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The GPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0180] A plurality of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, an optical disc, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0181] The GPU 701 executes the various methods and processes described above, such as the rendering method of a 3D digital human executed by a hardware rendering component configured in the GPU, that is:

[0182] In response to a Gaussian rendering command sent by a software engine, calculate dynamic texture data corresponding to a target image to be rendered, and store the dynamic texture data in the GPU, where the target image includes a 3D digital human;

[0183] By invoking a target neural network model implemented based on the GPU, calculate dynamic Gaussian attributes matching the dynamic texture data, and store the dynamic Gaussian attributes in the GPU;

[0184] Render a target image according to the dynamic Gaussian attributes calculated by the target neural network model, and display the target image.

[0185] For example, in some embodiments, a rendering method of a 3D digital human executed by a hardware rendering component configured in a GPU can be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the GPU 701, one or more steps of the rendering method of the 3D digital human described above can be executed. Alternatively, in other embodiments, the GPU 701 can be configured to execute the rendering method of the 3D digital human in any other suitable manner (e.g., by means of firmware).

[0186] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0187] As Figure 8 shown, the device 800 includes a CPU 801, which can execute various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit (memory) 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0188] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as a keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as a disk, optical disc, etc.; and communication unit 809, such as a network card, modem, wireless communication transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunications networks.

[0189] CPU 801 executes the various methods and processes described above. For example, the rendering method of the 3D digital human executed by the hardware rendering component configured in the CPU, that is:

[0190] When the dynamic Gaussian rendering condition for the content to be displayed is met, a Gaussian rendering command corresponding to each frame of the image to be rendered is constructed; wherein, the image to be rendered includes a 3D digital human;

[0191] Each of the Gaussian rendering commands is sequentially sent to the hardware rendering component in the GPU for the hardware rendering component to generate dynamic Gaussian attributes corresponding to the dynamic texture data of each frame of the image to be rendered by invoking a target neural network model implemented based on the GPU, and perform real-time rendering and display of the content to be displayed according to each of the dynamic Gaussian attributes.

[0192] For example, in some embodiments, the rendering method of the 3D digital human executed by the hardware rendering component configured in the CPU can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by CPU 801, one or more steps of the rendering method of the 3D digital human described above can be executed. Alternatively, in other embodiments, CPU 801 can be configured to execute the rendering method of the 3D digital human in any other suitable manner (e.g., by means of firmware).

[0193] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems on a chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0194] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0195] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0196] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the input received from the user can be in any form (including acoustic input, speech input, or tactile input).

[0197] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0198] A computer system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services. The server can also be a server of a distributed system, or a server combined with a blockchain.

[0199] Artificial intelligence is a discipline that studies how to make a computer simulate certain thinking processes and intelligent behaviors of humans (such as learning, reasoning, thinking, planning, etc.), and it has both hardware-level technologies and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning technology, big data processing technology, and knowledge graph technology.

[0200] Cloud computing refers to a technology system that accesses an elastic and scalable shared physical or virtual resource pool through a network. The resources can include servers, operating systems, networks, software, applications, storage devices, etc., and the resources can be deployed and managed in a demand-based and self-service manner. Through cloud computing technology, it is possible to provide efficient and powerful data processing capabilities for the application and model training of technologies such as artificial intelligence and blockchain.

[0201] It should be understood that the various forms of processes shown above can be used, and steps can be reordered, added, or deleted. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions provided in this disclosure can be achieved. There is no limitation herein.

[0202] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A 3D digital human rendering method, which is executed by a hardware rendering component configured in a graphics processor (GPU), and comprises: In response to a Gaussian rendering command sent by the software engine, calculating dynamic texture data corresponding to a target image to be rendered, and storing the dynamic texture data in the GPU, wherein the target image includes a 3D digital human; By calling a target neural network model implemented based on the GPU, a dynamic Gaussian attribute matching the dynamic texture data is calculated, and the dynamic Gaussian attribute is stored in the GPU; The target image is rendered according to the dynamic Gaussian properties calculated by the target neural network model, and the target image is displayed.

2. The method according to claim 1, wherein: The calculating of dynamic texture data corresponding to the target image to be rendered and storing the dynamic texture data in the GPU comprises: Calculate dynamic texture data corresponding to the target image, and store the dynamic texture data in a first memory area pre-applied by the hardware rendering component; By calling a standard computing component in the GPU, the first memory area is mapped to a standard memory area adapted to the standard computing component.

3. The method according to claim 2, wherein: The method of calculating a dynamic Gaussian attribute matching the dynamic texture data by calling a target neural network model implemented based on the GPU, and storing the dynamic Gaussian attribute in the GPU includes: By calling the standard computing group component, the dynamic texture data is copied from the first memory area to the second memory area pre-applied for the target neural network model using the standard memory area as a relay; By calling the target neural network model implemented based on the GPU, dynamic Gaussian attributes matching the dynamic texture data are calculated according to the dynamic texture data obtained from the second memory area and stored in the GPU.

4. The method according to claim 3, wherein: The method further includes: calculating a dynamic Gaussian attribute matching the dynamic texture data according to the dynamic texture data obtained from the second memory area by calling the target neural network model implemented based on the GPU, and storing the dynamic Gaussian attribute matching the dynamic texture data in the GPU. By calling and executing the target neural network model, the calculated dynamic Gaussian attribute matching the dynamic texture data is updated and stored in the second memory area; By calling a standard computing component and using the standard memory area as a relay, the dynamic Gaussian attribute is copied from the second memory area to the first memory area.

5. The method according to claim 3, wherein: After being triggered, the target neural network model calculates the dynamic Gaussian attribute based on the dynamic texture data obtained from the input data area of ​​the second memory area through at least one call to the standard computing component, and stores it in the output data area of ​​the second memory area.

6. The method according to claim 4, wherein: The step of rendering the target image according to the dynamic Gaussian attribute calculated by the target neural network model comprises: The dynamic Gaussian attribute calculated by the target neural network model is obtained from the first memory area, and the target image is rendered according to the dynamic texture data.

7. The method according to any one of claims 1 to 6, wherein: The Gaussian rendering command includes: an extended calculation instruction, a Gaussian attribute update instruction, and a standard rendering instruction. The extended calculation instruction is executed before the Gaussian attribute update instruction, and the Gaussian attribute update instruction is executed before the standard rendering instruction. The method specifically includes: In response to a Gaussian rendering command for a target image to be rendered, executing the instructions encapsulated in the Gaussian rendering command in sequence according to a preset instruction execution order; When executing the extended calculation instruction, calculating dynamic texture data corresponding to the target image, and storing the dynamic texture data in the GPU; When executing the Gaussian attribute update instruction, a dynamic Gaussian attribute matching the dynamic texture data is calculated by calling a target neural network model implemented based on the GPU, and the dynamic Gaussian attribute is stored in the GPU; When executing the standard rendering instruction, the target image is rendered according to the dynamic Gaussian properties calculated by the target neural network model, and the target image is displayed.

8. The method according to claim 7, wherein: The Gaussian rendering command is an extension of the standard rendering command, the standard rendering command includes a standard calculation instruction and a standard rendering instruction, the extended calculation instruction is constructed by adding a target flag bit to the standard calculation instruction; the Gaussian attribute update instruction is a newly added extended instruction; The target flag is used to instruct the hardware rendering component to establish a memory mapping relationship between the hardware rendering component and the standard computing component after completing the calculation and storage of dynamic texture data.

9. The method according to claim 7, wherein: In the Gaussian attribute update instruction, a target execution code is pre-imported. When the target execution code in the Gaussian attribute update instruction is executed, the target neural network model implemented based on the GPU is triggered to be called.

10. The method according to claim 7, wherein: Each instruction included in the Gaussian rendering command has a matching classification identification sequence; The binary size of the classification identification sequence is used to describe the execution order of the instructions to which the classification identification sequence belongs; the last bit in the classification identification sequence is used to describe whether the instruction to which the classification identification sequence belongs is the Gaussian attribute update instruction.

11. A method for rendering a 3D digital human, executed by a software engine configured in a central processing unit (CPU), the method comprising: When the dynamic Gaussian rendering conditions for the content to be displayed are met, constructing Gaussian rendering commands corresponding to each frame of the image to be rendered; wherein the image to be rendered includes a 3D digital human; Each of the Gaussian rendering commands is sent in sequence to the hardware rendering component in the GPU, so that the hardware rendering component generates dynamic Gaussian attributes corresponding to the dynamic texture data of each frame of the image to be rendered by calling the target neural network model implemented based on the GPU, and performs real-time rendering and display of the content to be displayed according to each of the dynamic Gaussian attributes.

12. The method according to claim 11, characterized in that The Gaussian rendering command includes an extended calculation instruction, a Gaussian attribute update instruction and a standard rendering instruction; the execution order of the extended calculation instruction is prior to the execution order of the Gaussian attribute update instruction, and the execution order of the Gaussian attribute update instruction is prior to the execution order of the standard rendering instruction.

13. The method according to claim 11 or 12, wherein: The dynamic Gaussian rendering conditions for the content to be displayed specifically include: When the content to be displayed includes a 3D digital human and a dynamic index of the 3D digital human in the content to be displayed exceeds a preset threshold condition, it is determined that a dynamic Gaussian rendering condition for the content to be displayed is met.

14. A 3D digital human rendering device, a hardware rendering component configured in a GPU, comprising: A command response module, configured to respond to a Gaussian rendering command sent by the software engine, calculate dynamic texture data corresponding to a target image to be rendered, and store the dynamic texture data in the GPU, wherein the target image includes a 3D digital human; An attribute calculation module, configured to calculate a dynamic Gaussian attribute matching the dynamic texture data by calling a target neural network model implemented based on the GPU, and store the dynamic Gaussian attribute in the GPU; An image rendering module is used to render the target image according to the dynamic Gaussian properties calculated by the target neural network model, and display the target image.

15. A 3D digital human rendering device, a software engine configured in a CPU, comprising: A command construction module, used for constructing Gaussian rendering commands corresponding to each frame of the image to be rendered when the dynamic Gaussian rendering conditions of the content to be displayed are met; wherein the image to be rendered includes a 3D digital human; A command sending module is used to send each of the Gaussian rendering commands to the hardware rendering component in the GPU in sequence, so that the hardware rendering component generates dynamic Gaussian attributes corresponding to the dynamic texture data of each frame of the image to be rendered by calling the target neural network model implemented based on the GPU, and performs real-time rendering and display of the content to be displayed according to each of the dynamic Gaussian attributes.

16. An electronic device, characterized in that: include: At least one GPU; as well as A memory communicatively connected to the at least one GPU; wherein, The memory stores a computer program executed by the at least one GPU, and the computer program is executed by the at least one GPU so that the at least one GPU can execute the 3D digital human rendering method according to any one of claims 1 to 10.

17. An electronic device, characterized in that: include: At least one CPU; as well as A memory in communication with the at least one CPU; wherein, The memory stores a computer program executed by the at least one CPU, and the computer program is executed by the at least one CPU so that the at least one CPU can execute the 3D digital human rendering method according to any one of claims 11 to 13.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause an electronic device to execute a method according to any one of claims 1-13.

19. A computer program product, comprising a computer program, wherein when the computer program is executed by a GPU or a CPU in an electronic device, the steps of the method according to any one of claims 1 to 13 are implemented.

Images