Memory management method, apparatus and related product

Abstract

Embodiments of the present application provide a memory management method, device and related product, wherein the memory management method comprises: determining an amount of data required for a character visualization in a graphics application; determining a target size of a first buffer according to the amount of data required for the character visualization, wherein the first buffer is continuous in a virtual address space; and adjusting a size of the first buffer for storing the data required for the character visualization according to the target size.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to International Application No. PCT / RU2023 / 000330, filed on October 27, 2023, which is incorporated herein by reference in its entirety. Technical Field

[0003] This application generally relates to the field of cloud computing technology, and in particular to a memory management method, apparatus and related products. Background Technology

[0004] Modern graphics applications tend to provide users with the best possible immersive experience, regardless of whether the application type is a top-tier AAA game or a simple virtual assistant that can provide a limited number of answers to simple questions. From a psychological perspective, communication with people, whether real or fictional, is more effective than text conversations. For example, using a virtual assistant can persuade potential customers faster than texting or making a phone call. In terms of gaming experiences, realistically-looking characters can draw players deeper into the game and immerse them in the story. Therefore, rendering realistically-looking characters is a very important part of 3D graphics processing today. Putting characters in static poses cannot provide an immersive effect, so animation becomes even more important. Older game applications used different parts of the body to animate characters independently, making the characters look more like dolls or robots, not realistic. Later, new technologies made it possible to "weight" animate parts of a character's body, thus simulating realistic body movement. This technique, a combination of artistic and programming methods, is called "rigging and skinning."

[0005] Skinning is a technique for animates a character using a character mesh and a pre-created internal skeleton. Skinning is primarily used for characters with flexible joints, such as humans or animals, while it's not used for rigid joint skeletons like robots. To perform proper skinned animation, different approaches are required. The first approach involves two instances of the same mesh: a bound pose used as a reference and the actual position calculated based on the bound pose with applied bone modifications. The second approach implements iterative animation: as the next position is calculated from the previous position, the incremental bone modifications from the previous frame are applied.

[0006] The storage of data required for character visualization (such as skinned mesh data) has a significant impact on the performance of graphics applications.

[0007] This background information is provided to disclose information that the applicant believes may be relevant to this application. It is not intended to acknowledge, nor should it be interpreted, that any of the foregoing information constitutes prior art to this application. Summary of the Invention

[0008] This application provides a memory management method, apparatus, and related products.

[0009] According to a first aspect, one embodiment of this application provides a memory management method, wherein the method includes:

[0010] Determine the amount of data required for character visualization in a graphical application;

[0011] The target size of the first buffer is determined based on the amount of data required for the visualization of the character, wherein the first buffer is contiguous in the virtual address space;

[0012] Adjust the size of the first buffer used to store the data required for the visualization of the character according to the target size.

[0013] In this way, the size of the first contiguous buffer in the virtual address space is adjusted according to the amount of data required for character visualization, and memory is allocated on demand to keep the graphics application's VRAM consumption to a minimum.

[0014] In one possible design of the first aspect, the method further includes:

[0015] Receive a role creation request, wherein the role creation request includes a first identifier of a first role to be created;

[0016] The amount of data required to determine the character visualization in the graphical application includes:

[0017] Determine the amount of data required for the visualization of the first role;

[0018] Determining the target size of the first buffer in the virtual address space based on the amount of data required for the character visualization includes:

[0019] Calculate the required GPU memory based on the amount of data required for the visualization of the first role;

[0020] The target size of the first buffer is determined based on the required GPU memory.

[0021] In one possible design of the first aspect, the method further includes:

[0022] Determine whether the first GPU memory in the first buffer is greater than or equal to the required GPU memory;

[0023] When the first GPU memory is greater than or equal to the required GPU memory, the data required for the visualization of the first role is stored in the first GPU memory;

[0024] The step of determining the target size of the first buffer based on the required GPU memory includes:

[0025] When the first GPU memory is less than the required GPU memory, the number M of GPU memory pages for the first role is determined, where M is calculated based on the first GPU memory and the required GPU memory, and M is a positive integer;

[0026] Adjusting the size of the first buffer used to store the data required for the character visualization according to the target size includes:

[0027] Allocate M GPU memory pages to adjust the size of the first buffer to the target size.

[0028] In one possible design of the first aspect, M is calculated based on the first GPU memory, the required GPU memory, and the added page size of the added GPU memory pages, wherein the added page size of the added GPU memory pages is configured by the user.

[0029] In this way, new roles can be added dynamically at runtime, with a minimum amount of GPU memory allocated to each new role. This enables flexible and dynamic addition of new roles and prevents excessive allocation of GPU memory.

[0030] In one possible design of the first aspect, the method further includes:

[0031] Receive a role removal request, wherein the role removal request includes a second identifier of the second role to be removed;

[0032] Mark the second GPU memory in the first buffer corresponding to the second role as removed GPU memory;

[0033] When no other character visualization data is stored in at least one of the N GPU memory pages corresponding to the second GPU memory, and the at least one of the N GPU memory pages is not the initial GPU memory page, the at least one of the N GPU memory pages is released, wherein the initial GPU memory page includes the initially allocated GPU memory page, and N is a positive integer.

[0034] In this way, by releasing the memory pages corresponding to the roles to be removed, the allocation of VRAM is reduced, thereby limiting the allocation of VRAM and using only the memory actually needed.

[0035] In one possible design of the first aspect, the method further includes:

[0036] Determine the size of an initial buffer, wherein the initial buffer is contiguous in the virtual address space;

[0037] The initial buffer is created according to the size of the initial buffer.

[0038] In one possible design of the first aspect, determining the size of the initial buffer includes:

[0039] The minimum size is determined to be the size of the initial buffer; or

[0040] Receive a first configuration instruction, wherein the first configuration instruction includes a user-configured size, and determine the user-configured size as the size of the initial buffer.

[0041] In this way, the initial buffer allocation can be as small as possible, so that the amount of GPU memory allocated is minimized, thus preventing memory waste.

[0042] In one possible design of the first aspect, the size of the initial buffer includes the page size of the initial buffer.

[0043] In one possible design of the first aspect, the first configuration instruction further includes a buffer limit and an added page size for the added GPU memory pages, wherein the buffer limit indicates the maximum amount of the first buffer.

[0044] In one possible design of the first aspect, the method further includes:

[0045] Receive a second configuration instruction, wherein the second configuration instruction includes the maximum number of roles configured by the user.

[0046] In one possible design of the first aspect, the method further includes:

[0047] A third configuration instruction is received, wherein the third configuration instruction includes a selected service function configured by the user, the selected service function being selected from an unlimited service function, a balanced service function, and a priority balanced service function, the unlimited service function indicating that the maximum number of roles allowed by the graphical application is not limited, and guaranteeing that all necessary roles are added to the graphical application; the balanced service function indicating that the maximum number of roles allowed by the graphical application is not limited, but does not guarantee that all necessary roles are added to the graphical application; the priority balanced service function indicating that the maximum number of roles allowed by the graphical application is not limited, and prioritizing the addition of third roles that need to be added to the graphical application to services with priority.

[0048] In one possible design of the first aspect, the data required for the character visualization includes skinned mesh data.

[0049] According to a second aspect, one embodiment of this application provides a memory management device, the memory management device comprising:

[0050] A determination module is used to determine the amount of data required for character visualization in a graphical application, and to determine the target size of a first buffer based on the amount of data required for character visualization, wherein the first buffer is contiguous in the virtual address space;

[0051] An adjustment module is used to adjust the size of the first buffer used to store the data required for the visualization of the character according to the target size.

[0052] In one possible design of the second aspect, the device further includes:

[0053] The first receiving module is used to receive a role creation request, wherein the role creation request includes a first identifier of a first role to be created;

[0054] The determining module is further configured to determine the amount of data required for the visualization of the first character, calculate the required GPU memory based on the amount of data required for the visualization of the first character, and determine the target size of the first buffer based on the required GPU memory.

[0055] In one possible design of the second aspect, the determining module is further configured to determine whether the first GPU memory in the first buffer is greater than or equal to the required GPU memory;

[0056] The device further includes:

[0057] A storage module is used to store the data required for the visualization of the first role into the first GPU memory when the first GPU memory is greater than or equal to the required GPU memory.

[0058] The determining module is further configured to determine the number M of GPU memory pages for the first role when the first GPU memory is less than the required GPU memory, wherein M is calculated based on the first GPU memory and the required GPU memory, and M is a positive integer;

[0059] The adjustment module is also used to allocate M GPU memory pages to adjust the size of the first buffer to the target size.

[0060] In one possible design of the second aspect, M is calculated based on the first GPU memory, the required GPU memory, and the added page size of the added GPU memory pages, wherein the added page size of the added GPU memory pages is configured by the user.

[0061] In one possible design of the second aspect, the device further includes:

[0062] The second receiving module is used to receive a role removal request, wherein the role removal request includes a second identifier of the second role to be removed;

[0063] A marking module is used to mark the second GPU memory in the first buffer corresponding to the second role as removed GPU memory;

[0064] A release module is configured to release the N GPU memory pages when no other data required for character visualization is stored in at least one of the N GPU memory pages corresponding to the second GPU memory and the at least one of the N GPU memory pages is not the initial GPU memory page, wherein the initial GPU memory page includes the initially allocated GPU memory page and N is a positive integer.

[0065] In one possible design of the second aspect, the determining module is further configured to determine the size of an initial buffer, wherein the initial buffer is contiguous in the virtual address space;

[0066] The device further includes:

[0067] A creation module is used to create the initial buffer according to the size of the initial buffer.

[0068] In one possible design of the second aspect, the determining module is further configured to:

[0069] The minimum size is determined to be the size of the initial buffer; or

[0070] The size of the user-configured buffer is determined as the size of the initial buffer, wherein the size of the user-configured buffer is included in the first configuration instruction received by the third receiving module of the device.

[0071] In one possible design of the second aspect, the size of the initial buffer includes the page size of the initial buffer.

[0072] In one possible design of the second aspect, the first configuration instruction further includes a buffer limit and an added page size for the added GPU memory pages, wherein the buffer limit indicates the maximum amount of the first buffer.

[0073] In one possible design of the second aspect, the device further includes:

[0074] The fourth receiving module is used to receive a second configuration instruction, wherein the second configuration instruction includes the maximum number of roles configured by the user.

[0075] In one possible design of the second aspect, the device further includes:

[0076] The fifth receiving module is configured to receive a third configuration instruction, wherein the third configuration instruction includes a selected service function configured by the user. The selected service function is selected from an unlimited service function, a balanced service function, and a priority balanced service function. The unlimited service function indicates that the maximum number of roles allowed by the graphical application is not limited, and guarantees that all necessary roles are added to the graphical application. The balanced service function indicates that the maximum number of roles allowed by the graphical application is not limited, but does not guarantee that all necessary roles are added to the graphical application. The priority balanced service function indicates that the maximum number of roles allowed by the graphical application is not limited, and prioritizes adding the third roles that need to be added to the graphical application to services with priority.

[0077] In one possible design of the second aspect, the data required for the character visualization includes skinned mesh data.

[0078] According to a third aspect, one embodiment of this application provides an electronic device including processing circuitry for performing the methods of the first aspect or any possible design of the first aspect.

[0079] According to a fourth aspect, one embodiment of this application provides a computing device cluster including processing circuitry for performing the methods of the first aspect or any possible design of the first aspect.

[0080] According to a fifth aspect, one embodiment of this application provides a chip including an input / output (I / O) interface and a processor, wherein the processor is configured to call and run a computer program stored in a memory to cause a device on which the chip is mounted to perform the methods of the first aspect or any possible design of the first aspect.

[0081] According to a sixth aspect, one embodiment of this application provides a graphics processing unit (GPU) driver, the GPU driver comprising:

[0082] processor;

[0083] The memory is coupled to the processor and stores a program executed by the processor, wherein, when executed by the processor, the program causes the GPU driver to perform the first aspect or any possible design of the first aspect.

[0084] According to a seventh aspect, one embodiment of this application provides a server system comprising: a graphics processing unit (GPU) driver as described in the sixth aspect, one or more CPUs, and multiple GPUs.

[0085] According to an eighth aspect, one embodiment of this application provides a non-transitory computer-readable medium carrying program code that, when executed by a computer device, causes the computer device to perform the methods of the first aspect or any possible design of the first aspect.

[0086] According to a ninth aspect, one embodiment of this application provides a computer program product including program code for performing the methods in the first aspect or any possible design of the first aspect. Attached Figure Description

[0087] This application can be further understood in conjunction with the accompanying drawings, which form part of the specification and are used together with the following specific exemplary embodiments to explain this application, but should not be construed as limiting this application.

[0088] Figure 1 An illustrative example of the initial position of a character model is shown.

[0089] Figure 2 These are illustrations of male dancers in different poses.

[0090] Figure 3 This is a diagram illustrating the submission of resources for textures with different levels of detail (LOD).

[0091] Figure 4 This is a schematic diagram of the memory layout for sparse textures.

[0092] Figure 5 This is a schematic flowchart illustrating a memory management method provided in one embodiment of this application.

[0093] Figure 6 This is a schematic flowchart of a memory management method provided in another embodiment of this application.

[0094] Figure 7 This is a schematic diagram of the role creation process in a specific implementation method.

[0095] Figure 8 This is a schematic flowchart of a memory management method provided in another embodiment of this application.

[0096] Figure 9 This is a schematic diagram of the initial allocation process in a specific implementation.

[0097] Figure 10 This is a schematic flowchart illustrating a memory management method provided in another embodiment of this application.

[0098] Figure 11 This is a schematic diagram of the role removal process in a specific implementation.

[0099] Figure 12 This is a schematic diagram of the frame drawing process in a specific implementation.

[0100] Figure 13 This is a schematic structural diagram of a memory management device provided in one embodiment of this application.

[0101] Figure 14 This is a schematic structural diagram of a graphics processing unit (GPU) driver provided in one embodiment of this application. Detailed Implementation

[0102] In the following description, reference is made to the accompanying drawings, which form part of this application, which illustratively show specific aspects of the examples of this application or aspects in which the examples of this application may be used. It should be understood that the examples of this application may be used in other aspects and may include structural or logical variations not depicted in the drawings. Therefore, the following detailed description is not to be construed as limiting, and the scope of this application is defined by the appended claims.

[0103] To more clearly illustrate the concepts of this application, exemplary concepts of some terms are provided below.

[0104] Exemplary concepts of some terms

[0105] The rendering pipeline or graphics pipeline: a fundamental framework in computer graphics that outlines the necessary steps to transform a three-dimensional (3D) scene into a two-dimensional (2D) representation on the screen.

[0106] Vertex: A point in 3D space, represented by the point's position in 3D space. It can include various other parameters, such as color, texture coordinates, surface normals, tangents, etc.

[0107] Mesh: Represents the set of vertices of an object / model in 3D space.

[0108] Role: Graphics processor unit (GPU) buffer, which contains data used to render a grid representing a living organism (such as a human or animal body).

[0109] Virtual avatar or player avatar: A 3D character model representing a user in 3D space, where the avatar is controlled by the user and can interact with the environment or other virtual avatars (similar to game scenes).

[0110] Non-playable character (NPC): A character in a 3D scene that is not directly controlled by the user. NPCs usually follow a predefined script, which is controlled by programs or algorithms such as AI algorithms.

[0111] T-pose (bound pose): The initial position of a character model, widely used by artists. In this pose, the human model is in a standing position with arms outstretched, and the human model's body forms the letter "T".

[0112] The A-pose is a variation of the T-pose, in which the human model stands upright with arms partially extended, forming the letter "A" with their body.

[0113] Skeleton: The internal non-geometric structure representing the actual position of the character model's body. The internal representation of the skeleton is a chain of linked quaternions or matrices. Moreover, the final transformation is the result of cumulative modifications performed on a series of bones (from the root to the current element).

[0114] Skeleton: A single element of the skeleton, represented by a single transformation entity, matrix, or quaternion.

[0115] Root: The immovable part of the skeleton, which can be a single bone or a single joint.

[0116] Skinned mesh / Skinned mesh: Each vertex is bound to the mesh formed by the skeleton. Binding is performed by creating weights for the vertices, which determine how the position of a mesh vertex is affected by a single bone or a set of skeleton bones.

[0117] Pipeline state object (PSO): A set of configuration parameters for the GPU, used by shaders or by other fixed functional stages of the rendering pipeline.

[0118] Shaders: Programs used by the GPU, where different shaders differ from one another depending on the stage of the graphics pipeline being executed.

[0119] Compute shader (CS): A general-purpose GPU program used to compute and simulate in-game processes that require direct processing of geometry / textures or other GPU data.

[0120] Rendering: The representation of 3D data (mesh, character) on a 2D screen.

[0121] 3D Scene (Scene): A combination of 3D object meshes, characters, and light sources in world space, representing a scene used for 3D rendering.

[0122] Scene budget: The total number of vertices, models, characters, video random access memory (VRAM), etc., all of which can be used in a single 3D scene. In practice, the scene budget can be defined by the product manager or game designer based on the target system requirements or application use case, and it is a suggestion to artists and / or scene developers, not a hard limit. Furthermore, violating the scene budget limit may result in a decrease in frames per second (FPS) or excessive VRAM usage.

[0123] Frame: A single snapshot of a 3D scene within a unit of time; all modifications are applied to all dynamic / animated objects.

[0124] Submitted resources: contiguous general-purpose buffers of fixed size, allocated as a single group of GPU memory blocks.

[0125] Parallel rendering pass or parallel channel (internal jargon): A single instance of a graphics application running on a server.

[0126] Service density: The total number of parallel channels running on a single server.

[0127] Figure 1 An illustrative example of the initial position of the character model is shown, where a T-pose is shown in part A and an A-pose is shown in part B.

[0128] Modern graphics applications strive to provide users with the best possible immersive experience, regardless of whether the application is a top-tier AAA game or a simple virtual assistant. Rendering realistic-looking characters is now a crucial part of 3D graphics processing.

[0129] Skinning is a technique used to create character animations using a character mesh and a pre-created internal skeleton. Skinning is primarily used for characters with flexible joints, such as humans or animals, while it is not used for rigid-jointed skeletal structures like robots. See also Figure 2 This provides illustrations of male dancer characters in different poses. Different methods can be used to perform correct skinned animation for the male dancer character. The first method involves two instances of the same mesh: a bound pose used as a reference, and the actual position calculated based on the bound pose with applied bone modifications. The second method implements iterative animation, applying the incremental bone modifications from the previous frame as the next position is calculated from the previous position.

[0130] In this application, the term "VRAM" can be understood as equivalent to GPU memory, or even simply as memory, used to store data required for scene visualization; the term "buffer" can also be understood as equivalent to a GPU buffer, allocated in VRAM. Moreover, without causing ambiguity, some buffers and memory may not be strictly distinguished.

[0131] For graphics applications, dynamic memory management on the GPU is a critical and challenging task. In related technologies, a common approach to memory and resource management on the GPU is based on using fixed-size buffers (also known as static / committed resources). These resources have specific memory layouts that vary depending on the resource type to provide the most efficient access to resources within the GPU. Figure 3 This is a diagram illustrating the submission of resources for textures with different Levels of Detail (LOD). For example... Figure 3 As shown, the layout of submitted resources is a contiguous buffer in memory, and varies depending on the textures with different Levels of Detail (LODs). Another approach is sparse texture technology, designed to manage textures that are too large to fit into GPU memory. The concept of sparse texture technology is that for a texture or rendering target, only the portion needed in the current frame / scene is loaded. Therefore, the memory layout for sparse textures differs from the layout of submitted resources described above. The memory for sparse textures can also be called a sparse buffer. Figure 4 This is a schematic diagram of memory layout for sparse textures. For example... Figure 4 As shown, physically, the memory used for sparse textures is represented by a group of blocks of memory that are randomly allocated and not connected to each other in VRAM, while in the virtual address space, the memory used for sparse textures is represented as a contiguous buffer in GPU memory to simulate the same layout as the submitted resources.

[0132] It's important to note that the term "contiguous buffer" in this article refers to a buffer that is contiguous in the virtual address space, while in physical space, portions of the buffer can be randomly allocated. Therefore, when a block group needs to be added to the buffer, the block group can be allocated at any physical address; however, it will be added to the end of the buffer in the virtual address space, and the buffer will remain contiguous. Conversely, in the way resources are committed in related technologies, GPU buffers are contiguous but have a fixed size, thus making them non-expandable.

[0133] In related technologies, different methods are applied to allocate GPU memory for skinned meshes. The main factors to consider when choosing a method depend on the use case scenario, such as how many characters will appear in the game / scene, whether characters can be added / removed during the job, or whether all characters should be created at startup and no characters can be added during the job.

[0134] Method 1: Allocate VRAM separately for each skinned mesh. This method is suitable for scenes where characters can appear / disappear during gameplay and the total number of possible characters in the scene is unknown. In this case, each buffer of each skinned mesh is allocated as a separate GPU resource, and the number of characters is limited by the internal scene budget or the total amount of video memory.

[0135] Method 2: Static GPU Buffer. In this case, when the maximum number of characters is known and the video memory is sufficient to create all characters, a static GPU buffer (pool) is allocated on the GPU to store all skinned meshes, thus avoiding memory allocation at runtime. Using a static pool, new characters can be added or removed at runtime without causing a drop in FPS. This method is widely used in services where users can connect or disconnect during runtime (e.g., large-scale multiplayer online games).

[0136] However, both of the above methods have limitations.

[0137] Method 1's separate VRAM allocation requires one or two independent allocations on the GPU (depending on the pose calculation method) to create the character. Allocating a large amount of memory on the GPU can block the execution of other rendering commands until the character is built. Therefore, adding a new character at runtime introduces a noticeable glitch (e.g., a drop in FPS), thus impacting the user experience.

[0138] Method 2's static GPU buffer does not introduce the problem of FPS drop. The main problem with static buffers relates to the minimum VRAM requirement allocated to each application instance, regardless of the number of characters available in the final scene. Furthermore, the total number of skinned characters is limited by the buffer size. Therefore, the VRAM constraint increases the artist's mesh optimization requirements, which increases the cost of scene development.

[0139] In summary, to achieve high-performance processing of the skin, it is necessary to store all skin meshes in a single buffer and increase or decrease the amount of memory allocated to the skin mesh.

[0140] One embodiment of this application provides a memory management method, such as... Figure 5 As shown, the memory management method may include the following steps.

[0141] S501: Determine the amount of data required for character visualization in a graphical application.

[0142] This method can be implemented by a memory management device. More specifically, the memory management device can be an electronic device, a chip, or a GPU driver.

[0143] The data required for character visualization includes skinned mesh data, which will be explained below as an example of the data required for character visualization.

[0144] The graphical application involves character animation and renders the animated characters into the main scene of the graphical application.

[0145] This application supports two common but contradictory approaches to rendering animated characters as 3D scenes. The first scenario is a minimal scene with a single character, which should consume as little memory as possible. The second scenario is one where a large number of characters can exist, and characters can be added or removed at runtime, requiring only the necessary amount of GPU memory.

[0146] Specifically, the first scenario could include a virtual assistant or MetaHuman: the user runs a graphical application and interacts with a virtual human / role capable of providing support, advice, or advertising services. Here, the virtual human / role executes scripted scenarios upon the user's request, rather than being controlled by a real user. This scenario requires allocating as little GPU memory as possible to increase the number of services that can run on a single server.

[0147] The second scenario can include interactive graphical applications that provide an immersive user experience when users have playable avatars. The avatar can move through an area of ​​3D space and interact with another player avatar. In this scenario, it's necessary to support adding and / or removing characters at runtime, and due to service density requirements and limited memory consumption, dynamic memory allocation is required. Existing examples of this second scenario include MetaSpace (which can be used for virtual concerts or virtual exhibitions).

[0148] S502: Determine the target size of the first buffer based on the amount of data required for character visualization, wherein the first buffer is contiguous in the virtual address space.

[0149] The first buffer mentioned above is contained in VRAM and is contiguous in the virtual address space. In physical memory, the first buffer can be a distributed memory allocation. The target size of the first buffer is determined based on the amount of data required for character visualization.

[0150] In the following text, without ambiguity, the first contiguous buffer within the virtual address space may also be simply referred to as the first contiguous buffer. Correspondingly, a contiguous buffer can refer to a buffer that is contiguous within the virtual address space.

[0151] S503: Adjust the size of the first buffer used to store the data required for character visualization based on the target size.

[0152] In this process, after determining the target size of the first buffer, the size of the first buffer is adjusted according to the target size. This means allocating additional GPU memory to expand the size of the first buffer to the target size, so that the data in the first buffer has enough capacity to store the data required for character visualization.

[0153] In one implementation, if a new character needs to be created, the amount of skinned mesh data for that character can be determined. Based on this determined amount, the required GPU memory for storing the character's skinned mesh data is calculated. To ensure that a first buffer has sufficient capacity to store the character's skinned mesh data, a target size for the first buffer is determined based on the required GPU memory for storing the new character's skinned mesh data. The first buffer is contiguous in the virtual address space. The size of the first buffer is then adjusted to match the target size; additional GPU memory is allocated to expand the first buffer to the target size. In this way, the amount of data required for character visualization is considered during memory allocation, and the size of the first buffer is adjusted accordingly, thereby ensuring the allocation of a minimum amount of GPU memory.

[0154] The first buffer, which is contiguous in the virtual address space, can be created using sparse texture techniques, which utilize distributed memory allocation in physical memory.

[0155] In addition, contiguous buffers in the virtual address space memory can be implemented using the pool method, which can also be called contiguous pools.

[0156] In one embodiment of the memory management method provided in this application, the amount of data required for character visualization in a graphics application is determined; a target size of a first buffer is determined based on the amount of data required for character visualization, wherein the first buffer is contiguous in the virtual address space; the size of the first buffer used to store the data required for character visualization is adjusted according to the target size. The size of the contiguous first buffer in the virtual address space can be adjusted according to the target size (determined based on the amount of data required for character visualization) to achieve the following: allocating a minimum amount of GPU memory for animated characters, dynamically adding new characters to the scene without causing significant performance problems to GPU allocation; and effectively managing GPU memory in a memory pool manner to avoid over-allocation when the allocated memory is sufficient to store newly added characters.

[0157] The method provided in this application embodiment is part of the resource management of the graphics engine core.

[0158] In addition, as a response to Figure 5 As a refinement and extension of the method shown, one embodiment of this application also provides a memory management method. Figure 6 This is a flowchart of a memory management method provided in another embodiment of this application. For example... Figure 6 As shown, the memory management method may include the following steps.

[0159] S601: Receive a role creation request, wherein the role creation request includes the first identifier of the first role to be created.

[0160] The character creation request is a request to create a new character, i.e., the first character to be created mentioned above. The character creation request can be entered by the user or triggered by an application or service. In one specific implementation, the character creation request can be entered by the user to request the creation of a user's virtual avatar. Alternatively, the character creation request can be triggered by the application to request the creation of an NPC for that application.

[0161] In a graphical application, each role corresponds to a specific identifier used to identify that role. For a first role, the first identifier of the first role can identify the first role and is included in the role creation request if the first role needs to be created.

[0162] It should be noted that the roles described in this application include, but are not limited to, playable characters, NPCs, or animals.

[0163] S602: Determine the amount of data required for the visualization of the first role.

[0164] For the specific implementation of S602, please refer to the description of S501 above, which will not be repeated here.

[0165] S603: Calculates the required GPU memory based on the amount of data needed for the visualization of the first role.

[0166] The specific implementation method for calculating the required memory based on the amount of data can refer to existing technologies, and this application does not impose any restrictions on it.

[0167] S604: Determine the target size of the first buffer based on the required GPU memory, wherein the first buffer is contiguous in the virtual address space.

[0168] Specifically, after obtaining the GPU memory required to store the data for the visualization of the first role, the target size of the first buffer is determined based on the required GPU memory.

[0169] S605: Adjust the size of the first buffer used to store the data required for character visualization based on the target size.

[0170] For the specific implementation of S605, please refer to the description of S503 above, which will not be repeated here.

[0171] In one specific implementation, after calculating the required GPU memory, it is determined whether the first GPU memory in the first buffer is greater than or equal to the required GPU memory. When the first GPU memory is greater than or equal to the required GPU memory, the data required for the visualization of the first role is stored in the first GPU memory. When the first GPU memory is less than the required GPU memory, the number M of GPU memory pages for the first role is determined, and M GPU memory pages are allocated to adjust the size of the first buffer to the target size. Here, M is calculated based on the first GPU memory and the required GPU memory, and M is a positive integer.

[0172] For example, during the runtime of a graphics application, a first contiguous buffer is allocated in VRAM, and a certain amount of data is already stored in the first contiguous buffer. Assuming the required GPU memory corresponding to the first character is calculated to be 270 kilobits (kb), the remaining memory in the first contiguous buffer (i.e., the first GPU memory) has two different scenarios: Scenario 1, the remaining memory in the first contiguous buffer is greater than or equal to the required GPU memory (e.g., 320 kb), the data required for the character visualization of the first character is stored in the remaining memory of the first contiguous buffer without allocating any GPU memory; Scenario 2, the remaining memory in the first contiguous buffer is less than the required GPU memory (e.g., 150 kb), one or more additional GPU memory pages need to be allocated to expand the first contiguous buffer so that the data required for the character visualization of the first character can be stored in the expanded first contiguous buffer. The number of additional GPU memory pages can be represented as M. The value of M can be calculated based on the difference between the remaining memory in the first contiguous buffer and the required GPU memory.

[0173] Furthermore, the size of the added GPU memory pages can be configured by the user or used to calculate the value of M. That is, M can be calculated based on the initial GPU memory, the required GPU memory, and the size of the added GPU memory pages, where the size of the added GPU memory pages is configured by the user.

[0174] Continuing with the example of scenario 2 above, the remaining memory in the first contiguous buffer (first GPU memory) is 150kb, while the required GPU memory is 270kb, meaning there is a shortage of 120kb of GPU memory. When the added page size is configured to 200kb, adding a single GPU memory page is sufficient, so the value of M is 1. When the added page size is configured to 80kb, two GPU memory pages are needed, so the value of M is 2.

[0175] Figure 7 This is a schematic diagram of the role creation process in a specific implementation method.

[0176] See Figure 7 In one specific implementation, a user can log into an interactive graphical application, requiring the creation of a user-specific role. This role can be a virtual avatar of the user within the 3D scene of the interactive graphical application, and is controlled by the user. In another specific implementation, a user can run a graphical application, such as a virtual assistant or MetaHuman, requiring the creation of an NPC that follows a predefined script. Both virtual avatars and NPCs can be collectively referred to as roles.

[0177] To create characters (including but not limited to avatars or NPCs), a character creation request is typically used to indicate a request to create a new character. During character creation, the character's skinned mesh data can be downloaded from a file and stored in random-access memory (RAM), and then further loaded into GPU memory. The required GPU memory can be calculated based on the amount of skinned mesh data for the character.

[0178] If the allocated buffer has enough space to hold the character's skinned mesh data (i.e., the allocated buffer's capacity is greater than or equal to the required GPU memory), the character will be placed in the allocated buffer without any memory allocation, thus preventing glitches and FPS drops. If the allocated buffer does not have enough memory to store the character's skinned mesh data (i.e., the allocated buffer's capacity is less than the required GPU memory), the size of the allocated buffer needs to be expanded by allocating the required number of GPU memory pages.

[0179] There are several ways to allocate the required number of GPU memory pages. In one approach, the required number of GPU memory pages is determined based on the total GPU memory needed and the page size of the added GPU memory pages. The page size of the added GPU memory pages can be pre-configured or specified by the user. The required number can be represented as a positive integer M. After determining the required number of GPU memory pages (M), M GPU memory pages are allocated.

[0180] In the second approach, the allocation of the required number of GPU memory pages can be achieved iteratively without calculating the required number M. More specifically, if the memory in the allocated buffer is insufficient to store the character's skinned mesh data, new GPU memory pages are allocated to expand the size of the allocated buffer. If the expanded buffer is still insufficient to store the character's skinned mesh data, multiple pages are allocated one by one until the expanded buffer is sufficient to store the entire skinned mesh data of the character.

[0181] The above method of allocating the required number of GPU memory pages is merely an example, and this application does not limit the specific implementation method.

[0182] In one embodiment of the memory management method provided in this application, by receiving a role creation request, wherein the role creation request includes a first identifier of a first role to be created; determining the amount of data required for the role visualization of the first role; calculating the required GPU memory based on the amount of data required for the role visualization of the first role; and allocating the required GPU memory to the first role, new roles can be added dynamically at runtime, and the minimum amount of GPU memory is allocated to the new roles, thereby achieving flexible and dynamic addition of new roles and preventing excessive allocation of GPU memory.

[0183] Furthermore, by determining whether the first GPU memory in the contiguous buffer is greater than or equal to the required GPU memory of the first character; when the first GPU memory is less than the required GPU memory, M GPU memory pages are allocated to the first character; when the first GPU memory is greater than or equal to the required GPU memory, the data required for the character visualization of the first character is stored in the first GPU memory, ensuring that if the allocated memory is sufficient to store the new character, GPU allocation is avoided, thereby significantly reducing the number of GPU allocations and thus improving the processing performance of the data required for character visualization.

[0184] At a high level, memory allocation in the dynamic buffer of the skinning mesh in memory management methods can be divided into three main logical parts. The first part is the initial allocation performed when the application starts; the second part is new character creation, either requested by the user or created by an NPC (the steps are the same for both); and the last part is character removal. The steps for creating a new character can be found in [reference needed]. Figure 7 .

[0185] The following will describe an example of initial allocation.

[0186] Figure 8 This is a schematic flowchart illustrating a memory management method provided in another embodiment of this application. Figure 8 As shown, the memory management method may also include the following steps.

[0187] S801: Determine the size of the initial buffer, wherein the initial buffer is contiguous in the virtual address space.

[0188] The buffer initialization occurs during application startup. In this application, the buffer is contained in VRAM and is contiguous in the virtual address space. In physical memory, the contiguous buffer can be distributed memory allocation. The initial buffer is the buffer initially allocated for the application, and its size can be configured by the user or service developer according to actual needs; this application does not impose any restrictions on this.

[0189] S802: Create an initial buffer based on the size of the initial buffer.

[0190] After determining the size of the initial buffer, the initial buffer is created based on that size. In one implementation, determining the size of the initial buffer may include setting a minimum size as the initial buffer size.

[0191] Hereinafter, without causing ambiguity, the initial contiguous buffer within the virtual address space may also be referred to simply as the initial contiguous buffer.

[0192] To prevent memory waste, the initial allocation of contiguous buffers should be as small as possible to allocate as little GPU memory as possible. For applications with a single character, minimal GPU memory consumption is appropriate; for other applications, such as interactive graphics applications with more than one character, if the allocated contiguous buffer is insufficient to store the skinned mesh data of a new character, the contiguous buffer can be expanded according to the amount of skinned mesh data of the new character so that the expanded contiguous buffer is sufficient to store the new character.

[0193] In one implementation, determining the size of the initial contiguous buffer may include: receiving a first configuration instruction, wherein the first configuration instruction includes a user-configured size, and determining the user-configured size as the size of the initial contiguous buffer.

[0194] Service developers / users can also configure the initial contiguous buffer size using the first configuration instruction based on specific rules or their own experience. In this way, upon receiving the first configuration instruction, the size configured by the user in the first configuration instruction is determined as the initial contiguous buffer size, thus allowing for flexible configuration of the initial contiguous buffer size.

[0195] In addition, the size of the initial buffer includes the page size of the initial buffer.

[0196] In one implementation, the initial contiguous buffer can be created by allocating initial GPU memory pages. In this way, the size of the initial buffer is equal to the size of the initial GPU memory pages, i.e., the page size of the initial buffer. Therefore, the size of the initial buffer includes the page size of the initial buffer.

[0197] Additionally, the first configuration instructions also include buffer limits and the added page size for the added GPU memory pages.

[0198] Here, the buffer limit represents the maximum size of the first buffer. Specifically, the buffer limit describes the maximum amount of memory the first buffer can be expanded to. For example, if the total allocated memory (including the initial contiguous buffer) is less than the buffer limit, at least one GPU memory page can be added to expand the capacity of the first buffer; if the total allocated memory is equal to or greater than the buffer limit, no more GPU memory pages are allocated so that the capacity of the contiguous buffer no longer expands.

[0199] The added page size of the GPU memory page indicates the page size used to allocate additional pages when the buffer needs to be expanded. It's important to note that the initially allocated GPU memory page when the first contiguous buffer is created is called the initial GPU memory page. Each contiguous buffer can only have one initial GPU memory page, and it cannot be removed from the contiguous buffer. The size of the initial GPU memory page can be independent of the sizes of other added pages used to expand the contiguous buffer's capacity; therefore, the added page size of the GPU memory page can be the same as or different from the page size of the initial contiguous buffer.

[0200] Figure 9 This is a schematic diagram of the initial allocation process in a specific implementation.

[0201] See Figure 9 In one specific implementation, when the interactive graphics application starts, default memory configurations are collected, such as the initial page size of the initial contiguous buffer, buffer limits, and the additional page size of the added GPU memory pages. These parameters are then configured to the application one by one, and the initial GPU memory pages are allocated as the initial contiguous buffer according to the initial page size.

[0202] In one embodiment of the memory management method provided in this application, for the initial allocation when the application starts, the minimum size is determined as the size of the initial contiguous buffer; or, a first configuration instruction is received, wherein the first configuration instruction includes a user-configured size; the user-configured size is determined as the size of the initial contiguous buffer; an initial contiguous buffer is created according to the size of the initial contiguous buffer, and the allocation of the initial contiguous buffer can be the minimum GPU memory allocated or the GPU memory required by the developer, thereby ensuring that the initial contiguous buffer is allocated according to actual needs, so as to avoid huge waste of memory consumption.

[0203] Figure 10 This is a schematic flowchart illustrating a memory management method provided in another embodiment of this application. Figure 10 As shown, a memory management method may include the following steps.

[0204] S1001: Receive a role removal request, wherein the role removal request includes a second identifier of the second role to be removed.

[0205] The role removal request is a request to remove a role, specifically the second role mentioned above that needs to be removed. Since each role corresponds to a specific identifier, the second identifier of the second role can identify the second role. When it is necessary to remove the second role, the second identifier is included in the role removal request.

[0206] S1002: Mark the second GPU memory in the first buffer corresponding to the second role as removed GPU memory.

[0207] Specifically, the data required for the visualization of the second role is stored in a first buffer, which is contiguous in the virtual address space. The specific memory occupied can be recorded as the second GPU memory within the first buffer. Upon receiving a role removal request that includes the second identifier of the second role to be removed, the second GPU memory corresponding to the second role in the first buffer is marked as the removed GPU memory.

[0208] S1003: When no other character visualization data is stored in at least one of the N GPU memory pages corresponding to the second GPU memory and at least one of the N GPU memory pages is not the initial GPU memory page, release at least one of the N GPU memory pages, where the initial GPU memory page includes the initially allocated GPU memory page, and N is a positive integer.

[0209] Specifically, for the second role to be removed, it is not necessary to retain the second GPU memory corresponding to the second role in the first contiguous buffer. However, there is a possibility that the data required for the visualization of the second role and one or more other roles may be stored on the same GPU memory page. In this case, the aforementioned same GPU memory page corresponding to both the second role and the one or more other roles should be retained, rather than being released as memory corresponding to the second role.

[0210] Furthermore, as mentioned above, there is only one initial GPU memory page in each contiguous buffer and it cannot be removed from the first contiguous buffer. When the GPU memory page corresponding to the second memory is the initial GPU memory page, that GPU memory page should also be retained instead of being released.

[0211] Therefore, when no other character visualization data is stored in at least one of the N GPU memory pages corresponding to the second GPU memory and at least one of the N GPU memory pages is not the initial GPU memory page, at least one of the N GPU memory pages is released, where the initial GPU memory page includes the initially allocated GPU memory page, and N is a positive integer.

[0212] See Figure 11In one implementation, a user can exit the interactive graphical application, corresponding to the removal of the user's avatar. In another implementation, an application NPC may need to be removed. To remove a character (including but not limited to avatars or NPCs), a character removal request is typically used to indicate the request to remove the character. Upon receiving a character removal request, the GPU memory in a contiguous buffer used to store the character's skinned mesh data is marked for removal. If one or more buffer pages consumed by the character's skinned mesh data do not contain allocated additional skinned mesh data for another character, and these pages are not the initial GPU memory pages of the contiguous buffer, then these pages are released. This can reduce VRAM allocation by limiting VRAM allocation to only the amount of memory actually needed.

[0213] In one embodiment of the memory management method provided in this application, a role removal request is received, wherein the role removal request includes a second identifier of a second role to be removed; a second GPU memory corresponding to the second role in a contiguous buffer is marked as the GPU memory to be removed; when no other role visualization data is stored in at least one GPU memory page among N GPU memory pages corresponding to the second GPU memory and at least one GPU memory page among N GPU memory pages is not the initial GPU memory page, at least one GPU memory page among N GPU memory pages is released, wherein the initial GPU memory page includes the initially allocated GPU memory page, and N is a positive integer; when removing a role, if no more roles also consume the same GPU page, the allocated GPU memory page can be released, thereby maintaining a minimum memory allocation.

[0214] In one embodiment of this application, the memory management method may further include receiving a second configuration instruction, wherein the second configuration instruction includes a maximum number of roles configured by a user.

[0215] In one embodiment of this application, the memory management method may further include: receiving a third configuration instruction, wherein the third configuration instruction includes a selected service function configured by a user, the selected service function being selected from an unlimited service function, a balanced service function, and a preferred balanced service function, the unlimited service function indicating that the maximum number of roles allowed by the graphics application is not limited, and guaranteeing that all necessary roles are added to the graphics application; the balanced service function indicating that the maximum number of roles allowed by the graphics application is not limited, but does not guarantee that all necessary roles are added to the graphics application; the preferred balanced service function indicating that the maximum number of roles allowed by the graphics application is not limited, and prioritizing the addition of third roles that need to be added to the graphics application to services with priority.

[0216] The term "balance" means that running a single service through several parallel channels can rely on the remaining available VRAM in the system. For example, a server may have several MetaSpace parallel channels running, and these channels may consume all of the system's VRAM. Because a large number of roles join all sessions, free slots only become available when a user leaves any session to bootstrap the service, allowing new users to join sessions.

[0217] Furthermore, a priority balancing service is a balancing service with higher priority so that users can connect to it as soon as resources become available. There can only be one priority balancing service on a single server. The priority balancing service will have a higher rate of acquiring available resources; that is, if idle VRAM becomes available, users waiting in the queue for the priority balancing service will be most likely to connect to that service rather than other users waiting to connect to the simple balancing service.

[0218] Figure 12 This is a schematic diagram of the frame drawing process in a specific implementation.

[0219] See Figure 12 In one specific implementation, the frame rendering process includes: at the beginning of the frame, calculating the skeletal pose of the characters in the scene, wherein the calculation is performed individually for each character in the scene; then, providing all skeletal information for calculating the skinned mesh, and applying the skeletal pose to the skinned mesh so that the characters can have the same pose as the skeleton, wherein this operation is also performed individually for each character in the scene; after completing the calculation of the skeletal pose of all characters in the scene, providing a modified skinned mesh for rendering and visualizing the characters in the 3D scene.

[0220] Because the contiguous buffer structure in this application is contiguous in the virtual address space, the processing of the contiguous buffer can be performed by calculating the shader for each character by passing a single PSO. Similar to the processing of animated characters stored in a static memory GPU pool, setting a different PSO for each animated character can avoid shader resource switching and pipeline restarts, which are resource-intensive. Therefore, using the memory management method provided in this application's embodiments, when all CS calls are merged into a single call, an additional performance improvement (1% to 2%) can be provided for buffer processing.

[0221] This application applies to skinning and animation, which will bring significant memory improvements, allowing a single implementation to be used in both dynamic and static scenes without recompiling the source code and reconfiguring the graphics engine.

[0222] The core concept of this application can be summarized as follows: the initial allocation of the contiguous buffer (pool) is configurable: the initial allocation of the contiguous buffer can be the minimum amount of GPU memory allocated or the GPU memory required by the developer, creating the minimum VRAM consumption memory for an application with a single role, and reverting to the static memory pool paradigm by requesting a large amount of initially allocated memory; new roles can be dynamically added at runtime, and the amount of GPU memory can be configured: the maximum amount of GPU memory that can be allocated to a role can be configured by the user, and the maximum amount of memory can exceed the total amount of GPU memory. If the allocated memory is insufficient, a new memory page will be allocated for the dynamically added role, and when a role is removed, if no more roles consume the same page, the memory page allocated to that role will be released to maintain a minimum memory allocation.

[0223] Therefore, the skinned mesh is placed in a single contiguous buffer, which is similar to a static memory pool on the GPU, but with the ability to increase or decrease the allocated memory space; skinned roles can be added or removed at runtime, depending on memory being allocated and released on demand to keep the graphics application's VRAM consumption to a minimum; and the maximum size of the buffer is not limited by the maximum amount of available VRAM, so that the theoretical number of skinned meshes can exceed the total capacity of VRAM.

[0224] The method disclosed in this application is implemented within Cloud MetaEngine and integrated into several cloud products. Specifically, the method is integrated into a virtual assistant (a collaborative project with a telecommunications company) to achieve minimal VRAM consumption for a single character scenario. By employing this method, the service density of the virtual assistant application can be increased to 79 parallel channels, which is 45% higher than other similar products that only allow 54 parallel channels under similar settings. The method is also applied to the virtual teacher scenario of the MetaHuman project, which can reduce the VRAM consumption of a single channel by 30% and run 12 parallel channels on a single server. Therefore, the solution of this invention is 4 times faster than solutions based on industry-leading game engines (3 to 4 channels).

[0225] Furthermore, the method disclosed in this application is used in the MetaSpace project—a fully interactive virtual exhibition scenario with a set of characters that can interact with the environment and with each other. Using this method, the number of characters is limited only by the amount of free VRAM, thus allowing any number of users to join a session and browse MetaSpace simultaneously, enabling all users to join or leave MetaSpace at any time. Moreover, MetaSpace's parallel channels use only the minimum amount of VRAM required to support characters already in a session, thus increasing the density of the MetaSpace service.

[0226] Figure 13 This is a schematic structural diagram of a memory management device provided in one embodiment of this application. Figure 13 As shown, the storage management device 1300 may include:

[0227] The determination module 1301 is used to determine the amount of data required for character visualization in a graphical application, and to determine the target size of a first buffer based on the amount of data required for character visualization, wherein the first buffer is contiguous in the virtual address space;

[0228] Adjustment module 1302 is used to adjust the size of the first buffer used to store the data required for character visualization according to the target size.

[0229] In one possible implementation, the device further includes:

[0230] The first receiving module is used to receive a role creation request, wherein the role creation request includes a first identifier of the first role to be created;

[0231] The determination module 1301 is also used to determine the amount of data required for the visualization of the first role, calculate the required GPU memory based on the amount of data required for the visualization of the first role, and determine the target size of the first buffer based on the required GPU memory.

[0232] In one possible implementation, the determining module 1301 is further configured to determine whether the first GPU memory in the first buffer is greater than or equal to the required GPU memory;

[0233] The device also includes:

[0234] The storage module is used to store the data required for the visualization of the first role into the first GPU memory when the first GPU memory is greater than or equal to the required GPU memory;

[0235] The determining module 1301 is also used to determine the number M of GPU memory pages for the first role when the first GPU memory is less than the required GPU memory, wherein M is calculated based on the first GPU memory and the required GPU memory, and M is a positive integer;

[0236] The adjustment module 1302 is also used to allocate M GPU memory pages to adjust the size of the first buffer to the target size.

[0237] In one possible implementation, M is calculated based on the first GPU memory, the required GPU memory, and the added page size of the added GPU memory pages, wherein the added page size of the added GPU memory pages is configured by the user.

[0238] In one possible implementation, the device further includes:

[0239] The second receiving module is used to receive a role removal request, wherein the role removal request includes a second identifier of the second role to be removed;

[0240] A marking module is used to mark the second GPU memory in the first buffer corresponding to the second role as removed GPU memory;

[0241] The release module is used to release N GPU memory pages when no other data required for character visualization is stored in at least one of the N GPU memory pages corresponding to the second GPU memory and at least one of the N GPU memory pages is not the initial GPU memory page, where the initial GPU memory page includes the initially allocated GPU memory page and N is a positive integer.

[0242] In one possible implementation, the determining module 1301 is further configured to determine the size of the initial buffer, wherein the initial buffer is contiguous in the virtual address space;

[0243] The device also includes:

[0244] Create a module to create an initial buffer based on the size of the initial buffer.

[0245] In one possible implementation, the determining module is also used for:

[0246] The minimum size is determined as the size of the initial buffer; or

[0247] The size configured by the user is determined as the size of the initial buffer, wherein the size configured by the user is included in the first configuration instruction received by the third receiving module of the device.

[0248] In one possible implementation, the size of the initial buffer includes the page size of the initial buffer.

[0249] In one possible implementation, the first configuration instruction also includes a buffer limit and an added page size for the added GPU memory pages, wherein the buffer limit indicates the maximum amount of the first buffer.

[0250] In one possible implementation, the device further includes:

[0251] The fourth receiving module is used to receive the second configuration instruction, wherein the second configuration instruction includes the maximum number of roles configured by the user.

[0252] In one possible implementation, the device further includes:

[0253] The fifth receiving module is used to receive a third configuration instruction, wherein the third configuration instruction includes a selected service function configured by the user. The selected service function is selected from the unlimited service function, the balanced service function, and the priority balanced service function. The unlimited service function indicates that the maximum number of roles allowed by the graphics application is not limited, and guarantees that all roles that need to be added are added to the graphics application. The balanced service function indicates that the maximum number of roles allowed by the graphics application is not limited, but does not guarantee that all roles that need to be added are added to the graphics application. The priority balanced service function indicates that the maximum number of roles allowed by the graphics application is not limited, and prioritizes adding the third roles that need to be added to the graphics application to the service with priority.

[0254] In one possible implementation, the data required for character visualization includes skinned mesh data.

[0255] This invention provides an electronic device that includes a processing circuit for performing any of the memory management methods described above.

[0256] In one possible implementation, the electronic device may include a transceiver, a processor, and a memory. The memory may be used to store code, instructions, etc., executed by the processor.

[0257] It should be understood that the processor can be an integrated circuit chip and has signal processing capabilities. In implementation, the various steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), a system-on-chip (SoC), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor, or it can be any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly executed and completed by a hardware decoding processor, or they can be executed and completed by a combination of hardware and software modules in the decoding processor. The software modules can be located in mature storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with the hardware in the processor, completes the steps of the above methods.

[0258] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0259] It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0260] An embodiment of this application also provides a computing device cluster including processing circuitry for performing any of the memory management methods described above.

[0261] One embodiment of this application also provides a chip including an input / output interface and a processor, wherein the processor is used to call and run a computer program stored in a memory, so that a device with the chip installed performs any of the memory management methods described above.

[0262] Figure 14 This is a schematic structural diagram of a graphics processing unit (GPU) driver provided in one embodiment of this application.

[0263] An embodiment of this application also provides a GPU driver 1400, which includes a processor 1401 and a memory 1402 coupled to the processor 1401 for storing a program executed by the processor 1401, wherein the program, when executed by the processor 1401, causes the GPU driver 1400 to perform any of the memory management methods described above.

[0264] One embodiment of this application also provides a server system comprising: the aforementioned graphics processing unit (GPU) driver, one or more CPUs, and multiple GPUs.

[0265] An embodiment of this application also provides a non-transitory computer-readable medium carrying program code that, when executed by a computer device, causes the computer device to perform any of the memory management methods described above.

[0266] Alternatively, the storage medium may specifically be a memory.

[0267] An embodiment of this application also provides a computer program product comprising program code for performing any of the memory management methods described above.

[0268] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electrical hardware, or a combination of computer software and electrical hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0269] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the above-described systems, devices, and units can be referred to the corresponding processes in the above-described method embodiments. Further details will not be repeated here.

[0270] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the described apparatus embodiments are merely illustrative. For example, the unit division is only a logical functional division, and other division methods may be used in actual implementation. For example, multiple units or components may be merged or integrated into another system, or some features may be ignored or not performed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be implemented through some interfaces. Indirect coupling or communication connection between devices or units can be implemented electronically, mechanically, or in other ways.

[0271] The units described as individual components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected based on actual needs to achieve the purpose of this embodiment.

[0272] In addition, the functional units in the embodiments of this application can be integrated into a processing unit. Each unit in the unit can exist physically separately, or two or more units can be integrated into a unit.

[0273] If implemented as software functional units and sold or used as independent products, these functions can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of this application, either substantially or contributing to the prior art, or some of the technical solutions within it, can be implemented in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to instruct a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the embodiments of this application. The aforementioned storage medium includes any medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0274] The above description is merely a specific implementation of this application and is not intended to limit the scope of protection of this application. Any variations or substitutions readily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0275] Although the present invention describes methods and processes by steps performed in a certain order, one or more steps in the methods and processes may be omitted or modified as appropriate. Where appropriate, one or more steps may be performed in an order other than that described.

[0276] It should be noted that the expression "at least one of A or B" as used herein is interchangeable with the expression "A and / or B". This refers to a list where either A or B, or both A and B, can be selected. Similarly, "at least one of A, B, or C" as used herein is interchangeable with "A and / or B and / or C" or "A, B, and / or C". This refers to a list where: either A or B or C, or both A and B, or both A and C, or both B and C, or all of A, B, and C. The same principle applies to longer lists with the same format.

[0277] Although the invention has been described at least partially in terms of method, those skilled in the art will understand that the invention is also directed to various components for performing at least some aspects and features of the method, whether by hardware components, software, or any combination thereof. Accordingly, the technical solutions of the invention can be embodied in the form of a software product. Suitable software products can be stored in pre-recorded storage devices or other similar non-volatile or non-transitory computer-readable media, including DVDs, CD-ROMs, USB flash drives, removable hard drives, or other storage media. The software product includes instructions tangibly stored thereon that cause a processing device (e.g., a personal computer, server, or network device) to perform examples of the methods disclosed herein. Machine-executable instructions can be in the form of sequences of code, configuration information, or other data that, when executed, cause a machine (e.g., a processor or other processing device) to perform the steps of the methods according to examples of the invention.

[0278] The invention may be embodied in other specific forms without departing from the subject matter of the claims. The exemplary embodiments described are merely illustrative in all respects and not restrictive. Features selected from one or more of the foregoing embodiments may be combined to create alternative embodiments not explicitly described, and features suitable for such combinations will be understood within the scope of the invention.

Claims

1. A memory management method, characterized in that, include: Determine the amount of data required for character visualization in a graphical application; The target size of the first buffer is determined based on the amount of data required for the visualization of the character, wherein the first buffer is contiguous in the virtual address space; Adjust the size of the first buffer used to store the data required for the visualization of the character according to the target size.

2. The method according to claim 1, characterized in that, Also includes: Receive a role creation request, wherein the role creation request includes a first identifier of a first role to be created; The amount of data required to determine the character visualization in the graphical application includes: Determine the amount of data required for the visualization of the first role; Determining the target size of the first buffer in the virtual address space based on the amount of data required for the character visualization includes: Calculate the required graphics processing unit (GPU) memory based on the amount of data required for the visualization of the first role; The target size of the first buffer is determined based on the required GPU memory.

3. The method according to claim 2, characterized in that, Also includes: Determine whether the first GPU memory in the first buffer is greater than or equal to the required GPU memory; When the first GPU memory is greater than or equal to the required GPU memory, the data required for the visualization of the first role is stored in the first GPU memory; Determining the target size of the first buffer based on the required GPU memory includes: When the first GPU memory is less than the required GPU memory, the number M of GPU memory pages for the first role is determined, where M is calculated based on the first GPU memory and the required GPU memory, and M is a positive integer; Adjusting the size of the first buffer used to store the data required for the character visualization according to the target size includes: Allocate M GPU memory pages to adjust the size of the first buffer to the target size.

4. The method according to claim 3, characterized in that, M is calculated based on the first GPU memory, the required GPU memory, and the added page size of the added GPU memory pages, wherein the added page size of the added GPU memory pages is configured by the user.

5. The method according to any one of claims 1 to 4, characterized in that, Also includes: Receive a role removal request, wherein the role removal request includes a second identifier of the second role to be removed; Mark the second GPU memory in the first buffer corresponding to the second role as removed GPU memory; When no other character visualization data is stored in at least one of the N GPU memory pages corresponding to the second GPU memory, and the at least one of the N GPU memory pages is not the initial GPU memory page, the at least one of the N GPU memory pages is released, wherein the initial GPU memory page includes the initially allocated GPU memory page, and N is a positive integer.

6. The method according to any one of claims 1 to 5, characterized in that, Also includes: Determine the size of an initial buffer, wherein the initial buffer is contiguous in the virtual address space; The initial buffer is created according to the size of the initial buffer.

7. The method according to claim 6, characterized in that, Determining the size of the initial buffer includes: The minimum size is determined to be the size of the initial buffer; or Receive a first configuration instruction, wherein the first configuration instruction includes a user-configured size; determine the user-configured size as the size of the initial buffer.

8. The method according to claim 7, characterized in that, The size of the initial buffer includes the page size of the initial buffer.

9. The method according to claim 7 or 8, characterized in that, The first configuration instruction also includes a buffer limit and an added page size for the added GPU memory pages, wherein the buffer limit indicates the maximum amount of the first buffer.

10. The method according to any one of claims 1 to 9, characterized in that, Also includes: Receive a second configuration instruction, wherein the second configuration instruction includes the maximum number of roles configured by the user.

11. The method according to any one of claims 1 to 10, characterized in that, Also includes: A third configuration instruction is received, wherein the third configuration instruction includes a selected service function configured by the user, the selected service function being selected from an unlimited service function, a balanced service function, and a priority balanced service function; the unlimited service function indicates that the maximum number of roles allowed by the graphical application is not limited, and guarantees that all necessary roles are added to the graphical application; the balanced service function indicates that the maximum number of roles allowed by the graphical application is not limited, but does not guarantee that all necessary roles are added to the graphical application; the priority balanced service function indicates that the maximum number of roles allowed by the graphical application is not limited, and prioritizes adding the third roles that need to be added to the graphical application to services with priority.

12. The method according to any one of claims 1 to 11, characterized in that, The data required for character visualization includes skinning mesh data.

13. A memory management device, characterized in that, include: The determination module is used to determine the amount of data required for character visualization in a graphical application; The target size of the first buffer is determined based on the amount of data required for the visualization of the character, wherein the first buffer is contiguous in the virtual address space; An adjustment module is used to adjust the size of the first buffer used to store the data required for the visualization of the character according to the target size.

14. The apparatus according to claim 13, characterized in that, Also includes: The first receiving module is used to receive a role creation request, wherein the role creation request includes a first identifier of a first role to be created; The determining module is further configured to determine the amount of data required for the visualization of the first character; calculate the required graphics processing unit (GPU) memory based on the amount of data required for the visualization of the first character; and determine the target size of the first buffer based on the required GPU memory.

15. The apparatus according to claim 14, characterized in that, The determining module is further configured to determine whether the first GPU memory in the first buffer is greater than or equal to the required GPU memory; The device further includes: A storage module is used to store the data required for the visualization of the first role into the first GPU memory when the first GPU memory is greater than or equal to the required GPU memory. The determining module is further configured to determine the number M of GPU memory pages for the first role when the first GPU memory is less than the required GPU memory, wherein M is calculated based on the first GPU memory and the required GPU memory, and M is a positive integer; The adjustment module is also used to allocate M GPU memory pages to adjust the size of the first buffer to the target size.

16. The apparatus according to claim 15, characterized in that, M is calculated based on the first GPU memory, the required GPU memory, and the added page size of the added GPU memory pages, wherein the added page size of the added GPU memory pages is configured by the user.

17. The apparatus according to any one of claims 13 to 16, characterized in that, Also includes: The second receiving module is used to receive a role removal request, wherein the role removal request includes a second identifier of the second role to be removed; A marking module is used to mark the second GPU memory in the first buffer corresponding to the second role as removed GPU memory; A release module is configured to release the N GPU memory pages when no other data required for character visualization is stored in at least one of the N GPU memory pages corresponding to the second GPU memory and the at least one of the N GPU memory pages is not the initial GPU memory page, wherein the initial GPU memory page includes the initially allocated GPU memory page and N is a positive integer.

18. The apparatus according to any one of claims 13 to 17, characterized in that, The determining module is further configured to determine the size of an initial buffer, wherein the initial buffer is contiguous in the virtual address space; The device further includes: A creation module is used to create the initial buffer according to the size of the initial buffer.

19. The apparatus according to claim 18, characterized in that, The determining module is also used for: The minimum size is determined to be the size of the initial buffer; or The size of the user-configured buffer is determined as the size of the initial buffer, wherein the size of the user-configured buffer is included in the first configuration instruction received by the third receiving module of the device.

20. The apparatus according to claim 19, characterized in that, The size of the initial buffer includes the page size of the initial buffer.

21. The apparatus according to claim 19 or 20, characterized in that, The first configuration instruction also includes a buffer limit and an added page size for the added GPU memory pages, wherein the buffer limit indicates the maximum amount of the first buffer.

22. The apparatus according to any one of claims 13 to 21, characterized in that, Also includes: The fourth receiving module is used to receive a second configuration instruction, wherein the second configuration instruction includes the maximum number of roles configured by the user.

23. The apparatus according to any one of claims 13 to 22, characterized in that, Also includes: The fifth receiving module is configured to receive a third configuration instruction, wherein the third configuration instruction includes a selected service function configured by the user. The selected service function is selected from an unlimited service function, a balanced service function, and a priority balanced service function. The unlimited service function indicates that the maximum number of roles allowed by the graphical application is not limited, and guarantees that all necessary roles are added to the graphical application. The balanced service function indicates that the maximum number of roles allowed by the graphical application is not limited, but does not guarantee that all necessary roles are added to the graphical application. The priority balanced service function indicates that the maximum number of roles allowed by the graphical application is not limited, and prioritizes adding the third roles that need to be added to the graphical application to services with priority.

24. The apparatus according to any one of claims 13 to 23, characterized in that, The data required for character visualization includes skinning mesh data.

25. An electronic device, characterized in that, It includes a processing circuit for performing the method according to any one of claims 1 to 12.

26. A computing device cluster, characterized in that, It includes a processing circuit for performing the method according to any one of claims 1 to 12.

27. A chip, characterized in that, It includes an input / output (I / O) interface and a processor, wherein the processor is used to call and run a computer program stored in a memory to cause a device on which the chip is mounted to perform the method according to any one of claims 1 to 12.

28. A graphics processing unit (GPU) driver, characterized in that, include: processor; A memory coupled to the processor and storing a program executed by the processor, wherein, when executed by the processor, the program causes the GPU driver to perform the method according to any one of claims 1 to 12.

29. A server system, characterized in that, It includes a graphics processing unit (GPU) driver as described in claim 28, one or more CPUs, and multiple GPUs.

30. A non-transitory computer-readable medium carrying program code, characterized in that, When the program code is executed by a computer device, it causes the computer device to perform the method according to any one of claims 1 to 12.

31. A computer program product including program code, characterized in that, The program code is used to perform the method according to any one of claims 1 to 12.

Images