Video memory management methods, devices, electronic devices, storage media, and software products

Abstract

This disclosure relates to a video memory management method, apparatus, electronic device, storage medium, and program product. The method includes: creating a target user memory context corresponding to a target user process, wherein the target user memory context is used to manage a target virtual address space and a target video memory space corresponding to the target user process; for at least two GPU firmwares used to execute tasks of the target user process, allocating virtual address subspaces in the target virtual address space to the at least two GPU firmwares respectively; for any one of the at least two GPU firmwares, the GPU firmware updates the page table information of the firmware memory context corresponding to the GPU firmware according to the page table information of the target user memory context and the virtual address subspace corresponding to the GPU firmware.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a video memory management method, video memory management device, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] GPU (Graphics Processing Unit) firmware collaboration refers to the process by which different firmware components within a GPU cooperate to complete graphics processing or other computational tasks. This collaborative working mode can improve the overall performance and efficiency of the system, especially when handling complex or large-scale graphics rendering and computational tasks. Memory management is crucial in multi-GPU firmware collaboration, as it not only relates to the effective utilization of memory resources but also directly affects the system's performance and stability. Summary of the Invention

[0003] This disclosure provides a video memory management technology solution.

[0004] According to one aspect of this disclosure, a video memory management method is provided, comprising:

[0005] Create a target user memory context corresponding to the target user process, wherein the target user memory context is used to manage the target virtual address space and the target video memory space corresponding to the target user process;

[0006] For at least two GPU firmwares used to execute the task of the target user process, a virtual address subspace in the target virtual address space is allocated to each of the at least two GPU firmwares.

[0007] For any one of the at least two GPU firmwares, the GPU firmware updates the page table information of the firmware memory context corresponding to the GPU firmware based on the page table information of the target user memory context and the virtual address subspace corresponding to the GPU firmware.

[0008] In one possible implementation, there is a one-to-one correspondence between user processes and user memory contexts, and a one-to-one correspondence between GPU firmware and firmware memory contexts.

[0009] In one possible implementation, the method further includes:

[0010] For any of the at least two GPU firmwares, the target virtual address corresponding to the task allocated to the GPU firmware is converted into a target physical address according to the page table information of the firmware memory context corresponding to the GPU firmware, and the target video memory space is accessed according to the target physical address, wherein the target virtual address is in the virtual address subspace corresponding to the GPU firmware.

[0011] In one possible implementation, the method further includes:

[0012] Within the target video memory space, a synchronization video memory space is allocated, wherein the synchronization video memory space is used for synchronization between tasks of the at least two GPU firmwares.

[0013] In one possible implementation, the virtual address subspaces corresponding to the at least two GPU firmwares respectively include the virtual address ranges corresponding to the synchronous video memory space.

[0014] In one possible implementation, the method further includes:

[0015] For the first GPU firmware in the at least two GPU firmwares, a mapping relationship between the first virtual address range and the physical address range of the synchronous video memory space is added to the page table information of the firmware memory context corresponding to the first GPU firmware, wherein the first virtual address range is a segment of virtual address in the virtual address subspace corresponding to the first GPU firmware.

[0016] For any GPU firmware other than the first GPU firmware among the at least two GPU firmwares, the first virtual address range is added to the virtual address subspace corresponding to the GPU firmware, and a mapping relationship between the first virtual address range and the physical address range of the synchronous video memory space is added to the page table information of the firmware memory context corresponding to the GPU firmware.

[0017] In one possible implementation, the method further includes:

[0018] Return the handle of the synchronization resource in the synchronization memory space to the target user process.

[0019] According to one aspect of this disclosure, a video memory management device is provided, comprising:

[0020] A creation module is used to create a target user memory context corresponding to a target user process, wherein the target user memory context is used to manage the target virtual address space and the target video memory space corresponding to the target user process;

[0021] The first allocation module is configured to allocate virtual address subspaces in the target virtual address space to at least two GPU firmwares for executing the task of the target user process, respectively;

[0022] An update module is configured to, for any one of the at least two GPU firmwares, update the page table information of the firmware memory context corresponding to the GPU firmware based on the page table information of the target user memory context and the virtual address subspace corresponding to the GPU firmware.

[0023] In one possible implementation, there is a one-to-one correspondence between user processes and user memory contexts, and a one-to-one correspondence between GPU firmware and firmware memory contexts.

[0024] In one possible implementation, the device further includes:

[0025] The conversion and access module is used to, for any one of the at least two GPU firmwares, convert the target virtual address corresponding to the task allocated to the GPU firmware into a target physical address according to the page table information of the firmware memory context corresponding to the GPU firmware, and access the target video memory space according to the target physical address, wherein the target virtual address is in the virtual address subspace corresponding to the GPU firmware.

[0026] In one possible implementation, the device further includes:

[0027] The second allocation module is used to allocate synchronous video memory space in the target video memory space, wherein the synchronous video memory space is used for synchronization between tasks of the at least two GPU firmwares.

[0028] In one possible implementation, the virtual address subspaces corresponding to the at least two GPU firmwares respectively include the virtual address ranges corresponding to the synchronous video memory space.

[0029] In one possible implementation, the device further includes:

[0030] The first adding module is used to add a mapping relationship between a first virtual address range and the physical address range of the synchronous video memory space in the page table information of the firmware memory context corresponding to the first GPU firmware among the at least two GPU firmwares, wherein the first virtual address range is a segment of virtual address in the virtual address subspace corresponding to the first GPU firmware.

[0031] The second adding module is used to add the first virtual address range to the virtual address subspace corresponding to the GPU firmware for any GPU firmware other than the first GPU firmware among the at least two GPU firmwares, and to add the mapping relationship between the first virtual address range and the physical address range of the synchronous video memory space in the page table information of the firmware memory context corresponding to the GPU firmware.

[0032] In one possible implementation, the device further includes:

[0033] The return module is used to return the handle of the synchronization resource in the synchronization memory space to the target user process.

[0034] According to one aspect of this disclosure, an electronic device is provided, comprising: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the executable instructions stored in the memory to perform the method described above.

[0035] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the above-described method.

[0036] According to one aspect of this disclosure, a computer program product is provided, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in an electronic device, a processor in the electronic device performs the above-described method.

[0037] In this embodiment, a target user memory context corresponding to a target user process is created. This target user memory context manages the target virtual address space and target video memory space corresponding to the target user process. For at least two GPU firmwares used to execute the task of the target user process, virtual address subspaces within the target virtual address space are allocated to each of the at least two GPU firmwares. For any one of the at least two GPU firmwares, the GPU firmware updates the page table information of its corresponding firmware memory context based on the page table information of the target user memory context and the virtual address subspace of the GPU firmware. Therefore, compared to the related art method of multiple user memory contexts corresponding to multiple GPU firmwares, this embodiment creates a unified user memory context for the user process, which manages the virtual address space and video memory space corresponding to the user process. This enables unified scheduling of multiple GPU firmwares by a single user process. That is, although multiple GPU firmwares may participate in task execution, they can coordinate their work through a shared user memory context to ensure successful task execution.

[0038] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0039] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0041] Figure 1 A flowchart illustrating the video memory management method provided in an embodiment of this disclosure is shown.

[0042] Figure 2 This diagram illustrates a strategy diagram of the user memory context under multi-GPU firmware collaborative operation in the video memory management method provided in this embodiment of the present disclosure.

[0043] Figure 3 This diagram illustrates a strategy diagram of firmware memory context under multi-GPU firmware collaborative operation in the video memory management method provided in this embodiment of the present disclosure.

[0044] Figure 4 A block diagram of a video memory management device provided in an embodiment of this disclosure is shown.

[0045] Figure 5A block diagram of an electronic device 1900 provided in an embodiment of this disclosure is shown. Detailed Implementation

[0046] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0047] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0048] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0049] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0050] In GPU firmware collaboration schemes in related technologies, task allocation is typically handled by the CPU (Central Processing Unit), i.e., the host side. The CPU breaks down the tasks and distributes them to the GPU side for execution by the GPU firmware. Specifically, the CPU transfers task data to the GPU's video memory (VRAM), and the GPU firmware executes these tasks by accessing VRAM. In GPU firmware VRAM management, VRAM is usually divided into independent regions, with each GPU firmware having its own dedicated VRAM space, and each GPU firmware's user memory context is also independent. This design is effective in scenarios where multiple GPU firmwares independently occupy VRAM resources because it simplifies VRAM management strategies and eliminates the need to consider VRAM sharing issues between different GPU firmwares.

[0051] However, with technological advancements, scenarios where multiple GPU firmwares share the same GPU memory resources are becoming increasingly common. In such scenarios, traditional memory management strategies are no longer applicable. During the collaborative work of GPU firmware, tasks between different GPU firmwares may have dependencies. Therefore, an effective mechanism is needed to manage and coordinate these dependencies to ensure the sequential execution of tasks.

[0052] To address the technical problems described above, this disclosure provides a video memory management method. By creating a target user memory context corresponding to a target user process, the target user memory context manages the target virtual address space and the target video memory space corresponding to the target user process. For at least two GPU firmwares used to execute the task of the target user process, virtual address subspaces within the target virtual address space are allocated to each of the at least two GPU firmwares. For any one of the at least two GPU firmwares, the GPU firmware updates the page table information of its corresponding firmware memory context based on the page table information of the target user memory context and the virtual address subspace of the GPU firmware. Therefore, compared to the related art method of multiple user memory contexts corresponding to multiple GPU firmwares, this disclosure creates a unified user memory context for the user process, which manages the virtual address space and video memory space corresponding to the user process. This enables unified scheduling of multiple GPU firmwares by a single user process. That is, although multiple GPU firmwares may participate in task execution, they can coordinate their work through a shared user memory context to ensure successful task execution.

[0053] The memory management method provided in the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0054] Figure 1 A flowchart illustrating a video memory management method provided in an embodiment of this disclosure is shown. In one possible implementation, the execution entity of the video memory management method can be a video memory management device. For example, the video memory management method can be executed by a terminal device, a server, or other electronic devices. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, or wearable device, etc. In some possible implementations, the video memory management method can be implemented by a processor calling computer-readable instructions stored in memory. Figure 1 As shown, the video memory management method includes steps S11 to S13.

[0055] In step S11, a target user memory context corresponding to the target user process is created, wherein the target user memory context is used to manage the target virtual address space and the target video memory space corresponding to the target user process.

[0056] In step S12, for at least two GPU firmwares used to execute the task of the target user process, a virtual address subspace in the target virtual address space is allocated to each of the at least two GPU firmwares.

[0057] In step S13, for any one of the at least two GPU firmwares, the GPU firmware updates the page table information of the firmware memory context corresponding to the GPU firmware based on the page table information of the target user memory context and the virtual address subspace corresponding to the GPU firmware.

[0058] In this embodiment of the disclosure, the target user process can be any user process. A user process can refer to a process that is started by the user in the operating system or created by the system for the user.

[0059] A target user memory context can represent the user memory context corresponding to a target user process. In this embodiment of the disclosure, a corresponding user memory context can be created for each user process, and any user memory context can be used to manage and track the video memory usage of the corresponding user process. For example, the target user memory context can be used to manage and track the video memory usage of the target user process.

[0060] The user memory context manages the virtual address space of a user process. The virtual address space represents the range of virtual addresses used by a user process during execution, and it is mapped to physical video memory. Through the virtual address space, a user process can access its corresponding video memory space without needing to know the actual layout of the physical video memory. For example, the target user memory context manages the target virtual address space of the target user process. Here, the target virtual address space represents the virtual address space of the target user process.

[0061] In this embodiment of the disclosure, the target video memory space can represent the video memory space corresponding to the target user process. The video memory space corresponding to any user process can represent the space allocated for the user process in video memory, which can be used to store graphics data, textures, frame buffers, and other information.

[0062] The user memory context corresponding to any user process can be used to manage the entire video memory space allocated to that user process. For example, the target user memory context corresponding to the target user process can be used to manage the entire video memory space allocated to the target user process, i.e., the target video memory space. For the target user process, by using a unified target user memory context (i.e., the target user process corresponds to only one user memory context, i.e., the target user memory context), it is possible to achieve unified scheduling of multiple GPU firmware by the target user process.

[0063] The user memory context includes page table information, which can include the mapping between the virtual address space and video memory space corresponding to the user process. For example, the page table information of the target user memory context can include the mapping between the target virtual address space and the target video memory space. In the case of multiple GPU firmware sharing video memory, the page table information of the user memory context can be shared by all relevant GPU firmware, so that each GPU firmware can correctly access video memory resources. For example, when the target user process uses multiple GPU firmware to execute tasks, the multiple GPU firmwares can share the page table information of the target user memory context, so that each GPU firmware can correctly access the target video memory space.

[0064] In this embodiment of the disclosure, the page table information of the target user memory context can be passed to each GPU firmware, so that each GPU firmware knows how to translate the virtual address into the actual physical address of the video memory.

[0065] In one possible implementation, a user MMU (Memory Management Unit) context corresponding to the target user process can also be created. This user MMU context can contain memory layout information of the target user process, as well as information controlling memory access permissions, such as which memory regions are readable, writable, or executable.

[0066] In this embodiment, multiple virtual address subspaces can be divided from the target virtual address space corresponding to the target user process, and each virtual address subspace can be allocated to different GPU firmware. In this embodiment, apart from the synchronized video memory space, the virtual address subspaces corresponding to each GPU firmware can be independent, thereby ensuring isolation between the virtual address subspaces corresponding to different GPU firmware. That is, each GPU firmware is provided with an independent working environment, allowing them to safely execute tasks without worrying about address conflicts or data corruption. Although each GPU firmware has an independent virtual address subspace, they can share physical video memory. That is, multiple GPU firmwares can share the target video memory space without needing to allocate dedicated physical video memory space to each GPU firmware separately.

[0067] In this embodiment of the disclosure, in an environment where multiple GPU firmwares share the same GPU memory resources, independent firmware memory contexts can be created for each GPU firmware.

[0068] In this embodiment of the disclosure, for any GPU firmware, the GPU firmware can determine the mapping relationship between the virtual address subspace corresponding to the GPU firmware and the actual physical address of the video memory based on the page table information of the target user memory context, and can update the mapping relationship to the page table information of the firmware memory context corresponding to the GPU firmware. Thus, the GPU firmware can access the task data written by the CPU to the video memory based on the page table information of the firmware memory context corresponding to the GPU firmware to execute the corresponding task.

[0069] In one possible implementation, there is a one-to-one correspondence between user processes and user memory contexts, and a one-to-one correspondence between GPU firmware and firmware memory contexts.

[0070] In this implementation, there is a one-to-one correspondence between user processes and user memory contexts; that is, each user process corresponds to only one user memory context, and different user processes correspond to different user memory contexts. For any given user process, by using a unified user memory context, unified scheduling of multiple GPU firmware can be achieved.

[0071] In this implementation, there is a one-to-one correspondence between GPU firmware and firmware memory context; that is, each GPU firmware corresponds to only one firmware memory context, and different GPU firmwares correspond to different firmware memory contexts. The page table information in the firmware memory context corresponding to any GPU firmware can include the mapping relationship between the virtual address and the physical address of the GPU firmware.

[0072] In one possible implementation, the method further includes: for any one of the at least two GPU firmwares, converting the target virtual address corresponding to the task allocated to the GPU firmware into a target physical address according to the page table information of the firmware memory context corresponding to the GPU firmware, and accessing the target video memory space according to the target physical address, wherein the target virtual address is in the virtual address subspace corresponding to the GPU firmware.

[0073] In this implementation, when a task is assigned to the GPU firmware for execution, this task has one or more virtual addresses within the GPU firmware's virtual address subspace. The GPU firmware can use the page table information of its firmware memory context to determine the physical address corresponding to the virtual address. The page table information of the firmware memory context contains the mapping relationship between virtual addresses and physical addresses in the virtual address subspace corresponding to the GPU firmware, enabling the GPU firmware to know how to locate task data in the GPU's video memory. By querying the page table information of the firmware memory context corresponding to the GPU firmware, the GPU firmware can translate the virtual address into an actual physical address. This physical address is the actual location in video memory, where the data required by the GPU firmware to execute the task is stored. Once the physical address is determined, the GPU firmware can access the correct location in video memory to read or write data to execute the task assigned to it.

[0074] By adopting this implementation method, each GPU firmware can securely and efficiently access the required video memory resources in a multi-GPU firmware environment, while avoiding address conflicts and data corruption. Furthermore, this implementation method enables the system to support complex graphics rendering and computational tasks, improving overall performance and efficiency.

[0075] In one possible implementation, the method further includes: allocating a synchronization memory space in the target video memory space, wherein the synchronization memory space is used for synchronization between tasks of the at least two GPU firmwares.

[0076] In multi-GPU environments, different GPU firmwares may need to collaborate to complete complex graphics rendering or computation tasks, thus requiring a mechanism to synchronize the execution of these tasks. In this implementation, a portion of the target video memory can be allocated as synchronization memory. Synchronization memory can represent a dedicated video memory area for managing task synchronization between different GPU firmwares.

[0077] As an example of this implementation, the driver can allocate synchronous memory space in the target video memory space in response to a command issued by the user process to allocate synchronous resources.

[0078] During GPU firmware collaboration, tasks between different GPU firmware may have dependencies, rendering the synchronization strategies used in related technologies inapplicable. In this implementation, synchronization memory space is allocated within the target video memory space, allowing synchronization signals, status flags, or other data structures used for synchronization to be stored.

[0079] Different GPU firmware can coordinate their task execution by reading and writing data in the synchronized video memory space. For example, after one GPU firmware completes a task, it can update a flag in the synchronized video memory space to notify other GPU firmware that depends on this task that it can begin executing its own tasks.

[0080] This implementation provides a flexible synchronization mechanism by allocating synchronized video memory space within the target video memory space. This mechanism can adapt to different task dependencies and execution orders. In a multi-GPU environment, the synchronization mechanism ensures the correctness of task execution order and data access, preventing data races and inconsistencies. Compared to complex software synchronization logic, using synchronized video memory space provides a hardware-level synchronization solution, simplifying system design. This implementation supports multi-GPU firmware collaborative processing of complex or large-scale graphics rendering and computation tasks, improving the system's ability to handle complex tasks. Furthermore, effective synchronization reduces system errors caused by improper task execution order or data access conflicts, enhancing system stability and reliability.

[0081] As an example of this implementation, the virtual address subspaces corresponding to the at least two GPU firmwares respectively include the virtual address ranges corresponding to the synchronous video memory space.

[0082] In this example, in a scenario where multiple GPU firmwares work together, synchronized video memory space is allocated and mapped into the virtual address subspace of each GPU firmware. That is, the firmware memory context of each GPU firmware can be updated to include the virtual address range of the synchronized video memory space and its corresponding physical address mapping.

[0083] In this example, each GPU firmware's virtual address subspace contains not only its own task data but also the virtual address range of the synchronization memory space. This allows each GPU firmware to access the synchronization memory space through its virtual address subspace. By mapping the synchronization memory space to the virtual address subspaces of all GPU firmware, all GPU firmware participating in the collaborative process can access the same physical memory space. This allows GPU firmware to exchange state information and synchronization signals through the synchronization memory space, thereby coordinating their task execution.

[0084] In this example, by sharing the synchronization memory space, different GPU firmwares can more effectively coordinate their task execution, avoiding issues such as incorrect execution order or data inconsistency. Furthermore, this implementation allows the system to adapt to new synchronization requirements when adding more GPU firmware by simply expanding the virtual address subspace and updating the page table mapping, without requiring large-scale modifications to the existing architecture. Moreover, sharing the synchronization memory space reduces the amount of synchronization resources that each GPU firmware needs to manage independently, thereby optimizing the utilization of memory resources.

[0085] As an example of this implementation, the method further includes: for the first GPU firmware among the at least two GPU firmwares, adding a mapping relationship between a first virtual address range and the physical address range of the synchronous video memory space in the page table information of the firmware memory context corresponding to the first GPU firmware, wherein the first virtual address range is a segment of virtual addresses in the virtual address subspace corresponding to the first GPU firmware; for any GPU firmware other than the first GPU firmware among the at least two GPU firmwares, adding the first virtual address range to the virtual address subspace corresponding to the GPU firmware, and adding a mapping relationship between the first virtual address range and the physical address range of the synchronous video memory space in the page table information of the firmware memory context corresponding to the GPU firmware.

[0086] In this example, the first GPU firmware can be any one of the at least two GPU firmwares.

[0087] In one example, after the driver allocates synchronous memory space in the target video memory space, it can map the physical address range of the synchronous memory space to the first virtual address range in the virtual address subspace corresponding to the first GPU firmware.

[0088] For any GPU firmware other than the first GPU firmware among the at least two GPU firmwares, a first virtual address range can be added to the virtual address subspace corresponding to the GPU firmware, and a mapping relationship between the first virtual address range and the physical address range of the synchronous video memory space can be added in the virtual address subspace corresponding to the GPU firmware and in the page table information of the firmware memory context corresponding to the GPU firmware.

[0089] In this example, the virtual address subspaces corresponding to each GPU firmware include the same first virtual address range, and the first virtual address range in the virtual address subspaces corresponding to each GPU firmware corresponds to the same physical address range, that is, the physical address range of the synchronous video memory space.

[0090] Therefore, the at least two GPU firmwares can access the synchronized memory space using the same virtual address. This configuration ensures the consistency of synchronized resources and coordination between GPU firmwares, allowing user-space control over access to synchronized resources by multiple GPU firmwares through a single handle.

[0091] As another example of this implementation, the method further includes: allocating a first virtual address range corresponding to the synchronous video memory space in the target virtual address space, wherein the first virtual address range is different from the virtual address subspace allocated to each GPU firmware; for any of the at least two GPU firmwares, adding the first virtual address range to the virtual address subspace corresponding to the GPU firmware, and adding a mapping relationship between the first virtual address range and the physical address range of the synchronous video memory space in the page table information of the firmware memory context corresponding to the GPU firmware.

[0092] As an example of this implementation, the method further includes: returning a handle to the synchronized video memory space to the target user process.

[0093] In this implementation, the handle can be a unique identifier used to reference and manage resources. The handle can be used to identify the synchronized video memory space. The handle to the synchronized video memory space can be a reference to that space, and it can contain all the information needed to access the synchronized video memory space, such as its address, size, and access permissions.

[0094] In a multi-GPU firmware environment, once the synchronized video memory space is allocated and configured, its handle can be returned to the target user process. This means the target user process gains access to and control of the synchronized video memory space. The target user process can use the returned handle to manage and coordinate access to the synchronized video memory space by different GPU firmware. For example, the target user process can use the handle to check, request, or release access permissions to synchronized resources. The target user process can use the handle to perform various operations, such as waiting for a certain condition, notifying other GPU firmware that a task has completed, or synchronizing the execution flow of multiple tasks. Through the handle, the target user process can more flexibly control synchronization operations and implement complex synchronization strategies.

[0095] In this implementation, the target user process can directly control the synchronized video memory space through handles, thus enabling more flexible implementation of various synchronization strategies to meet different task execution requirements. The use of handles allows the target user process to effectively coordinate the execution of tasks across multiple GPU firmware instances, ensuring task synchronization and ordering. Synchronization operations via handles reduce the overhead of system calls and context switching, thereby improving the overall system performance and efficiency.

[0096] The memory management method provided in this disclosure can be applied to technical fields such as GPU, firmware collaboration, memory management, and page table sharing, and is not limited thereto.

[0097] The following describes the video memory management method provided in this embodiment through a specific application scenario.

[0098] Figure 2 This diagram illustrates a strategy diagram for user memory context under multi-GPU firmware collaborative operation in the video memory management method provided in this embodiment of the present disclosure. For example... Figure 2 As shown, a target user memory context and a target user MMU context corresponding to the target user process can be created. A first virtual address subspace VA1 and a second virtual address subspace VA2 can be partitioned from the target virtual address space corresponding to the target user process. The first virtual address subspace VA1 can be allocated to the first GPU firmware, and the second virtual address subspace VA2 can be allocated to the second GPU firmware. The first GPU firmware can determine the physical address range PA1 corresponding to the first virtual address subspace VA1 in the target video memory space based on the page table information of the target user memory context; the second GPU firmware can determine the physical address range PA2 corresponding to the second virtual address subspace VA2 in the target video memory space based on the page table information of the target user memory context.

[0099] Figure 3 This diagram illustrates a firmware memory context strategy under multi-GPU firmware collaborative operation in the video memory management method provided in this embodiment. For example... Figure 3 As shown, the driver can respond to a command from the user process to allocate synchronized resources and allocate synchronized video memory space in the target video memory space. The first GPU firmware can add a mapping relationship between the virtual address range VA0 and the physical address range PA0 of the synchronized video memory space in the page table information of the firmware memory context corresponding to the first GPU firmware. The second GPU firmware can obtain the virtual address range VA0 of the synchronized video memory space and add a mapping relationship between the virtual address range VA0 and the physical address range PA0 of the synchronized video memory space in the page table information of the firmware memory context corresponding to the second GPU firmware. Therefore, both the first and second GPU firmware can access the synchronized video memory space. Figure 3 As shown, it can also return a handle to the synchronized video memory space to the target user process.

[0100] This application scenario enables memory management under multi-GPU firmware collaborative work in an environment where multiple firmwares share video memory, solving problems such as memory allocation, GPU firmware task scheduling, data consistency, and multi-GPU firmware synchronization.

[0101] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0102] In addition, this disclosure also provides a video memory management device, an electronic device, a computer-readable storage medium, and a computer program product, all of which can be used to implement any of the video memory management methods provided in this disclosure. The corresponding technical solutions and effects can be found in the relevant descriptions in the method section, and will not be repeated here.

[0103] Figure 4 A block diagram of a video memory management device provided in an embodiment of this disclosure is shown. Figure 4 As shown, the video memory management device includes:

[0104] The creation module 41 is used to create a target user memory context corresponding to the target user process, wherein the target user memory context is used to manage the target virtual address space and the target video memory space corresponding to the target user process;

[0105] The first allocation module 42 is configured to allocate virtual address subspaces in the target virtual address space to at least two GPU firmwares for executing the task of the target user process, respectively;

[0106] The update module 43 is used to update the page table information of the firmware memory context corresponding to the GPU firmware for any of the at least two GPU firmwares, based on the page table information of the target user memory context and the virtual address subspace corresponding to the GPU firmware.

[0107] In one possible implementation, there is a one-to-one correspondence between user processes and user memory contexts, and a one-to-one correspondence between GPU firmware and firmware memory contexts.

[0108] In one possible implementation, the device further includes:

[0109] The conversion and access module is used to, for any one of the at least two GPU firmwares, convert the target virtual address corresponding to the task allocated to the GPU firmware into a target physical address according to the page table information of the firmware memory context corresponding to the GPU firmware, and access the target video memory space according to the target physical address, wherein the target virtual address is in the virtual address subspace corresponding to the GPU firmware.

[0110] In one possible implementation, the device further includes:

[0111] The second allocation module is used to allocate synchronous video memory space in the target video memory space, wherein the synchronous video memory space is used for synchronization between tasks of the at least two GPU firmwares.

[0112] In one possible implementation, the virtual address subspaces corresponding to the at least two GPU firmwares respectively include the virtual address ranges corresponding to the synchronous video memory space.

[0113] In one possible implementation, the device further includes:

[0114] The first adding module is used to add a mapping relationship between a first virtual address range and the physical address range of the synchronous video memory space in the page table information of the firmware memory context corresponding to the first GPU firmware among the at least two GPU firmwares, wherein the first virtual address range is a segment of virtual address in the virtual address subspace corresponding to the first GPU firmware.

[0115] The second adding module is used to add the first virtual address range to the virtual address subspace corresponding to the GPU firmware for any GPU firmware other than the first GPU firmware among the at least two GPU firmwares, and to add the mapping relationship between the first virtual address range and the physical address range of the synchronous video memory space in the page table information of the firmware memory context corresponding to the GPU firmware.

[0116] In one possible implementation, the device further includes:

[0117] The return module is used to return the handle of the synchronization resource in the synchronization memory space to the target user process.

[0118] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation and technical effects can be referred to the description of the above method embodiments. For the sake of brevity, they will not be repeated here.

[0119] This disclosure also provides a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the above-described method. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium.

[0120] This disclosure also proposes a computer program including computer-readable code, wherein when the computer-readable code is run in an electronic device, a processor in the electronic device executes the above-described method.

[0121] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in an electronic device, the processor in the electronic device executes the above-described method.

[0122] This disclosure also provides an electronic device, including: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the executable instructions stored in the memory to perform the above-described method.

[0123] Electronic devices can be provided as terminals, servers, or other forms of devices.

[0124] Figure 5 A block diagram of an electronic device 1900 provided according to an embodiment of this disclosure is shown. For example, the electronic device 1900 may be provided as a server or a terminal. (Refer to...) Figure 5 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0125] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). Electronic device 1900 can operate on an operating system stored in memory 1932, such as Microsoft Server operating system (Windows Server). TM Apple's graphical user interface-based operating system (MacOS X) TM ), a multi-user, multi-process computer operating system (Unix) TM Linux is a free and open-source Unix-like operating system. TM ), the open-source Unix-like operating system (FreeBSD) TM (or similar.)

[0126] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0127] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0128] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0129] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0130] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0131] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0132] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0133] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0135] The computer program product can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0136] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0137] If the technical solution of this disclosure involves personal information, the product applying the technical solution of this disclosure has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this disclosure involves sensitive personal information, the product applying the technical solution of this disclosure has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to indicate that the user has entered the scope of personal information collection and that personal information will be collected. If the user voluntarily enters the collection scope, it is deemed to have consented to the collection of their personal information; or on the personal information processing device, with clear signs / information informing the user of the personal information processing rules, authorization is obtained from the user through pop-up information or by asking the user to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0138] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for managing a video memory, the method comprising: include: Create a target user memory context corresponding to the target user process, wherein the target user memory context is used to manage the target virtual address space and the target video memory space corresponding to the target user process; For at least two GPU firmwares used to execute the task of the target user process, a virtual address subspace in the target virtual address space is allocated to each of the at least two GPU firmwares. For any one of the at least two GPU firmwares, the GPU firmware updates the page table information of the firmware memory context corresponding to the GPU firmware based on the page table information of the target user memory context and the virtual address subspace corresponding to the GPU firmware.

2. The method of claim 1, wherein, There is a one-to-one correspondence between user processes and user memory contexts, and a one-to-one correspondence between GPU firmware and firmware memory contexts.

3. The method of claim 1, wherein, The method further includes: For any of the at least two GPU firmwares, the target virtual address corresponding to the task allocated to the GPU firmware is converted into a target physical address according to the page table information of the firmware memory context corresponding to the GPU firmware, and the target video memory space is accessed according to the target physical address, wherein the target virtual address is in the virtual address subspace corresponding to the GPU firmware.

4. The method of claim 1, wherein, The method further includes: Within the target video memory space, a synchronization video memory space is allocated, wherein the synchronization video memory space is used for synchronization between tasks of the at least two GPU firmwares.

5. The method of claim 4, wherein, The virtual address subspaces corresponding to the at least two GPU firmwares each include the virtual address range corresponding to the synchronous video memory space.

6. The method according to claim 4 or 5, characterized in that, The method further includes: For the first GPU firmware in the at least two GPU firmwares, a mapping relationship between the first virtual address range and the physical address range of the synchronous video memory space is added to the page table information of the firmware memory context corresponding to the first GPU firmware, wherein the first virtual address range is a segment of virtual address in the virtual address subspace corresponding to the first GPU firmware. For any GPU firmware other than the first GPU firmware among the at least two GPU firmwares, the first virtual address range is added to the virtual address subspace corresponding to the GPU firmware, and a mapping relationship between the first virtual address range and the physical address range of the synchronous video memory space is added to the page table information of the firmware memory context corresponding to the GPU firmware.

7. The method according to claim 4 or 5, characterized in that, The method further includes: Return the handle of the synchronization resource in the synchronization memory space to the target user process.

8. A video memory management device, comprising: include: A creation module is used to create a target user memory context corresponding to a target user process, wherein the target user memory context is used to manage the target virtual address space and the target video memory space corresponding to the target user process; The first allocation module is configured to allocate virtual address subspaces in the target virtual address space to at least two GPU firmwares for executing the task of the target user process, respectively; An updating module is configured to update, for any GPU firmware of the at least two GPU firmwares, page table information of a firmware memory context corresponding to the GPU firmware according to page table information of the target user memory context and a virtual address subspace corresponding to the GPU firmware.

9. An electronic device, comprising: Comprise: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to invoke the executable instructions stored in the memory to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by a processor, implement the method of any one of claims 1 to 7.

11. A computer program product comprising computer readable code, or a non-transitory computer readable storage medium having computer readable code embodied thereon, the computer readable code comprising instructions for causing a computer to perform the method of any one of claims 1 to 10. When the computer readable code runs in an electronic device, a processor in the electronic device performs the method of any one of claims 1 to 7.

Images