Processing resource allocation method and computing device

Abstract

The invention relates to the technical field of storage management, and discloses a processing resource allocation method and computing device.The method comprises the steps that a memory allocation request of a first process to a second processor is intercepted, and the memory allocation request is used for requesting the second processor to allocate a storage area of a first storage size for the first process; determining a lock holding state of the first process to a dynamic time slice DTL lock; and under the condition that the first process holds the DTL lock, calling the second processor based on the memory allocation request, so that the second processor allocates a first storage area in the memory of the second processor for the first process. Therefore, process scheduling can be carried out through the DTL lock, and the storage area is managed under the condition that the process holds the DTL lock, so that frequent page immigration and emigration are avoided, and the jitter phenomenon is reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of storage management, and in particular, to a processing resource allocation method and a computing device. Background Art

[0002] In order to optimize the resource allocation method of a processor, processors that support resource sharing have emerged, that is, multiple processes can share the resources of the same processor at the same time, avoiding the inefficiency and latency problems caused by the exclusive allocation of processor resources. During the process of the processor allocating resources to a process, when there is a new process, if the resources currently available to the processor are not enough to meet the resource request of the new process, it is easy to cause an Out of Memory (OOM) error, resulting in the crash of the new process during operation.

[0003] To solve the OOM error, it is necessary to page out the data in the processor to free up space for the new process. However, when these data need to be accessed again, since the data has been paged out, a PageFault will occur, and at this time, the paged-out data needs to be reloaded into the processor.

[0004] However, due to limited space and the fact that each process frequently schedules processor resources, that is, frequently switches the occupation of the processor, the page migration behavior is very frequent and the thrashing phenomenon is serious. Summary of the Invention

[0005] The embodiments of the present application provide a processing resource allocation method and a computing device, which can schedule each process through a DTL lock and can allocate and schedule processing resources to reduce the thrashing phenomenon.

[0006] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, the embodiments of the present application provide a processing resource allocation method, which is applied to a first processor and includes: intercepting a memory allocation request of a first process for a second processor, where the memory allocation request is used to request the second processor to allocate a storage area with a first storage size for the first process; determining the lock-holding state of the first process for a dynamic time slice DTL lock; and when the first process holds the DTL lock, calling the second processor based on the memory allocation request, so that the second processor allocates a first storage area in the second processor memory for the first process.

[0008] Based on this solution, the operation of the process can be controlled through the scheduling of the DTL lock, avoiding the frequent page in and page out caused by the frequent switching of the process.

[0009] In a possible implementation, after determining the lock-holding state of the first process on the dynamic time slice DTL lock, it further includes: when the first process does not hold the DTL lock and the target condition is satisfied, blocking the first process, where the target condition includes that there is a process holding the DTL lock, and / or the holding time slice of at least one second process holding the DTL lock has not expired; or, when the first process does not hold the DTL lock and the target condition is not satisfied, issuing the DTL lock to the first process; when the first process holds the DTL lock and the holding time slice of the DTL lock has expired, blocking the first process.

[0010] Based on this solution, the DTL lock can be managed dynamically in a serialized manner, avoiding frequent process switching.

[0011] In a possible implementation, when the first process holds the DTL lock, calling the second processor based on a memory allocation request to enable the second processor to allocate a first storage area in the second processor's memory for the first process, including: redirecting the memory allocation request to a memory allocation API, where the memory allocation API is used to enable the second processor to allocate a first storage area in the second processor's memory for the first process based on the memory allocation API. In this way, the process of allocating the second processor's memory can be managed flexibly.

[0012] In a possible implementation, before calling the second processor based on a memory allocation request to enable the second processor to allocate a first storage area in the second processor's memory for the first process when the first process holds the DTL lock, it further includes: when the size of the unallocated storage area in the second processor's memory is less than the first storage size, migrating the data in the second processor's memory to the memory of the first processor. When the size of the unallocated storage area in the second processor's memory is less than the first storage size, migrating the data in the second processor's memory to the memory of the first processor specifically includes: when the size of the unallocated storage area in the second processor's memory is less than the first storage size, migrating the first target data in the second processor's memory to the memory of the first processor, where the first target data is the data in the storage area with the lowest occupancy priority in the second processor's memory. In this way, flexible allocation of processing resources can be achieved by flexibly migrating data, reducing the jitter phenomenon.

[0013] In a possible implementation, the method further includes: intercepting a memory release request of a first process for a second processor, where the memory release request is used to request the second processor to release a storage area of a second storage size occupied by the first process; when the first process holds a DTL lock, calling the second processor based on the memory release request to release the storage area of the second storage size in the second processor occupied by the first process. When the first process holds a DTL lock, calling the second processor based on the memory release request to release the storage area of the second storage size in the second processor occupied by the first process includes: redirecting the memory release request to a memory release API, where the memory release API is used to enable the second processor to reclaim the storage area of the second storage size in the second processor occupied by the first process based on the memory release API. Based on this solution, the release of processing resources can be flexibly managed, and resource utilization can be improved.

[0014] In a possible implementation, after calling the second processor based on the memory release request to release the storage area of the second storage size in the second processor occupied by the first process when the first process holds a DTL lock, it further includes: migrating the data in the memory of the first processor to the memory of the second processor. Migrating the data in the memory of the first processor to the memory of the second processor specifically includes: migrating the second target data in the memory of the first processor to the memory of the second processor, where the second target data is the data in the storage area with the highest occupancy priority in the first processor. In this way, after the storage area of the second processor is released, the data in the memory of the first processor can be prefetched to the second processor, improving the utilization rate of the resources of the second processor.

[0015] In a possible implementation, the occupancy priority is positively correlated with the access frequency and / or the latest access time of the process accessing the storage area, and negatively correlated with the size and / or access latency of the storage area. Based on this solution, the priority of each storage area can be overall measured based on various key indicators.

[0016] In a possible implementation, after calling the second processor based on a memory allocation request to allocate a first storage area in the memory of the second processor for the first process when the first process holds a DTL lock, it further includes: generating a storage record instance for the first storage area; the storage record instance includes one or more of the identifier of the first process, a pointer to the first storage area, the size of the first storage area, or the occupancy priority of the first storage area; storing the storage record instance.

[0017] Based on this solution, a storage record instance can be constructed, and thus the memory management efficiency can be improved.

[0018] In a possible implementation, the method further includes: determining the process priority of each process at a first preset frequency, where the process priority is equal to the sum of the occupation priorities of the storage areas occupied by the process; and determining the length of the lock-holding time slice corresponding to each process when holding the DTL lock next time based on the process priority.

[0019] Based on this solution, the lock-holding time slice length can be dynamically adjusted by periodically calculating the process priority, optimizing the memory management strategy.

[0020] In a possible implementation, after determining the length of the lock-holding time slice corresponding to each process when holding the DTL lock next time based on the process priority, it further includes: writing the length of the lock-holding time slice into a configuration file; the method further includes: in response to a first operation, updating the length of the lock-holding time slice corresponding to the target process based on the configuration file; the first operation is to read the configuration file and modify the length of the lock-holding time slice of the target process in the configuration file, and the target process is one of the processes.

[0021] Based on this solution, the dynamic duration can be managed through the configuration file to achieve flexible configuration and update; moreover, the lock-holding time of the process supports modification to meet specific requirements, enhancing adaptability and flexibility.

[0022] In a second aspect, an embodiment of the present application further provides a processing resource allocation device, including: an interception module configured to intercept a memory allocation request of a first process for a second processor, where the memory allocation request is used to request the second processor to allocate a storage area with a first storage size for the first process; and determine the lock-holding state of the first process for the dynamic time slice DTL lock; a control module configured to, when the first process holds the DTL lock, call the second processor based on the memory allocation request, so that the second processor allocates a first storage area in the second processor memory for the first process.

[0023] In a third aspect, an embodiment of the present application further provides a computing device, including: a memory for storing computer instructions; a processor, where the processor includes a first processor and a second processor, the first processor is used to execute the computer instructions to implement the processing resource allocation method in the foregoing first aspect and its various implementation manners, and the second processor is used to allocate a first storage area in the second processor memory for the first process in response to the call of the first processor when the first processor executes the computer instructions.

[0024] In a fourth aspect, an embodiment of the present application provides a chip, and the chip is used to execute the method in any one of the foregoing first aspects.

[0025] Fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a computer, the method according to any one of the first aspect is implemented.

[0026] Sixth aspect, an embodiment of the present application provides a program product, including a computer program, and when the computer program is executed by a processor, the method according to any one of the first aspect is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a system architecture diagram of a computing device provided by an embodiment of the present application;

[0028] Figure 2 It is a scheduling schematic diagram of a controller provided by an embodiment of the present application;

[0029] Figure 3 It is a first flowchart of a processing resource allocation method provided by an embodiment of the present application;

[0030] Figure 4 It is an interaction schematic diagram of a processing resource allocation method provided by an embodiment of the present application;

[0031] Figure 5 It is a schematic diagram of a redirected call request provided by an embodiment of the present application;

[0032] Figure 6 It is a working flowchart of an interceptor provided by an embodiment of the present application;

[0033] Figure 7 It is an interaction schematic diagram between an interceptor and a controller provided by an embodiment of the present application;

[0034] Figure 8 It is a schematic diagram of a priority queue and a hash table provided by an embodiment of the present application;

[0035] Figure 9 It is a flowchart of dynamic memory migration provided by an embodiment of the present application;

[0036] Figure 10 It is a schematic diagram of storage area occupied by process A provided by an embodiment of the present application;

[0037] Figure 11 It is a schematic diagram of storage area occupied by process B provided by an embodiment of the present application;

[0038] Figure 12 It is a second flowchart of a processing resource allocation method provided by an embodiment of the present application;

[0039] Figure 13 It is a structural schematic diagram of a processing resource allocation device provided by an embodiment of the present application;

[0040] Figure 14 Schematic diagram of a computing device provided for some embodiments of the present application. Detailed implementation manners

[0041] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application. To facilitate a clear description of the technical solutions in the embodiments of the present application, the first, second, etc. descriptions that appear in the embodiments of the present application are only for schematic and distinguishing the described objects, without an order, and do not represent a special limitation on the number of devices in the embodiments of the present application, and shall not constitute any limitation to the embodiments of the present application.

[0042] Before introducing the technical solutions in the embodiments of the present application, exemplary introductions are first made to the terms involved in the embodiments of the present application.

[0043] 1. Graphics Processing Unit (GPU): A hardware device dedicated to processing graphics and image computing. It is applied to execute accelerated graphics rendering tasks, such as 3D graphics and video processing, and is also widely used in scientific computing, artificial intelligence, deep learning and other fields.

[0044] 2. Compute Unified Device Architecture (CUDA): A parallel computing platform and programming model. CUDA allows developers to write programs using programming languages such as C, C++, and Python, and these programs can directly run on the GPU, giving full play to the parallel computing power of the GPU.

[0045] 3. Container: A virtualization technology in a computer operating system. This technology enables a process to run in a relatively independent and isolated environment (including an independent file system, namespace, resource view, etc.), thereby simplifying the software deployment process, enhancing the portability and security of the software, and improving system resource utilization. Container technology is widely used in service-oriented scenarios in the cloud computing field.

[0046] 4. Unified Memory (UM): A memory management mechanism provided by CUDA, designed to simplify memory access and sharing in a multi-processor system, especially between a GPU and a central processing unit (CPU). That is, it allows the CPU and GPU to share the same memory space. With unified memory, developers can simplify memory management and no longer need to manually copy data between the CPU and GPU. Unified memory automatically migrates data between the CPU and GPU according to the program's needs, thus improving development efficiency and optimizing program performance. It is suitable for application scenarios that require frequent data exchange between the CPU and GPU.

[0047] 5. Memory Super-Resolution: In the management of GPU video memory, Memory Super-Resolution refers to the operating system or application allowing the allocation of memory that exceeds the actual physical video memory capacity, providing virtual video memory that exceeds the physical video memory size by utilizing CPU memory resources.

[0048] 6. Compute Unified Device Architecture Application Programming Interface (CUDA API): A programming interface for general-purpose computing on the GPU. The CUDA API includes the CUDA Driver API and the Runtime API, providing functions such as device management, memory management, kernel execution, event, and stream management.

[0049] 7. CUDA API Forwarding: CUDA API Forwarding is a technique that forwards calls to the CUDA API to the business layer for hijacking and appending custom business logic through encapsulation or proxy. Through CUDA API Forwarding, more flexible resource management and task scheduling can be achieved, while hiding the complexity of the underlying hardware from users, thus enhancing the scalability and usability of the system.

[0050] Generally speaking, the resource allocation method of the GPU has the characteristic of exclusivity. Resources such as cores and video memory inside the GPU can usually only be fully occupied and used by one task or process at a certain moment to ensure the stability and efficiency of graphics processing. For example, during the operation of the GPU, GPU resources are usually allocated to a specific process A for use. When other processes (such as process B, process C, etc.) also need to utilize GPU resources, due to the exclusive allocation method of GPU resources, these processes can only wait for process A to release the GPU resources before they can be used.

[0051] To optimize the way of allocating resources of a processor, processors that support resource sharing have emerged, that is, multiple processes can share the resources of the same processor at the same time, avoiding the inefficiency and latency problems caused by the exclusive allocation of processor resources. However, during the process of the processor allocating video memory resources to processes, when there is a new process, if the currently available resources of the processor are not enough to meet the resource requests of the new process, it is easy to cause an out-of-memory (OOM) error, resulting in the new process crashing during operation. Moreover, when the time slice of the underlying driver scheduling is only a few milliseconds, when using processing resources, each application occupies the processor for a very short time, which will lead to very frequent page migration behaviors. Page migration behavior refers to an application evicting the pages (data blocks) in the resources occupied by another application to other locations (such as CPU memory) through an evict page operation, and then moving the data it needs to access to the current processor through a fetch page operation. This causes unacceptable performance losses and serious jitter phenomena.

[0052] To solve this problem, an embodiment of the present application provides a processing resource allocation method, which can be applied to a computing device.

[0053] The computing device can run in a physical machine environment, which includes components such as physical hardware, an operating system, a driver, an infrastructure management tool, an application, and user management. The core feature of this environment is to directly use the physical resources of the computer without relying on virtualization layers or containerization technologies. The physical hardware includes a processor, a memory, a hard disk and other memories, a network interface, and input / output devices. The processor can include one or more processing units. For example, the processor can include a CPU and a GPU, etc. Among them, different processing units can be independent devices or integrated in one or more processors. In the physical machine environment, the operating system is the core for managing physical hardware and software resources. The operating system, such as Windows, Linux, macOS, etc., can be directly installed on the physical machine. The operating system can communicate with hardware devices (such as hard disks, graphics cards, network interfaces) through corresponding drivers. In the physical machine environment, applications run directly on the operating system and use hardware resources to complete tasks.

[0054] The computing device can also run in a host environment. The host refers to a physical computer, and the host environment refers to a virtualization platform built on the physical computer (host) for managing and coordinating the allocation of computing resources so as to effectively allocate them to virtual machines (VMs) or containers. The host environment relies on virtualization technology, usually runs on physical hardware, allows multiple virtual instances (such as virtual machines or containers) to run in parallel, and shares the host's hardware resources, including CPU, GPU, memory, storage, and network, etc. These resources are virtualized and then allocated to virtual machines or containers, and the operating system and hypervisor running on the host are responsible for coordinating the allocation and management of resources.

[0055] Furthermore, the computing device can also run in a virtualization environment or a container environment built on the basis of the host environment. The virtualization environment refers to creating multiple virtual machines in the host environment through virtualization technology. Each virtual machine runs its own operating system and can simulate a complete computer. Each virtual machine runs its own operating system and can simulate a complete computer, and virtual machines can share the host's hardware resources. The container environment refers to creating multiple containers in the host environment through containerization technology. Containers share the operating system kernel of the host but have independent file systems, networks, and process spaces, and containers can also utilize the host's hardware resources more efficiently.

[0056] In addition, the computing device provided by the embodiments of the present application can also be used to build a cloud environment, and the embodiments of the present application do not make specific limitations on this.

[0057] Figure 1 It is the system architecture diagram of the computing device provided by the embodiments of the present application.

[0058] As Figure 1 shown, furthermore, the embodiments of the present application provide a system architecture of a computing device so that the computing device can be used to execute the processing resource allocation method provided by the embodiments of the present application.

[0059] The computing device provided by the embodiment of the present application at least includes a first processor 400, a controller 100, a client 200, and at least one second processor 300. Among them, the first processor 400 and the second processor 300 can initialize a unified memory, and this storage area includes a second processor memory 301 and a first processor memory 401. The first processor 400 and the second processor 300 can jointly access this storage area. That is to say, the unified memory can be jointly accessed and operated by the first processor 400 and the second processor 300.

[0060] It is worth noting that the controller 100 can be a sub-module in the first processor 400.

[0061] In addition, the second processor 300 can include a processor driver (processor Driver) 302. It can be understood that each hardware device has a corresponding hardware driver. For example, a display screen has a corresponding display driver, and the second processor 300 can have a corresponding second processor driver 302. The second processor driver 302 is responsible for underlying hardware management and resource scheduling, and is used to implement direct interaction with the hardware of the second processor 300.

[0062] In the embodiment of the present application, the first processor 400 can specifically be a central processing unit CPU. Correspondingly, the first processor memory 401 can be a CPU memory RAM. The second processor 300 can be a graphics processing unit GPU. Correspondingly, the second processor memory 301 can be a GPU video memory VRAM. In the following, taking the first processor 400 as the CPU, the second processor 300 as the GPU, the first processor memory 401 as the CPU memory, and the second processor memory 301 as the GPU video memory as examples, the processing resource allocation method provided by the embodiment of the present application will be further introduced.

[0063] Furthermore, the second processor 300 can be configured with an API library. The application 201 or the first processor 400 can make the second processor 300 execute corresponding functions by calling the API library. For example, the API library can include a memory management API and a non-memory management API. The memory management API can include an allocate memory API and a release memory API. Among them, by calling the allocate memory API, the memory resources of the second processor 300 can be allocated for the application, and by calling the release memory API, the memory resources of the second processor 300 occupied by the application can be released. Through these APIs, the application 201 or the first processor 400 can interact with the second processor driver 302 to implement the management of the resources of the second processor 300.

[0064] Exemplarily, when the second processor is a GPU, the GPU can be configured with a GPU API library. The GPU API library can include, for example, a video memory management API and a non-video memory management API. The video memory management API can include an API for allocating video memory and an API for releasing video memory. Through these APIs, the application can interact with the GPU driver to manage the GPU video memory resources.

[0065] Furthermore, in the embodiments of the present application, the GPU API library can be a CUDA API library. For example, the CUDA API includes the underlying CUDA driver API (GPU Driver API) and the CUDA runtime API (GPU Runtime API). The CUDA driver API can communicate with the second processor 300 through the libcuda.so library and is responsible for managing underlying operations such as starting the GPU, executing CUDA programs, and handling memory access.

[0066] The CUDA API can include a device management API, a memory management API, a thread management API, etc. Exemplarily, the memory management API includes cudaMalloc() and cudaMemcpy(). cudaMalloc() is used to allocate memory in the video memory of the GPI 300, and cudaMemcpy() is used to transfer data between the memory of the host first processor 400 and the video memory of the second processor 300. In addition, the CUDA API is also used to define data structures and constants. By calling the functions in the CUDA API, interactions with the hardware of the second processor 300 can be achieved, including operations such as device initialization, memory allocation and release, and starting kernel functions, so that the second processor 300 can execute computing tasks.

[0067] Furthermore, various applications (APP) 201 are deployed on the client 200. The client 200 is an interface for users to interact with the computing device. The user can start various applications 201 through the client 200 and submit tasks, such as submitting an image processing task using the second processor 300. After that, the client 200 can receive the execution result of the GPU task and display it to the user.

[0068] Furthermore, the client 200 can also include an interceptor 202. At the software structure level, the interceptor 202 is a GPU API forwarding layer and is configured with an interception library for intercepting the call requests of the application 201.

[0069] Furthermore, the client 200 can include a client A 200A and a client B 200B.

[0070] In a containerized environment, client A 200A and client B 200B can run in different container groups (pods), and controller 100 can run in a control pod. In a non-containerized environment, such as in a virtual machine environment, client A 200A and client B 200B can run in different virtual machine instances, and controller 100 can run in a control instance. Also, in a physical machine environment with multiple users on a single machine, client A 200A and client B 200B can run in different user sessions, and controller 100 can run in a dedicated process for managing user sessions.

[0071] Further, controller 100 is used to manage and schedule memory resources, such as unified memory. Specifically, it can include a memory manager 101, a priority evaluator 102, a monitor 103, a scheduler 104, and a policy controller 105. The detailed working steps of each component in controller 100 will be described in detail below.

[0072] Figure 2 It is a scheduling schematic diagram of the controller provided by the embodiment of the present application.

[0073] Further, as Figure 2 shown, the processing resource allocation method provided by the embodiment of the present application includes the following steps S101 - S102.

[0074] S101: Controller 100 starts and controls the start of scheduler 104 and monitor 103.

[0075] As Figure 2 shown, the code for starting controller 100 is represented as "Run Controller", the code for controlling the start of scheduler 104 is represented as "Run Scheduler", and the code for controlling the start of monitor 103 is represented as "Run Monitor".

[0076] S102: After starting, scheduler 104 initializes memory manager 101 and priority evaluator 102.

[0077] Continue to refer to Figure 2 , the code for initializing memory manager 101 is represented as "Init MemoryManager", and the code for initializing priority evaluator 102 is represented as "Init Evaluator".

[0078] Through these two steps, the initialization process of controller 100 is completed.

[0079] Further, asFigure 1 As shown, after step S102, step S103 may further be included.

[0080] S103: The controller 100 accesses the application 201 and injects target environment variables (env) into the application 201.

[0081] Among them, the target environment variable may be LD_PRELOAD, which can point to an interception library (also known as a shared library), and the interception library is a ".so file". In this way, after the controller 100 is started, the interception library can be preferentially loaded. In the embodiments of the present application, the interception library includes APIs with the same names as the GPU APIs in the GPU API library. When the GPU API library is the CUDA API library, the interception library includes APIs with the same names as the CUDA APIs in the CUDA API library. Then, when the application starts to call the GPU API, due to the effect of the LDPRELOAD environment variable, the actually called API is the API with the same name in the interception library, realizing the interception of the original API call. In this way, after the interception, the API call can be flexibly processed according to different business requirements, such as redirecting the API to other APIs and no longer calling the GPI API library. If redirection is required, the custom target API can be directly called; if redirection is not required, the original API can be selected for calling.

[0082] In some implementation manners, the interception library includes APIs with the same names as the Driver APIs in the CUDA Driver API library, that is, the APIs in the interception library correspond one-to-one with the Driver APIs in the CUDA Driver API library. In this way, when the Driver API is called, the interception library can perform API interception based on the API name, realizing Driver-level interception.

[0083] The CUDA Driver API library is one of the CUDA API libraries, and the corresponding one is the CUDA Runtime API library, and the CUDA Runtime API library is obtained by encapsulating the CUDA Driver API library.

[0084] The embodiments of the present application can use the interception library to intercept the Driver-level APIs instead of intercepting the Runtime-level APIs. This is because: the CUDA Runtime API library is highly encapsulated, and it is difficult to finely control the API after interception, and Driver-level interception can achieve more efficient and flexible GPU operation control and optimization.

[0085] In some implementations, the CUDA Driver API library can be the libcuda.so library.

[0086] In some implementations, the interception library can also include an API with the same name as the RuntimeAPI in the CUDA Rnntime API library to intercept the Runtime API.

[0087] In a containerized environment, the controller 100 can inject target environment variables into each Pod.

[0088] Exemplarily, the method of injecting target environment variables is as follows:

[0089] export LD_PRELOAD= / path / to / libmemshare.so

[0090] . / gpu_application

[0091] Among them, libmemshare.so represents the interception library. When gpu_application is executed, it will load the functions in libmymemory.so instead of the default functions in libcuda.so.

[0092] Figure 3 This is the first flowchart of the processing resource allocation method provided by the embodiments of the present application.

[0093] Figure 4 This is the interaction diagram of the processing resource allocation method provided by the embodiments of the present application.

[0094] Further, as Figure 3 and Figure 4 shown, the processing resource allocation method provided by the embodiments of the present application includes the following steps S201-S205.

[0095] S201: The first application 201a creates and runs at least one first process 203.

[0096] It is understandable that the operating mode of the first application 201a varies with different operating environments of the computing device. In a containerized environment, the first application 201a can be deployed inside a container of a certain pod, and the container can allocate resources such as the first processor 400 to the first process 203. In a virtual machine environment, the first application 201a can run in a virtual machine instance, and the creation and operation of the first process 203 depend on the operating system and resource allocation of the virtual machine. In a physical machine environment with multiple single - user sessions, the first application 201a can create the first process 203 under a user session, and the physical machine operating system can allocate corresponding resources to the first process 203. Each user session is isolated from others, and the first process 203 can only access the resources allowed to be accessed under that user session.

[0097] The first application 201a is, for example, an application that executes an inference service. At different running stages of the first application 201a, different threads can be concurrently executed in the first process, and different threads can be used to call different CUDA APIs to perform different tasks. For example, in the first process, thread A can be used to call the cuInit() function, which is used to initialize the CUDA runtime environment; thread B can be used to call the cuDeviceGet() function, which is used to obtain information about available GPU devices; and thread C can be used to call the cuMemAlloc() function to allocate memory space for the data and intermediate results of the inference task in the GPU's video memory.

[0098] S202: The first process 203 initiates a call request to the second processor 300.

[0099] Among them, the call request can be to call a video memory management API and / or a non - video memory management API. The video memory management API and the non - video memory management API belong to the GPU API library.

[0100] Therefore, step S202 can specifically include the following step S2021: The first process 203 initiates a memory allocation request to the second processor 300. The memory allocation request can be used to request the second processor 300 to allocate a storage area of the first storage size for the first process 203. It is understandable that the specific value of the first storage size depends on the memory requirements of the first process 203, and the embodiments of the present application do not make specific limitations on this.

[0101] It is understandable that the call request can also be other requests except the memory allocation request, such as a device status query request, an event notification request, a memory release request, etc. The embodiments of the present application do not make specific limitations on this.

[0102] S203: Interceptor 202 intercepts the memory allocation request of the first process 203 for the second processor 300 and determines the lock-holding status of the first process 203 for the Dynamic Time-Sliced Lock (DTL).

[0103] In practical applications, the steps of intercepting the memory call request and determining the lock-holding status can be two independent sub-steps. That is to say, step S203 can include the following S2031 - S2032.

[0104] S2031: Interceptor 202 intercepts the memory allocation request of the first process 203 for the second processor 300; the memory allocation request is used to request the second processor 300 to allocate a storage area of the first storage size for the first process 203.

[0105] S2032: Interceptor 202 determines the lock-holding status of the first process 203 for the DTL lock.

[0106] In the embodiments of the present application, the GPU API library may include a video memory management API, and the video memory management API may include an API for allocating video memory and an API for releasing video memory. Further, when the GPU API library is specifically the CUDA API library, the API for allocating video memory may be cuMemAlloc(), which is used to allocate a storage area of a specified size in the second processor memory 301. The API for releasing video memory is, for example, cuMemFree(), which is used to release the previously allocated video memory resources so as to recycle the video memory space for other processes or tasks. Therefore, the memory allocation request of the first process 203 for the second processor 300 may refer to initiating a request to call the API for allocating video memory, and specifically, the memory allocation request may be used to request the second processor 300 to allocate a storage area of the first storage size for the first process 203.

[0107] In addition to the video memory management API, the GPU API library may also include a non-video memory management API. The non-video memory management API may include a device management API, a stream management API, an event management API, an error management API, etc. The device management API can be used to query and manage the information of the GPU device. A stream refers to a parallel execution mechanism. When the GPU API library is specifically the CUDA API library, the stream management API is, for example, cudaStreamCreate(), which is used to create a new stream. The event management API is, for example, cudaEventCreate(), which is used to create a new event. The error management API is, for example, cudaGetErrorString(), which is used to obtain error information.

[0108] In the embodiments of the present application, the first process 203 may also initiate a memory release request, a stream management request, etc., and the embodiments of the present application do not make specific limitations thereto.

[0109] Further, when the first process 203 calls the video memory management API and / or the non-video memory management API, since the interception library has been preferentially loaded, the interceptor 202 may first search for the API in the interception library. Further, APIs with the same names as the GPU APIs in the GPU API library are defined in the interception library. Therefore, the interceptor 202 may determine that the call request is a call to a GPU API based on the interception library, and thus may intercept the call request.

[0110] Further, the DTL lock is a lock mechanism, and the holding time of the lock is dynamic and can be controlled based on a time-slice. The interceptor 202 may determine the lock-holding state of the first process 203 based on the lock state information. The lock-holding information may include two types, namely the running state and the waiting state. The running state indicates that the first process 203 is holding the lock, and the waiting state indicates that the first process 203 is not holding the lock.

[0111] S204: When the first process 203 does not hold the DTL lock and the target condition is satisfied, the interceptor 202 blocks the first process 203; when the first process 203 does not hold the DTL lock and the target condition is not satisfied, the controller 100 issues the DTL lock to the first process 203.

[0112] Among them, the target condition includes that the holding time slice of the DTL lock held by at least one second process 204 expires. That is to say, if at least one second process 204 does not release the DTL lock, then the first process 203 will be in a blocked state. If at least one second process 204 releases the DTL lock, then the first process 203 may obtain the DTL lock. In this way, it is possible to prevent the first process 203 from preempting GPU resources with the second process 204 and avoid frequent page in and page out operations.

[0113] It can be understood that the second process 204 is any other process that shares (accesses jointly) the second processor 300 with the first process 203. In a non-containerized environment, the first process 203 and the second process 204 may be processes in the same application program, such as the first application program 201a. In a containerized environment, the first process 203 and the second process 204 may run in different pods.

[0114] In one implementation, the target condition may further include that a process is holding the DTL lock. When a process is holding the DTL lock, the target condition is satisfied; when no process is holding the DTL lock, the target condition is not satisfied. For example, when the first process 203 is the first to access the second processor 300, the DTL lock is in the idle state and the target condition is not satisfied, and the DTL lock can be issued to the first process 203.

[0115] Further, step S204 specifically includes the following steps S2041 - S2044.

[0116] S2041: When the interceptor 202 determines that the first process 203 does not hold the DTL lock, the interceptor 202 sends a first lock request to the controller 100.

[0117] It can be understood that after the interceptor 202 intercepts the call request, if the first process 203 does not hold the lock, the interceptor 202 can first send a lock signal to the controller 100, and then request the DTL lock for the first process 203.

[0118] S2042: In response to the first lock request, the controller 100 determines whether the target condition is satisfied.

[0119] S2043: When the target condition is satisfied, the controller 100 returns a first failure message to the interceptor 202, and the first failure message is used to indicate that the first process 203 has not obtained the lock.

[0120] In one implementation, step S2042 may include the following steps S2042a - S2042b.

[0121] S2042a: In response to the first lock request, the controller 100 determines whether the holding time slice of the DTL lock held by the second process 204 has expired, and / or determines whether no process is holding the lock.

[0122] Among them, the controller 100 can determine whether a process holds the DTL lock by maintaining a lock status table. Exemplarily, the lock status table can be a hash table, where the key is the identifier of the process (process ID), and the value is the lock status information of the boolean type. "true" indicates that the process holds the DTL lock, and "false" indicates that it does not hold. After the controller 100 issues a lock to a process, it can update the lock status table. Further, in response to the first lock request, the controller 100 can traverse the lock status table to check whether there is an entry with a value of "true", and then determine whether a process holds the DTL lock.

[0123] S2042b: If the holding time slice of the DTL lock held by the second process 204 has not expired, the controller 100 determines that the target condition is satisfied and returns a first failure message to the interceptor 202.

[0124] In some implementations, if the holding time slice of the DTL lock held by the second process 204 has not expired, that is, if a process holds the DTL lock, the controller 100 may add the process ID (pid) of the first process 203 to a waiting queue, where the waiting queue is used to store all processes waiting to acquire the DTL lock, so as to achieve an orderly allocation of the DTL lock.

[0125] S2044: The interceptor 202 blocks the first process 203 based on the first failure information.

[0126] Further, after step S2042, step S2045 is further included.

[0127] S2045: If the holding time slice of the DTL lock held by the second process 204 expires, and / or if no process is holding the lock, the controller 100 determines that the target condition is not satisfied, issues the DTL lock to the first process 203, and returns the first lock acquisition information to the interceptor 202 to inform the interceptor 202 that the first process 203 has acquired the DTL lock.

[0128] In this step, the controller 100 may initialize the DTL lock. Moreover, the controller 100 may also record the timestamp of the initialization completion and start a timer after initializing the DTL lock. The start time of the timer may be the timestamp of the initialization completion, and the timing duration of the timer is equal to the active duration of the DTL lock, that is, the duration of the holding time slice of the DTL lock held by the first process 203. It should be noted that the timing duration of the timer is greater than the context time slice length of the underlying driver of the second processor 300.

[0129] Further, the controller 100 may also update the lock status table, mark the value of the process ID corresponding to the first process 203 as "true", thereby indicating that the first process 203 has successfully acquired the DTL lock.

[0130] Further, the interceptor 202 may modify the lock holding information from the waiting state to the running state in response to the first lock acquisition information.

[0131] In the embodiment of the present application, after step S203, step S205 may further be included.

[0132] S205: After the first process 203 holds the DTL lock and the holding time slice of the DTL lock expires, the interceptor 202 blocks the first process 203.

[0133] In one implementation, step S205 may include the following steps S2051 - S2052.

[0134] S2051: When the holding time slice of the DTL lock held by the first process 203 expires, the controller 100 sends a lock-holding expiration message to the interceptor 202.

[0135] Specifically, the controller 100 can determine that the holding time slice of the DTL lock held by the first process 203 has expired in response to the timer timeout, and then can generate and send the lock-holding expiration message.

[0136] In addition, the controller 100 can modify the lock status table to change the lock-holding status of the first process 203 in the table.

[0137] Moreover, the controller 100 can also issue the DTL lock to the process waiting to acquire the DTL lock based on the waiting queue.

[0138] In this way, the processes sharing the same second processor 300 can be serialized through the serializable lock-holding logic of the dynamically configurable DTL lock. And since the interceptor 202 can intercept the API called by the process and not make the call when the process does not hold the lock, the DTL lock can have a locking effect, which can cover the context time slice scheduling of the underlying driver of the second processor 300. Furthermore, it can avoid the frequent page in and out caused by the frequent time slice scheduling, reduce the jitter phenomenon, and improve the performance of the second processor 300.

[0139] S2052: The interceptor 202 blocks the first process 203 based on the lock-holding expiration message, and modifies the lock-holding information from the running state to the waiting state.

[0140] In this way, the interceptor 202 can block the first process 203 in time, avoiding a single process monopolizing the GPU resources for a long time, enabling multiple processes to share the second processor 300, and improving the resource utilization rate.

[0141] Further, after step S204, step S206 can also be included.

[0142] S206: When the first process 203 holds the DTL lock, the interceptor 202 calls the second processor 300 based on the memory allocation request, so that the second processor 300 allocates the first storage area in the second processor memory 301 for the first process 203.

[0143] The step of allocating the first storage area in the second processor memory 301 for the first process 203 can include: the interceptor 202 redirects the memory allocation request to call the memory allocation API to allocate the first storage area in the second processor memory 301 for the first process 203.

[0144] Specifically, the first storage area can be an unallocated storage area in the second processor memory 301 and / or a storage area released through an eviction operation. The eviction operation refers to migrating the data stored by the third process in the second processor memory 301 to the first processor memory 401. The data to be migrated is the data in the storage area with the lowest occupancy priority in the second processor memory 301. The third process is a process that shares the resources of the second processor 300 with the first process 203. The occupancy priority is an index for comprehensively evaluating the importance of the storage areas in the second processor memory 301. The calculation method of the occupancy priority will be described in detail below and will not be elaborated here.

[0145] In the embodiment of the present application, the memory allocation API can specifically be a unified memory allocation API. The first storage area belongs to the unified memory resource, and a migration (evict) operation can be performed on the first storage area. Migration refers to an operation of moving the data stored in the storage area between two different storage areas, namely the second processor memory 301 and the first processor memory 401.

[0146] In this way, super-resolution of the video memory of the second processor 300 can be achieved, and the occurrence of OOM errors can be avoided.

[0147] Further, when the first process 203 holds the DTL lock, in order to occupy the storage resources, step S206 may include the following steps S2061 - S2065.

[0148] S2061: The interceptor 202 redirects the memory allocation request to the memory allocation API.

[0149] Among them, the memory allocation API is one of the memory management APIs. Specifically, it is used to enable the second processor 300 to allocate the first storage area in the second processor memory 301 for the first process 203 based on the memory allocation API.

[0150] When the CPU API library is the CUDA API library, the video memory allocation API can be cuMemAlloc(), and the memory management API can be cudaMallocManaged(). In this way, when the first process 203 calls cuMemAlloc(), the embodiment of the present application can redirect the call request to cudaMallocManaged() to allocate unified memory through cudaMallocManaged(). The storage area (i.e., unified memory) allocated in this way can be shared between the first processor 400 and the second processor 300 and can be automatically migrated between the two as needed, optimizing the memory usage efficiency and data access speed.

[0151] S2062: The second processor driver 302 allocates a first storage area for the first process 203 based on the memory allocation API and generates an allocation result.

[0152] Among them, the first storage area may be a storage area in the second processor memory 301, and the allocation result at least includes a pointer ptr pointing to the allocated memory (the first storage area) and the size size of the first storage area.

[0153] It can be understood that the second processor driver 302 can return the allocation result to the interceptor 202, and at this time, it can indicate that the redirection is successful.

[0154] In this way, a unified memory resource management mechanism can be introduced in the hijacking layer to realize the allocation of unified memory for the first process 203.

[0155] Furthermore, the embodiments of the present application may further include the following steps S207 - S208 to recycle the storage area occupied by the first process 203. Steps S207 - S208 may be steps executed after step S206.

[0156] S207: The interceptor 202 intercepts the memory release request of the first process 203 for the second processor 300. The memory release is used to request the second processor 300 to release the storage area of the second storage size occupied by the first process 203.

[0157] Among them, the memory release request may specifically refer to the request initiated by the first process 203 to call the release video memory API to apply for video memory resources.

[0158] S208: When the first process 203 holds the DTL lock, the interceptor 202 calls the second processor 300 based on the memory release request to release the storage area of the second storage size in the second processor 300 occupied by the first process 203.

[0159] Furthermore, step S208 may specifically include the following steps S2081 - S2082.

[0160] S2081: The interceptor 202 redirects the memory release request to the memory release API. The memory release API is used to enable the second processor 300 to recycle the storage area of the second storage size in the second processor 300 occupied by the first process 203 based on the memory release API.

[0161] Among them, the memory release API is one of the memory management APIs. When the CPU API library is the CUDA API library, the video memory release API can be cuMemFree(), and the memory release API can be cudaFreeManaged(). In practical applications, when the first process 203 calls the video memory release API cuMemFree(), the embodiment of the present application can redirect the call request to the memory release API cudaFreeManaged() to recycle the unified memory through cudaFreeManaged().

[0162] Figure 5 It is a schematic diagram of redirecting a call request provided by an embodiment of the present application.

[0163] When the first process 203 of the first application 201a initiates a call request to call the video memory allocation API cuMemAlloc(), if the interceptor 202 is not included, then the call request will be called to the lower-level driver layer libcuda.so.xxx.xx via the CUDA API library libcuda.so to execute. At this time, unified memory cannot be allocated for the first process 203. However, the interceptor 202 provided by the embodiment of the present application can intercept the call request for calling cuMemAlloc() based on the interception library libmemshare.so and redirect it to cudaMallocManaged(), and then call the lower-level driver layer libcuda.so.xxx.xx based on cudaMallocManaged() to allocate unified memory.

[0164] S2082: The second processor driver 302 recycles the storage area of the second storage size occupied by the first process 203 based on the memory release API and generates a release result.

[0165] Among them, the storage area released by the first process 203 can be the storage area in the second processor memory 301 or the first processor memory 401. The release result at least includes the pointer ptr of at least the released storage area and the size size of the released storage area.

[0166] It can be understood that the second processor driver 302 can return the release result to the interceptor 202, and at this time, it can indicate that the redirection is successful.

[0167] It should be noted that since the data in the second - processor memory 301 and the data in the first - processor memory 401 can be migrated to each other, when the first process 203 releases a storage area, the released storage area may be the storage area of the second - processor memory 301 or the storage area of the first - processor memory 401. After the second - processor driver 302 receives a request to release a storage area, it can check the physical location of the storage area to be released (such as the second - processor memory 301 or the first - processor memory 401). If the physical location of the storage area to be released is the first - processor memory 401, then the second - processor driver 302 can send a memory - release instruction to the memory - management module of the first processor 400, and the memory - release instruction can carry information such as the address and size of the storage area to be released. After that, the memory - management module can perform the release operation.

[0168] In this way, unified memory recycling can be achieved.

[0169] Furthermore, after step S202, the following steps S2091 - S2092 may further be included.

[0170] S2091: The interceptor 202 intercepts non - memory - management requests of the first process 203 for the second processor 300.

[0171] Among them, the non - memory - management request may refer to a request for calling a non - memory - management API.

[0172] S2092: The interceptor 202 forwards the non - memory - management request to the non - memory - management API.

[0173] It should be noted that as Figure 1 shown, in the case where the call request is for calling a non - video - memory - management API, the embodiment of the present application can forward the call request to the non - video - memory - management API in the original GPU API library for processing. Exemplarily, in the case where the GPU API library is the CUDA API library, the call request can be forwarded to the original CUDA Driver API library, and the code is expressed as "call original cuda driver api". In this way, by directly calling the functions in the original CUDA API library, it can be ensured that these operations that do not involve video - memory management can be executed normally without being affected by the interception and redirection mechanisms, so as to ensure the normal operation of the second processor 300 without affecting video - memory management.

[0174] Figure 6 This is a schematic diagram of the working process of the interceptor provided by the embodiment of the present application.

[0175] As Figure 6As shown, in the embodiment of the present application, after the interceptor 202 intercepts the call request of the first process 203, if the first process 203 does not hold the DTL lock, it can request a DTL lock for the first process 203 (request DTL Lock). Further, when the first process 203 holds the DTL lock, it can continue to call the intercepted call request (call hijacked CUDA API). If the call request is to call a non-memory management API, then the call request can be directly forwarded to the non-memory management API in the API library of the original second processor 300, for example, forwarded to the non-video memory management API in the original CUDA API library, and the code is represented as "call original cuda api".

[0176] If the call request is to call a video memory management API, then the memory management API can be called. Further, the interceptor 202 can interact with the controller 100 after calling the memory management API so that the controller 100 records the memory allocation (recordmem allocation) and updates the memory allocation (update mem allocation).

[0177] Recording the memory allocation means creating an allocation record based on the unified memory allocation, which can be implemented by the controller 100 scheduling the memory manager 101. Updating the memory allocation means updating the occupancy priority of the allocated storage area. Specifically, the scheduler 104 inside the controller 100 can regularly schedule the priority evaluator 102 to perform a priority evaluation on the allocated storage areas managed by the memory manager 101 to achieve the update of the occupancy priority.

[0178] The above process will be introduced in detail below with reference to the accompanying drawings.

[0179] Figure 7 It is a schematic diagram of the interaction between the interceptor and the controller provided by the embodiment of the present application.

[0180] In the embodiment of the present application, as Figure 7 shown, after step S206, the following step S210 may further be included.

[0181] S210: The memory manager 101 generates an instance of the storage record (memory record) of the first storage area; the storage record instance includes one or more of the identifier of the first process 203, a pointer to the first storage area, the size of the first storage area, or the occupancy priority of the first storage area.

[0182] S211: Store the storage record instance.

[0183] In an embodiment of the present application, the storage record instance can be stored in the memory priority queue 1011.

[0184] Further, after step S210, the following step S212 may further be included: adding a hash node (hashnode) to the hash table 1012 based on the storage record instance.

[0185] Further, before S210, the following steps S210a and S210b may be included.

[0186] S210a: The interceptor 202 sends a first post-operation signal to the controller 100 based on the allocation result, where the first post-operation signal carries the allocation result.

[0187] S210b: The controller 100 forwards the allocation result to the memory manager 101 in response to the first post-operation signal.

[0188] Further, step S210 may include the following steps S2101 - S2012.

[0189] S2101: The memory manager 101 adds a new storage record instance to the priority queue PriorityQueue 1011 based on the allocation result.

[0190] S2102: The memory manager 101 adds a hash node to the hash table 1012 based on the storage record instance.

[0191] It can be understood that in response to a memory allocation request of a process once, the second processor 300 can allocate a storage area for the process. At this time, the memory manager 101 can maintain memory information related to the storage area. Specifically, the memory manager 101 can maintain two data structures, namely the priority queue 1011 and the hash table 1012.

[0192] Figure 8 It is a schematic diagram of the priority queue and the hash table provided by the embodiment of the present application.

[0193] As Figure 8 shown, the priority queue 1011 is used to store the memory information of all processes for the storage areas already allocated on the second processor 300. The structure of the priority queue 1011 can be a max heap maxHeap. The memory manager 101 can add a new element to the maxHeap after the storage area is successfully allocated. Each element in the maxHeap is a storage record instance, representing an allocated storage record. Further, the elements in the max heap can be sorted according to the priority evaluation result of the storage area by the priority evaluator 102, and the element with the highest occupancy priority is located at the top of the heap.

[0194] Continue to refer toFigure 8 , the struct PriorityQueue is a structure used to implement the priority queue 1011. In this structure, "int size" represents the number of elements in the current priority queue 1011, that is, the number of memory allocation records that have been stored. "int capacity" represents the capacity of the current priority queue 1011, that is, the maximum number of memory allocation records that can be stored. "MemoryRecord maxHeap[MAX_HEAP_SIZE]" is an array used to implement the max heap data structure. This array is used to store elements of the MemoryRecord type. "MAX_HEAP_SIZE" is a predefined constant representing the maximum capacity of the max heap. "pthread_mutex_t lock" is a mutex used to protect access to the priority queue 1011 in a multi-threaded environment, preventing multiple threads from operating on the queue simultaneously and causing data inconsistency.

[0195] Furthermore, the storage record instance at least includes the following member variables:

[0196] Cudeviceptr ptr: A pointer to the allocated memory;

[0197] size_t size: The size of the allocated memory;

[0198] unsigned int accessCount: The access frequency counter;

[0199] time_t lastAccessTime: The time of the last access;

[0200] int pid: The process ID to which the current memory belongs;

[0201] float priority: The occupancy priority calculated by the priority evaluator 102;

[0202] float latency: The access latency, updated by the monitor 103;

[0203] pthread_mutex_t lock: The mutex.

[0204] In summary, the storage record instance at least includes the following key information: ptr (a pointer to the allocated storage area), size (the allocated size), accessCount (the access frequency), lastAccessTime (the latest access time), pid (the process ID to which the current memory belongs), priority (the occupancy priority calculated by the priority evaluator 102), and latency (the access latency updated by the monitor 103).

[0205] Exemplarily, if the allocated storage area refers to the first storage area, then ptr is a pointer to the first storage area, size is the size of the first storage area, accessCount is the frequency of the first process 203 accessing the first storage area, lastAccessTime is the latest time when the first process 203 accesses the first storage area, pid is the process ID of the first process 203, priority is the occupancy priority of the first storage area, and latency is the access latency of the first process 203 accessing the first storage area.

[0206] In some implementation manners, when the first process 203 calls an API to perform an operation of accessing the storage space, this call will be hijacked by the interceptor 202. After that, the interceptor 202 can notify the memory manager 101 to update the accessCount and the lastAccessTime.

[0207] Further, continue to refer to Figure 8 , struct HashTable is a structure for implementing the hash table 1012. In this structure, "pthread_mutex_t lock" represents a mutex lock used to protect the access to the hash table 1012 in a multi-threaded environment and prevent data conflicts when multiple threads simultaneously perform operations such as insertion, deletion, and search on the hash table 1012. "HashNode*table[HASH_TABLE_SIZE]" represents an array of pointers, which is the storage structure of the hash table 1012. "HASH_TABLE_SIZE" is a predefined constant representing the size of the hash table 1012. Each array element is a pointer to a HashNode, used to link the nodes in the hash table 1012.

[0208] Further, "struct HashNode" is the node structure in the hash table 1012, including the following member variables:

[0209] Cudeviceptr ptr: A pointer to the allocated memory;

[0210] MemoryRecord value: Stores record information related to the storage area, including pointers, memory size, access frequency, access time, process ID, occupancy priority, access latency, etc., that is, the memory allocation record corresponding to this hash node.

[0211] struct HashNode*next: A pointer used to link the next node on the conflict chain in the hash table. When a hash conflict occurs (i.e., different key values are mapped to the same hash bucket), the conflicting hash nodes can be linked together through this pointer to form a linked list structure.

[0212] pthread_mutex_t lock: A mutex lock used to protect access and modification to this hash node in a multi-threaded environment to prevent data inconsistency.

[0213] It can be understood that when querying, updating, or deleting the data with the highest priority in the priority queue 1011, the time complexity is O(1). However, when querying a certain element in the priority queue 1011, since the priority queue 1011 is in a heap shape, the worst-case time complexity for querying is O(n), where n represents the number of elements in the maxHeap. This means that in the worst case, the query operation may need to traverse the entire max heap. When querying a certain element in the hash node of the hash table 1012, the time complexity is usually O(1). Therefore, the hash table 1012 can be used as an auxiliary data structure for the priority queue 1011 for auxiliary query. In practical applications, the hash node of the hash table 1012 at least includes a pointer ptr pointing to the allocated memory and the MemoryRecord value. Therefore, when accessing the memory manager 101 to query information, the hash table can be queried first. Through parameters such as the pointer information of the hash table, the storage record instance can be quickly located to achieve the fast query function.

[0214] In some implementation manners, the monitor 103 can also be used for monitoring and collecting performance metrics to calculate bandwidth, etc., and the embodiments of the present application do not make specific limitations on this.

[0215] In the embodiments of the present application, after step S208, the following step S212 may further be included.

[0216] S212: The memory manager 101 deletes the storage record instance corresponding to the released storage area in the priority queue 1011, and deletes the hash node corresponding to the released storage area in the hash table 1102.

[0217] It can be understood that the released storage area can be a storage area reclaimed based on a call request. The released storage area can be a storage area in the second processor memory 301 or a storage area in the first processor memory 401.

[0218] Furthermore, before step S212, the following steps S212a - S212b may be included.

[0219] S212a: The interceptor 202 sends second post - operation information to the controller 100 based on the release result, where the second post - operation information carries the release result.

[0220] S212b: The controller 100 forwards the release result to the memory manager 101 in response to the second post - operation signal.

[0221] Furthermore, step S212 may include the following step S2121.

[0222] S2121: The memory manager 101 deletes the storage record instance corresponding to the released storage area in the priority queue 1011 based on the release result, and deletes the hash node corresponding to the released storage area in the hash table 1102.

[0223] It can be seen that after allocating a storage area, the memory manager 101 can create a storage record matching the allocated storage area and continuously maintain this storage record until the allocated storage area is released by the process.

[0224] Furthermore, continue to refer to Figure 1 and Figure 2 , the scheduler 104 can schedule the priority evaluator 102 to calculate the occupancy priority (calculate priority) to achieve the update of the occupancy priority (refresh priority). The scheduler 104 can perform dynamic memory migration (dynamic mem migrate) based on the updated occupancy priority. The steps of dynamic memory migration include a migration operation and a pre - fetch operation. The migration operation means moving the data of the storage area stored in the second processor memory 301 to the first processor memory 401. The pre - fetch operation means fetching in advance (fetch) the data of the storage area stored in the first processor memory 401 to the second processor memory 301 before the process accesses it. The above steps can be executed periodically in a loop, that is, executed once per cycle (every cycle).

[0225] Specifically, the method provided by the embodiments of the present application may further include the following steps S301 - S302.

[0226] Among them, steps S301 - S302 can be executed after step S102.

[0227] S301: The priority evaluator 102 calculates the occupancy priority of each allocated storage area at a second preset frequency.

[0228] Furthermore, step S301 can specifically include the following steps S3011 - S3012.

[0229] S3011: The scheduler 104 periodically schedules the priority evaluator 102 at a second preset frequency, enabling it to periodically access the memory manager 101 and traverse each storage record instance in the priority queue, and calculates the occupancy priority of each allocated storage area based on the information recorded in the storage record instance. The information recorded in the storage record instance includes the size of the allocated storage area, access frequency, latest access time, and / or access latency.

[0230] Among them, the second preset frequency is, for example, once every 30ms, once per second, once every ten seconds, or once per minute. The embodiments of the present application do not make specific limitations on this.

[0231] The occupancy priority is calculated based on the following priority formula.

[0232]

[0233] Among them, priority represents the occupancy priority, access_count represents the access frequency, current_time represents the current time, last_time represents the latest access time, size represents the size of the allocated storage area, latency represents the access latency, stream_priority represents the stream priority of the stream to which the allocated storage area belongs, α, β, γ, and λ represent weight coefficients, and factor represents an empirical coefficient.

[0234] The weight coefficients and the empirical coefficient can be adjusted based on the actual situation. The embodiments of the present application do not make specific limitations on this.

[0235] It can be seen from the priority formula that the evaluation of the occupancy priority follows the following basic strategy: The occupancy priority is positively correlated with the access frequency. The higher the access frequency, the higher the priority; the occupancy priority is positively correlated with the latest access time. The closer the latest access time is to the current time, the higher the occupancy priority, and the longer the unaccessed time, the lower the occupancy priority; the occupancy priority is negatively correlated with the size of the storage area. The larger the size of the storage area, the lower the occupancy priority; the occupancy priority is negatively correlated with the access latency. The higher the access latency, the lower the occupancy priority; the occupancy priority is positively correlated with the stream priority. The higher the CUDA stream priority bound to the storage area, the smaller the value of the stream priority, and the higher the occupancy priority.

[0236] In some implementations, the embodiments of the present application may also introduce an artificial intelligence (AI) tool to train a model, and use the model to further evaluate the occupancy priorities under different parameters, so as to continuously optimize and refine the priority evaluation algorithm.

[0237] S3012: The priority evaluator 102 calls the memory manager 101 to record the occupancy priority in the storage record instance.

[0238] It can be seen that after calculating the latest occupancy priority, the priority evaluator 102 can call the memory manager 101 to update each storage record instance.

[0239] In some implementations, the priority evaluator 102 can directly modify the storage record instance to update the occupancy priority.

[0240] In this way, periodic update of the storage record instance can be achieved.

[0241] Further, after step S301, the following steps S303 - S304 may also be included.

[0242] S303: If the occupancy priority of the allocated storage area exceeds the first priority threshold, and the allocated storage area is a storage area in the first processor memory 401, the scheduler 104 prefetches the data stored in the allocated storage area to the second processor memory 301.

[0243] Figure 9 It is a schematic flow diagram of the dynamic memory migration provided by the embodiments of the present application.

[0244] As Figure 9 shown in (a) of , step S303 may include steps S3031 - S3033.

[0245] S3031: For the allocated storage area whose occupancy priority exceeds the first priority threshold, the scheduler 104 determines whether it is a storage area in the second processor memory 301.

[0246] It can be understood that the scheduler 104 can determine whether the allocated storage area is a storage area of the second processor memory 301 based on the storage area pointer recorded in the storage record instance.

[0247] S3032: If it is a storage area in the first processor memory 401, the scheduler 104 prefetches the data stored in it to the second processor memory 301;

[0248] It can be understood that after the prefetch step, the storage area for storing the prefetched data is converted from the first processor memory 401 to the second processor memory 301.

[0249] S3033: In the case where it is a storage area in the second processor memory 301, keep resident and do not perform data migration.

[0250] It should be noted that after step S303, it may further include: The memory manager 101 updates the storage record instance to update the pointer. At this time, the pointer originally pointing to the first processor memory 401 in the storage record instance can be updated to point to the second processor memory 301.

[0251] S304: If the occupancy priority of the allocated storage area does not exceed the second priority threshold, and the allocated storage area is a storage area in the second processor memory 301, the scheduler 104 migrates the data stored in the allocated storage area to the first processor memory 401.

[0252] It should be noted that the second priority threshold is less than the first priority threshold, and the specific values of the first priority threshold and the second priority threshold are adjustable.

[0253] As Figure 9 shown in (b) below, step S304 may include steps S3041 - S3043.

[0254] S3041: For the allocated storage area whose occupancy priority does not exceed the second priority threshold, the scheduler 104 determines whether it is a storage area in the second processor memory 301;

[0255] S3042: In the case where it is a storage area in the second processor memory 301, the scheduler 104 migrates the data stored in it to the first processor memory 401;

[0256] It can be understood that after the migration step, the storage area for storing the migrated data is converted from the second processor memory 301 to the first processor memory 401.

[0257] S3043: In the case where it is a storage area in the first processor memory 401, keep resident and do not perform data migration.

[0258] It should be noted that after step S304, it may further include: The memory manager 101 updates the storage record instance to update the pointer. At this time, the pointer originally pointing to the second processor memory 301 in the storage record instance can be updated to point to the first processor memory 401.

[0259] It can be seen that the embodiments of the present application can divide the allocated storage areas by the priority threshold, determine the allocated storage areas with occupancy priorities exceeding the first priority threshold as high-priority storage areas, and determine the allocated storage areas with occupancy priorities not exceeding the second priority threshold as low-priority storage areas. That is to say, for the high-priority storage areas, if they belong to the storage areas of the second processor memory 301, they are kept resident and no data migration is performed. Otherwise, the data stored in them is prefetched to the second processor memory 301 in advance. For the low-priority storage areas, if they belong to the storage areas of the second processor memory 301, the data stored in them is migrated to the first processor memory 401, otherwise they are kept resident and no data migration is performed.

[0260] In some implementation manners, the allocated storage areas can be sorted based on the occupancy priority, the allocated storage areas ranked at the front are determined as high-priority storage areas, and the allocated storage areas ranked at the back are determined as low-priority storage areas. The embodiments of the present application do not make specific limitations on this.

[0261] In this way, based on various key indicators, the occupancy priority of each unified memory can be overall measured, and then automatic data prefetching and data migration can be realized, making the dynamic migration behavior of the storage areas more reasonable and efficient, realizing the scheduling of the unified memory, avoiding frequent page in and out, reducing the occurrence of jitter, and avoiding the performance loss problem caused by jitter, thereby further improving the GPU resource utilization rate.

[0262] Further, after the first process 203 releases the unified memory, the scheduler 104 can immediately trigger a dynamic migration, select a storage area with the highest occupancy priority and appropriate size in the first processor memory 401, and prefetch the data stored in it to the second processor memory 301.

[0263] Specifically, after step S208, the following steps S401-S402 may further be included.

[0264] S401: The scheduler 104 migrates the data in the first processor memory 401 to the second processor memory 301.

[0265] Specifically, step S401 may include S4011: The scheduler 104 accesses the memory manager 101, traverses each storage record instance in the priority queue, and obtains the storage area information of each un-released allocated storage area. The storage area information includes the occupancy priority, the pointer to the allocated storage area, and the size of the allocated storage area.

[0266] It should be noted that after receiving the release result returned by the interceptor 202, the interceptor 202 may send the release result to the scheduler 104, so that the scheduler 104 can determine that a certain storage area has been released.

[0267] S402: Migrate the second target data in the first processor memory 401 to the second processor memory 301, where the second target data is the data in the storage area with the highest occupancy priority in the first processor 400.

[0268] Based on the storage area information, the scheduler 104 determines, from the allocated storage areas in the first processor memory 401 whose sizes are less than or equal to the released storage area, the one with the highest occupancy priority as the storage area to be migrated. Then, the data stored in the storage area to be migrated is the second target data. Prefetching the second target data in the storage area to be migrated to the second processor memory 301 can complete the dynamic migration.

[0269] It can be understood that after the migration step, the storage area for storing the migrated data is converted from the second processor memory 301 to the first processor memory 401.

[0270] It is worth noting that after step S402, S403 may further be included: The memory manager 101 updates the storage record instance to update the pointer. At this time, the pointer originally pointing to the first processor memory 401 in the storage record instance can be updated to point to the second processor memory 301.

[0271] In this way, data prefetching can be achieved, frequent page migrations can be avoided, and jitter can be reduced.

[0272] In some implementation manners, when the video memory is insufficient, the embodiments of the present application may also migrate the data in the storage area of the second processor memory 301 based on the occupancy priority.

[0273] Specifically, before step S206, the following steps S501 - S504 may further be included.

[0274] S501: When the size of the unallocated storage area in the second processor memory 301 is less than the first storage size, the scheduler 104 migrates the data in the second processor memory 301 to the first processor memory 401.

[0275] In this step, specifically, the first target data in the second processor memory 301 may be migrated to the first processor memory 401, where the first target data is the data in the storage area with the lowest occupancy priority in the second processor memory 301.

[0276] Specifically, step S501 may specifically include the following steps S5011 - S5013.

[0277] S5011: When the size of the unallocated storage area (remaining storage area) in the second - processor memory 301 is less than the size of the storage area requested by the first process 203 for allocation, the second - processor driver 302 returns a failure result to the interceptor 202.

[0278] S5012: In response to the failure result, the interceptor 202 sends a migration request to the scheduler 104.

[0279] S5013: In response to the migration request, the scheduler 104 migrates the first target data stored in the second - processor memory 301 by the third process to the first - processor memory 401. The first target data is the data in the storage area with the lowest occupancy priority in the second - processor memory 301.

[0280] In this step, the scheduler 104 can sort the occupancy priorities of the target storage areas. Among them, the target storage areas are the allocated storage areas in the second - processor memory 301, and the target storage areas are the allocated storage areas occupied by processes other than the first process 203. Then, the scheduler 104 can determine the target storage area with the lowest occupancy priority as the storage area to be migrated out. The process occupying this storage area to be migrated out can be referred to as the third process.

[0281] In some implementation manners, the scheduler 104 can also migrate the target data stored in the second - processor memory 301 by some processes to the first - processor memory 401 when the size of the remaining storage area in the second - processor memory 301 is lower than a preset space threshold. The preset space threshold is adjustable. For example, it can be equal to 2GB, 4GB, or 8GB. The embodiments of the present application do not make specific limitations on this. In this way, early migration can be achieved.

[0282] Further, step S5013 may include the following steps S5013 - 1 - S5013 - 3.

[0283] S5013 - 1: The scheduler 104 accesses the memory manager 101 and traverses each storage record instance in the priority queue to obtain the storage area information of each allocated storage area.

[0284] S5013 - 2: Based on the storage area information, the scheduler 104 determines the storage area to be migrated out.

[0285] In some implementation manners, the storage area to be migrated out may be the storage area occupied by the first process 203, or may be the storage area occupied by other processes other than the first process 203. The embodiments of the present application do not make specific limitations on this.

[0286] S5013 - 3: The scheduler 104 migrates the data in the storage area to be migrated to the first processor memory 401 and returns the migration result to the interceptor 202.

[0287] Further, after step S5013, step S5014 is included: The interceptor 202 calls the memory allocation API again based on the migration result.

[0288] In the embodiment of the present application, steps S5011 - S5014 can be executed in a loop until the video memory application of the first process 203 can be satisfied, and then step S206 is continued to be executed.

[0289] It can be understood that through the migration steps, the storage area for storing the migrated data is converted from the second processor memory 301 to the first processor memory 401.

[0290] In some implementation manners, after step S5013, it may further include: The memory manager 101 updates the storage record instance to update the pointer. At this time, the pointer in the storage record instance originally pointing to the second processor memory 301 can be updated to point to the first processor memory 401.

[0291] It should also be noted that in the embodiment of the present application, the duration of the lock - holding time slice when the process first holds the DTL lock can be equal to a preset duration, and the preset duration is greater than the context time slice length of the underlying driver of the second processor 300, for example, equal to 20 ms, 50 ms, or 100 ms. The embodiment of the present application does not make specific limitations on this. Further, the lock - holding time slice for each subsequent time the process holds the DTL lock can be calculated by the scheduler 104. That is to say, the lock - holding time slice for each process holding the DTL lock can be dynamically adjusted to weaken the jitter effect under the unified memory architecture.

[0292] Continue to refer to Figure 2 , specifically, the scheduler 104 can query the priority (querypriority) through the priority evaluator 102, and then update the lock - holding time slice of the DTL lock based on the occupancy priority (Refresh DTL). The above steps can also be executed in a periodic loop, that is, executed once in each loop.

[0293] Specifically, the method provided in the embodiment of the present application may further include the following steps S601 - S602.

[0294] Among them, steps S601 - S603 can be executed after step S102.

[0295] S601: The scheduler 104 determines the process priority of each process at a first preset frequency. The process priority is equal to the sum of the occupancy priorities of the storage areas occupied by the process.

[0296] Further, step S601 may include the following steps S6011 - S6012.

[0297] S6011: The scheduler 104 accesses the memory manager 101 at a first preset frequency, and periodically obtains the process ID and occupancy priority corresponding to each allocated storage area from the storage record instance.

[0298] Wherein, the second preset frequency is, for example, once every 30 ms, once per second, once every ten seconds, or once per minute. The embodiments of the present application do not make specific limitations thereto.

[0299] S6012: The scheduler 104 sums up the occupancy priorities of the allocated storage areas occupied by the same process based on the process ID to obtain the process priority.

[0300] S602: The scheduler 104 determines the duration of the lock - holding time slice corresponding to each process when holding the DTL lock next time based on the process priority.

[0301] In this step, the scheduler 104 may determine the dynamic duration corresponding to each process based on the process priority. The dynamic duration is equal to the duration of the lock - holding time slice corresponding to the process when holding the DTL lock next time.

[0302] It can be seen that the scheduler 104 can calculate the occupancy priorities of all allocated storage areas under each process, and control the time for each process to hold the DTL lock based on this. The process with a higher total storage area priority (process priority) will obtain more running time when occupying the GPU next time.

[0303] In addition, the policy controller 105 is a configured storage center and can store a configuration file. The scheduler 104 can also maintain information such as the lock - holding time slice in the configuration file.

[0304] Specifically, after step S602, the following step S603 may further be included.

[0305] S603: The scheduler 104 accesses the policy controller 105 and writes the duration of the lock - holding time slice into the configuration file.

[0306] Further, the user can perform a configured override update on the policy controller 105 through the client 200. Continue to refer to Figure 1 and Figure 2, the user can perform a first operation through the client 200. The first operation is to read a configuration file (readconfig) and inject configuration. Injecting configuration means modifying the duration of the lock-holding time slice corresponding to the target process in the configuration file. The target process is one of the processes sharing the second processor 300.

[0307] At this time, the scheduler 104 can, in response to the first operation, update the duration of the lock-holding time slice corresponding to the target process based on the configuration file. In this way, when the target process holds the DTL lock next time, the lock-holding duration is equal to the duration specified by the first operation.

[0308] In some implementation manners, if the user specifies the lock-holding duration of the target process, then before the target process is destroyed, the lock-holding duration of the target process each time it holds the DTL lock can be equal to the duration specified by the user.

[0309] The following introduces the process of automatic video memory resource allocation in combination with an example in the scenario of multiple processes sharing the GPU.

[0310] Exemplarily, there can be two processes sharing a second processor 300. The two processes are process A (pid: 10020) and process B (pid: 10030) respectively. The memory size of the second processor 300 can be 32GB. After the initialization process of the controller 100 is completed, the process of automatic video memory resource allocation can include the following stage one and stage two.

[0311] Stage one: Process A occupies the second processor 300 and starts to execute tasks.

[0312] Process A can preferentially initiate a task call request, such as calling cuInit(). cuInit() is a CUDA Driver API used to initialize the CUDA driver.

[0313] The interceptor 202 can intercept this API and check the lock-holding status of process A.

[0314] In the case where process A does not hold the DTL lock, the interceptor 202 can send a first lock request to the controller 100. Since the DTL lock is idle (i.e., the target condition is not established), the controller 100 can issue the DTL lock to process A, and process A can enter the running state.

[0315] At this time, if process B also initiates a task call request, the interceptor 202 intercepts this call request and applies to the controller 100 for locking. Since the DTL lock is already held by process A, the controller 100 will reject the lock request of process B, and the task call of process B will be blocked.

[0316] Process A can continuously initiate CUDA calls, and the memory allocation requests therein will be redirected to call the memory management API.

[0317] Figure 10 Schematic diagram of the storage area occupied by Process A provided by the embodiment of the present application.

[0318] As Figure 10 shown in (a) therein, Process A can initiate two video memory allocations, respectively applying to occupy 8GB and 16GB of memory.

[0319] As Figure 10 shown in (b) therein, the memory manager 101 can record the results of these two allocations. Among them, the occupancy priority "NA" indicates that the occupancy priority is empty.

[0320] Furthermore, after triggering the loop of the scheduler 104, the priority evaluator 102 can calculate the occupancy priority of each allocated storage area, and the calculation results are as Figure 10 shown in (c) therein.

[0321] Phase two: Process B occupies the second processor 300 and starts to execute tasks.

[0322] After the holding time slice of Process A expires, the controller 100 will recycle the DTL lock and allocate the DTL lock to Process B. At this time, Process B can enter the running state and Process A is blocked.

[0323] The memory allocation requests initiated by Process B will be redirected to call the memory management API, and the memory manager 101 can record the allocation results of Process B.

[0324] Figure 11 Schematic diagram of the storage area occupied by Process B provided by the embodiment of the present application.

[0325] For example, Process B applies for two video memory resources, which are 12GB and 4GB respectively. Since there is only 8GB of available video memory left on the second processor 300, which is not enough to allocate the resources requested by Process B, the scheduler 104 can trigger dynamic migration at this time, that is, select the one with a lower occupancy priority in the storage area occupied by Process A and migrate it out to the first processor memory 401. As Figure 11 shown in (a) therein, since the occupancy priority of the storage area corresponding to number 1 is 4, which is relatively smaller, memory migration can be triggered to migrate the data in the 8G video memory allocated on the second processor 300 to the first processor memory 401, while the data in the storage area corresponding to number 2 with a higher occupancy priority still remains in the second processor memory 301.

[0326] After that, the video memory allocation of Process B is as Figure 11 shown in (b) therein, and the occupancy priority evaluation is asFigure 11 as shown in (c) below.

[0327] During the execution of Process B, if the storage area corresponding to No. 3 is released, the scheduler 104 can immediately trigger a dynamic migration, that is, select a storage area with the highest occupancy priority and appropriate size in the first processor memory 401, and prefetch the stored data to the second processor memory 301. In this example, since only the data in one storage area is migrated to the first processor memory 401, the data in the storage area corresponding to No. 1 will be prefetched to the second processor memory 301, as Figure 11 shown in (d) below.

[0328] After that, the lock-holding time slice of Process B expires, Process B can release the second processor 300, and Process A can hold the DTL lock and continue to execute.

[0329] In addition, the scheduler 104 can also update the active time of the DTL lock. As Figure 11 shown in (c) below, the total memory priority (process priority) of Process A is higher. Therefore, when Process A holds the DTL lock for the second time, it can obtain a longer lock-holding time than the first time. In contrast, the time for Process B to hold the DTL lock for the second time may be dynamically reduced.

[0330] Through the above Phase 1 and Phase 2, unified memory allocation, as well as dynamic prefetching and migration operations, can be achieved to minimize performance loss.

[0331] In summary, the embodiments of the present application can utilize unified memory to allow oversubscription of the GPU. Moreover, the embodiments of the present application cover the time slice scheduling effect of the CUDA underlying driver through the scheduling of the DTL lock, avoiding frequent page in and out. And the embodiments of the present application introduce a memory management and priority evaluation algorithm, and through a dynamic migration strategy, prefetching and migration are achieved, avoiding the occurrence of page faults and the performance loss problem caused by jitter, thereby further improving the utilization rate of GPU resources.

[0332] Figure 12 This is the second process schematic diagram of the processing resource allocation method provided by the embodiments of the present application.

[0333] As Figure 12 shown, the processing resource allocation method provided by the embodiments of the present application may include the following steps S100 - S300.

[0334] S100: Intercept the memory allocation request of the first process 203 for the second processor 300. The memory allocation request is used to request the second processor 300 to allocate a storage area with a first storage size for the first process 203;

[0335] S200: Determine the lock-holding status of the first process 203 on the dynamic time slice DTL lock;

[0336] S300: When the first process 203 holds the DTL lock, call the second processor 300 based on the memory allocation request, so that the second processor 300 allocates a first storage area in the second processor memory 301 for the first process 203.

[0337] The method provided by the embodiments of the present application can utilize unified memory to achieve oversubscription of the GPU. Moreover, this solution controls the blocking or running of the process through the scheduling of the DTL lock, avoiding frequent page in and out caused by frequent process switching.

[0338] In the above embodiments provided by the present application, the various solutions of the processing resource allocation method provided by the present application are introduced from the perspective of the computing device. It can be understood that in order for the computing device to implement the above functions, it includes the corresponding hardware structure and / or software module for executing each function. Those skilled in the art should easily realize that, combining the units and algorithm steps of the examples described in the embodiments disclosed in this article, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but this implementation should not be considered to exceed the scope of the present application.

[0339] Figure 13 This is a schematic structural diagram of the processing resource allocation device provided by the embodiments of the present application.

[0340] Corresponding to the foregoing embodiments of the processing resource allocation method, the present application also provides an embodiment of the processing resource allocation device 1100. As Figure 13 shown, the processing resource allocation device 1100 may include an interception module 1101 and a control module 1102. Among them,

[0341] The interception module 1101 is configured to: intercept the memory allocation request of the first process 203 for the second processor 300, and the memory allocation request is used to request the second processor 300 to allocate a storage area with a first storage size for the first process 203;

[0342] In addition, the interception module 1101 is further configured to: determine the lock-holding status of the first process 203 on the dynamic time slice DTL lock;

[0343] The control module 1102 is configured to: when the first process holds the DTL lock, call the second processor 300 based on the memory allocation request, so that the second processor 300 allocates a first storage area in the second processor memory 301 for the first process 203.

[0344] In a possible implementation, the control module 1102 is further configured to: block the first process 203 when the first process 203 does not hold the DTL lock and the target condition is satisfied, where the target condition includes that there is a process holding the DTL lock and / or the holding time slice of at least one second process holding the DTL lock has not expired; or, issue the DTL lock to the first process 203 when the first process 203 does not hold the DTL lock and the target condition is not satisfied; or, block the first process 203 after the holding time slice of the DTL lock held by the first process 203 has expired.

[0345] In a possible implementation, the control module 1102 is further configured to: redirect the memory allocation request to the memory allocation application programming interface API, and the memory allocation API is used to enable the second processor 300 to allocate a first storage area in the memory of the second processor 300 for the first process 203 based on the memory allocation API.

[0346] In a possible implementation, the control module 1102 is further configured to: migrate the data in the second processor memory 301 to the first processor memory 401 when the size of the unallocated storage area in the second processor memory 301 is less than the first storage size.

[0347] In a possible implementation, the control module 1102 is further configured to: migrate the first target data in the second processor memory 301 to the first processor memory 401 when the size of the unallocated storage area in the second processor memory 301 is less than the first storage size, and the first target data is the data in the storage area with the lowest occupancy priority in the second processor memory 301.

[0348] In a possible implementation, the interception module 1101 is further configured to: intercept the memory release request of the first process 203 for the second processor 300, and the memory release request is used to request the second processor 300 to release the storage area of the second storage size occupied by the first process 203. The control module 1102 is further configured to: call the second processor 300 based on the memory release request to release the storage area of the second storage size in the second processor 300 occupied by the first process 203 when the first process 203 holds the DTL lock.

[0349] In a possible implementation, the control module 1102 is further configured to: redirect the memory release request to the memory release API, and the memory release API is used to enable the second processor 300 to reclaim the storage area of the second storage size in the second processor 300 occupied by the first process 203 based on the memory release API.

[0350] In a possible implementation, the control module 1102 is further configured to migrate the data in the first processor memory 401 to the second processor memory 301.

[0351] In a possible implementation, the control module 1102 is further configured to migrate the second target data in the first processor memory 401 to the second processor memory 301, where the second target data is the data in the storage area with the highest occupancy priority in the first processor 400.

[0352] In a possible implementation, the occupancy priority is positively correlated with the access frequency and / or the latest access time of the process accessing the storage area, and negatively correlated with the size and / or access latency of the storage area.

[0353] In a possible implementation, the control module 1102 is further configured to generate a storage record instance of the first storage area; the storage record instance includes one or more of the identity of the first process 203, a pointer to the first storage area, the size of the first storage area, or the occupancy priority of the first storage area; and store the storage record instance.

[0354] In a possible implementation, the control module 1102 is further configured to determine the process priority of each process at a first preset frequency, where the process priority is equal to the sum of the occupancy priorities of the storage areas occupied by the process; and based on the process priority, determine the duration of the lock-holding time slice corresponding to each process when holding the DTL lock next time.

[0355] Figure 14 Schematic diagram of a computing device provided for some embodiments of the present application.

[0356] As Figure 14 shown, the computing device may include devices such as a server, a terminal, etc.; the computing device includes one or more processors 1201 and a memory 1202. The memory 1202 is configured to store one or more programs. Wherein, when the one or more programs are executed by the one or more processors 1201, the one or more processors 1201 implement the processing resource allocation method in the above embodiments.

[0357] Continue to refer to Figure 14 , the computing device 1200 may further include: a communication interface 1203 and a communication bus 1204.

[0358] Wherein, the processor 1201, the memory 1202, and the communication interface 1203 complete communication with each other through the communication bus 1204. The communication interface 1203 is used to communicate with network elements of other devices such as clients or other servers.

[0359] In some embodiments, one or more processors 1201 are used to execute one or more programs 1205, and specifically, can execute the relevant steps in the above-mentioned embodiments of the processing resource allocation method. Specifically, the program 1205 may include program code, and the program code includes computer-executable instructions.

[0360] Exemplarily, the processor 1201 may include a central processing unit 1021a (CPU) and a graphics processing unit 1021b (GPU). The processor 1201 may also be an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement some embodiments of the present application. The one or more processors that the computing device 1200 may include may be of the same type of processor, such as one or more CPUs; or may be different types of processors, such as one or more CPUs, one or more GPUs, and one or more ASICs.

[0361] In some embodiments, the memory 1202 is used to store one or more programs 1205. The memory 1202 may include high-speed RAM memory, and may also include non-volatile memory (NVM), such as at least one disk memory.

[0362] The program 1205 can specifically be called by the processor 1201 to cause the computing device 1200 to perform the operations of the processing resource allocation method.

[0363] Some embodiments of the present application provide a computer-readable storage medium. The computer-readable storage medium stores at least one executable instruction. When the executable instruction runs on the computing device 1200, it causes the computing device 1200 to execute the processing resource allocation method in the above-mentioned embodiments.

[0364] The executable instruction can specifically be used to cause the computing device 1200 to perform the operations of the processing resource allocation method.

[0365] For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0366] The beneficial effects that can be achieved by the readable storage medium provided in some embodiments of the present application can refer to the beneficial effects in the corresponding processing resource allocation method provided above, and will not be elaborated here.

[0367] It should be noted that in the application, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0368] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and reference can be made to the relevant parts of the method embodiments for the related content.

[0369] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus or device), or in conjunction with such instruction execution systems, apparatus or devices.

[0370] For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device.

[0371] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion having one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM).

[0372] In addition, a computer-readable medium can even be paper or other suitable media on which a program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation or other suitable processing as necessary, and then stored in a computer memory. It should be understood that various parts of the present application can be implemented by hardware, software, firmware or a combination thereof.

[0373] In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having suitable combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0374] The above embodiments are only specific embodiments of the present application and are not used to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the present application shall be included in the protection scope of the present application.

Claims

1. A method for processing resource allocation, characterized in that, Applied to a first processor, including: Intercept a memory allocation request of a first process for a second processor, where the memory allocation request is used to request the second processor to allocate a storage area with a first storage size for the first process; Determine the lock-holding status of the first process for a dynamic time slice DTL lock; When the first process holds the DTL lock, call the second processor based on the memory allocation request, so that the second processor allocates a first storage area in the second processor memory for the first process.

2. The processing resource allocation method according to claim 1, wherein After determining the lock-holding status of the first process for the dynamic time slice DTL lock, it further includes: When the first process does not hold the DTL lock and a target condition is satisfied, block the first process, where the target condition includes that there is a process holding the DTL lock, and / or the lock-holding time slice of at least one second process holding the DTL lock has not expired; Alternatively, when the first process does not hold the DTL lock and the target condition is not satisfied, issue the DTL lock to the first process; Alternatively, when the first process holds the DTL lock and the lock-holding time slice of holding the DTL lock has expired, block the first process.

3. The processing resource allocation method according to claim 2, wherein When the first process holds the DTL lock, calling the second processor based on the memory allocation request, so that the second processor allocates a first storage area in the second processor memory for the first process, includes: Redirect the memory allocation request to a memory allocation application programming interface API, where the memory allocation API is used to enable the second processor to allocate the first storage area in the second processor memory for the first process based on the memory allocation API.

4. The processing resource allocation method according to any one of claims 1-3, characterized in that Before, when the first process holds the DTL lock, calling the second processor based on the memory allocation request, so that the second processor allocates a first storage area in the second processor memory for the first process, it further includes: When the size of the unallocated storage area in the second processor memory is less than the first storage size, migrate the data in the second processor memory to the memory of the first processor.

5. The processing resource allocation method according to claim 4, wherein When the size of the unallocated storage area in the second processor memory is less than the first storage size, migrating the data in the second processor memory to the memory of the first processor specifically includes: When the size of the unallocated storage area in the second processor memory is less than the first storage size, migrate the first target data in the second processor memory to the memory of the first processor, where the first target data is the data in the storage area with the lowest occupancy priority in the second processor memory.

6. The processing resource allocation method according to claim 1, characterized in that The method further includes: Intercept a memory release request of the first process for the second processor, where the memory release request is used to request the second processor to release a storage area with a second storage size occupied by the first process; When the first process holds the DTL lock, call the second processor based on the memory release request to release a storage area of a second storage size in the second processor occupied by the first process.

7. The processing resource allocation method according to claim 6, wherein When the first process holds the DTL lock, calling the second processor based on the memory release request to release a storage area of a second storage size in the second processor occupied by the first process includes: Redirect the memory release request to a memory release API, which is used to cause the second processor to reclaim the storage area of the second storage size in the second processor occupied by the first process based on the memory release API.

8. The processing resource allocation method according to any one of claims 6-7, characterized in that, After calling the second processor based on the memory release request to release a storage area of a second storage size in the second processor occupied by the first process when the first process holds the DTL lock, it further includes: Migrate the data in the memory of the first processor to the memory of the second processor.

9. The processing resource allocation method according to claim 8, wherein The migrating the data in the memory of the first processor to the memory of the second processor specifically includes: Migrate the second target data in the memory of the first processor to the memory of the second processor, where the second target data is the data in the storage area with the highest occupancy priority in the first processor.

10. The processing resource allocation method according to claim 5 or 9, characterized in that, The occupancy priority is positively correlated with the access frequency and / or the latest access time of the process accessing the storage area, and negatively correlated with the size and / or access latency of the storage area.

11. The processing resource allocation method according to claim 1, wherein After calling the second processor based on the memory allocation request to allocate a first storage area in the second processor memory for the first process when the first process holds the DTL lock, it further includes: Generate a storage record instance for the first storage area; the storage record instance includes one or more of the identifier of the first process, a pointer to the first storage area, the size of the first storage area, or the occupancy priority of the first storage area; Store the storage record instance.

12. The processing resource allocation method according to claim 1, wherein The method further includes: Determine the process priority of each process at a first preset frequency, where the process priority is equal to the sum of the occupancy priorities of the storage areas occupied by the process; Based on the process priority, determine the duration of the lock-holding time slice corresponding to each process when the DTL lock is held next time.

13. A computing device, characterized in that, It includes: A memory for storing computer instructions; A processor, where the processor includes a first processor and a second processor. The first processor is used to execute the computer instructions to implement the processing resource allocation method according to any one of claims 1-12, and the second processor is used to allocate a first storage area in the second processor memory for the first process in response to the call of the first processor when the first processor executes the computer instructions.

Images