Memory resource allocation method and device, storage medium and electronic device

By providing the deep learning framework with the basic interface group determined by memory type information, the problem of high coupling between memory management algorithms and memory types is solved, the universality and multi-card support of memory management algorithms are realized, and the development cost is reduced.

CN119025281APending Publication Date: 2024-11-26SHANGHAI SENSETIME INTELLIGENT TECH CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411217135.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In the prior art, memory management algorithms in deep learning frameworks are highly coupled with memory types, resulting in memory management algorithms that cannot be generalized and need to develop special algorithms for different memory types, resulting in large adaptation workloads, high technical requirements, and many repetitive work.

Method used

By providing a memory resource allocation method for deep learning frameworks, the pre-configured basic interface groups are determined using memory type information, and these interface groups are called through memory management algorithms to allocate memory resources for training tasks, realizing memory management algorithms and memory types Decoupling.

Benefits of technology

The memory management algorithm is decoupled from memory types, so that the memory management algorithm can be called across different memory types, simplifying memory management logic, reducing development and maintenance costs, and supporting memory resource management of multiple cards and multiple AI-accelerated chips .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025281A_ABST
    Figure CN119025281A_ABST
Patent Text Reader

Abstract

The invention discloses a memory resource allocation method and device, a storage medium and an electronic device, and the method comprises the steps: allocating a first memory management algorithm for a memory call request sent by a deep learning framework, the memory call request being used for requesting to call a target memory for a training task, the memory calling request at least carries memory type information of the target memory and memory size information of the first memory resource; a first basic interface group configured for a target memory in advance is determined according to the memory type information, memories of at least two memory types are deployed in the device comprising the deep learning framework, the at least two memories comprise the target memory, the first basic interface group is a packaging interface group of a special interface of the target memory, and the packaging interface group is a packaging interface group of a special interface of the target memory. The packaging interface group is allowed to be called by any memory management algorithm; and calling the first basic interface group through a first memory management algorithm to allocate a first memory resource with the memory size indicated by the memory size information for the training task.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of deep learning frameworks, and more specifically, to a method and device for allocating memory resources, a storage medium, and an electronic device. Background Art

[0002] A new round of scientific and technological revolution and industrial transformation is reshaping the global economic structure. Computing power (i.e. computing capacity), as a new productivity in the digital economy era, has become a new focus of global strategic competition and an important engine for national economic development. In related technologies, artificial intelligence (AI) acceleration chips are used to provide faster and more efficient computing power for AI tasks, and memory management algorithms are used to better utilize the performance of AI acceleration chips.

[0003] At present, there are many mature memory management algorithms that are widely used in various operating systems and applications. In the deep learning framework, the memory management algorithm is mainly used to manage different memory types such as host memory, device memory (i.e., memory in AI acceleration chips), and page-locked memory. However, the memory management algorithms used in the deep learning framework are often specially developed for different memory types and are highly coupled with different memory types. For example, in the related art, dedicated memory management algorithms are developed for AI acceleration chips from different manufacturers.

[0004] Regarding the problem of high coupling between memory management algorithms and memory types used in deep learning frameworks in related technologies, no effective solution has been proposed yet. Summary of the invention

[0005] Embodiments of the present application provide a method and device for allocating memory resources, a storage medium, and an electronic device.

[0006] According to an embodiment of the embodiments of the present application, a method for allocating memory resources is provided, comprising: allocating a first memory management algorithm for a memory call request sent by a deep learning framework, wherein the memory call request is used to request to call a target memory for a training task, and the memory call request carries at least memory type information of the target memory and memory size information of a first memory resource that needs to be called from the target memory; determining a first basic interface group pre-configured for the target memory based on the memory type information, wherein a device including the deep learning framework is deployed with at least two memory types of memories, at least two memories including the target memory, wherein the first basic interface group is an encapsulation interface group of a proprietary interface of the target memory, and the encapsulation interface group allows being called by any memory management algorithm; calling the first basic interface group through the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information to the training task.

[0007] In an exemplary embodiment, a first memory management algorithm is assigned to a memory call request sent by a deep learning framework, including: when the memory call request carries an algorithm identifier, assigning the first memory management algorithm to the memory call request through a global list, wherein the algorithm identifier is used to uniquely identify the first memory management algorithm, and the global list includes the algorithm identifier; when the memory call request does not carry the algorithm identifier, using a default memory management algorithm as the first memory management algorithm assigned to the memory call request.

[0008] In an exemplary embodiment, the first memory management algorithm is assigned to the memory call request through a global list, including: obtaining registration information of a second memory management algorithm recorded in the global list, wherein the second memory management algorithm is encapsulated in a memory algorithm manager through a preset registration mechanism, and the second memory management algorithm includes: the first memory management algorithm, the registration information at least includes an algorithm identifier of the second memory management algorithm, and the memory algorithm manager is at least used to pre-configure the first basic interface group for the target memory; and assigning the first memory management algorithm to the memory call request through the registration information.

[0009] In an exemplary embodiment, before assigning a first memory management algorithm to a memory call request sent by a deep learning framework, the method further includes: receiving the second memory management algorithm registered through the preset registration mechanism; performing an encapsulation operation on the second memory management algorithm, and updating a historical global list through the second memory management algorithm to obtain the global list.

[0010] In an exemplary embodiment, an encapsulation operation is performed on the second memory management algorithm, including: encapsulating the second memory management algorithm into an algorithm implementation object; adding a calling interface to the algorithm implementation object to complete the encapsulation operation, wherein the calling interface is used to call the second basic interface group pre-configured by the memory algorithm manager for the at least two memories, at least two of the second basic interface groups include the first basic interface group.

[0011] In an exemplary embodiment, the historical global list is updated by the second memory management algorithm to obtain the global list, including: determining the registration information of the second memory management algorithm; assigning annotation information to the second memory management algorithm according to the registration information, wherein the annotation information includes at least the priority assigned to the second memory management algorithm; and adding the registration information and the annotation information to the historical global list to obtain the global list.

[0012] In an exemplary embodiment, determining a first basic interface group pre-configured for the target memory based on the memory type information includes: obtaining configuration information of the memory algorithm manager for the at least two memories, wherein the configuration information is used to indicate a second basic interface group corresponding to each memory, respectively, the second basic interface group being a packaged interface group of a proprietary interface pre-configured for each memory; determining the first basic interface group from the second basic interface group through the memory type information, wherein the memory type of the first basic interface group is consistent with the memory type indicated by the memory type information.

[0013] In an exemplary embodiment, calling the first basic interface group through the first memory management algorithm to allocate a first memory resource with the memory size to the training task includes: calling the first basic interface group through the first memory management algorithm to allocate a first memory resource with the memory size indicated by the memory size information to the training task, including: determining a first asynchronous resource pool instantiated by the memory algorithm manager for the first memory management algorithm, wherein the at least two memory types correspond one-to-one to at least two of the first asynchronous resource pools; determining a second asynchronous resource pool from at least two of the first asynchronous resource pools according to the memory type information, wherein the second asynchronous resource pool is used to store a second memory resource divided from the target memory corresponding to the memory type information; allocating a first memory resource with the memory size to the training task through the second asynchronous resource pool and the first basic interface group.

[0014] In an exemplary embodiment, a first memory resource with the memory size is allocated to the training task through the second asynchronous resource pool and the first basic interface group, including: determining whether a third memory resource exists in a third asynchronous resource pool corresponding to the first memory management algorithm, wherein the third memory resource is a second memory resource in a ready state recovered from the second asynchronous resource pool by the first memory management algorithm, and the first asynchronous resource pool includes the third asynchronous resource pool; in a case where the third memory resource exists in the third asynchronous resource pool, a first memory resource with the memory size is allocated to the training task from the third memory resource; in a case where the third memory resource does not exist in the third asynchronous resource pool, dividing the first memory resource from the target memory through the first basic interface group.

[0015] In an exemplary embodiment, allocating a first memory resource with the memory size to the training task from the third memory resource includes: determining the task type of the training task; determining a fourth memory resource in the third memory resource that has executed a historical training task with the task type; and allocating a first memory resource with the memory size to the training task from the fourth memory resource.

[0016] According to another embodiment of the embodiments of the present application, a memory resource allocation device is also provided, including: an allocation module, used to allocate a first memory management algorithm for a memory call request sent by a deep learning framework, wherein the memory call request is used to request to call a target memory for a training task, and the memory call request carries at least memory type information of the target memory and memory size information of a first memory resource that needs to be called from the target memory; a determination module, used to determine a first basic interface group pre-configured for the target memory according to the memory type information, wherein the device including the deep learning framework is deployed with at least two memory types of memory, at least two memories including the target memory, wherein the first basic interface group is an encapsulation interface group of a proprietary interface of the target memory, and the encapsulation interface group allows being called by any memory management algorithm; a calling module, used to call the first basic interface group through the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information to the training task.

[0017] According to another embodiment of the embodiment of the present application, a memory resource allocation system is also provided, including: a memory algorithm manager, and a deep learning framework connected to the memory algorithm manager, wherein the memory algorithm manager is used to allocate a first memory management algorithm for a memory call request sent by the deep learning framework, wherein the memory call request is used to request to call a target memory for a training task, and the memory call request carries at least memory type information of the target memory and memory size information of a first memory resource that needs to be called from the target memory; it is also used to determine a first basic interface group pre-configured for the target memory based on the memory type information, wherein a device including the deep learning framework is deployed with at least two memory types of memory, at least two memories including the target memory, wherein the first basic interface group is an encapsulation interface group of a proprietary interface of the target memory, and the encapsulation interface group allows being called by any memory management algorithm; it is also used to call the first basic interface group through the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information to the training task.

[0018] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned memory resource allocation method when running.

[0019] According to another aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the memory resource allocation method through the computer program.

[0020] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, wherein the computer program is executed by a processor to perform the method.

[0021] In an embodiment of the present application, when a memory call request sent by a deep learning framework is received to request to call a target memory in a memory of at least two memory types deployed in a device including the deep learning framework for a training task, a first memory management algorithm is allocated to the memory call request; and a first basic interface group pre-configured for the target memory is determined according to the memory type information carried in the memory call request, the first basic interface group being a packaged interface group of a proprietary interface of the target memory, and the packaged interface group is allowed to be called by any memory management algorithm; the first basic interface group is called by the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information carried in the memory call request to the training task. That is, through the first basic interface group, memories of different memory types are decoupled from the memory management algorithm, so that the memory management algorithm can call the target memory by calling the first basic interface group. Furthermore, through the embodiment of the present application, the problem of high coupling between the memory management algorithm and the memory type applied to the deep learning framework in the related art can be solved. The technical effect of decoupling the memory management algorithm from the memory type is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0024] Figure 1 is a flowchart of a method for allocating memory resources according to an embodiment of the present application;

[0025] Figure 2 is a schematic diagram of a memory resource allocation architecture according to an optional embodiment of the present application;

[0026] Figure 3 is a structural block diagram of a memory resource allocation device according to an embodiment of the present application (I);

[0027] Figure 4 is a structural block diagram of a memory resource allocation device according to an embodiment of the present application (II);

[0028] Figure 5 is a schematic diagram of an electronic device for implementing a method for allocating memory resources according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] In this embodiment, a method for allocating memory resources is provided, which is applied to a device deployed with a deep learning framework. Figure 1 : is a flowchart of a method for allocating memory resources according to an embodiment of the present application, the process comprising the following steps:

[0032] Step S102: allocating a first memory management algorithm to a memory call request sent by a deep learning framework, wherein the memory call request is used to request to call a target memory for a training task, and the memory call request carries at least memory type information of the target memory and memory size information of a first memory resource to be called from the target memory;

[0033] It should be noted that a deep learning framework is a software framework that simplifies and accelerates the development and deployment process of deep learning models. These frameworks provide a series of pre-implemented deep learning algorithms and model structures, allowing users to easily build, train and test their own deep learning models. Common deep learning frameworks include TensorFlow, PyTorch, Keras, etc. These frameworks usually provide rich APIs and tools, allowing users to flexibly customize their own deep learning models and run them efficiently on different hardware platforms. The above-mentioned devices deployed with deep learning frameworks include these different hardware platforms.

[0034] Optionally, in an embodiment of the present application, a memory algorithm manager is further deployed in the device including the deep learning framework, and the memory algorithm manager provides a unified memory usage interface for the deep learning framework, and the memory call request sent by the deep learning framework is sent to the memory algorithm manager through the unified memory usage interface. The memory algorithm manager assigns a first memory management algorithm to the memory call request.

[0035] Step S104: determining a first basic interface group pre-configured for the target memory according to the memory type information, wherein the device including the deep learning framework is deployed with at least two memory types of memories, at least two memories including the target memory, wherein the first basic interface group is a package interface group of a proprietary interface of the target memory, and the package interface group allows being called by any memory management algorithm;

[0036] like Figure 2 As shown, at least two memory types include: host memory (i.e. Figure 2 CPU Host memory in), page-locked memory (i.e. Figure 2 The CPU locked page memory in the deep learning framework), device memory; device memory refers to the memory in the AI ​​acceleration chip deployed in the device including the deep learning framework. The deep learning framework in the embodiment of the present application is equivalent to Figure 2 The upper frame in .

[0037] The first basic interface group is pre-configured by the memory algorithm manager for the target memory, and the memory algorithm manager is pre-configured with a second basic interface group for at least two types of memory deployed in a device including a deep learning framework, and at least two (or at least two) second basic interface groups include: the first basic interface group.

[0038] Step S106: calling the first basic interface group through the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information to the training task.

[0039] Through the above steps, when a memory call request sent by a deep learning framework is received to request to call a target memory in a memory of at least two memory types deployed in a device including the deep learning framework for a training task, a first memory management algorithm is allocated to the memory call request; and a first basic interface group pre-configured for the target memory is determined according to the memory type information carried in the memory call request, the first basic interface group being a packaged interface group of a proprietary interface of the target memory, and the packaged interface group is allowed to be called by any memory management algorithm; the first basic interface group is called by the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information carried in the memory call request to the training task. That is, through the first basic interface group, memories of different memory types are decoupled from the memory management algorithm, so that the memory management algorithm can call the target memory by calling the first basic interface group. Furthermore, through the embodiments of the present application, the problem of high coupling between the memory management algorithm and the memory type applied to the deep learning framework in the related art can be solved, and the technical effect of decoupling the memory management algorithm from the memory type is achieved.

[0040] In an exemplary embodiment, a first memory management algorithm is allocated to a memory call request sent by a deep learning framework, including two allocation methods:

[0041] Allocation method 1: when the memory call request carries an algorithm identifier, the first memory management algorithm is allocated to the memory call request through a global list, wherein the algorithm identifier is used to uniquely identify the first memory management algorithm, and the algorithm identifier is included in the global list;

[0042] If the deep learning framework needs to adopt a specific memory management algorithm (i.e., a specific first memory management algorithm), the memory call request sent by the deep learning framework to the memory algorithm manager will also carry the algorithm identifier of the specific memory management algorithm, so that the specific memory management algorithm can be assigned to the memory call request from the global list through the algorithm identifier.

[0043] It should be noted that before receiving the memory call request, at least one second memory management algorithm has been registered in the memory algorithm manager, and the global list is a list generated by registration information of at least one memory management algorithm, and the registration information includes the algorithm identifier.

[0044] Specifically, the first memory management algorithm is assigned to the memory call request through a global list, including: traversing the global list through the algorithm identifier until a field identical to the algorithm identifier is determined, and the memory management algorithm corresponding to the identical field is the first memory management algorithm.

[0045] Allocation method two: when the algorithm identifier is not carried in the memory call request, a default memory management algorithm is used as the first memory management algorithm allocated to the memory call request.

[0046] Optionally, when the first memory management algorithm cannot be determined through the global list and the algorithm flag, a default memory management algorithm may be used as the first memory management algorithm allocated to the memory call request.

[0047] In the case where the deep learning framework does not need to adopt a specific memory management algorithm, the memory algorithm manager can automatically assign a memory management algorithm for the received memory call request. Automatically assigning a memory management algorithm includes: assigning a default memory management algorithm. For example, the memory management algorithm corresponding to the first piece of information in the global list can be used as the default memory management algorithm. The first piece of information in the global list at least includes: the registration information of the memory management algorithm corresponding to the first piece of information.

[0048] Optionally, allocating the first memory management algorithm to the memory call request through a global list also includes: obtaining registration information of a second memory management algorithm recorded in the global list, wherein the second memory management algorithm is encapsulated in a memory algorithm manager through a preset registration mechanism, and the second memory management algorithm includes: the first memory management algorithm, the registration information at least includes an algorithm identifier of the second memory management algorithm, and the memory algorithm manager is at least used to pre-configure the first basic interface group for the target memory; allocating the first memory management algorithm to the memory call request through the registration information.

[0049] Optionally, the preset registration mechanism includes: allocator registration mechanism. The second memory management algorithm after the encapsulation operation is equivalent to Figure 2 The memory type in the single-device memory management algorithm A, B, etc. is independent.

[0050] The allocator registration mechanism is a mechanism for managing memory allocators (i.e., memory management algorithms), through which different memory allocators can be dynamically registered and switched. When the program is running, different memory allocators can be selected as needed to achieve more efficient memory allocation and management. The allocator registration mechanism can select the appropriate memory allocator in different scenarios, such as selecting a thread-safe memory allocator in a multi-threaded environment, and selecting a more efficient memory allocator in scenarios with frequent memory allocation. The allocator registration mechanism allows for more flexible management and optimization of memory allocation.

[0051] In an optional embodiment, before assigning a first memory management algorithm to a memory call request sent by a deep learning framework, the method further includes: receiving the second memory management algorithm registered through the preset registration mechanism; performing an encapsulation operation on the second memory management algorithm, and updating the historical global list through the second memory management algorithm to obtain the global list.

[0052] Optionally, new memory management algorithms can continue to be registered in the memory algorithm manager through a preset registration mechanism.

[0053] Furthermore, an encapsulation operation is performed on the second memory management algorithm, including: encapsulating the second memory management algorithm into an algorithm implementation object; adding a calling interface to the algorithm implementation object to complete the encapsulation operation, wherein the calling interface is used to call the second basic interface group pre-configured by the memory algorithm manager for the at least two memories, and at least two of the second basic interface groups include the first basic interface group.

[0054] It can be understood that when adding a calling interface for the algorithm implementation object that can call the second basic interface group that the memory algorithm manager pre-configured for at least two of the memories, if there are M second basic interface groups, M calling interfaces will also be added to the algorithm implementation object; where M is a positive integer.

[0055] Furthermore, the historical global list is updated by the second memory management algorithm to obtain the global list, including: determining the registration information of the second memory management algorithm; assigning annotation information to the second memory management algorithm according to the registration information, wherein the annotation information at least includes the priority assigned to the second memory management algorithm; and adding the registration information and the annotation information to the historical global list to obtain the global list.

[0056] Optionally, the registration information of the second memory management algorithm also includes: the algorithm name of the second memory management algorithm, the applicable device type (for example, the type of applicable AI acceleration chip). The memory algorithm manager will also assign priority and other annotation information to the second memory management algorithm based on the registration information. By default, the registration information and annotation information of the second memory management algorithm are added to the first blank row from top to bottom in the historical global list to obtain a global list. Optionally, the registration information and annotation information can also be added to the target row according to the priority assigned to the second memory management algorithm, the priority of the previous row of the target row is higher than the priority assigned to the second memory management algorithm, and the priority of the next row of the target row is lower than the priority assigned to the second memory management algorithm.

[0057] Specifically, determining a first basic interface group pre-configured for the target memory according to the memory type information includes: obtaining configuration information of the memory algorithm manager for the at least two memories, wherein the configuration information is used to indicate a second basic interface group corresponding to each memory, respectively, and the second basic interface group is a packaged interface group of a proprietary interface pre-configured for each memory; determining the first basic interface group from the second basic interface group according to the memory type information, wherein the memory type of the first basic interface group is consistent with the memory type indicated by the memory type information.

[0058] It is understandable that at least two memory types can be used as identifiers of the second basic interface group pre-configured for each memory. Optionally, when the memory algorithm manager and at least two memories are connected through the second basic interface group respectively, the configuration information is the connection topology information between the memory algorithm manager and the at least two memories.

[0059] In an exemplary embodiment, calling the first basic interface group through the first memory management algorithm to allocate a first memory resource with the memory size to the training task includes: calling the first basic interface group through the first memory management algorithm to allocate a first memory resource with the memory size indicated by the memory size information to the training task, including: determining a first asynchronous resource pool instantiated by the memory algorithm manager for the first memory management algorithm, wherein the at least two memory types correspond one-to-one to at least two of the first asynchronous resource pools; determining a second asynchronous resource pool from at least two of the first asynchronous resource pools according to the memory type information, wherein the second asynchronous resource pool is used to store a second memory resource divided from the target memory corresponding to the memory type information; allocating a first memory resource with the memory size to the training task through the second asynchronous resource pool and the first basic interface group.

[0060] Optionally, the memory algorithm manager instantiates an asynchronous resource pool for each memory type. For example, when at least two memory types include: host memory, page-locked memory, and device memory, the first asynchronous resource pool includes: an asynchronous resource pool instantiated for host memory, an asynchronous resource pool instantiated for page-locked memory, and an asynchronous resource pool instantiated for device memory. For example, when the memory type is device memory, the asynchronous resource pool instantiated for device memory is used as the second asynchronous resource pool. The first memory management algorithm can interact with the first asynchronous resource pool through abstract virtual functions.

[0061] It should be noted that the first asynchronous resource pool is a resource pool instantiated by the memory algorithm manager for the first memory management algorithm after the first memory management algorithm is registered with the memory algorithm manager and before the historical global list is updated. The annotation information of the first memory management algorithm generated according to the registration information of the first memory management algorithm also includes: a resource pool handle of the first asynchronous resource pool, which is also added to the historical global list together with the registration information to obtain a global list.

[0062] Furthermore, allocating a first memory resource with the memory size to the training task through the second asynchronous resource pool and the first basic interface group includes: determining whether a third memory resource exists in the third asynchronous resource pool corresponding to the first memory management algorithm, wherein the third memory resource is a second memory resource in a ready state recovered from the second asynchronous resource pool by the first memory management algorithm, and the first asynchronous resource pool includes the third asynchronous resource pool; in a case where the third memory resource exists in the third asynchronous resource pool, allocating a first memory resource with the memory size to the training task from the third memory resource; in a case where the third memory resource does not exist in the third asynchronous resource pool, dividing the first memory resource from the target memory through the first basic interface group.

[0063] It is understandable that the second memory resources allocated for each training task corresponding to the memory call request requesting to call the target memory will be placed in the second asynchronous resource pool. When the corresponding operation of the training task is completed, the second memory resources allocated for the training task will be in a ready state, wherein the corresponding operation includes: an operator operation using the second memory resource in the training task, and a collective communication operation using the second memory resource in the training task. The first memory management algorithm recycles the second memory resources in a ready state from the second asynchronous resource pool to the third asynchronous resource pool, so that when the next memory call request to the target memory is received, the target memory can be allocated without dividing it through the calling interface and the first basic interface group, but through the third memory resource that has been divided out in the third asynchronous resource pool and is not occupied by the training task at the current moment (i.e., in a ready state) for the training task corresponding to the next memory call request to the target memory. The fifth memory resource is allocated.

[0064] Furthermore, allocating a first memory resource with the memory size to the training task from the third memory resource includes: determining the task type of the training task; determining a fourth memory resource in the third memory resource that has executed historical training tasks with the task type; and allocating a first memory resource with the memory size to the training task from the fourth memory resource.

[0065] Optionally, allocating a first memory resource having the memory size to the training task from the third memory resource further includes: if there is a sixth memory resource greater than or equal to the memory size in the third memory resource, using the sixth memory resource as the first memory resource. If there is no sixth memory resource in the third memory resource, it is necessary to re-partition memory resources that meet or exceed the memory size from the target memory through the first basic interface group.

[0066] It should be noted that the second asynchronous resource pool is used to manage the divided second memory resources in an orderly manner. At the current moment, if the second memory resource is in a ready state, the memory resource before the second memory resource is considered to be ready; if the memory resource is not ready at the current moment, the memory resource before the second memory resource is considered to be not ready. That is, further judgment on the memory resource before the second memory resource is abandoned, so as to recycle the second memory resource in a ready state as soon as possible and improve resource utilization.

[0067] In order to better understand the process of the above-mentioned memory resource allocation method, the implementation method flow of the above-mentioned memory resource allocation is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of the present application.

[0068] In the related technology, there are many mature memory management algorithms, which are widely used in various operating systems and applications. In the deep learning framework, memory management algorithms are usually used to manage host memory, device memory, and page-locked memory.

[0069] Compared with the use of memory management algorithms in other operating systems and applications, the application of memory management algorithms in deep learning frameworks usually involves some issues that require special consideration: 1) In a training process (a process that executes training tasks), the central processing unit (CPU) and the device run asynchronously; in a training process, multiple CPU threads may run simultaneously; in a training process, multiple devices (multiple cards) may run asynchronously; in a training process, each device (card) may have multiple streams at the same time, and operators, communications and other operations are run sequentially on each stream, and different streams are performed asynchronously; the memory of each device is exclusive, and the memory on a device cannot be allocated to other devices for direct use; multiple memory types need to be managed (equivalent to at least two memory types in the above embodiment): host memory, page-locked memory, and device memory (multiple cards must be supported).

[0070] It should be noted that in an optional embodiment of the present application, both the device and the card are used to indicate an AI acceleration chip.

[0071] The main issue among these issues that require special consideration is: In the related technology, dedicated memory management algorithm implementations are provided for different memory types, and the memory management algorithms applicable to different memory types cannot be universal. If the memory management logic of multiple cards is coupled in one memory management algorithm, it will be difficult to debug. The memory management algorithm is coupled with the interface of proprietary devices and cannot be universal across devices. In addition, the interface design is complex, difficult to peel off, and difficult to debug and maintain. In short, the memory management algorithm of the current AI training framework (ie, deep learning framework) is highly coupled with the AI ​​acceleration chip. The training framework needs to develop a set of related logic for memory management for each AI chip. There is a large workload for adaptation, high technical requirements, and a lot of repetitive work.

[0072] To solve the above problems, the optional embodiments of the present application provide a set of solutions that can uniformly process asynchronous resources and synchronous resources, and maximize the reuse of various memory management algorithms in the existing synchronous mode without introducing additional overhead. A convenient and expandable allocator registration mechanism is also provided. It is only necessary to implement the memory management algorithm of a single device in synchronous mode, and after registration, multi-card and different types (i.e. different memory types) of memory resource management (device_memory, pin_memory, normal_memory) can be performed. By introducing an asynchronous resource pool, unready resources are placed in the resource pool, and subsequent processing algorithms directly take ready resources from the resource pool, so that the memory management algorithm is decoupled from the specific device, and synchronous resources and asynchronous resources are uniformly processed, which is convenient for access to a variety of AI acceleration training chips and a variety of memory management algorithms.

[0073] Specifically, Figure 2 As shown, an optional embodiment of the present application introduces a set of the most basic interface sets (equivalent to the multiple second basic interface groups in the above embodiments). By introducing a set of the most basic interface sets, accessing a new AI acceleration chip only requires adapting a set of the simplest basic interfaces, thereby realizing the decoupling of the AI ​​acceleration chip, locked page memory, ordinary memory and other memory management algorithms. The memory management algorithm focuses on implementing the logic of memory management without having to worry about specific devices and memory types. The specific AI acceleration chip only needs to implement the most basic interfaces without having to get involved in the specific details of the memory management algorithm. In the case where a new AI acceleration chip needs to be adapted, there is no need to modify the memory management algorithm.

[0074] Optionally, each second basic interface group may include but is not limited to the following calling functions:

[0075] Call function 1), event constructor, no input, output is the constructed event address;

[0076] Call function 2), event destructor, no output, input is the address of the event to be destructed;

[0077] Call function 3), the CPU waits for the event to complete the function, and the input is the address of the event to be waited for;

[0078] Call function 4) to record the event function on the stream;

[0079] Call function 5), the stream waits for the event to complete the function;

[0080] Among them, calling functions 1) to 5) is used for synchronization between the CPU and the AI ​​acceleration chip and between different streams of the AI ​​acceleration chip;

[0081] Call function 6), allocate page-locked memory function: input the required memory size and return the address of the allocated memory;

[0082] Call function 7), release the locked page memory function: input the address of the memory to be released, no output;

[0083] Among them, calling function 6) is used to apply for page-locked memory. Calling function 7) is used to release page-locked memory;

[0084] Call function 8) to allocate memory for the AI ​​acceleration chip: input the required memory size and return the allocated memory address;

[0085] Call function 9) to release the memory function on the AI ​​acceleration chip: input the memory address to be released, no output.

[0086] Among them, calling function 8) is used to apply for device memory, and calling function 9) is used to release device memory.

[0087] Furthermore, in the optional embodiment of the present application, an asynchronous resource pool is also introduced to achieve the unification of synchronous operation scenarios and asynchronous operation scenarios, so that a large number of mature memory management algorithms in synchronous scenarios can also be used in asynchronous scenarios, simplifying the design and implementation of memory management algorithms in asynchronous scenarios. The resources used asynchronously are first placed in the resource pool (equivalent to the second asynchronous resource pool in the above embodiment), and the cache algorithm only recycles the ready resources in the asynchronous resource pool (equivalent to the second memory resource in the above embodiment). The asynchronous resource pool is used to manage resources (i.e., memory resources) in an orderly manner. The currently ready resources assume that the previous resources are also ready, and the currently unready resources assume that the previous resources are also unready. At this time, further judgment is abandoned to improve performance. The ready resources are recycled as soon as possible to improve resource utilization. Specifically, for the ready memory resources in the asynchronous resources, the memory management algorithm can be processed like an ordinary memory management algorithm, including recycling to the system or reallocating to other requests, without considering the asynchronous operation. All the problems that need to be considered for asynchronous memory management on AI acceleration chips are solved by asynchronous resource pools. The design and implementation of the entire module have become independent, easier to implement, easier to modularize, and modules are decoupled from each other. At the same time, the processing logic of the asynchronous resource pool can be optimized to optimize performance. For the objective differences in synchronous and asynchronous processing between ordinary host memory and host page-locked memory, AI acceleration chip memory can be targeted to design a more matching asynchronous resource pool to obtain better performance.

[0088] In the optional embodiments of the present application, sufficient abstraction is made for the asynchronous resource pool. The resource type can be used as a template parameter and can be specialized to the resource type required for the memory management algorithm to implement. The memory management algorithm interacts with the asynchronous resource pool through a unified abstract virtual function. The memory management algorithm can customize the asynchronous resource pool for different devices and different algorithms according to actual conditions. It only needs to specialize AsyncResourcePoolImpl according to the device type and algorithm id. Each resource in the asynchronous resource pool corresponds to an actual event status queue. When all states in the event status queue are ready, it means that there is no object using this resource, and the memory management algorithm (that is, the first memory management algorithm mentioned above) can perform recycling operations. If a non-default stream uses this resource, when adding this resource to the resource pool, add the event corresponding to this stream, which means that this resource is in use by this stream.

[0089] The way to design an asynchronous resource pool is as follows:

[0090] The asynchronous resource pool is designed as a template class, which inherits from the abstract resource pool template base class. The reason for designing it as a template class is that the resource types required to be saved by the asynchronous resource pool can be customized as needed, and different memory management algorithms may require different resource types to be saved. For example, in the above embodiment, at least two asynchronous resource pools can be instantiated based on at least two types of memory. The abstract resource pool template base class provides four interfaces, which are the conventions for the interaction between the memory management algorithm and the asynchronous resource pool. To implement the asynchronous resource pool, you need to inherit this base class. The four interfaces are:

[0091] The first interface is the interface for adding resources to the asynchronous resource pool. It needs to give the resources that need to be temporarily managed by asynchronous resources (resources that are not convenient to immediately determine whether they can be recycled, or resources that cannot be immediately recycled by the memory management algorithm, or resources that cannot be immediately allocated to other memory requests, such as the second memory resource in the unready state in the above embodiment, the first memory resource just allocated for the training task, etc.), and the event set that records the use of this resource by the stream. This function is a pure virtual function and needs to be implemented by subclasses.

[0092] The second interface is used by the asynchronous resource pool for the memory management algorithm to determine whether there are ready resources. It returns true or false, where true indicates that there are ready resources and false indicates that there are no ready resources.

[0093] The third interface is used to obtain ready resources from the asynchronous resource pool and is used to return ready resources.

[0094] The fourth interface is used to determine how many resources are stored in the asynchronous resource pool, and is used to return the number of all resources stored in the resource pool.

[0095] Furthermore, the optional embodiments of the present application provide a flexible and convenient expansion mechanism through a registration mechanism (equivalent to the preset registration mechanism in the above embodiments), so as to add memory management algorithms to existing and future AI acceleration chips. It only needs to register the new implementation without modifying or recompiling the original code.

[0096] When registering a memory management algorithm, it is necessary to specify the device type that the algorithm is to support and the name of the memory management algorithm. When necessary, the registrar (the memory algorithm manager includes the registrar) specializes and instantiates an asynchronous resource pool (equivalent to the first asynchronous resource pool in the above embodiment) according to the device type and device ID, and then instantiates the memory management algorithm implementation object (equivalent to the algorithm implementation object in the above embodiment), and gives the resource pool handle (the address of the abstract base class) to the memory management algorithm for use. The algorithm name, device type, priority, memory management algorithm resource pool handle (abstract base class address) and other information are then put into a global list for maintenance. The upper-level user directly obtains the appropriate instance handle (i.e., resource pool handle) from the global list for use.

[0097] Optionally, different log masks are set in the process of calling the basic interface set (i.e., the first basic interface group), the asynchronous resource pool (i.e., the first asynchronous resource pool) interface, and obtaining the memory management implementation handle (i.e., obtaining the resource pool handle) from the registration manager through the memory management algorithm. In actual applications, masks can be set according to needs to view call log information and call parameters at different levels to quickly locate which module has a problem, making the entire module extremely transparent and easy to understand.

[0098] Through the memory algorithm implementation manager in the optional embodiment of the present application (equivalent to the memory algorithm manager in the above embodiment), the basic interface set, the appropriate asynchronous resource pool and the specific memory management algorithm can be organically combined, so that various types of memory such as synchronous and asynchronous memory, memory on AI acceleration chips from different manufacturers, and page-locked memory can be managed. And the memory management algorithm is decoupled from the multi-card scenario. The memory management algorithm only needs to implement the memory management logic on one card, without considering the applicability of multiple cards, which simplifies the design and implementation of the memory management algorithm.

[0099] Therefore, the optional embodiment of the present application realizes the decoupling of specific devices from memory management algorithms, and solves the problem of the need to develop dedicated memory management algorithms for specific devices in related technologies. The optional embodiment of the present application also provides an asynchronous resource pool, which simplifies the solution of asynchronous use of memory management algorithms, eliminates the need to design special memory management algorithms for asynchronous scenarios, reduces development costs and maintenance costs, and allows the generation of memory management algorithms that have been designed to be reused. In addition, the optional embodiment of the present application also designs a set of flexible and convenient extension mechanisms for different memory management algorithms. The newly added memory management algorithm does not need to modify the original code, and does not need to recompile the original code, and can be used in various memory management scenarios such as AI acceleration chips and page-locked memory. To switch the memory management algorithm, you only need to set an environment variable and restart; it solves the problems of difficulty in switching different memory management algorithms, high switching costs, high coupling between the implementation of the memory management algorithm and the calling code, and difficulty in separation.

[0100] Optionally, based on the solution proposed in the above contents of the optional embodiments of the present application, a memory management algorithm that is independent of the memory type of a single device can also be developed for the management of memory, page-locked memory, and ordinary memory on different AI acceleration chips in synchronous and asynchronous usage scenarios.

[0101] To sum up, through the optional embodiments of the present application, it will be possible to decouple different AI acceleration chips and memory management algorithms, and convert the traditional single chip and proprietary memory management algorithm adaptation process into a unified implementation that does not depend on the chip architecture, so that the training framework can provide memory management for multiple chips from different chip manufacturers through one adaptation.

[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0103] Figure 3 is a structural block diagram of a memory resource allocation device according to an embodiment of the present application (I); Figure 3 As shown, including:

[0104] An allocation module 32 is used to allocate a first memory management algorithm for a memory call request sent by a deep learning framework, wherein the memory call request is used to request to call a target memory for a training task, and the memory call request carries at least memory type information of the target memory and memory size information of a first memory resource to be called from the target memory;

[0105] A determination module 34 is used to determine a first basic interface group pre-configured for the target memory according to the memory type information, wherein the device including the deep learning framework is deployed with at least two memory types of memories, at least two memories including the target memory, wherein the first basic interface group is a package interface group of a proprietary interface of the target memory, and the package interface group allows being called by any memory management algorithm;

[0106] The calling module 36 is used to call the first basic interface group through the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information to the training task.

[0107] Through the above-mentioned device, when a memory call request sent by a deep learning framework is received to request to call a target memory in a memory of at least two memory types deployed in a device of the deep learning framework for a training task, a first memory management algorithm is allocated to the memory call request; and a first basic interface group pre-configured for the target memory is determined according to the memory type information carried in the memory call request, the first basic interface group is a package interface group of a proprietary interface of the target memory, and the package interface group allows to be called by any memory management algorithm; the first basic interface group is called by the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information carried in the memory call request to the training task. That is, through the first basic interface group, the memory of different memory types is decoupled from the memory management algorithm, so that the memory management algorithm can call the target memory by calling the first basic interface group. Furthermore, through the embodiment of the present application, the problem of high coupling between the memory management algorithm and the memory type applied to the deep learning framework in the related art can be solved. The technical effect of decoupling the memory management algorithm from the memory type is achieved.

[0108] In an exemplary embodiment, the allocation module 32 is also used to: when the memory call request carries an algorithm identifier, allocate the first memory management algorithm to the memory call request through a global list, wherein the algorithm identifier is used to uniquely identify the first memory management algorithm, and the global list includes the algorithm identifier; when the memory call request does not carry the algorithm identifier, use a default memory management algorithm as the first memory management algorithm allocated to the memory call request.

[0109] In an exemplary embodiment, the allocation module 32 is also used to: obtain registration information of a second memory management algorithm recorded in the global list, wherein the second memory management algorithm is encapsulated in a memory algorithm manager through a preset registration mechanism, and the second memory management algorithm includes: the first memory management algorithm, the registration information at least includes an algorithm identifier of the second memory management algorithm, and the memory algorithm manager is at least used to pre-configure the first basic interface group for the target memory; and allocate the first memory management algorithm to the memory call request through the registration information.

[0110] In an exemplary embodiment, Figure 4 As shown, the device also includes a registration module 38, which is used to: receive the second memory management algorithm registered through the preset registration mechanism; perform a packaging operation on the second memory management algorithm, and update the historical global list through the second memory management algorithm to obtain the global list.

[0111] In an exemplary embodiment, the registration module is also used to: encapsulate the second memory management algorithm into an algorithm implementation object; add a calling interface to the algorithm implementation object to complete the encapsulation operation, wherein the calling interface is used to call the second basic interface group pre-configured by the memory algorithm manager for the at least two memories, at least two of the second basic interface groups include the first basic interface group.

[0112] In an exemplary embodiment, the registration module is also used to: determine the registration information of the second memory management algorithm; assign annotation information to the second memory management algorithm based on the registration information, wherein the annotation information at least includes the priority assigned to the second memory management algorithm; and add the registration information and the annotation information to the historical global list to obtain the global list.

[0113] In an exemplary embodiment, the determination module 34 is also used to: obtain configuration information of the memory algorithm manager for the at least two memories, wherein the configuration information is used to indicate a second basic interface group corresponding to each memory, and the second basic interface group is a packaged interface group of a proprietary interface pre-configured for each memory and for each memory; determine the first basic interface group from the second basic interface group through the memory type information, wherein the memory type of the first basic interface group is consistent with the memory type indicated by the memory type information.

[0114] In an exemplary embodiment, the calling module 36 is also used to: call the first basic interface group through the first memory management algorithm to allocate a first memory resource with the memory size to the training task, including: calling the first basic interface group through the first memory management algorithm to allocate a first memory resource with the memory size indicated by the memory size information to the training task, including: determining a first asynchronous resource pool instantiated by the memory algorithm manager for the first memory management algorithm, wherein the at least two memory types correspond one-to-one to at least two of the first asynchronous resource pools; determining a second asynchronous resource pool from at least two of the first asynchronous resource pools according to the memory type information, wherein the second asynchronous resource pool is used to store a second memory resource divided from the target memory corresponding to the memory type information; and allocating a first memory resource with the memory size to the training task through the second asynchronous resource pool and the first basic interface group.

[0115] In an exemplary embodiment, the calling module 36 is also used to: determine whether there is a third memory resource in the third asynchronous resource pool corresponding to the first memory management algorithm, wherein the third memory resource is a second memory resource in a ready state recovered from the second asynchronous resource pool by the first memory management algorithm, and the first asynchronous resource pool includes the third asynchronous resource pool; in the case where the third memory resource exists in the third asynchronous resource pool, allocate a first memory resource with the memory size to the training task from the third memory resource; in the case where the third memory resource does not exist in the third asynchronous resource pool, divide the first memory resource from the target memory through the first basic interface group.

[0116] In an exemplary embodiment, the calling module 36 is also used to: determine the task type of the training task; determine a fourth memory resource in the third memory resource that has executed a historical training task with the task type; and allocate a first memory resource with the memory size to the training task from the fourth memory resource.

[0117] The embodiment of the present application also provides a memory resource allocation system, which is characterized by comprising:

[0118] A memory algorithm manager, a deep learning framework connected to the memory algorithm manager, wherein:

[0119] The memory algorithm manager is used to allocate a first memory management algorithm for a memory call request sent by a deep learning framework, wherein the memory call request is used to request to call a target memory for a training task, and the memory call request carries at least memory type information of the target memory and memory size information of a first memory resource to be called from the target memory;

[0120] The memory algorithm manager is further used to determine a first basic interface group pre-configured for the target memory according to the memory type information, wherein the device including the deep learning framework is deployed with at least two memory types of memories, at least two memories including the target memory, wherein the first basic interface group is a package interface group of a proprietary interface of the target memory, and the package interface group allows being called by any memory management algorithm;

[0121] The memory algorithm manager is further used to call the first basic interface group through the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information to the training task.

[0122] Through the above system, the memory algorithm manager connected to the deep learning framework, when receiving a memory call request sent by the deep learning framework for requesting to call a target memory in a memory of at least two memory types deployed in a device including the deep learning framework for a training task, allocates a first memory management algorithm to the memory call request; and determines a first basic interface group pre-configured for the target memory according to the memory type information carried in the memory call request, the first basic interface group is a package interface group of a proprietary interface of the target memory, and the package interface group allows to be called by any memory management algorithm; the first basic interface group is called by the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information carried in the memory call request to the training task. That is, through the first basic interface group, the memory of different memory types is decoupled from the memory management algorithm, so that the memory management algorithm can call the target memory by calling the first basic interface group. Furthermore, through the embodiments of the present application, the problem of high coupling between the memory management algorithm and the memory type applied to the deep learning framework in the related art can be solved. The technical effect of decoupling the memory management algorithm from the memory type is achieved.

[0123] In an exemplary embodiment, the memory algorithm manager is also used to: when the memory call request carries an algorithm identifier, assign the first memory management algorithm to the memory call request through a global list, wherein the algorithm identifier is used to uniquely identify the first memory management algorithm, and the global list includes the algorithm identifier; when the memory call request does not carry the algorithm identifier, use a default memory management algorithm as the first memory management algorithm assigned to the memory call request.

[0124] In an exemplary embodiment, the memory algorithm manager is also used to: obtain registration information of a second memory management algorithm recorded in the global list, wherein the second memory management algorithm is encapsulated in the memory algorithm manager through a preset registration mechanism, and the second memory management algorithm includes: the first memory management algorithm, the registration information at least includes an algorithm identifier of the second memory management algorithm, and the memory algorithm manager is at least used to pre-configure the first basic interface group for the target memory; and allocate the first memory management algorithm to the memory call request through the registration information.

[0125] In an exemplary embodiment, the memory algorithm manager is further used to: receive the second memory management algorithm registered through the preset registration mechanism; perform encapsulation operations on the second memory management algorithm, and update the historical global list through the second memory management algorithm to obtain the global list.

[0126] In an exemplary embodiment, the memory algorithm manager is also used to: encapsulate the second memory management algorithm into an algorithm implementation object; add a calling interface to the algorithm implementation object to complete the encapsulation operation, wherein the calling interface is used to call the second basic interface group pre-configured by the memory algorithm manager for the at least two types of memory, at least two of the second basic interface groups include the first basic interface group.

[0127] In an exemplary embodiment, the memory algorithm manager is also used to: determine the registration information of the second memory management algorithm; assign annotation information to the second memory management algorithm based on the registration information, wherein the annotation information at least includes the priority assigned to the second memory management algorithm; and add the registration information and the annotation information to the historical global list to obtain the global list.

[0128] In an exemplary embodiment, the memory algorithm manager is further used to: obtain configuration information of the memory algorithm manager for the at least two memories, wherein the configuration information is used to indicate a second basic interface group corresponding to each memory, and the second basic interface group is a packaged interface group of a proprietary interface pre-configured for each memory; determine the first basic interface group from the second basic interface group through the memory type information, wherein the memory type of the first basic interface group is consistent with the memory type indicated by the memory type information.

[0129] In an exemplary embodiment, the memory algorithm manager is also used to: call the first basic interface group through the first memory management algorithm to allocate a first memory resource with a memory size indicated by the memory size information to the training task, including: determining a first asynchronous resource pool instantiated by the memory algorithm manager for the first memory management algorithm, wherein the at least two memory types correspond one-to-one to at least two of the first asynchronous resource pools; determining a second asynchronous resource pool from at least two of the first asynchronous resource pools according to the memory type information, wherein the second asynchronous resource pool is used to store a second memory resource divided from the target memory corresponding to the memory type information; and allocating a first memory resource with the memory size to the training task through the second asynchronous resource pool and the first basic interface group.

[0130] In an exemplary embodiment, the memory algorithm manager is also used to: determine whether there is a third memory resource in the third asynchronous resource pool corresponding to the first memory management algorithm, wherein the third memory resource is a second memory resource in a ready state recovered from the second asynchronous resource pool by the first memory management algorithm, and the first asynchronous resource pool includes the third asynchronous resource pool; in the case where the third memory resource exists in the third asynchronous resource pool, allocate a first memory resource with the memory size to the training task from the third memory resource; in the case where the third memory resource does not exist in the third asynchronous resource pool, divide the first memory resource from the target memory through the first basic interface group.

[0131] In an exemplary embodiment, the memory algorithm manager is also used to: determine the task type of the training task; determine a fourth memory resource in the third memory resource that has executed a historical training task with the task type; and allocate a first memory resource with the memory size to the training task from the fourth memory resource.

[0132] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0133] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0134] S1, allocating a first memory management algorithm for a memory call request sent by a deep learning framework, wherein the memory call request is used to request to call a target memory for a training task, and the memory call request carries at least memory type information of the target memory and memory size information of a first memory resource to be called from the target memory;

[0135] S2, determining a first basic interface group pre-configured for the target memory according to the memory type information, wherein the device including the deep learning framework is deployed with at least two memory types of memories, at least two memories including the target memory, wherein the first basic interface group is a package interface group of a proprietary interface of the target memory, and the package interface group allows being called by any memory management algorithm;

[0136] S3: Call the first basic interface group through the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information to the training task.

[0137] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0138] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0139] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0140] S1, allocating a first memory management algorithm for a memory call request sent by a deep learning framework, wherein the memory call request is used to request to call a target memory for a training task, and the memory call request carries at least memory type information of the target memory and memory size information of a first memory resource to be called from the target memory;

[0141] S2, determining a first basic interface group pre-configured for the target memory according to the memory type information, wherein the device including the deep learning framework is deployed with at least two memory types of memories, at least two memories including the target memory, wherein the first basic interface group is a package interface group of a proprietary interface of the target memory, and the package interface group allows being called by any memory management algorithm;

[0142] S3: Call the first basic interface group through the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information to the training task.

[0143] An embodiment of the present application further provides a computer program product, including a computer program, which is used by a processor to execute the steps in any of the above method embodiments.

[0144] Optionally, in this embodiment, the computer program product may be executed by a processor to perform the following steps:

[0145] S1, allocating a first memory management algorithm for a memory call request sent by a deep learning framework, wherein the memory call request is used to request to call a target memory for a training task, and the memory call request carries at least memory type information of the target memory and memory size information of a first memory resource to be called from the target memory;

[0146] S2, determining a first basic interface group pre-configured for the target memory according to the memory type information, wherein the device including the deep learning framework is deployed with at least two memory types of memories, at least two memories including the target memory, wherein the first basic interface group is a package interface group of a proprietary interface of the target memory, and the package interface group allows being called by any memory management algorithm;

[0147] S3: Call the first basic interface group through the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information to the training task.

[0148] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0149] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0150] Optionally, an embodiment of the present application further provides another electronic device for implementing the above-mentioned memory resource allocation method, such as Figure 5 As shown, the electronic device includes a memory 502 and a processor 504. The memory 502 stores a computer program, and the processor 504 is configured to execute the steps in any of the above method embodiments through the computer program.

[0151] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network, or may be located in a device including a deep learning framework.

[0152] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0153] S1, allocating a first memory management algorithm for a memory call request sent by a deep learning framework, wherein the memory call request is used to request to call a target memory for a training task, and the memory call request carries at least memory type information of the target memory and memory size information of a first memory resource to be called from the target memory;

[0154] S2, determining a first basic interface group pre-configured for the target memory according to the memory type information, wherein the device including the deep learning framework is deployed with at least two memory types of memories, at least two memories including the target memory, wherein the first basic interface group is a package interface group of a proprietary interface of the target memory, and the package interface group allows being called by any memory management algorithm;

[0155] S3: Call the first basic interface group through the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information to the training task.

[0156] Alternatively, a person skilled in the art may understand that: Figure 5 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 5 The structure of the electronic device is not limited. Figure 5 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 5 Different configurations shown.

[0157] Among them, the memory 502 can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for allocating memory resources in the embodiments of the present application. The processor 504 executes various functional applications and data processing by running the software programs and modules stored in the memory 502, that is, realizing the above-mentioned method for allocating memory resources. The memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 502 may further include a memory remotely located relative to the processor 504, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, as an example, Figure 5As shown, the memory 502 may include but is not limited to the allocation module 32, determination module 34, and calling module 36 in the memory resource allocation device. In addition, it may also include but is not limited to other module units in the memory resource allocation device, which will not be repeated in this example.

[0158] Optionally, the transmission device 506 is used to receive or send data via a network. Specific examples of the network may include wired networks and wireless networks. In one example, the transmission device 506 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 506 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0159] In addition, the electronic device further includes: a display 508 for displaying the memory call request, the allocation result of the memory resources, etc.; and a connection bus 510 for connecting the various module components in the electronic device.

[0160] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0161] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for allocating memory resources, characterized in that: include: Allocating a first memory management algorithm for a memory call request sent by a deep learning framework, wherein the memory call request is used to request to call a target memory for a training task, and the memory call request carries at least memory type information of the target memory and memory size information of a first memory resource to be called from the target memory; Determine, according to the memory type information, a first basic interface group pre-configured for the target memory, wherein the device including the deep learning framework is deployed with at least two memory types of memories, and the at least two memories include the target memory, wherein the first basic interface group is a package interface group of a proprietary interface of the target memory, and the package interface group allows being called by any memory management algorithm; The first basic interface group is called through the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information to the training task.

2. The method for allocating memory resources according to claim 1, characterized in that: Assigning a first memory management algorithm to a memory call request sent by a deep learning framework includes: In a case where the memory call request carries an algorithm identifier, allocating the first memory management algorithm to the memory call request through a global list, wherein the algorithm identifier is used to uniquely identify the first memory management algorithm, and the algorithm identifier is included in the global list; When the algorithm identifier is not carried in the memory call request, a default memory management algorithm is used as the first memory management algorithm allocated to the memory call request.

3. The method for allocating memory resources according to claim 2, characterized in that: Allocating the first memory management algorithm to the memory call request through a global list includes: Acquire registration information of a second memory management algorithm recorded in the global list, wherein the second memory management algorithm is encapsulated in a memory algorithm manager through a preset registration mechanism, the second memory management algorithm comprises: the first memory management algorithm, the registration information comprises at least an algorithm identifier of the second memory management algorithm, and the memory algorithm manager is at least used to pre-configure the first basic interface group for the target memory; The first memory management algorithm is allocated to the memory call request through the registration information.

4. The method for allocating memory resources according to claim 3, characterized in that: Before allocating the first memory management algorithm for the memory call request sent by the deep learning framework, the method further includes: Receiving the second memory management algorithm registered through the preset registration mechanism; An encapsulation operation is performed on the second memory management algorithm, and the historical global list is updated through the second memory management algorithm to obtain the global list.

5. The method for allocating memory resources according to claim 4, characterized in that: Performing a packaging operation on the second memory management algorithm includes: Encapsulating the second memory management algorithm as an algorithm implementation object; Add a calling interface to the algorithm implementation object to complete the encapsulation operation, wherein the calling interface is used to call the second basic interface group pre-configured by the memory algorithm manager for the at least two memories, and at least two of the second basic interface groups include the first basic interface group.

6. The method for allocating memory resources according to claim 4, characterized in that: The historical global list is updated by using the second memory management algorithm to obtain the global list, including: Determining registration information of the second memory management algorithm; Assigning annotation information to the second memory management algorithm according to the registration information, wherein the annotation information at least includes a priority assigned to the second memory management algorithm; The registration information and the annotation information are added to the historical global list to obtain the global list.

7. The method for allocating memory resources according to claim 1, characterized in that: Determining a first basic interface group pre-configured for the target memory according to the memory type information includes: Obtaining configuration information of the memory algorithm manager for the at least two memories, wherein the configuration information is used to indicate a second basic interface group corresponding to each memory, respectively, and the second basic interface group is a package interface group of a proprietary interface pre-configured for each memory; The first basic interface group is determined from the second basic interface group according to the memory type information, wherein the memory type of the first basic interface group is consistent with the memory type indicated by the memory type information.

8. The method for allocating memory resources according to claim 1, characterized in that: Calling the first basic interface group through the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information to the training task includes: Determine a first asynchronous resource pool in which the memory algorithm manager instantiates a first memory management algorithm, wherein the at least two memory types correspond one-to-one to at least two of the first asynchronous resource pools; Determining a second asynchronous resource pool from at least two of the first asynchronous resource pools according to the memory type information, wherein the second asynchronous resource pool is used to store second memory resources divided from the target memory corresponding to the memory type information; A first memory resource having the memory size is allocated to the training task through the second asynchronous resource pool and the first basic interface group.

9. The method for allocating memory resources according to claim 8, characterized in that: Allocating a first memory resource having the memory size to the training task through the second asynchronous resource pool and the first basic interface group includes: Determine whether there is a third memory resource in the third asynchronous resource pool corresponding to the first memory management algorithm, wherein the third memory resource is a second memory resource in a ready state that is recovered from the second asynchronous resource pool by the first memory management algorithm, and the first asynchronous resource pool includes the third asynchronous resource pool; In a case where the third memory resource exists in the third asynchronous resource pool, allocating a first memory resource having the memory size to the training task from the third memory resource; In a case where the third memory resource does not exist in the third asynchronous resource pool, the first memory resource is divided from the target memory through the first basic interface group.

10. The method for allocating memory resources according to claim 9, characterized in that: Allocating a first memory resource having the memory size to the training task from the third memory resource includes: Determining a task type of the training task; Determine a fourth memory resource in the third memory resource that has executed a historical training task having the task type; A first memory resource having the memory size is allocated to the training task from the fourth memory resource.

11. A memory resource allocation device, characterized in that: include: An allocation module, configured to allocate a first memory management algorithm for a memory call request sent by a deep learning framework, wherein the memory call request is used to request to call a target memory for a training task, and the memory call request carries at least memory type information of the target memory and memory size information of a first memory resource to be called from the target memory; A determination module, configured to determine a first basic interface group pre-configured for the target memory according to the memory type information, wherein the device including the deep learning framework is deployed with at least two memory types of memories, at least two memories including the target memory, wherein the first basic interface group is a package interface group of a proprietary interface of the target memory, and the package interface group allows being called by any memory management algorithm; A calling module is used to call the first basic interface group through the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information to the training task.

12. A memory resource allocation system, characterized in that: include: A memory algorithm manager, a deep learning framework connected to the memory algorithm manager, wherein the memory algorithm manager is used to assign a first memory management algorithm to a memory call request sent by the deep learning framework, wherein the memory call request is used to request to call a target memory for a training task, and the memory call request carries at least memory type information of the target memory and memory size information of a first memory resource to be called from the target memory; The memory algorithm manager is further used to determine a first basic interface group pre-configured for the target memory according to the memory type information, wherein the device including the deep learning framework is deployed with at least two memory types of memories, at least two memories including the target memory, wherein the first basic interface group is a package interface group of a proprietary interface of the target memory, and the package interface group allows being called by any memory management algorithm; The memory algorithm manager is further used to call the first basic interface group through the first memory management algorithm to allocate a first memory resource having a memory size indicated by the memory size information to the training task.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 10 when executed.

14. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 10 through the computer program.

Citation Information

Cited By

  • Memory control method, computing device, storage medium and program product

    CN121501518A

  • Memory control method, computing device, storage medium, and program product

    CN121501518B