Kunlun XPU chip memory allocation optimization method and system on AI framework
By introducing a video memory pool mechanism into the PaddlePaddle framework, the problem of video memory fragmentation and resource waste in model training of Kunlun XPU chip is solved, and the recycling of video memory blocks is realized, and the memory allocation efficiency is improved.
Patent Information
- Application Number
- CN202411259448.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-10
- Publication Date
- 2025-01-03
- Estimated Expiration
- 2044-09-10
AI Technical Summary
In the artificial intelligence framework, especially the PaddlePaddle framework, the frequent xpu_malloc and xpu_free operations of Kunlun XPU chips during model training have caused video memory fragments to waste computing resources, and there is a lack of memory allocation optimization methods for domestic chips.
The video memory pool mechanism is introduced in the PaddlePaddle framework. By opening up the video memory block before the model training begins, and using the video memory pool to recycle the XPU video memory blocks during the training process, avoiding frequent xpu_malloc and xpu_free operations. The video memory blocks are returned after use Go to the video memory pool and return the system uniformly at the end of training.
It reduces the generation of video memory fragments on Kunlun XPU, reduces the waste of computing resources, and improves the efficiency of memory allocation.
Smart Images
Figure CN118796477B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to hardware accelerator optimization technology in the field of artificial intelligence, and specifically relates to a Kunlun XPU chip memory allocation optimization method and system on an AI framework. Background Art
[0002] In recent years, artificial intelligence (AI) technology has been rapidly popularized in various fields. Especially with the emergence and rapid development of large models, the parameters and training data of deep learning models have shown explosive growth, which requires more and more computing and memory resources in the training and reasoning process of the model. However, in practical applications, resources are limited, and limited resources cannot meet the operation requirements of the model in many cases. Therefore, it is necessary to optimize the memory allocation method in the artificial intelligence framework.
[0003] At present, mainstream artificial intelligence frameworks (AI frameworks) have taken certain measures to optimize memory allocation. For example, TesnsorFlow and PaddlePaddle frameworks use the BFC allocation algorithm and design a flexible memory pool mechanism; in order to avoid the overhead caused by repeated memory allocation during training, the Caffe framework will calculate the input and output of each node, as well as the size of the model parameters, and perform a global allocation before the training task begins; in Pytorch, the cache will be cleaned up regularly and unnecessary memory will be released. In addition, for the optimization of GPU memory in Pytorch, the Ant team also open-sourced the source code of GMLake. Although many artificial intelligence frameworks currently have methods to optimize memory allocation, most of them are for GPUs, and there are few optimization methods for domestic acceleration chips.
[0004] Compared with foreign chips, domestic chips have many shortcomings, especially high-end chips. In addition, domestic chips have a low market share and an imperfect ecosystem, which further restricts the development of domestic chips and makes optimization research on domestic chips very rare. However, in the context of the country's active development of the information and innovation industry, achieving independent control, and ensuring national information security, research on the optimization of domestic chips is of great significance.
[0005] Currently, the PaddlePaddle framework can well support hardware accelerators such as Kunlun XPU. However, the PaddlePaddle framework does not optimize the memory allocation for Kunlun XPU during training. During model training, as Tensors are constantly created and destroyed, the PaddlePaddle framework will frequently perform xpu_malloc and xpu_free operations directly on Kunlun XPU, which will not only bring additional overhead to model training, but also generate a large amount of video memory fragmentation on Kunlun XPU, seriously wasting Kunlun XPU's computing resources. Summary of the invention
[0006] Technical problem to be solved by the present invention: In view of the above-mentioned problems in the prior art, a method and system for optimizing the memory allocation of Kunlun XPU chip on an AI framework are provided. The present invention aims to reduce the video memory fragmentation generated on Kunlun XPU during the training of AI frameworks including PaddlePaddle, and reduce the waste of Kunlun XPU computing resources.
[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:
[0008] A method for optimizing memory allocation of Kunlun XPU chip on an AI framework comprises the following steps:
[0009] S101, before the AI model training of the PaddlePaddle framework begins, a series of XPU memory blocks are opened up and a memory pool is established;
[0010] S102, during the AI model training process of the PaddlePaddle framework, if an XPU video memory block needs to be applied for, a free XPU video memory block is searched in the video memory pool and the XPU video memory block is allocated. After the applied XPU video memory block is used up, the XPU video memory is returned to the video memory pool and marked as free;
[0011] S103, after the AI model training of the PaddlePaddle framework is completed, the XPU video memory block in the video memory pool is returned to the system.
[0012] Optionally, opening up a series of XPU video memory blocks in step S101 includes: calling an xpu_smi command to query basic information of the XPU graphics card, obtaining the size of the XPU video memory available for allocation in the system according to the basic information of the XPU graphics card, and opening up a series of XPU video memory blocks for establishing a video memory pool.
[0013] Optionally, establishing a memory pool in step S101 refers to using the existing data structure MemoryBlock of the PaddlePaddle framework to build a memory pool for the Kunlun XPU chip, and the established memory pool is a bidirectional linked list with the data structure MemoryBlock as a node, each data structure MemoryBlock represents an XPU memory block in the memory pool, and the data structure MemoryBlock contains the sequence number, size, pointers to the left and right adjacent nodes, and the status of the memory block of the XPU memory block.
[0014] Optionally, when it is necessary to apply for an XPU video memory block in step S102, the step further includes:
[0015] S201, obtaining the request size request_size required to apply for the XPU video memory block;
[0016] S202, determining whether the request size request_size is greater than the preset maximum XPU memory block size max_chunk_size that can be applied for, if not, searching for free XPU memory blocks in the memory pool and allocating XPU memory blocks; otherwise, applying for XPU memory blocks that need to be applied for is directly allocated by the system and is no longer managed through the memory pool.
[0017] Optionally, searching for an idle XPU video memory block in the video memory pool and allocating the XPU video memory block in step S102 includes:
[0018] S301, obtaining the request size request_size required to apply for the XPU video memory block;
[0019] S302, determining whether the request size request_size is smaller than the preset minimum XPU memory block size min_chunk_size that can be applied for, if it is smaller than the preset minimum XPU memory block size min_chunk_size that can be applied for, then using the preset minimum XPU memory block size min_chunk_size that can be applied for as the memory block size for searching the XPU memory block; otherwise, using the request size request_size as the memory block size for searching the XPU memory block;
[0020] S303, searching for an unoccupied XPU memory block of corresponding size in the memory pool according to the memory block size of the XPU memory block to be searched; if the search is successful, marking the unoccupied XPU memory block of corresponding size found as occupied, and completing the allocation of the XPU memory block; if the search fails, applying for an XPU memory block of corresponding size from the memory pool, marking the applied XPU memory block of corresponding size as occupied, and completing the allocation of the XPU memory block.
[0021] Optionally, after returning the XPU video memory to the video memory pool in step S102, the following steps are included:
[0022] S401, obtaining the size of the XPU memory block returned to the memory pool;
[0023] S402, if the size of the XPU video memory block returned to the video memory pool is between [min_chunk_size, max_chunk_size], the state of the XPU video memory block returned to the video memory pool is marked as free, and then the total video memory information in the video memory pool is updated, wherein min_chunk_size is the minimum XPU video memory block size that can be applied, and max_chunk_size is the maximum XPU video memory block size that can be applied, and jump to step S403; if the size of the XPU video memory block returned to the video memory pool is larger than the maximum XPU video memory block size max_chunk_size that can be applied, the XPU video memory block returned to the video memory pool is returned to the system, and the system's XPU video memory block release function xpu_free is used to release and delete all information of the XPU video memory block, and the total video memory information in the video memory pool is updated, and the process ends and exits;
[0024] S403, determining whether the status of the XPU video memory blocks adjacent to the XPU video memory block on the left and right in the video memory pool is idle, if the status of the XPU video memory blocks adjacent to the XPU video memory block on the left and right is idle, merging the XPU video memory block with the adjacent XPU video memory block, and putting the merged XPU video memory block back into the video memory pool.
[0025] Optionally, returning the XPU video memory blocks in the video memory pool to the system in step S103 includes: traversing all XPU video memory blocks in the video memory pool, releasing and deleting all information of the XPU video memory blocks one by one through the system's XPU video memory block release function xpu_free.
[0026] In addition, the present invention also provides a Kunlun XPU chip memory allocation optimization system on an AI framework, comprising a microprocessor and a Kunlun XPU chip connected to each other, wherein the microprocessor is programmed or configured to execute the Kunlun XPU chip memory allocation optimization method on the AI framework.
[0027] In addition, the present invention also provides a computer-readable storage medium, which stores a computer program / instruction, and the computer program / instruction is programmed or configured to execute the Kunlun XPU chip memory allocation optimization method on the AI framework through a processor.
[0028] In addition, the present invention also provides a computer program product, including a computer program / instruction, which is programmed or configured to execute the Kunlun XPU chip memory allocation optimization method on the AI framework through a processor.
[0029] Compared with the prior art, the present invention mainly has the following advantages: Since the PaddlePaddle framework does not optimize the XPU video memory allocation process during training, a large amount of video memory fragments will be generated on the Kunlun XPU during training, wasting computing resources. To address this problem, the present invention adds a video memory pool mechanism for the Kunlun XPU chip in the PaddlePaddle framework, and uses the state to realize the recycling of XPU video memory blocks, avoiding frequent xpu_malloc and xpu_free operations during training, thereby reducing the video memory fragments generated on the XPU during training and reducing the waste of XPU computing resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 Schematic diagram of the basic flow of the method of the embodiment of the present invention.
[0031] Figure 2 The figure is a schematic diagram of the allocation process of the video memory pool in an embodiment of the present invention.
[0032] Figure 3 The figure is a schematic diagram of a process of returning an XPU video memory block to a video memory pool in an embodiment of the present invention.
[0033] Figure 4 Schematic diagram of comparison of XPU memory block allocation strategies in an embodiment of the present invention. DETAILED DESCRIPTION
[0034] The following will take PaddlePaddle2.5.0 of the PaddlePaddle framework as an example to further explain in detail the Kunlun XPU chip memory allocation optimization method and system on the AI framework of the present invention.
[0035] like Figure 1 As shown, the Kunlun XPU chip memory allocation optimization method on the AI framework of this embodiment includes the following steps:
[0036] S101, before the AI model training of the PaddlePaddle framework begins, a series of XPU memory blocks are opened up and a memory pool (Chunk memory pool) is established;
[0037] S102, during the AI model training process of the PaddlePaddle framework, if an XPU video memory block needs to be applied for, a free XPU video memory block is searched in the video memory pool and the XPU video memory block is allocated. After the applied XPU video memory block is used up, the XPU video memory is returned to the video memory pool and marked as free;
[0038] S103, after the AI model training of the PaddlePaddle framework is completed, the XPU video memory block in the video memory pool is returned to the system.
[0039] In step S101 of this embodiment, a series of XPU video memory blocks are opened up, including: calling the xpu_smi command to query the basic information of the XPU graphics card, obtaining the size of the XPU video memory available for allocation in the system according to the basic information of the XPU graphics card, and opening up a series of XPU video memory blocks for establishing a video memory pool. In this embodiment, the XPUAvailableMemToAlloc() function is specifically added to the PaddlePaddle framework, and the XPUAvailableMemToAlloc() function is used to query the size of the XPU video memory available for allocation in the system. When building the XPU video memory pool, it is necessary to query the size of the available XPU video memory in the system. However, the external interface provided by Kunlun XPU is limited, and the XPU video memory size information cannot be directly queried through the external interface provided by XPU. Therefore, in this patent, the xpu_smi command is called by C++ to query the basic information of the XPU graphics card. In this embodiment, the RecordedXPUMallocHelper class is specifically added to the PaddlePaddle framework, which is a singleton class, and methods such as Malloc() and Free() are defined. The xpu_malloc function is called in Malloc(). When adding a memory block to the XPU memory pool, you can use Malloc() to request the XPU memory block from the system and add it to the memory pool. The xpu_free method is called in Free(). When the memory pool is no longer needed, Free() can be used to release the memory blocks in the memory pool to the system one by one.
[0040] In step S101 of this embodiment, establishing a video memory pool means using the existing data structure MemoryBlock of the PaddlePaddle framework (PaddlePaddle2.5.0) to build a video memory pool for the Kunlun XPU chip, and the established video memory pool is a bidirectional linked list with the data structure MemoryBlock as a node. Each data structure MemoryBlock represents an XPU video memory block in the video memory pool, and the data structure MemoryBlock contains the sequence number, size, pointers to the left and right adjacent nodes, and the status of the video memory block of the XPU video memory block. The bidirectional linked list forms a bidirectional linked list structure by combining the XPU video memory blocks with the left and right adjacent nodes. Each time the video memory pool allocates XPU video memory, it will search for appropriate nodes in the bidirectional linked list and divide them.
[0041] This embodiment adds the XPUMaxChunkSize() function to the PaddlePaddle framework. The XPUMaxChunkSize() function defines the maximum XPU video memory block size max_chunk_size that can be applied for in the video memory pool. When applying for video memory, if the applied size satisfies request_size > max_chunk_size, then this video memory block is defined as an extra-large video memory block and is directly allocated by the system without being managed through the video memory pool. Specifically, as shown in Figure 2 the uppermost judgment box in, first determine whether the XPU video memory block to be released is an extra-large video memory block, that is, determine whether the size of the video memory block is within [min_chunk_size, max_chunk_size]. If size > max_chunk_size, then it is determined that this XPU video memory block is an extra-large video memory block. At this time, this video memory block needs to be returned to the system and directly released through xpu_free. Specifically, when applying for an XPU video memory block in step S102, it also includes:
[0042] S201, obtain the request size request_size for the XPU video memory block to be applied for;
[0043] S202, determine whether it is established that the request size request_size is greater than the preset maximum XPU video memory block size max_chunk_size that can be applied for. If it is not established (request_size>max_chunk_size), only then search for an idle XPU video memory block in the video memory pool and allocate the XPU video memory block; otherwise (request_size ≤ max_chunk_size), for this XPU video memory block to be applied for, it is directly allocated by the system without being managed through the video memory pool.
[0044] In addition, this embodiment adds the XPUMinChunkSize() function to the PaddlePaddle framework. The XPUMinChunkSize() defines the minimum XPU video memory block size min_chunk_size that can be applied for in the Chunk video memory pool. When applying for XPU video memory, if request_size < min_chunk_size, then a video memory block of size min_chunk_size is allocated in the video memory pool. At the same time, min_chunk_size is also the unit of memory alignment during the XPU video memory allocation process. See Figure 2, when it is necessary to apply for an XPU video memory block in step S102, it also includes: judging whether an XPU video memory block that meets the size can be found in the video memory pool, if an XPU video memory block that meets the size is found, marking the XPU video memory block as occupied, and completing the allocation of the XPU video memory block; otherwise, dividing the XPU video memory block in the video memory pool into XPU video memory blocks that meet the size, marking the XPU video memory block as occupied, and completing the allocation of the XPU video memory block. Further, in this embodiment, it also includes combining the request size request_size and the minimum XPU video memory block size that can be applied for min_chunk_size for comparison to apply for an XPU video memory block, specifically including:
[0045] S301, obtaining the request size request_size required to apply for the XPU video memory block;
[0046] S302, determine whether the request size request_size is less than the preset minimum XPU video memory block size min_chunk_size that can be applied. If it is less than the preset minimum XPU video memory block size min_chunk_size that can be applied, the preset minimum XPU video memory block size min_chunk_size that can be applied is used as the video memory block size for searching the XPU video memory block; otherwise, the request size request_size is used as the video memory block size for searching the XPU video memory block; because when the request size request_size is less than the preset minimum XPU video memory block size min_chunk_size that can be applied, there is no available XPU video memory block, so in this embodiment, the minimum XPU video memory block size min_chunk_size that can be applied is used, and the preset minimum XPU video memory block size min_chunk_size that can be applied is used as the video memory block size for searching the XPU video memory block, ensuring that the video memory block size of the XPU video memory block is available;
[0047] S303, searching for an unoccupied XPU memory block of corresponding size in the memory pool according to the memory block size of the XPU memory block to be searched; if the search is successful, marking the unoccupied XPU memory block of corresponding size found as occupied, and completing the allocation of the XPU memory block; if the search fails, applying for an XPU memory block of corresponding size from the memory pool, marking the applied XPU memory block of corresponding size as occupied, and completing the allocation of the XPU memory block.
[0048] After adding the video memory pool, releasing video memory is no longer simply returning the used XPU video memory to the system, but returning the XPU video memory to the video memory pool, such as Figure 3 As shown, in step S102 of this embodiment, after returning the XPU video memory to the video memory pool, the following steps are included:
[0049] S401, obtaining the size of the XPU memory block returned to the memory pool;
[0050] S402, if the size of the XPU video memory block returned to the video memory pool is between [min_chunk_size, max_chunk_size], the state of the XPU video memory block returned to the video memory pool is marked as free, and then the total video memory information in the video memory pool is updated, wherein min_chunk_size is the minimum XPU video memory block size that can be applied, and max_chunk_size is the maximum XPU video memory block size that can be applied, and jump to step S403; if the size of the XPU video memory block returned to the video memory pool is larger than the maximum XPU video memory block size max_chunk_size that can be applied, the XPU video memory block returned to the video memory pool is returned to the system, and the system's XPU video memory block release function xpu_free is used to release and delete all information of the XPU video memory block, and the total video memory information in the video memory pool is updated, and the process ends and exits;
[0051] S403, determining whether the status of the XPU video memory blocks adjacent to the XPU video memory block on the left and right in the video memory pool is idle, if the status of the XPU video memory blocks adjacent to the XPU video memory block on the left and right is idle, merging the XPU video memory block with the adjacent XPU video memory block, and putting the merged XPU video memory block back into the video memory pool.
[0052] See also Figure 3, this embodiment first determines whether the XPU video memory block to be released is an extra-large video memory block, that is, determines whether the size of the video memory block is between [min_chunk_size, max_chunk_size]. If size>max_chunk_size, the XPU video memory block is determined to be an extra-large video memory block. At this time, the video memory block needs to be returned to the system and released directly through xpu_free. If the size of the XPU video memory block to be released is between [min_chunk_size, max_chunk_size], the XPU video memory block will not be returned directly to the system, but will be returned to the Chunk video memory pool. At this time, the status of the XPU video memory block will be marked as idle, and then the total video memory information in the Chunk video memory pool will be updated. After the XPU video memory block is returned to the video memory pool, in order to reduce the video memory fragmentation in the video memory pool, it is also necessary to determine whether the XPU video memory block can be merged with the idle XPU video memory blocks adjacent to the left and right. If it can be merged, it is also necessary to merge the XPU memory block with the free memory blocks adjacent to it, and then put the merged XPU memory block back into the Chunk memory pool. When the training process reaches a stable state, most of the XPU memory blocks can be found in the memory pool. At this time, the application and release of XPU memory can be managed through the Chunk memory pool, and there is no need to perform frequent xpu_malloc and xpu_free operations through the system.
[0053] In the release phase of the XPU video memory, the super large video memory block is directly released to the system by calling xpu_free, but the video memory in the Chunk video memory pool is not actually released to the system. Only when the model training is completed will the XPU video memory in the Chunk video memory pool be truly returned to the system. At this time, all XPU video memory blocks in the Chunk video memory pool will be traversed, and released one by one through xpu_free, and all information of the XPU video memory block will be deleted. In step S103 of this embodiment, returning the XPU video memory block in the video memory pool to the system includes: traversing all XPU video memory blocks in the video memory pool, releasing and deleting all information of the XPU video memory block one by one through the system's XPU video memory block release function xpu_free.
[0054] Figure 4 FIG. 1 is a comparative diagram of the XPU memory block allocation strategy in this embodiment. Figure 4 As shown in (a) in the original PaddlePaddle framework, each time an XPU memory is requested during model training, it is directly requested from the system through xpu_malloc. When the XPU memory is used up, it is directly released through xpu_free. Figure 4As shown in (b), when the Chunk memory pool mechanism for XPU is added to the PaddlePaddle framework using the method of this embodiment, the system will first use the Malloc() method in the RecordedXPUMallocHelper class to open up a series of XPU memory blocks (the size of the XPU memory pool is chunk_size) and establish a memory pool. After that, when XPU memory needs to be applied for during model training, the appropriate XPU memory blocks will be found in the memory pool and divided, instead of applying directly to the system through xpu_malloc. When the applied XPU memory is used up, the XPU memory will be returned to the Chunk memory pool instead of being released to the system through xpu_free.
[0055] To summarize, since the PaddlePaddle framework does not optimize the XPU video memory allocation process during training, a large amount of video memory fragments will be generated on the Kunlun XPU during training, wasting computing resources. To address this problem, the Kunlun XPU chip memory allocation optimization method on the AI framework of this embodiment adds a video memory pool mechanism for the Kunlun XPU chip in the PaddlePaddle framework, and uses the state to realize the recycling of XPU video memory blocks, avoiding frequent xpu_malloc and xpu_free operations during training, thereby reducing the video memory fragmentation generated on the XPU during training and reducing the waste of XPU computing resources.
[0056] In addition, this embodiment also provides a Kunlun XPU chip memory allocation optimization system on an AI framework, comprising a microprocessor and a Kunlun XPU chip connected to each other, wherein the microprocessor is programmed or configured to execute the Kunlun XPU chip memory allocation optimization method on the AI framework.
[0057] In addition, this embodiment also provides a computer-readable storage medium, which stores a computer program / instruction, and the computer program / instruction is programmed or configured to execute the Kunlun XPU chip memory allocation optimization method on the AI framework through a processor.
[0058] In addition, this embodiment also provides a computer program product, including a computer program / instruction, which is programmed or configured to execute the Kunlun XPU chip memory allocation optimization method on the AI framework through a processor.
[0059] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0060] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.
Claims
1. A Kunlun XPU chip memory allocation optimization method on an AI framework, characterized in that: The steps include: S101, before the AI model training of the PaddlePaddle framework begins, a series of XPU memory blocks are opened up and a memory pool is established; S102, during the AI model training process of the PaddlePaddle framework, if an XPU video memory block needs to be applied for, a free XPU video memory block is searched in the video memory pool and the XPU video memory block is allocated. After the applied XPU video memory block is used up, the XPU video memory is returned to the video memory pool and marked as free; S103, after the AI model training of the PaddlePaddle framework is completed, the XPU memory block in the memory pool is returned to the system; In step S102, searching for an idle XPU video memory block in the video memory pool and allocating the XPU video memory block includes: S301, obtaining the request size request_size required to apply for the XPU video memory block; S302, determining whether the request size request_size is smaller than the preset minimum XPU memory block size min_chunk_size that can be applied for, if it is smaller than the preset minimum XPU memory block size min_chunk_size that can be applied for, then using the preset minimum XPU memory block size min_chunk_size that can be applied for as the memory block size for searching the XPU memory block; otherwise, using the request size request_size as the memory block size for searching the XPU memory block; S303, searching for an unoccupied XPU video memory block of a corresponding size in the video memory pool according to the video memory block size of the searched XPU video memory block, if the search is successful, marking the unoccupied XPU video memory block of the corresponding size found as occupied, and completing the allocation of the XPU video memory block; if the search fails, applying for an XPU video memory block of a corresponding size from the video memory pool, marking the applied XPU video memory block of the corresponding size as occupied, and completing the allocation of the XPU video memory block; After returning the XPU video memory to the video memory pool in step S102, the following steps are included: S401, obtaining the size of the XPU memory block returned to the memory pool; S402, if the size of the XPU video memory block returned to the video memory pool is between [min_chunk_size, max_chunk_size], the state of the XPU video memory block returned to the video memory pool is marked as free, and then the total video memory information in the video memory pool is updated, wherein min_chunk_size is the minimum XPU video memory block size that can be applied, and max_chunk_size is the maximum XPU video memory block size that can be applied, and jump to step S403; if the size of the XPU video memory block returned to the video memory pool is larger than the maximum XPU video memory block size max_chunk_size that can be applied, the XPU video memory block returned to the video memory pool is returned to the system, and the system's XPU video memory block release function xpu_free is used to release and delete all information of the XPU video memory block, and the total video memory information in the video memory pool is updated, and the process ends and exits; S403, determining whether the status of the XPU video memory blocks adjacent to the XPU video memory block on the left and right in the video memory pool is idle, if the status of the XPU video memory blocks adjacent to the XPU video memory block on the left and right is idle, merging the XPU video memory block with the adjacent XPU video memory block, and putting the merged XPU video memory block back into the video memory pool.
2. The Kunlun XPU chip memory allocation optimization method on the AI framework according to claim 1, characterized in that: In step S101, opening up a series of XPU video memory blocks includes: calling the xpu_smi command to query basic information of the XPU graphics card, obtaining the size of the XPU video memory that can be allocated in the system according to the basic information of the XPU graphics card, and opening up a series of XPU video memory blocks for establishing a video memory pool.
3. The Kunlun XPU chip memory allocation optimization method on the AI framework according to claim 1, characterized in that: Establishing the memory pool in step S101 refers to using the existing data structure MemoryBlock of the PaddlePaddle framework to build a memory pool for the Kunlun XPU chip, and the established memory pool is a bidirectional linked list with the data structure MemoryBlock as a node. Each data structure MemoryBlock represents an XPU memory block in the memory pool, and the data structure MemoryBlock contains the sequence number, size, pointers to the left and right adjacent nodes, and the status of the memory block of the XPU memory block.
4. The Kunlun XPU chip memory allocation optimization method on the AI framework according to claim 1, characterized in that: When the XPU video memory block needs to be applied for in step S102, the following steps are also included: S201, obtaining the request size request_size required to apply for the XPU video memory block; S202, determining whether the request size request_size is greater than the preset maximum XPU memory block size max_chunk_size that can be applied for, if not, searching for free XPU memory blocks in the memory pool and allocating XPU memory blocks; otherwise, applying for XPU memory blocks that need to be applied for is directly allocated by the system and is no longer managed through the memory pool.
5. The Kunlun XPU chip memory allocation optimization method on the AI framework according to claim 1, characterized in that: In step S103, returning the XPU video memory blocks in the video memory pool to the system includes: traversing all XPU video memory blocks in the video memory pool, releasing and deleting all information of the XPU video memory blocks one by one through the system's XPU video memory block release function xpu_free.
6. A Kunlun XPU chip memory allocation optimization system on an AI framework, comprising a microprocessor and a Kunlun XPU chip connected to each other, characterized in that: The microprocessor is programmed or configured to execute the Kunlun XPU chip memory allocation optimization method on the AI framework described in any one of claims 1 to 5.
7. A computer-readable storage medium having a computer program / instruction stored therein, characterized in that: The computer program / instructions are programmed or configured to execute the Kunlun XPU chip memory allocation optimization method on the AI framework described in any one of claims 1 to 5 through a processor.
8. A computer program product comprising a computer program / instructions, characterized in that The computer program / instructions are programmed or configured to execute the Kunlun XPU chip memory allocation optimization method on the AI framework described in any one of claims 1 to 5 through a processor.