NPU memory management method and device and chip
By building linked list nodes in the NPU system and using linked list data structures for memory management, the conflict problem in NPU memory resource allocation is solved, and efficient and reliable memory resource allocation and release are achieved.
Patent Information
- Application Number
- CN202411880675.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-18
- Publication Date
- 2025-03-25
AI Technical Summary
The existing NPU memory management method cannot ensure that the allocated memory resources do not conflict in space, resulting in improper resource utilization.
By building a linked list node and using a linked list data structure, the memory resources in the NPU system are managed, the memory resource size requested by the NPU instruction is obtained, and when the memory space of the memory resources managed by the current node that is not used in the preset linked list node is greater than or equal to the requested memory resource size, memory allocation is performed, and the usage state is configured as the used state.
Ensure that the allocated memory resources do not conflict in space, avoid resource reuse, and improve the efficiency and reliability of memory management.
Smart Images
Figure CN119690677A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of chip technology, and in particular to an NPU memory management method, device and chip. Background Art
[0002] With the continuous development of computer technology and chip technology, the demand for memory resources by applications is also increasing. NPU (Neural Processing Unit) is a processor used for neural network calculations. In the common NPU architecture, one or two layers of on-chip cache are set between the computing core and the off-chip memory. On-chip cache resources generally need to be managed manually, that is, the cache resources to be used must be allocated in the NPU instructions.
[0003] Related technologies often use a method for generating NPU instructions based on random constraints in System Verilog (SV language). This method can randomly allocate memory address fields within a single NPU instruction. However, when randomly generating NPU instructions, the cache resources allocated to the current NPU may have been occupied by previously generated NPU instructions or overlap with the cache space used by previously generated NPU instructions, thereby causing memory resource conflicts in space.
[0004] Therefore, the NPU memory management method of the related art cannot ensure that the allocated memory resources do not conflict in space. Summary of the invention
[0005] In order to solve or partially solve the problems existing in the related art, the present application provides an NPU memory management method, device and chip, which can avoid spatial conflicts in allocated memory resources.
[0006] The first aspect of the present application provides an NPU memory management method, including:
[0007] Get the memory resource size requested by the NPU instruction;
[0008] When the memory space of the memory resources managed by the unused current node in the preset linked list node is greater than or equal to the requested memory resource size, memory is allocated to the NPU instruction through the current node, and the usage status of the current node is configured as a used status.
[0009] In one embodiment, the preset linked list nodes are preset in the following manner:
[0010] Abstract all memory resources that have not been allocated as linked list nodes;
[0011] The linked list node stores the starting address of the memory resource managed by the current node, the usage status of the current node and the pointer of the successor node adjacent to the current node.
[0012] In one embodiment, when the memory space of the memory resource managed by the unused current node in the preset linked list node is equal to the requested memory resource size, allocating memory for the NPU instruction through the current node, and configuring the use state of the current node to be in a used state, includes:
[0013] When the memory space of the memory resources managed by the unused current node in the preset linked list node is equal to the requested memory resource size, the memory space of the memory resources of the current node is used as the memory allocated for the NPU instruction, and the usage status of the current node is configured as a used status.
[0014] In one embodiment, when the memory space of the memory resource managed by the unused current node in the preset linked list node is greater than the requested memory resource size, allocating memory for the NPU instruction through the current node and setting the use state of the current node configuration to the used state includes:
[0015] When the memory space of the memory resource managed by the current node that is not in use in the preset linked list nodes is larger than the requested memory resource size, the memory space of the memory resource of the current node is split into the requested space and the remaining space,
[0016] The request space is used as memory allocated for the NPU instruction, and continues to be managed as the current node, and the use state of the current node is configured as a used state;
[0017] The remaining space is taken as a new node, and the new node is configured as a successor node adjacent to the current node.
[0018] In one embodiment, the method further comprises:
[0019] When the memory space of the memory resources managed by the unused current node in the preset linked list node is smaller than the requested memory resource size, the successor node adjacent to the current node is obtained according to the pointer of the successor node adjacent to the current node, and memory is allocated to the NPU instruction through the successor node.
[0020] In one embodiment, allocating memory for the NPU instruction through the current node includes:
[0021] Searching a preset memory management table for a memory address symbol assigned to the instruction address of the NPU instruction;
[0022] After finding the memory address symbol, determining the current node corresponding to the memory address symbol recorded in the preset memory management table, and allocating memory to the NPU instruction by the current node; after the NPU instruction obtains the allocated memory, modifying the instruction address to the allocated memory address; or,
[0023] After the memory address symbol is not found, a new node is allocated to the NPU instruction, and the new node is recorded in the preset memory management table.
[0024] In one embodiment, when the instruction address is an instruction address with a dependency relationship, the memory address symbols allocated to the instruction addresses of the NPU instructions are the same.
[0025] In one embodiment, the instruction address of the NPU instruction includes a destination address of data moved by the data moving instruction and a source address of data acquired by the computing instruction.
[0026] In one embodiment, the method further comprises:
[0027] After the memory resources allocated for the NPU instruction are used up, the memory resources are released and / or reclaimed.
[0028] In one implementation, the releasing of memory resources includes:
[0029] Configure the usage status of the node that currently needs to be released to be a usable status; and / or,
[0030] The reclaiming of memory resources comprises:
[0031] When it is determined that the predecessor node or successor node adjacent to the node currently to be released in the linked list node is no longer in use, the node currently to be released is merged with the adjacent predecessor node or successor node.
[0032] A second aspect of the present application provides an NPU memory management device, including:
[0033] The acquisition module is used to obtain the memory resource size requested by the NPU instruction;
[0034] The first allocation module is used to allocate memory for the NPU instruction through the current node when the memory space of the memory resources managed by the unused current node in the preset linked list node is greater than or equal to the requested memory resource size, and configure the usage status of the current node to be in a used state.
[0035] In one embodiment, the device further comprises:
[0036] The second allocation module is used to obtain the successor node adjacent to the current node according to the pointer of the successor node adjacent to the current node when the memory space of the memory resources managed by the unused current node in the preset linked list node is smaller than the requested memory resource size, and allocate memory for the NPU instruction through the successor node.
[0037] In one embodiment, the device further comprises:
[0038] The release and recovery module releases the memory resources and / or recovers the memory resources after the memory resources allocated for the NPU instructions are used up.
[0039] A third aspect of the present application provides a chip, comprising the NPU memory management device as described above.
[0040] A fourth aspect of the present application provides an electronic device, including:
[0041] Processor; and
[0042] The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.
[0043] A fifth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.
[0044] The technical solution provided by this application may have the following beneficial effects:
[0045] The method of the present application manages the memory resources in the NPU system by constructing a linked list node and using a linked list data structure. When the memory space of the memory resources managed by the unused current node in the preset linked list node is greater than or equal to the requested memory resource size, the memory is allocated to the NPU instruction through the current node, and the use status of the current node is configured as being used. Through this process, the memory space of the node that has been allocated for use will not be allocated again until it is released, so that the allocated memory resources can be guaranteed not to conflict in space.
[0046] Furthermore, the present application can implement memory allocation for NPU instructions with data dependencies by combining the data structure of the memory management table with a globally unique memory address symbol, thereby improving the applicability of the scenario.
[0047] Furthermore, the present application completes the release of memory resources and the recovery of memory fragments in one process, which can improve the efficiency of release and recovery.
[0048] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail the exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.
[0050] Figure 1 It is a flowchart of the NPU memory management method shown in this application;
[0051] Figure 2 is another flowchart of the NPU memory management method shown in the present application;
[0052] Figure 3 is another flowchart of the NPU memory management method shown in the present application;
[0053] Figure 4 It is a schematic diagram of the process of allocating and releasing memory resources in an NPU system using a linked list data structure as shown in the present application;
[0054] Figure 5 It is a schematic diagram of the NPU inter-instruction address constraint processing shown in the present application;
[0055] Figure 6 It is a schematic diagram of the structure of the NPU memory management device shown in this application;
[0056] Figure 7 is another structural schematic diagram of the NPU memory management device shown in the present application;
[0057] Figure 8 It is a schematic diagram of the structure of the electronic device shown in this application. DETAILED DESCRIPTION
[0058] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.
[0059] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0060] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the third information, and similarly, the third information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0061] The NPU memory management method of the related art cannot ensure that the allocated memory resources do not conflict in space.
[0062] In response to the above problems, the present application provides an NPU memory management method, which can avoid spatial conflicts in allocated memory resources.
[0063] The technical solution of the present application is described in detail below with reference to the accompanying drawings.
[0064] See also Figure 1 , the present application shows a NPU memory management method, which includes:
[0065] S110, obtaining the memory resource size requested by the NPU instruction.
[0066] When allocating memory resources, the present application obtains the memory resource size requested by the NPU instruction initiated by the user, that is, the memory resource size.
[0067] S120, when the memory space of the memory resources managed by the unused current node in the preset linked list nodes is greater than or equal to the requested memory resource size, memory is allocated to the NPU instruction through the current node, and the use state of the current node is configured as a used state.
[0068] In some implementations, the preset linked list nodes may be preset in the following manner:
[0069] All memory resources that have not been allocated are abstracted into linked list nodes; the linked list nodes store the starting address of the memory resources managed by the current node, the usage status of the current node, and the pointer of the successor node adjacent to the current node.
[0070] In some implementations, the usage status of the current node can be reflected by a usage status flag. For example, setting the usage status flag to 1 indicates a used status, and setting it to 0 indicates a usable status (i.e., an unused status). It should be noted that, as required, the usage status flag can also be set to 0 to indicate a used status, and set to 1 to indicate a usable status.
[0071] In some embodiments, when the memory space of the memory resources managed by the unused current node in the preset linked list node is equal to the requested memory resource size, the memory space of the memory resources of the current node is used as memory allocated for the NPU instructions, and the usage status of the current node is configured as a used state.
[0072] In some implementations, when the memory space of the memory resource managed by the current node that is not in use in the preset linked list nodes is larger than the requested memory resource size, the memory space of the memory resource of the current node is split into the requested space and the remaining space.
[0073] The requested space is used as memory allocated for NPU instructions, and continues to be managed as the current node, and the use state of the current node is configured as being used;
[0074] The remaining space is used as a new node and the new node is configured as a successor node adjacent to the current node.
[0075] From this example, it can be seen that the method of the present application manages the memory resources in the NPU system by constructing a linked list node and using a linked list data structure. When the memory space of the memory resources managed by the unused current node in the preset linked list node is greater than or equal to the requested memory resource size, the memory is allocated for the NPU instruction through the current node, and the use status of the current node is configured as being used. Through this process, the memory space of the node that has been allocated for use will not be allocated again until it is released, so that the allocated memory resources can be guaranteed not to conflict in space.
[0076] See also Figure 2 , is another flowchart of the NPU memory management method shown in the present application. The method includes:
[0077] S210, obtaining the memory resource size requested by the NPU instruction.
[0078] When allocating memory resources, the present application obtains the memory resource size requested by the NPU instruction initiated by the user, that is, the memory resource size.
[0079] S220, when the memory space of the memory resources managed by the unused current node in the preset linked list nodes is greater than or equal to the requested memory resource size, memory is allocated to the NPU instruction through the current node, and the use status of the current node is configured as a used state.
[0080] The description of step S220 can refer to the description of step S120, which will not be repeated here.
[0081] S230, when the memory space of the memory resources managed by the unused current node in the preset linked list node is smaller than the requested memory resource size, the successor node adjacent to the current node is obtained according to the pointer of the successor node adjacent to the current node, and memory is allocated to the NPU instruction through the successor node.
[0082] If the memory space of the memory resources managed by the current node is smaller than the requested resource size, the current node does not allocate memory resources. At this time, the pointer of the successor node in the linked list node saved in the current node can be used to obtain the next node, i.e., the successor node, and the memory resource allocation process can be continued through the successor node. The successor node can be allocated with reference to the processing method of the current node in step S120, which will not be repeated here.
[0083] From this example, it can be seen that the method of the present application manages the memory resources in the NPU system by constructing linked list nodes and using linked list data structures. When the memory space of the memory resources managed by the unused current node in the preset linked list node is greater than or equal to the requested memory resource size, memory can be allocated for the NPU instruction through the current node, and the use status of the current node can be configured as the used state; when the memory space of the memory resources managed by the unused current node in the preset linked list node is less than the requested memory resource size, memory can be allocated for the NPU instruction through the successor node. Through the above processing, the current node and subsequent nodes in the linked list node can allocate memory for the NPU instruction, and the nodes that have been allocated for use will not be allocated again until they are released, so that it can be ensured that the allocated memory resources do not conflict in space.
[0084] See also Figure 3 , another flow chart of the NPU memory management method shown in this application.
[0085] The method of the present application can be applied to a chip, for example, to a SoC chip. It can be understood that a SoC chip is a system-level chip, also known as a system on a chip.
[0086] like Figure 3 As shown, the method includes:
[0087] S310, pre-set linked list nodes.
[0088] The present application can pre-set linked list nodes in the following manner: abstract all memory resources that have not been allocated as linked list nodes; wherein the linked list nodes store the starting address of the memory resources managed by the current node, the usage status of the current node, and the pointer to the successor node adjacent to the current node.
[0089] This application uses a linked list data structure to manage on-chip and off-chip memory resources in the NPU system. The linked list nodes store the starting address of the memory resources managed by the current node, the usage status of the current node, and the pointer to the successor node of the current node. Among them, the usage status of the current node can be reflected by the usage status flag (uded flag). For example, the usage status flag (uded flag) is set to 1 to indicate the used state, and set to 0 to indicate the usable state (i.e., unused state). It should be noted that, as needed, the usage status flag (uded flag) can also be set to 0 to indicate the used state, and set to 1 to indicate the usable state.
[0090] A linked list is a non-continuous and non-sequential storage structure on a physical storage unit. The logical order of data elements is realized by the order of pointer links in the linked list. A linked list consists of a series of nodes (each element in the linked list is called a node), and the nodes can be dynamically generated at runtime.
[0091] See also Figure 4 , is a schematic diagram of the process of allocating and releasing memory resources in the NPU system using a linked list data structure as shown in the present application. Figure 4 As shown in the leftmost part, when no memory resource allocation is performed, the entire memory resource is abstracted as a linked list node, namely the Head Node. Figure 4 In the Head Node shown in the leftmost part, the uded flag is set to 0 to indicate a usable state, and the memory space of the memory resource is 0 to 'h5FFFFF (in hexadecimal as an example).
[0092] S320, obtaining the memory resource size requested by the NPU instruction.
[0093] When allocating memory resources, the present application obtains the memory resource size requested by the NPU instruction initiated by the user, that is, the memory resource size.
[0094] S330, determine whether the memory space of the memory resources managed by the unused current node in the preset linked list node is greater than or equal to the requested memory resource size. If it is greater than or equal to the requested memory resource size, enter S340 or enter S350. If it is less than the requested memory resource size, enter S360.
[0095] This application can compare whether the memory space size of the memory resource managed by the first unused memory node in the linked list node (at the beginning of the program, this node is the Head Node) meets the requested memory resource size, that is, the memory resource size.
[0096] If the memory space of the memory resource managed by the current node is greater than or equal to the requested memory resource size, that is, the memory resource size, then the process proceeds to S340 or S350 to perform a memory allocation.
[0097] S340, when the memory space of the memory resource managed by the current node that is not in use in the preset linked list nodes is equal to the requested memory resource size, a first allocation process is performed. Enter S370.
[0098] In some implementations, performing the first allocation process may include:
[0099] The memory space of the memory resource of the current node is used as the memory allocated for the NPU instruction, and the use state of the current node is configured as the used state.
[0100] For example, if the size of the requested memory resource is exactly equal to the memory space size of the memory resource managed by the current node, the usage status flag (uded flag) of the current node can be set to 1 to indicate the used status, and then the memory space starting address and space size of the allocated memory resource are returned to the NPU instruction that initiated the request for use by the NPU instruction that initiated the request.
[0101] In some implementations, the process of allocating memory for NPU instructions may include:
[0102] Searching for a memory address symbol assigned to the instruction address of the NPU instruction in a preset memory management table;
[0103] After finding the memory address symbol, determine the current node corresponding to the memory address symbol recorded in the preset memory management table, and allocate memory for the NPU instruction by the current node; after the NPU instruction obtains the allocated memory, modify the instruction address to the allocated memory address; or,
[0104] After the memory address symbol is not found, a new node is allocated for the NPU instruction, and the new node is recorded in a preset memory management table.
[0105] In some embodiments, when the instruction address is a dependent instruction address, the memory address symbol allocated to the instruction address of the NPU instruction is the same. The instruction address of the NPU instruction includes the destination address of the data transport instruction and the source address of the data obtained by the computing instruction.
[0106] See also Figure 5 , is a schematic diagram of the NPU inter-instruction address constraint processing shown in the present application.
[0107] The instructions of the NPU instruction flow network include computing instructions and data handling instructions coordinated with computing instructions, etc. That is to say, in addition to computing instructions, data handling instructions are also required in the instruction flow network executed by the NPU, and there is a fixed data dependency relationship between these instructions.
[0108] Among them, the function of the calculation instruction is to complete the calculation work on the data, such as convolution operation.
[0109] The data transfer instructions are used to transfer the data required by the computing instructions from the memory to the cache in advance during the operation of the NPU, so as to provide the computing core with the data. That is, the data required by the computing instructions are transferred from the memory to the cache in advance through the data transfer instructions.
[0110] like Figure 5 As shown, the computing instruction depends on the data moved by the data moving instruction. Therefore, the destination address (dst_addr_symbol) of the data moved by the data moving instruction must be the same as the source address (src_addr_symbol) of the computing instruction to obtain the data (i.e., dst_addr_symbol = src_addr_symbol). When allocating addresses to computing instructions and data moving instructions, the allocated memory addresses can be constrained.
[0111] The constraint process can be processed as follows: the instruction address in the instruction (for example, the destination address (dst_addr_symbol) of the data moving instruction and the source address (src_addr_symbol) of the data obtaining instruction) is first replaced with a globally unique symbol, that is, a globally unique memory address symbol (memory_addr_symbol) is allocated, and the instruction addresses with dependencies are allocated the same memory address symbol.
[0112] After the memory address symbol (memory_addr_symbol) is allocated, the memory resources are allocated. The process may include:
[0113] 1) Searching for the memory address symbol assigned to the instruction address of the NPU instruction in a preset memory manager table.
[0114] like Figure 5 As shown, each time memory resources are allocated, a memory address symbol (memory_addr_symbol) is first searched in a preset memory management table (memorymanager table).
[0115] If a memory_addr_symbol has never been allocated, a new memory resource node is allocated and recorded in the memory manager table.
[0116] If the memory_addr_symbol is found in the memory manager table, the memory resource node is no longer reallocated, but the node recorded in the memory manager table is directly returned as the node for allocating memory resources. For example, if memory_addr_symbol_1 is found in the memory manager table, and its corresponding node is memory node 1, memory allocation is performed by node memory node 1. After the NPU instruction that initiates the request obtains the allocated memory resources, the memory address field of the instruction address in the instruction is modified to the actual memory resource address.
[0117] Through the above processing, the constraint allocation process of the instruction address is completed.
[0118] It should be noted that if a memory resource node is released later, the memory manager table also needs to be traversed to delete the record of this node in the memory manager table.
[0119] S350, when the memory space of the memory resource managed by the unused current node in the preset linked list node is larger than the requested memory resource size, a second allocation process is performed. Enter S370.
[0120] In some implementations, performing the second allocation process may include:
[0121] Split the memory space of the current node's memory resources into requested space and remaining space.
[0122] The requested space is used as memory allocated for NPU instructions, and continues to be managed as the current node, and the use state of the current node is configured as being used;
[0123] The remaining space is used as a new node and the new node is configured as a successor node adjacent to the current node.
[0124] For example, if the memory space size of the memory resources managed by the current node is larger than the requested memory resource size, the memory allocation process of this application will split the original node, and split the memory space of the memory resources of the current node into request space and remaining space, that is, the first part of the split is the resource space requested by the user (called request space), and the second part of the split is the remaining memory space (called remaining space). This application continues to use the current node management for the first part of the memory space (request space), and sets the usage status flag (uded flag) of the current node to 1, indicating that the current node has been used. This application uses a new node management for the remaining memory space (remaining space) of the second part, and sets the new node as the successor node of the current node. After the node splitting is completed, the memory space starting address and space size of the allocated memory resource are returned to the NPU instruction that initiated the request for use by the NPU instruction that initiated the request.
[0125] like Figure 4 As shown, if the memory space size of the memory resources managed by the node Head Node is larger than the requested memory resource size, i.e., the memory resource size, 1M memory space (i.e., 1M memory space) is allocated to the NPU instruction that initiates the request, that is, in the memory space 0 to 'h5FFFFF of the memory resources of the node Head Node, the memory space from 0 to 'hFFFF is allocated to the NPU instruction that initiates the request, and the usage status flag (uded flag) of the current node is set to 1 to indicate the used state, i.e., the current node has been used, and the remaining memory resources are the memory space from h100000 to 'h5FFFFF, which is managed as a new node Mem Node1, and the usage status flag (uded flag) of Mem Node1 is set to 0 to indicate the usable state, i.e., the node Mem Node1 is not used.
[0126] Continue to see Figure 4 If the node Mem Node1 continues to allocate 1M memory space (i.e., 1M memory space) for the NPU instruction that initiates the request, the node Mem Node1 can allocate the memory space of h100000'h1 FFFFF in the memory resource to the NPU instruction that initiates the request, and set the usage status flag (uded flag) of the node Mem Node1 to 1 to indicate the used state, that is, the node Mem Node1 has been used, and the remaining memory space of h200000'h5FFFFF in the memory resource is managed as a new node Mem Node2, and the usage status flag (uded flag) of Mem Node2 is set to 0 to indicate the usable state, that is, the node Mem Node2 is not used.
[0127] It can be found that the above process completes the allocation of memory space once. The nodes that have been used will not be allocated again until they are released, which ensures that the memory resources will not be reused in space and time, and ensures that the allocated memory resources will not conflict in space or time.
[0128] S360: When the memory space of the memory resource managed by the unused current node in the preset linked list node is smaller than the requested memory resource size, a third allocation process is performed. Then, the process proceeds to S370.
[0129] In some implementations, performing the third allocation process may include:
[0130] According to the pointer of the successor node adjacent to the current node, the successor node adjacent to the current node is obtained, and memory is allocated for the NPU instruction through the successor node.
[0131] If the memory space of the memory resources managed by the current node is smaller than the requested resource size, the current node does not allocate memory resources. At this time, the pointer of the successor node in the linked list node saved in the current node can be used to obtain the next node, that is, the successor node, and the memory resource allocation process can be continued through the successor node. The memory allocation process of the subsequent nodes can refer to the memory allocation process of the current node, and the principle is the same.
[0132] If all nodes have been traversed and none of them match the requested resource size, an error indication may be returned to inform the user that memory resource allocation has failed.
[0133] S370: After the memory resources allocated for the NPU instructions are used up, the memory resources are released and / or the memory resources are recycled.
[0134] In some implementations, releasing memory resources includes:
[0135] Configure the usage status of the node that currently needs to be released to be a usable status; and / or,
[0136] In some implementations, reclaiming memory resources includes:
[0137] When it is determined that the predecessor node or successor node adjacent to the node currently to be released in the linked list node is no longer in use, the node currently to be released is merged with the adjacent predecessor node or successor node.
[0138] In the embodiment of the present application, if the memory resources can be released after use, they can be further recycled after release.
[0139] In some implementations, the process of releasing a node may include:
[0140] The used flag of the node to be released is set to 0, indicating that it is in a usable state (ie, unused state), thereby releasing the memory space of the node.
[0141] In some implementations, the node recycling process may include:
[0142] Determine whether the adjacent nodes (adjacent predecessor nodes or successor nodes) of the linked lists adjacent to the released node are still in use. If the adjacent node (adjacent predecessor node or successor node) is no longer in use (the used status flag (uded flag) is set to 0), merge the node that needs to be released with the adjacent node; otherwise, do nothing.
[0143] See also Figure 4 As shown, if the memory space of Mem Node 1 is to be released, the usage status flag (uded flag) of Mem Node 1 can be set to 0, indicating that it is in a usable state (i.e., not in use), thereby releasing the memory space of the node Mem Node 1; further checking finds that the usage status flag (uded flag) of the successor node Mem Node 2 adjacent to the node Mem Node 1 is also set to 0, indicating that the successor node Mem Node 2 is no longer in use, so the memory space of Mem Node 1 and Mem Node 2 can be merged.
[0144] Finally, if the Head Node memory space is released, the unused memory space of the Head Node is restored to 0 to 'h5FFFFF, and the used flag of the Head Node is set to 0.
[0145] Through the above processing, the present application can relatively efficiently avoid the generation of memory resource fragments caused by multiple memory resource allocation and release processes.
[0146] This application completes the release of memory resources and the recovery of memory fragments in one process, which no longer requires a separate process, and recovers memory fragments by traversing all nodes, thereby also improving the efficiency of memory fragment recovery.
[0147] From this example, it can be seen that the dynamic memory management method for NPU instructions provided in the present application uses a linked list data structure and linked list nodes to manage the on-chip and off-chip memory resources used by the NPU chip. Memory can be allocated for NPU instructions through the current node, and the usage status of the current node can be configured as being used. Since the memory space of the node that has been allocated for use will not be allocated again until it is released, conflicts in space and time of memory resources used by NPU instructions can be avoided.
[0148] The present application realizes memory allocation for NPU instructions with data dependencies by combining the memory management table (memory manager table) data structure and the memory address symbol (memory_addr_symbol), and also improves the applicability of application scenarios.
[0149] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides an NPU memory management device, chip, equipment and corresponding embodiments.
[0150] Figure 6 It is a schematic diagram of the structure of the NPU memory management device shown in this application.
[0151] See also Figure 6 The NPU memory management device 600 shown in this application includes: an acquisition module 610 and a first allocation module 620.
[0152] The acquisition module 610 is used to acquire the memory resource size requested by the NPU instruction. The acquisition module 610 acquires the memory resource size requested by the NPU instruction initiated by the user, that is, the memory resource size.
[0153] The first allocation module 620 is used to allocate memory for NPU instructions through the current node and configure the use status of the current node to be used when the memory space of the memory resources managed by the unused current node in the preset linked list node is greater than or equal to the requested memory resource size.
[0154] In some embodiments, the preset linked list nodes can be pre-set in the following manner: all memory resources that have not been allocated are abstracted as linked list nodes; wherein the linked list nodes store the starting address of the memory resources managed by the current node, the usage status of the current node, and the pointer to the successor node adjacent to the current node.
[0155] Figure 7 It is another structural schematic diagram of the NPU memory management device shown in this application.
[0156] like Figure 7As shown, the NPU memory management device 600 shown in the present application includes: an acquisition module 610, a comparison module 630, a first allocation module 620, a second allocation module 640, and a release recovery module 650.
[0157] The acquisition module 610 is used to acquire the memory resource size requested by the NPU instruction.
[0158] The comparison module 630 is used to compare the memory space of the memory resource managed by the current node that is not in use in the preset linked list nodes with the requested memory resource size.
[0159] The first allocation module 620 is used to allocate memory for NPU instructions through the current node and configure the use status of the current node to be used when the memory space of the memory resources managed by the unused current node in the preset linked list node is greater than or equal to the requested memory resource size.
[0160] In some embodiments, when the memory space of the memory resources managed by the unused current node in the preset linked list node is equal to the requested memory resource size, the first allocation module 620 uses the memory space of the memory resources of the current node as the memory allocated for the NPU instruction, and configures the usage status of the current node to be in a used state.
[0161] In some embodiments, when the memory space of the memory resources managed by the unused current node in the preset linked list node is larger than the requested memory resource size, the first allocation module 620 splits the memory space of the memory resources of the current node into requested space and remaining space, uses the requested space as the memory allocated for the NPU instructions, and continues to manage it as the current node, and configures the usage status of the current node as a used status; uses the remaining space as a new node, and configures the new node as a successor node adjacent to the current node.
[0162] The second allocation module 640 is used to obtain the successor node adjacent to the current node according to the pointer of the successor node adjacent to the current node when the memory space of the memory resources managed by the unused current node in the preset linked list node is smaller than the requested memory resource size, and allocate memory for the NPU instruction through the successor node.
[0163] The release and recovery module 650 releases the memory resources and / or recovers the memory resources after the memory resources allocated for the NPU instructions are used up.
[0164] In some implementations, the release and recovery module 650 releases memory resources, including:
[0165] Configure the usage status of the node that currently needs to be released to be a usable status; and / or,
[0166] The release and recovery module 650 recovers memory resources, including:
[0167] When it is determined that the predecessor node or successor node adjacent to the node currently to be released in the linked list node is no longer in use, the node currently to be released is merged with the adjacent predecessor node or successor node.
[0168] In summary, the device of the present application uses linked list data structures and linked list nodes to manage the on-chip and off-chip memory resources used by the NPU chip. Memory can be allocated for NPU instructions through the current node, and the usage status of the current node can be configured as being used. Since the memory space of the node that has been allocated for use will not be allocated again until it is released, conflicts in space and time of memory resources used by NPU instructions can be avoided.
[0169] The device of the present application realizes memory allocation for NPU instructions with data dependencies by combining the memory management table (memory manager table) data structure and the memory address symbol (memory_addr_symbol), thereby improving the applicability of application scenarios.
[0170] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.
[0171] The present application also provides a chip, which includes Figure 6 or Figure 7 The NPU memory management device is shown.
[0172] Figure 8 It is a schematic diagram of the structure of the electronic device shown in this application.
[0173] See also Figure 8 , the electronic device 1000 includes a memory 1010 and a processor 1020 .
[0174] The processor 1020 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.
[0175] The memory 1010 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 1020 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1010 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.
[0176] The memory 1010 stores executable codes, and when the executable codes are processed by the processor 1020 , the processor 1020 can execute part or all of the methods described above.
[0177] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.
[0178] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.
[0179] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A NPU memory management method, characterized in that: include: Get the memory resource size requested by the NPU instruction; When the memory space of the memory resources managed by the unused current node in the preset linked list node is greater than or equal to the requested memory resource size, memory is allocated to the NPU instruction through the current node, and the usage status of the current node is configured as a used status.
2. The method according to claim 1, characterized in that The preset linked list nodes are preset in the following manner: Abstract all memory resources that have not been allocated as linked list nodes; The linked list node stores the starting address of the memory resource managed by the current node, the usage status of the current node and the pointer of the successor node adjacent to the current node.
3. The method according to claim 1, characterized in that When the memory space of the memory resource managed by the unused current node in the preset linked list node is equal to the requested memory resource size, memory is allocated to the NPU instruction through the current node, and the use state of the current node is configured as a used state, including: When the memory space of the memory resources managed by the unused current node in the preset linked list node is equal to the requested memory resource size, the memory space of the memory resources of the current node is used as the memory allocated for the NPU instruction, and the usage status of the current node is configured as a used status.
4. The method according to claim 1, characterized in that When the memory space of the memory resource managed by the unused current node in the preset linked list node is larger than the requested memory resource size, memory is allocated to the NPU instruction through the current node, and the use state of the current node configuration is set to the used state, including: When the memory space of the memory resource managed by the current node that is not in use in the preset linked list nodes is larger than the requested memory resource size, the memory space of the memory resource of the current node is split into the requested space and the remaining space, The request space is used as memory allocated for the NPU instruction, and continues to be managed as the current node, and the use state of the current node is configured as a used state; The remaining space is taken as a new node, and the new node is configured as a successor node adjacent to the current node.
5. The method according to claim 2, characterized in that: The method further comprises: When the memory space of the memory resources managed by the unused current node in the preset linked list node is smaller than the requested memory resource size, the successor node adjacent to the current node is obtained according to the pointer of the successor node adjacent to the current node, and memory is allocated to the NPU instruction through the successor node.
6. The method according to claim 1, characterized in that The allocating memory for the NPU instruction through the current node includes: Searching a preset memory management table for a memory address symbol assigned to the instruction address of the NPU instruction; After finding the memory address symbol, determining the current node corresponding to the memory address symbol recorded in the preset memory management table, and allocating memory to the NPU instruction by the current node; after the NPU instruction obtains the allocated memory, modifying the instruction address to the allocated memory address; or, After the memory address symbol is not found, a new node is allocated to the NPU instruction, and the new node is recorded in the preset memory management table.
7. The method according to claim 6, characterized in that: When the instruction address is an instruction address with a dependency relationship, the memory address symbols allocated to the instruction addresses of the NPU instructions are the same.
8. The method according to claim 7, characterized in that: The instruction address of the NPU instruction includes the destination address of the data transported by the data transport instruction and the source address of the data acquired by the calculation instruction.
9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: After the memory resources allocated for the NPU instruction are used up, the memory resources are released and / or reclaimed.
10. The method according to claim 9, characterized in that The releasing of memory resources comprises: Configure the usage status of the node that currently needs to be released to be a usable status; and / or, The reclaiming of memory resources comprises: When it is determined that the predecessor node or successor node adjacent to the node currently to be released in the linked list node is no longer in use, the node currently to be released is merged with the adjacent predecessor node or successor node.
11. A NPU memory management device, characterized in that: include: The acquisition module is used to obtain the memory resource size requested by the NPU instruction; The first allocation module is used to allocate memory for the NPU instruction through the current node when the memory space of the memory resources managed by the unused current node in the preset linked list node is greater than or equal to the requested memory resource size, and configure the usage status of the current node to be in a used state.
12. The device according to claim 11, characterized in that The device also includes: The second allocation module is used to obtain the successor node adjacent to the current node according to the pointer of the successor node adjacent to the current node when the memory space of the memory resources managed by the unused current node in the preset linked list node is smaller than the requested memory resource size, and allocate memory for the NPU instruction through the successor node.
13. A chip, characterized in that: Including the NPU memory management device as described in claim 11 or 12.
14. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 10.
15. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 10.