A memory management method, system, device and computer readable storage medium

Abstract

The application discloses a memory management method, system, device and computer readable storage medium. The method comprises the following steps: obtaining a target neural network model; based on the operation support of each target computing device on the operators in the target neural network model, the target neural network model is dissected into sub-functions corresponding to each target computing device; the sub-functions are distributed to the corresponding target computing devices; for each target computing device, based on the operation information of each computing unit in the target computing device, the corresponding sub-function is dissected into execution units corresponding to each computing unit, and the memory management of the target computing device is performed in the granularity of the execution units. The sub-function required by the operation of each target computing device is consistent with the operation performance of the target computing device, and the memory management of the target computing device is performed in the granularity of the execution units, so that the accurate management of the memory of the computing device is realized. The memory management system, device and computer readable storage medium provided by the application also solve the corresponding technical problems.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, more particularly, to a memory management method, system, device and computer readable storage medium. BACKGROUND

[0002] In the use process of the neural network model, the corresponding calculation in the neural network model needs to be run by means of a computing device with computing capability, because the input and output of the operator in the neural network model will occupy a certain storage space, and the memory space of the computing device is limited, if the memory allocation is unreasonable, it will reduce the operation speed of the neural network model, therefore, in order to improve the operation speed of the neural network model, the memory of the computing device needs to be accurately managed.

[0003] In summary, how to accurately manage the memory of the computing device is a problem that needs to be solved by the technical personnel in the field at present. SUMMARY

[0004] The purpose of the present application is to provide a memory management method which can solve the technical problem of how to accurately manage the memory of the computing device to some extent. The present application also provides a memory management system, device and computer readable storage medium.

[0005] In order to achieve the above purpose, the present application provides the following technical scheme:

[0006] A memory management method, comprising:

[0007] obtaining a target neural network model;

[0008] based on the operation support of each target computing device to the operator in the target neural network model, the target neural network model is dissected into sub-functions corresponding to each target computing device;

[0009] distributing the sub-functions to the corresponding target computing device;

[0010] for each target computing device, based on the operation information of each computing unit in the target computing device, the corresponding sub-function is dissected into execution units corresponding to each computing unit, and the target computing device is managed in the granularity of the execution unit.

[0011] Preferably, the memory management of the target computing device in the granularity of the execution unit comprises:

[0012] dividing the memory of the target computing device into target memory and reserved memory;

[0013] determining the memory occupation information of the execution unit in the target memory;

[0014] managing the memory of the target computing device based on the memory occupation information.

[0015] Preferably, the determining the memory occupation information of the execution unit in the target memory comprises:

[0016] dividing the target memory into memory blocks corresponding to each execution unit;

[0017] taking the correspondence between the execution unit and the memory block as the memory occupation information.

[0018] Preferably, the dividing the target memory into memory blocks corresponding to each execution unit comprises:

[0019] dividing the target memory into memory blocks corresponding to each execution unit based on a memory reuse principle.

[0020] Preferably, the managing the memory of the target computing device based on the memory occupation information comprises:

[0021] counting the occurrence times of each execution unit in the sub-function, taking the occurrence times as the usage times of the memory block corresponding to the execution unit, and if the execution unit occurs once in the target computing device, reducing the value of the usage times by 1;

[0022] for each execution unit, judging whether the value of the usage times of the corresponding memory block is 0;

[0023] if the value of the usage times is 0, allowing the reuse of the memory block corresponding to the execution unit;

[0024] if the value of the usage times is not 0, prohibiting the reuse of the memory block corresponding to the execution unit, and returning to execute the step of if the execution unit occurs once in the target computing device, reducing the value of the usage times by 1.

[0025] Preferably, the dividing the target memory into memory blocks corresponding to each execution unit comprises:

[0026] dividing the target memory into memory blocks corresponding to each execution unit based on a fastest execution speed principle.

[0027] Preferably, after the managing the memory of the target computing device based on the memory occupation information, the method further comprises:

[0028] applying the reserved memory to memory compensation of the target memory.

[0029] A memory management system comprises:

[0030] A first obtaining module is configured to obtain a target neural network model.

[0031] A first splitting module is configured to split the target neural network model into sub-functions corresponding to each target computing device based on operation support of each target computing device for operators in the target neural network model.

[0032] A first distribution module is configured to distribute the sub-functions to the corresponding target computing devices.

[0033] A second splitting module is configured to split, for each target computing device, the corresponding sub-function into execution units corresponding to each computing unit in the target computing device based on operation information of each computing unit, and perform memory management on the target computing device in the granularity of the execution units.

[0034] A memory management device comprises:

[0035] A memory is configured to store a computer program.

[0036] A processor is configured to implement the steps of the memory management method according to any one of the above when executing the computer program.

[0037] A computer readable storage medium stores a computer program, and the computer program is configured to implement the steps of the memory management method according to any one of the above when executed by a processor.

[0038] The memory management method provided by the present application comprises the following steps: obtaining a target neural network model; splitting the target neural network model into sub-functions corresponding to each target computing device based on operation support of each target computing device for operators in the target neural network model; distributing the sub-functions to the corresponding target computing devices; and splitting, for each target computing device, the corresponding sub-function into execution units corresponding to each computing unit in the target computing device based on operation information of each computing unit, and performing memory management on the target computing device in the granularity of the execution units. In the present application, the target neural network model is first split into sub-functions corresponding to each target computing device based on operation support of each target computing device for operators in the target neural network model, so that the sub-function required for operation of each target computing device is consistent with the operation performance of the target computing device itself, and then for each target computing device, the corresponding sub-function is split into execution units corresponding to each computing unit in the target computing device based on operation information of each computing unit, and memory management is performed on the target computing device in the granularity of the execution units, thereby realizing accurate management of the memory of the computing device. The memory management system, device and computer readable storage medium provided by the present application also solve the corresponding technical problems. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0040] Figure 1 A first flowchart of a memory management method provided in an embodiment of this application;

[0041] Figure 2 A second flowchart of a memory management method provided in an embodiment of this application;

[0042] Figure 3 This is a diagram illustrating memory allocation.

[0043] Figure 4 This is a schematic diagram of a sub-function;

[0044] Figure 5 This is a schematic diagram of the structure of a memory management system provided in an embodiment of this application;

[0045] Figure 6 This is a schematic diagram of the structure of a memory management processing device provided in an embodiment of this application;

[0046] Figure 7 This is another structural schematic diagram of a memory management processing device provided in an embodiment of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] Please see Figure 1 , Figure 1 This is a first flowchart of a memory management method provided in an embodiment of this application.

[0049] This application provides a memory management method that may include the following steps:

[0050] Step S101: Obtain the target neural network model.

[0051] In actual application, the target neural network model to be operated can be acquired first. The type of the target neural network model can be determined according to actual needs, which is not limited herein.

[0052] Step S102: based on the operation support of each target computing device on the operators in the target neural network model, the target neural network model is dissected into sub-functions corresponding to each target computing device.

[0053] In actual application, after the target neural network model is acquired, the target neural network model can be dissected into sub-functions corresponding to each target computing device based on the operation support of each target computing device on the operators in the target neural network model. For example, a target computing device is suitable for convolution operation, and then the convolution operator in the target neural network model can be dissected into corresponding sub-functions to be distributed to the target computing device. For another example, a target computing device is suitable for pooling operation, and then the pooling operator in the target neural network model can be dissected into corresponding sub-functions to be distributed to the target computing device.

[0054] It should be noted that the target computing device refers to a device with computing capability. The type of the target computing device and the operator in the target neural network model can be determined according to actual needs. For example, the target computing device can be a CPU (central processing unit), a GPU (graphics processing unit), an FPGA (Field Programmable Gate Array), or the like. The operator in the target neural network model can be a convolution operator, a pooling operator, an activation operator, or the like, which is not limited herein.

[0055] Step S103: distribute the sub-functions to the corresponding target computing devices.

[0056] In actual application, after the target neural network model is dissected into sub-functions corresponding to each target computing device based on the operation support of each target computing device on the operators in the target neural network model, the obtained sub-functions can be distributed to the corresponding target computing devices, so that the target computing devices can process the corresponding sub-functions.

[0057] Step S104: for each target computing device, based on the operation information of each computing unit in the target computing device, the corresponding sub-function is dissected into execution units corresponding to each computing unit, and the target computing device is managed in memory with the execution unit as the granularity.

[0058] In actual application, after the sub-functions are distributed to the corresponding target computing devices, for each target computing device, the corresponding sub-function is also divided into execution units corresponding to each computing unit based on operation information of each computing unit in the target computing device, and the target computing device is managed in memory in units of execution units. Since the granularity of the execution units is smaller than that of the sub-functions, the target computing device can be managed in memory in units of execution units more meticulously and accurately.

[0059] It should be noted that the execution subject of the memory management method provided in the present application can be determined according to actual needs. For example, the execution subject can be a deep learning compiler. In this case, when the target neural network model is obtained, the deep learning compiler can read the target neural network model generated by the deep learning framework to obtain the target neural network model in the IR (Intermediate Representation) form that meets the deep learning compiler, thereby facilitating the deep learning compiler to process the target neural network model. Of course, the execution subject can also be a computer device on which the deep learning compiler is deployed, or a computer device on which the target neural network model needs to be run, and the like, which are not limited in the present application.

[0060] The memory management method provided in the present application comprises the following steps: obtaining a target neural network model; dividing the target neural network model into sub-functions corresponding to each target computing device based on operation support of each target computing device on operators in the target neural network model; distributing the sub-functions to the corresponding target computing devices; for each target computing device, dividing the corresponding sub-function into execution units corresponding to each computing unit in the target computing device based on operation information of each computing unit in the target computing device, and managing the target computing device in memory in units of execution units. In the present application, the target neural network model can be first divided into sub-functions corresponding to each target computing device based on operation support of each target computing device on operators in the target neural network model, so that the sub-function required for operation of each target computing device is consistent with the operation performance of the target computing device itself. Then, for each target computing device, the corresponding sub-function is also divided into execution units corresponding to each computing unit based on operation information of each computing unit in the target computing device, and the target computing device is managed in memory in units of execution units, thereby realizing accurate management of the memory of the computing device.

[0061] Please refer to Figure 2 , Figure 2 The second flowchart of the memory management method provided in the present application embodiment.

[0062] The memory management method provided in the present application embodiment can comprise the following steps:

[0063] Step S201: Obtain a target neural network model.

[0064] Step S202: Based on the operation support of each target computing device on the operators in the target neural network model, subdivide the target neural network model into sub-functions corresponding to each target computing device.

[0065] Step S203: Distribute the sub-functions to the corresponding target computing devices.

[0066] Step S204: For each target computing device, based on the operation information of each computing unit in the target computing device, subdivide the corresponding sub-function into execution units corresponding to each computing unit, divide the memory of the target computing device into target memory and reserved memory; determine the memory occupation information of the execution units in the target memory; and manage the memory of the target computing device based on the memory occupation information.

[0067] In actual applications, in the process of managing the memory of the target computing device at the granularity of the execution units, the memory of the target computing device can be divided into target memory and reserved memory; the memory occupation information of the execution units in the target memory can be determined; and the memory of the target computing device can be managed based on the memory occupation information. That is, the target memory is used to process the execution units first, and the reserved memory is used as the remaining memory when the target memory is insufficient, such as when the target memory is insufficient to meet the memory demand of the execution units, the reserved memory is used to compensate the target memory.

[0068] In actual applications, in the process of determining the memory occupation information of the execution units in the target memory, the target memory can be divided into memory blocks corresponding to each execution unit; and the correspondence between the execution units and the memory blocks can be used as the memory occupation information. In this way, the memory of the execution units can be accurately managed based on the correspondence between the execution units and the memory blocks. Assuming that there are four execution units, the allocation of the memory can be as shown in Figure 3 Figure 3 The memory space 1 in indicates the memory block corresponding to the first execution unit, Figure 3 The memory space 2 in indicates the memory block corresponding to the second execution unit, Figure 3 The memory space 3 in indicates the memory block corresponding to the third execution unit, Figure 3 The memory space 4 in indicates the memory block corresponding to the fourth execution unit, Figure 3 The unnumbered memory space in indicates the reserved memory, etc.

[0069] ​In actual application, in the process of dividing the target memory into memory blocks corresponding to each execution unit, the target memory can be divided into memory blocks corresponding to each execution unit based on the memory reuse principle. It should be noted that the memory reuse principle can reuse the same memory block between different execution units, or reuse the same memory block for input and output of the same execution unit, and the present application does not make specific limitation here.

[0070] In a specific application scenario, after the target memory is divided into memory blocks corresponding to each execution unit based on the memory reuse principle, the same memory block can be used by multiple execution units, which can cause the storage data of the memory block to be overwritten. If the overwritten data is still applied later, the subsequent operation cannot be performed, which can cause the final operation result to be unable to be obtained. In order to avoid this situation, in the process of managing the memory of the target computing device based on the memory occupation information, the number of occurrences of each execution unit in the sub-function can be counted, and the number of occurrences is used as the number of times of using the memory block corresponding to the execution unit. If the execution unit occurs once in the target computing device, the value of the number of times of using is reduced by 1, and if the execution unit does not occur in the target computing device, the value of the number of times of using is kept unchanged. For each execution unit, it is judged whether the value of the number of times of using the corresponding memory block is 0. If the value of the number of times of using is 0, the memory block corresponding to the execution unit is allowed to be reused. If the value of the number of times of using is not 0, the memory block corresponding to the execution unit is prohibited to be reused, and the step of executing the number of times of using is returned. If the execution unit occurs once in the target computing device, the value of the number of times of using is reduced by 1. In this way, in the case that the value of the number of times of using the memory block is not 0, that is, in the case that the memory block still needs to be used, the memory block cannot be reused, and the data in the memory block cannot be overwritten, which can ensure that the data in the memory block is used later.

[0071] For ease of understanding, it is assumed that the type of the sub-function is as shown in Figure 4 The number of times of using each operator in different paths of the computation graph is counted in a depth-first traversal manner from the third operator and the fourth operator of the sub-function. This number of times indicates the number of times that the output of the operator needs to be used by the subsequent operator, which is counted by Figure 4It can be seen that the first operator in the top-down direction appears in two paths, and its output is used by the third operator and the second operator respectively, and the use count of the first operator is 2. After completing the statistical count of each operator, the depth-first traversal is performed from the output as the first time, and the input and output of each operator are counted according to the memory number, so the first operator is processed first, and the input of the first operator is the input of the entire sub-function, and it is assumed that the memory number occupied by the first operator is 0, and the number of times of updating the memory number is the statistical number of the current data used by it. The input of the entire function is used only once, so the number of times of the memory number 0 is 1. For the output of the first operator, the existing memory number is traversed first, and it is compared whether the use count of the memory number is 0, and if it is 0 and the block size of the memory number is greater than or equal to the space required by the output of the operator, the memory number of the output is set to the memory number, otherwise a new space is opened in the memory for the new number. In the process of traversing the existing memory number, the use count of the memory block storing the input of the current operator is reduced by 1. In this way, the input and output numbers of each operator can be determined, and the reuse of memory can be realized.

[0072] In practical applications, in the process of dividing the target memory into memory blocks corresponding to each execution unit, the target memory can also be divided into memory blocks corresponding to each execution unit based on the principle of the fastest execution speed, and the present application does not make specific limitations here.

[0073] Please refer to Figure 5 , Figure 5 The structure diagram of a memory management system provided by the embodiment of the present application.

[0074] The memory management system provided by the embodiment of the present application can include:

[0075] The first obtaining module 101 is configured to obtain a target neural network model.

[0076] The first profiling module 102 is configured to profile the target neural network model into sub-functions corresponding to each target computing device based on the operation support of each target computing device for the operators in the target neural network model.

[0077] The first distribution module 103 is configured to distribute the sub-functions to the corresponding target computing devices.

[0078] The second profiling module 104 is configured to, for each target computing device, profile the corresponding sub-function into execution units corresponding to each computing unit in the target computing device based on the operation information of each computing unit, and perform memory management on the target computing device with the execution unit as the granularity.

[0079] The memory management system provided by the embodiment of the present application can include:

[0080] The first division unit is configured to divide the memory of the target computing device into a target memory and a reserved memory.

[0081] The first determination unit is configured to determine memory occupation information of the execution unit in the target memory.

[0082] The first management unit is configured to manage the memory of the target computing device based on the memory occupation information.

[0083] The memory management system provided by the embodiment of the present application can be specifically configured to: divide the target memory into memory blocks corresponding to the execution units; and take the correspondence between the execution units and the memory blocks as the memory occupation information.

[0084] The memory management system provided by the embodiment of the present application can be specifically configured to: divide the target memory into memory blocks corresponding to the execution units based on the memory reuse principle.

[0085] The memory management system provided by the embodiment of the present application can be specifically configured to: count the number of occurrences of each execution unit in the sub-function, take the number of occurrences as the usage frequency of the memory block corresponding to the execution unit, and reduce the value of the usage frequency by 1 if the execution unit occurs once in the target computing device; for each execution unit, determine whether the value of the usage frequency of the corresponding memory block is 0; if the value of the usage frequency is 0, allow the reuse of the memory block corresponding to the execution unit; if the value of the usage frequency is not 0, prohibit the reuse of the memory block corresponding to the execution unit, and return to the step of reducing the value of the usage frequency by 1 if the execution unit occurs once in the target computing device.

[0086] The memory management system provided by the embodiment of the present application can be specifically configured to: divide the target memory into memory blocks corresponding to the execution units based on the execution speed principle.

[0087] The memory management system provided by the embodiment of the present application can further include:

[0088] The first compensation module is configured to perform memory compensation on the target memory by using the reserved memory after the second profiling module manages the memory of the target computing device based on the memory occupation information.

[0089] The present application also provides a memory management device and a computer readable storage medium, which have the corresponding effects of the memory management method provided by the embodiment of the present application. Please refer to Figure 6 , Figure 6 The structure diagram of the memory management processing device provided by the embodiment of the present application is shown.

[0090] The embodiment of the application provides a memory management device, comprising a memory 201 and a processor 202, the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program:

[0091] Obtaining a target neural network model;

[0092] Based on operation support of each target computing device on an operator in the target neural network model, the target neural network model is dissected into sub-functions corresponding to each target computing device;

[0093] Distributing the sub-functions to the corresponding target computing devices;

[0094] For each target computing device, based on operation information of each computing unit in the target computing device, the corresponding sub-function is dissected into execution units corresponding to each computing unit, and the target computing device is managed in the granularity of the execution unit.

[0095] The embodiment of the application provides a memory management device, comprising a memory 201 and a processor 202, the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program: dividing the memory of the target computing device into a target memory and a reserved memory; determining memory occupation information of the execution unit in the target memory; and managing the memory of the target computing device based on the memory occupation information.

[0096] The embodiment of the application provides a memory management device, comprising a memory 201 and a processor 202, the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program: dividing the target memory into memory blocks corresponding to each execution unit; and taking the correspondence between the execution unit and the memory block as the memory occupation information.

[0097] The embodiment of the application provides a memory management device, comprising a memory 201 and a processor 202, the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program: based on a memory reuse principle, dividing the target memory into memory blocks corresponding to each execution unit.

[0098] The memory management device provided in the embodiment of the present application comprises a memory 201 and a processor 202, the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program: counting the occurrence times of each execution unit in a sub-function, taking the occurrence times as the usage times of the memory block corresponding to the execution unit, and reducing the value of the usage times by 1 if the execution unit occurs once in the target computing device; for each execution unit, judging whether the value of the usage times of the corresponding memory block is 0; if the value of the usage times is 0, allowing the reuse of the memory block corresponding to the execution unit; if the value of the usage times is not 0, prohibiting the reuse of the memory block corresponding to the execution unit, and returning to the step of reducing the value of the usage times by 1 if the execution unit occurs once in the target computing device.

[0099] The memory management device provided in the embodiment of the present application comprises a memory 201 and a processor 202, the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program: based on the principle of the fastest execution speed, dividing the target memory into memory blocks corresponding to each execution unit.

[0100] The memory management device provided in the embodiment of the present application comprises a memory 201 and a processor 202, the memory 201 stores a computer program, and the processor 202 implements the following steps when executing the computer program: based on the principle of the fastest execution speed, dividing the target memory into memory blocks corresponding to each execution unit.

[0101] Please refer to Figure 7 In another memory management device provided in the embodiment of the present application, the memory management device can further comprise: an input port 203 connected with the processor 202, used for transmitting the command input by the outside to the processor 202; a display unit 204 connected with the processor 202, used for displaying the processing result of the processor 202 to the outside; and a communication module 205 connected with the processor 202, used for realizing the communication between the memory management device and the outside. The display unit 204 can be a display panel, a laser scanning display, etc. The communication mode adopted by the communication module 205 includes but is not limited to the mobile high-definition link technology (HML), the universal serial bus (USB), the high-definition multimedia interface (HDMI), the wireless connection, the wireless fidelity technology (WiFi), the Bluetooth communication technology, the low-power Bluetooth communication technology, and the communication technology based on IEEE 802.11s.

[0102] The computer readable storage medium provided in the embodiment of the present application stores a computer program, and the computer program is executed by the processor to implement the following steps:

[0103] Obtaining a target neural network model;

[0104] based on operation support of each target computing device on operators in the target neural network model, the target neural network model is dissected into sub-functions corresponding to each target computing device;

[0105] the sub-functions are distributed to the corresponding target computing devices;

[0106] for each target computing device, based on operation information of each computing unit in the target computing device, the corresponding sub-function is dissected into execution units corresponding to each computing unit, and the target computing device is managed in the granularity of the execution units.

[0107] The computer readable storage medium provided in the embodiment of the application stores a computer program, and the computer program is executed by a processor to implement the following steps: dividing a memory of a target computing device into a target memory and a reserved memory; determining memory occupation information of an execution unit in the target memory; and managing the memory of the target computing device based on the memory occupation information.

[0108] The computer readable storage medium provided in the embodiment of the application stores a computer program, and the computer program is executed by a processor to implement the following steps: dividing a target memory into memory blocks corresponding to each execution unit; and taking a correspondence relationship between the execution unit and the memory block as memory occupation information.

[0109] The computer readable storage medium provided in the embodiment of the application stores a computer program, and the computer program is executed by a processor to implement the following steps: based on a memory reuse principle, dividing a target memory into memory blocks corresponding to each execution unit.

[0110] The computer readable storage medium provided in the embodiment of the application stores a computer program, and the computer program is executed by a processor to implement the following steps: counting a number of occurrences of each execution unit in a sub-function, taking the number of occurrences as a usage number of a memory block corresponding to the execution unit, and if the execution unit occurs once in a target computing device, reducing a value of the usage number by 1; for each execution unit, judging whether a value of the usage number of the corresponding memory block is 0; if the value of the usage number is 0, allowing reuse of the memory block corresponding to the execution unit; if the value of the usage number is not 0, prohibiting reuse of the memory block corresponding to the execution unit, and returning to the step of reducing the value of the usage number by 1 if the execution unit occurs once in the target computing device.

[0111] The computer readable storage medium provided by the embodiment of the present application stores a computer program, and the computer program is executed by a processor to implement the following steps: based on the principle of the fastest execution speed, the target memory is divided into memory blocks corresponding to each execution unit.

[0112] The computer readable storage medium provided by the embodiment of the present application stores a computer program, and the computer program is executed by a processor to implement the following steps: based on the principle of the fastest execution speed, the target memory is divided into memory blocks corresponding to each execution unit.

[0113] The computer readable storage medium involved in the present application includes a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0114] The memory management system, device and related part of the computer readable storage medium provided by the embodiment of the present application are described in detail in the corresponding part of the memory management method provided by the embodiment of the present application, and will not be described here. In addition, the part of the above technical solution provided by the embodiment of the present application which is consistent with the implementation principle of the corresponding technical solution in the prior art is not described in detail, so as not to be too verbose.

[0115] It should also be noted that, in this document, relational terms such as first and second and the like can only be used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations exist in any such actual relationship or order. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements does not only include those elements, but also includes other elements not explicitly listed or other elements inherent in such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0116] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A memory management method, characterized in that, include: Obtain the target neural network model; Based on the computational support of each target computing device for the operators in the target neural network model, the target neural network model is divided into sub-functions corresponding to each target computing device; Distribute the sub-function to the corresponding target computing device; For each target computing device, based on the computation information of each computing unit in the target computing device, the corresponding sub-function is divided into execution units corresponding to each computing unit, and memory management of the target computing device is performed at the granularity of the execution units; The step of distributing the sub-functions to the corresponding target computing devices includes: if the target computing device is suitable for convolution operations, then the convolution operator in the target neural network model is divided into corresponding sub-functions and distributed to the target computing device; if the target computing device is suitable for pooling operations, then the pooling operator in the target neural network model is divided into corresponding sub-functions and distributed to the target computing device. The step of managing the memory of the target computing device at the execution unit level includes: The memory of the target computing device is divided into target memory and reserved memory; Determine the memory usage information of the execution unit in the target memory; The memory of the target computing device is managed based on the memory usage information; The step of managing the memory of the target computing device based on the memory usage information includes: The number of times each execution unit appears in the sub-function is counted, and the number of times it appears is used as the number of times the memory block corresponding to the execution unit is used. If the execution unit appears once in the target computing device, the value of the number of times it is used is decremented by 1. For each execution unit, determine whether the value of the number of times the corresponding memory block is used is 0; If the value of the number of uses is 0, then the memory block corresponding to the execution unit is allowed to be reused; If the value of the number of uses is not 0, then the reuse of the memory block corresponding to the execution unit is prohibited, and the process returns to the step of decrementing the value of the number of uses by 1 if the execution unit appears once in the target computing device.

2. The method according to claim 1, characterized in that, Determining the memory usage information of the execution unit in the target memory includes: The target memory is divided into memory blocks corresponding to each of the execution units; The correspondence between the execution unit and the memory block is used as the memory usage information.

3. The method according to claim 2, characterized in that, The step of dividing the target memory into memory blocks corresponding to each of the execution units includes: Based on the principle of memory reuse, the target memory is divided into memory blocks corresponding to each of the execution units.

4. The method according to any one of claims 1 to 3, characterized in that, After managing the memory of the target computing device based on the memory usage information, the method further includes: The reserved memory is used to compensate the target memory.

5. A memory management system, characterized in that, include: The first acquisition module is used to acquire the target neural network model; The first partitioning module is used to partition the target neural network model into sub-functions corresponding to each of the target computing devices, based on the computational support of each target computing device for the operators in the target neural network model. The first distribution module is used to distribute the sub-function to the corresponding target computing device; The second partitioning module is used to partition the corresponding sub-function into execution units corresponding to each computing unit based on the operation information of each computing unit in the target computing device for each target computing device, and to perform memory management of the target computing device at the granularity of the execution units. The first distribution module is configured to: if the target computing device is suitable for convolution operations, then divide the convolution operator in the target neural network model into corresponding sub-functions and distribute them to the target computing device; if the target computing device is suitable for pooling operations, then divide the pooling operator in the target neural network model into corresponding sub-functions and distribute them to the target computing device. The second segmentation module includes: The first partitioning unit is used to divide the memory of the target computing device into target memory and reserved memory; The first determining unit is used to determine the memory usage information of the execution unit in the target memory; The first management unit is used to manage the memory of the target computing device based on the memory usage information; The first management unit is configured to: count the number of times each execution unit appears in the sub-function, use the number of appearances as the number of times the memory block corresponding to the execution unit is used, and decrement the number of times the execution unit appears once in the target computing device by 1; for each execution unit, determine whether the number of times the corresponding memory block is used is 0; if the number of times is used is 0, reuse of the memory block corresponding to the execution unit is allowed; if the number of times is not 0, reuse of the memory block corresponding to the execution unit is prohibited, and return to the step of decrementing the number of times the execution unit appears once in the target computing device by 1.

6. A memory management device, characterized in that, include: Memory, used to store computer programs; A processor for implementing the steps of the memory management method as described in any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the memory management method as described in any one of claims 1 to 4.

Images