Memory application method, device and equipment applied to neural network

Abstract

The application provides a memory application method and device applied to a neural network and equipment, the equipment comprises a memory and a processor; wherein the memory is used for storing processor executable instructions, and the processor is used for calling program instructions in the memory to execute the memory application method applied to the neural network. The memory application method, device and equipment applied to the neural network provided by the application optimize memory fragmentation and effectively avoid waste of memory space.

Description

Technical Field

[0001] This application relates to the field of neural network technology, and in particular to a memory allocation method, apparatus, and device for use in neural networks. Background Technology

[0002] A neural network is a mathematical model that processes information by adjusting the connections between a large number of nodes within a system. During training or inference computation, neural networks require a significant amount of memory to store the data being processed.

[0003] Currently, the common method for memory allocation is to send a memory allocation instruction to the memory manager when the target task needs memory for its current operation. This allows the memory manager to allocate the memory required by the target task and use a memory caching mechanism to retain the allocated memory for repeated use during the current operation of the target task.

[0004] However, as the target task runs, it will continuously generate new memory requirements. When the size of the new memory requirements generated by the target task is greater than the size of the unused memory reserved by the memory caching mechanism, the target task will request new memory from the memory manager again. In other words, the unused memory reserved by the memory caching mechanism cannot be utilized, thus forming memory fragmentation and wasting memory space. Summary of the Invention

[0005] This application provides a memory allocation method, apparatus, and device for neural networks to optimize memory fragmentation and avoid wasting memory space.

[0006] In a first aspect, embodiments of this application provide a memory allocation method applied to a neural network, comprising:

[0007] Receive a first memory request sent by the target task; wherein the first memory request includes the memory size required for the target task to run currently;

[0008] Based on the memory size currently required for the target task to run, determine the memory requirement level of the target task; wherein, the memory requirement level includes low, medium and high.

[0009] The amount of memory to be requested is determined based on the memory requirement level of the target task; different memory requirement levels correspond to different amounts of memory to be requested.

[0010] A second memory request is sent to the memory manager so that the memory manager allocates memory for the target task that is consistent with the size of the memory to be requested; wherein the second memory request includes the size of the memory to be requested.

[0011] Secondly, embodiments of this application provide a memory allocation device applied to a neural network, comprising:

[0012] A receiving module is configured to receive a first memory request sent by a target task; wherein the first memory request includes the memory size required for the target task to currently run;

[0013] The judgment module is used to determine the memory requirement level of the target task based on the memory size currently required for the target task to run; wherein the memory requirement level includes low, medium and high.

[0014] The determination module is used to determine the amount of memory to be requested based on the memory requirement level of the target task; wherein, different memory requirement levels correspond to different amounts of memory to be requested.

[0015] The sending module is configured to send a second memory request to the memory manager so that the memory manager allocates memory for the target task that is consistent with the size of the memory to be requested; wherein the second memory request includes the size of the memory to be requested.

[0016] Thirdly, embodiments of this application provide a processor and a memory communicatively connected to the processor;

[0017] The memory stores computer-executed instructions;

[0018] The processor executes computer execution instructions stored in the memory to implement the method as described in the first aspect.

[0019] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first aspect.

[0020] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0021] This application provides a memory allocation method, apparatus, and device for neural networks. It receives a first memory allocation request from a target task, wherein the first memory allocation request includes the memory size currently required for the target task's operation. Based on the memory size currently required for the target task's operation, it determines the memory requirement level of the target task, thereby determining the memory size to be allocated, and sends a second memory allocation request to the memory manager, so that the memory manager allocates memory to the target task that matches the memory size to be allocated. In other words, this application determines the memory requirement level of the target task based on its current memory requirements, then determines the memory size required for the corresponding requirement level, i.e., the memory size to be allocated, and requests the memory size required for the corresponding requirement level from the memory manager. This eliminates the need for the target task to request memory multiple times during operation, thereby optimizing memory fragmentation and effectively avoiding memory waste. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0023] Figure 1 A schematic diagram of a system architecture provided for an embodiment of this application;

[0024] Figure 2 A flowchart illustrating a memory allocation method applied to a neural network, provided in an embodiment of this application;

[0025] Figure 3 A flowchart illustrating a memory allocation method applied to a neural network, provided in an embodiment of this application;

[0026] Figure 4 A schematic diagram of the structure of a memory allocation device applied to a neural network provided in an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of the electronic device of this application.

[0028] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application.

[0030] Neural networks require a large amount of memory to store their computational data. Currently, a common memory allocation method is to send a memory request instruction to the memory manager when the target task needs memory. This allows the memory manager to allocate the required memory for the target task and use a memory caching mechanism to retain the allocated memory for repeated use during the current execution of the target task. In other words, the memory requirements of the target task can be responded to with low latency during the current execution of the target task.

[0031] However, as the target task runs, it will continuously generate new memory requirements. When the size of the new memory requirement generated by the target task is less than or equal to the size of the unused memory reserved by the memory caching mechanism, the memory reserved by the memory caching mechanism can successfully respond to the new memory requirement. However, when the size of the new memory requirement generated by the target task is greater than the size of the unused memory reserved by the memory caching mechanism, the target task will request new memory from the memory manager again. In other words, the unused memory previously reserved by the memory caching mechanism cannot be utilized, thus forming memory fragmentation and wasting memory space.

[0032] The memory allocation method, apparatus, and device for neural networks provided in this application aim to solve the above-mentioned technical problems in the prior art.

[0033] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0034] Figure 1 A schematic diagram of a system architecture provided in an embodiment of this application, such as... Figure 1 As shown, the Figure 1 The system architecture shown may specifically include target task 1, server 2 and memory manager 3, wherein server 2 is equipped with a memory allocation device.

[0035] The target task 1 can be a task such as training or inference computation of a neural network. The memory allocation device can be mounted on server 2 and is used to receive a first memory allocation request sent by target task 1, including the memory size currently required for the target task's operation. Based on the memory size currently required for the target task's operation, it determines the memory requirement level of the target task, thereby determining the memory size to be allocated, and sends a second memory allocation request including the memory size to be allocated to memory manager 3, so that memory manager 3 allocates memory consistent with the memory size to be allocated for the target task.

[0036] Example 1

[0037] Figure 2 A flowchart of a memory allocation method for neural networks provided in embodiments of this application is shown below. Figure 2 As shown, the memory allocation method for neural networks provided in this embodiment includes the following steps:

[0038] Step 101: Receive the first memory request sent by the target task; wherein the first memory request includes the memory size required for the target task to run.

[0039] Step 102: Determine the memory requirement level of the target task based on the memory size currently required for the target task to run; wherein the memory requirement level includes low, medium and high.

[0040] Step 103: Determine the amount of memory to be requested based on the memory requirement level of the target task; where different memory requirement levels correspond to different amounts of memory to be requested.

[0041] Step 104: Send a second memory request to the memory manager so that the memory manager allocates memory for the target task that is consistent with the size of the memory to be requested; wherein, the second memory request includes the size of the memory to be requested.

[0042] It should be noted that the execution entity of the memory allocation method for neural networks provided in this embodiment can be a memory allocation device for neural networks. In practical applications, this memory allocation device for neural networks can be implemented by a computer program, such as application software or a computer program, or by a medium storing the relevant computer program, such as a USB flash drive or optical disc; or it can be implemented by a physical device that integrates or installs the relevant computer program, such as a chip.

[0043] In this embodiment, since the neural network needs to request a large amount of memory to store the running data during the training or inference calculation process (that is, during the operation of the target task), and multiple memory requests will lead to a large number of memory fragments, resulting in a waste of memory space. Therefore, in order to avoid the waste of memory space, the memory request device can determine the memory requirement level of the target task, thereby determining the size of the memory to be requested, so that the target task does not need to request memory multiple times during the operation.

[0044] Specifically, the memory allocation device can first receive a first memory allocation request sent by the target task. This first memory allocation request includes the amount of memory required for the target task to currently run.

[0045] In practical applications, when the target task starts running, it sends a first memory request to the memory requesting device. The first memory request includes the memory size required for the target task to run. The unit of memory size can be bytes (B).

[0046] Next, after receiving the first memory request, the memory allocation device can determine the memory requirement level of the target task based on the memory size currently required for its operation. The memory requirement levels include low, medium, and high.

[0047] In practical applications, since the memory required by the target task during the entire operation of the neural network training or inference computation is greater than or equal to the memory required by the target task at a certain moment within a certain range, the memory requirement level of the target task can be determined based on the memory required by the target task at the moment of operation.

[0048] Next, after the memory allocation device determines the memory requirement level of the target task, it can determine the size of the memory to be allocated based on the memory requirement level of the target task. Different memory requirement levels correspond to different sizes of memory to be allocated. The specific determination method will be described in detail in Embodiment 2 of this application, and will not be repeated here.

[0049] Finally, after determining the size of the memory to be requested, the memory allocation device can send a second memory request to the memory manager, so that the memory manager can allocate memory for the target task that matches the size of the memory to be requested. The second memory request may include the size of the memory to be requested.

[0050] In addition to any other embodiment, in one optional implementation, before step 102, the method further includes: aligning the memory size required for the current operation of the target task based on the alignment requirements of the device where the target task is located.

[0051] In this embodiment, because different hardware platforms handle memory space differently, different devices typically have different memory alignment requirements. This means the requested memory size must be a common multiple of a certain integer, such as a common multiple of 64. Therefore, before the memory requesting device determines the memory requirement level of the target task based on the current memory size required by the target task, it needs to perform alignment processing on the memory size required by the device where the target task is located. This alignment processing refers to aligning the memory size required by the target task to a common multiple of the integer required by the device.

[0052] In one example, if the target task currently requires 500 bytes of memory, and the device requires the requested memory size to be aligned to a common multiple of 64 bytes, the memory requesting device can align the memory size currently required by the target task to 512 bytes, which is 8 times 64 bytes.

[0053] Accordingly, based on the above embodiment one, in an optional implementation, step 102 includes: determining the memory requirement level of the target task based on the memory size currently required for the target task to run after alignment processing.

[0054] In this embodiment, continuing with the previous example, if the memory size required for the current operation of the target task is 500 bytes, the memory allocation device can determine the memory requirement level of the target task based on the memory size required for the current operation of the target task after alignment processing, which is 512 bytes. This ensures the accuracy of the memory requirement level determination and thus accurately determines the size of the memory to be allocated.

[0055] This embodiment provides a memory allocation method for neural networks. It receives a first memory allocation request from a target task, which includes the memory size currently required for the target task's operation. Based on this memory size, the method determines the target task's memory requirement level, determines the required memory size, and then sends a second memory allocation request to the memory manager. This allows the memory manager to allocate memory to the target task that matches the required memory size. In other words, this embodiment determines the target task's memory requirement level based on its current memory needs, determines the required memory size for that level (the required memory size), and then requests that size from the memory manager. This eliminates the need for the target task to request memory multiple times during operation, thus optimizing memory fragmentation and effectively avoiding memory waste.

[0056] Example 2

[0057] Based on the above embodiment one, in an optional implementation, step 102 specifically includes: determining whether the memory size required for the current operation of the target task is less than a preset first threshold; wherein, the preset first threshold is determined based on the distribution of the expected memory size required by the target task throughout the entire operation; if the memory size required for the current operation of the target task is less than the preset first threshold, then the memory requirement level of the target task is determined to be low.

[0058] In this embodiment, the memory allocation device can determine whether the memory size currently required for the target task to run is less than a preset first threshold. The preset first threshold can be determined by the distribution of the expected memory size required by the target task throughout its operation.

[0059] Furthermore, it should be noted that the distribution of the expected memory size required by different target tasks during the entire operation may differ. For a target task requiring multiple memory, including multiple memory requests, the sizes of these multiple required memory are statistically analyzed, and a first threshold is determined based on their distribution. Optionally, the first threshold can be the average of the multiple required memory sizes, or the median of the multiple required memory sizes, or the most frequently occurring value among the multiple required memory sizes, etc., and can be analyzed according to specific circumstances; this application does not impose any limitations on this.

[0060] For example, in one example, the target task is to train a neural network. Since neural network training often requires a lot of memory, or often tries to use all the memory of the device to make full use of existing resources, the memory size required by the target task during the entire operation may be evenly distributed within a certain memory size range, such as 500 megabytes (MB). In this case, the preset first threshold can be 500 megabytes.

[0061] In another example, the target task is to perform neural network inference calculations. The memory size required by the target task during the entire operation is expected to be evenly distributed within 5 megabytes. Therefore, the preset first threshold can be 5 megabytes.

[0062] In one example, if the memory allocation device determines that the memory required for the current operation of the target task is less than a preset first threshold, it indicates that the memory required by the target task during the entire operation is as expected, and the memory allocation device can determine that the memory requirement level of the target task is low.

[0063] Based on the above embodiment 2, in an optional embodiment, step 102 further includes: if the memory size required for the current operation of the target task is not less than a preset first threshold, then locate the memory manager of the device where the target task is located and obtain the current unused memory size of the device; determine the memory requirement level of the target task based on the current memory size required for the current operation of the target task and the current unused memory size of the device.

[0064] Figure 3 A flowchart of a memory allocation method for neural networks provided in embodiments of this application is shown below. Figure 3 As shown, based on the above embodiment two, in an optional implementation, determining the memory requirement level of the target task according to the memory size currently required for the target task to run and the amount of unused memory on the device includes the following steps:

[0065] Step 201: Determine whether the ratio of the memory size required for the current operation of the target task to the current unused memory size of the device is less than a preset second threshold.

[0066] Step 202: If yes, then determine that the memory requirement level of the target task is medium.

[0067] Step 203: If not, then determine that the memory requirement level of the target task is high.

[0068] In this embodiment, if the memory request device determines that the memory size required for the current operation of the target task is not less than (i.e., greater than or equal to) a preset first threshold, it indicates that the memory required by the target task during the entire operation process may exceed expectations. Therefore, it is necessary to further determine the memory requirement level of the target task by combining the size of the currently unused (i.e., free) memory of the device where the target task is located.

[0069] Specifically, the memory allocation device first locates the memory manager of the device where the target task resides and obtains the current unused memory size of the device. Next, the memory allocation device determines whether the ratio of the memory size currently required for the target task to the current unused memory size of the device is less than a preset second threshold. The preset second threshold can be one-quarter or other suitable thresholds; this embodiment does not limit this.

[0070] In one example, if the memory allocation device determines that the ratio of the memory size required by the target task to the amount of unused memory on the device is less than a preset second threshold, it indicates that the memory size required by the target task during the entire operation is slightly higher than expected. In this case, the memory allocation device can determine that the memory requirement level of the target task is medium.

[0071] In another example, if the memory allocation device determines that the ratio of the memory size required by the target task to the amount of unused memory on the device is not less than (i.e., greater than or equal to) a preset second threshold, it indicates that the memory required by the target task during the entire operation process is significantly greater than expected, and the memory allocation device can determine that the memory requirement level of the target task is high.

[0072] For example, if the preset second threshold is one-quarter, and the current unused memory size of the device is 5 gigabytes (GB), and in one possible case, the memory size required for the target task to run is 1GB, then the memory allocation device can determine that the ratio of the memory size required for the target task to the current unused memory size of the device is one-fifth, which is less than the preset second threshold of one-quarter. Therefore, the memory allocation device can determine that the memory requirement level of the target task is medium.

[0073] In another possible scenario, if the target task currently requires 2GB of memory, the memory allocation device can determine that the ratio of the target task's current memory requirement to the device's currently unused memory is two-fifths, which is greater than a preset second threshold of one-quarter. Therefore, the memory allocation device can determine that the target task's memory requirement level is high.

[0074] Based on the above embodiment 2, in an optional embodiment, the memory request device can also determine the memory requirement level of the target task based on the difference between the memory size currently required for the target task to run and the memory size currently not used by the device.

[0075] Specifically, the memory allocation device can also determine whether the difference between the memory size currently required for the target task to run and the memory size currently not used by the device is greater than a preset third threshold. If so, the memory requirement level of the target task is determined to be medium; otherwise, the memory requirement level of the target task is determined to be high.

[0076] In addition, based on the above embodiment two, in another optional embodiment, the memory request device can also determine the memory requirement level of the target task based on the memory size currently required for the target task to run and the memory size currently used by the device.

[0077] Specifically, in one example, the memory allocation device can determine whether the ratio of the memory size currently required for the target task to the memory size currently used by the device is less than a preset fourth threshold. If so, the memory requirement level of the target task is determined to be medium; otherwise, the memory requirement level of the target task is determined to be high.

[0078] In another example, the memory allocation device can determine whether the difference between the memory size currently required for the target task to run and the memory size currently used by the device is greater than a preset fifth threshold. If so, the memory requirement level of the target task is determined to be medium; otherwise, the memory requirement level of the target task is determined to be high.

[0079] In an optional embodiment, the memory requirement level of the target task can also be determined based on factors including but not limited to the memory consumed by the program, the maximum memory of the device, the program's past running experience value, the number of programs running on the device, and the number of devices. This application does not limit this.

[0080] Based on the above embodiment two, in an optional implementation, step 103 specifically includes: if the memory requirement level of the target task is low, then the memory size to be requested is determined to be a first memory size; wherein, the first memory size is greater than or equal to a preset first threshold; if the memory requirement level of the target task is medium, then the memory size to be requested is determined to be a second memory size; wherein, the second memory size is greater than or equal to the product of the currently unused memory size of the device and the preset second threshold; if the memory requirement level of the target task is high, then the memory size to be requested is determined to be equal to the memory size currently required for the target task to run.

[0081] In this embodiment, in one example, if the memory allocation device determines that the memory requirement level of the target task is low, the memory allocation device can determine the size of the memory to be allocated as a first memory size. The first memory size can be greater than or equal to a preset first threshold.

[0082] For example, if the preset first threshold is 1MB, then the first memory size can be 1MB or other suitable values ​​greater than 1MB. This embodiment does not limit this.

[0083] In another example, if the memory allocation device determines that the memory requirement level of the target task is medium, the memory allocation device can determine the size of the memory to be allocated as a second memory size. This second memory size is greater than or equal to the product of the device's currently unused memory size and a preset second threshold.

[0084] For example, if the preset second threshold is one-fifth and the current unused memory size of the device is 5GB, then the second memory size can be 1GB or other suitable values ​​greater than 1GB. This embodiment does not limit this.

[0085] In another example, if the memory allocation device determines that the memory requirement level of the target task is high, it means that the target task requires a large amount of memory during the entire operation. Therefore, if the memory allocated to the target task is larger than the memory required for its current operation, it may lead to excessive memory consumption, thereby affecting the normal operation of other programs. Therefore, the memory allocation device can determine that the memory to be allocated is equal to the memory required for the target task to currently run.

[0086] The memory allocation method for neural networks provided in this embodiment determines the memory requirement level of the target task by judging whether the memory size currently required for the target task to run is less than a preset first threshold, and whether the ratio of the memory size currently required for the target task to the current unused memory size of the device is less than a preset second threshold. Based on the memory requirement level of the target task, the amount of memory to be allocated is determined, thereby enabling the target task to avoid multiple memory allocations during operation and effectively avoiding the waste of memory space.

[0087] Example 3

[0088] Figure 4 A schematic diagram of the structure of a memory allocation device applied to a neural network provided in an embodiment of this application is shown below. Figure 4 As shown, the memory allocation device for neural networks provided in this embodiment includes a receiving module 41, a judging module 42, a determining module 43, and a sending module 44. The receiving module 41 is used to receive a first memory allocation request sent by a target task; wherein the first memory allocation request includes the memory size currently required for the target task to run. The judging module 42 is used to determine the memory requirement level of the target task based on the memory size currently required for the target task to run; wherein the memory requirement level includes low, medium, and high. The determining module 43 is used to determine the memory size to be allocated based on the memory requirement level of the target task; wherein different memory requirement levels correspond to different memory sizes to be allocated. The sending module 44 is used to send a second memory allocation request to the memory manager, so that the memory manager allocates memory to the target task that is consistent with the memory size to be allocated; wherein the second memory allocation request includes the memory size to be allocated.

[0089] The memory allocation device for neural networks provided in this embodiment receives a first memory allocation request sent by a target task. The first memory allocation request includes the memory size currently required for the target task's operation. Based on the memory size currently required for the target task's operation, the device determines the memory requirement level of the target task, thereby determining the memory size to be allocated. A second memory allocation request is then sent to the memory manager, so that the memory manager allocates memory to the target task that matches the memory size to be allocated. In other words, in this embodiment, the memory requirement level of the target task is determined based on its current memory requirements, and the memory size required for that requirement level, i.e., the memory size to be allocated, is then allocated to the memory manager. This eliminates the need for the target task to allocate memory multiple times during operation, thereby optimizing memory fragmentation and effectively avoiding memory waste.

[0090] In an optional embodiment, the judgment module 42 is further configured to determine whether the memory size required for the current operation of the target task is less than a preset first threshold; wherein, the preset first threshold is determined by the distribution of the expected memory size required by the target task throughout the entire operation; if the memory size required for the current operation of the target task is less than the preset first threshold, then the memory requirement level of the target task is determined to be low.

[0091] In an optional embodiment, the judgment module 42 is further configured to: if the memory size required for the current operation of the target task is not less than a preset first threshold, then search the memory manager of the device where the target task is located and obtain the current unused memory size of the device; and determine the memory requirement level of the target task based on the current memory size required for the current operation of the target task and the current unused memory size of the device.

[0092] In an optional embodiment, the judgment module 42 is further configured to determine whether the ratio of the memory size currently required for the target task to the memory size currently not used by the device is less than a preset second threshold; if yes, the memory requirement level of the target task is determined to be medium; if no, the memory requirement level of the target task is determined to be high.

[0093] In an optional embodiment, the determining module 43 is further configured to: if the memory requirement level of the target task is low, determine the memory size to be requested as a first memory size; wherein the first memory size is greater than or equal to a preset first threshold; if the memory requirement level of the target task is medium, determine the memory size to be requested as a second memory size; wherein the second memory size is greater than or equal to the product of the currently unused memory size of the device and the preset second threshold; if the memory requirement level of the target task is high, determine the memory size to be requested is equal to the memory size currently required for the target task to run.

[0094] In an optional embodiment, the device further includes a processing module, configured to perform alignment processing on the memory size required for the current operation of the target task based on the alignment requirements of the device where the target task is located.

[0095] In an optional embodiment, the judgment module 42 is further configured to determine the memory requirement level of the target task based on the memory size required for the target task to run after the alignment process.

[0096] It should be noted that the technical solution and effects of the memory allocation device applied to the neural network provided in this embodiment can be found in the relevant content of the foregoing method embodiments, and will not be repeated here.

[0097] Example 4

[0098] Figure 5 This is a schematic diagram of the structure of the electronic device of this application, as shown below. Figure 5 As shown, this application also provides an electronic device 500, including: a memory 501 and a processor 502.

[0099] Memory 501 is used to store programs. Specifically, the program may include program code, which includes computer-executable instructions. Memory 501 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0100] Processor 502 is used to execute programs stored in memory 501.

[0101] The computer program is stored in memory 501 and configured to be executed by processor 502 to implement the memory allocation method for neural networks provided in any embodiment of this application. Related descriptions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the accompanying drawings, and will not be elaborated upon here.

[0102] In this embodiment, the memory 501 and the processor 502 are connected via a bus. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0103] Example 5

[0104] This application also provides a computer-readable storage medium storing a computer program thereon, which is executed by a processor to implement the memory allocation method for neural networks provided in any embodiment of this application.

[0105] Example 6

[0106] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the memory allocation method for neural networks provided in any embodiment of this application.

[0107] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0108] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0109] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in a combination of hardware and software functional modules.

[0110] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable memory allocation device applied to a neural network, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0111] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0112] Furthermore, although the operations are described in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this application. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0113] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0114] The foregoing can be better understood in accordance with the following terms:

[0115] Clause A1. A memory allocation method applied to a neural network, comprising:

[0116] Receive a first memory request sent by the target task; wherein the first memory request includes the memory size required for the target task to run currently;

[0117] Based on the memory size currently required for the target task to run, determine the memory requirement level of the target task; wherein, the memory requirement level includes low, medium and high.

[0118] The amount of memory to be requested is determined based on the memory requirement level of the target task; different memory requirement levels correspond to different amounts of memory to be requested.

[0119] A second memory request is sent to the memory manager so that the memory manager allocates memory for the target task that is consistent with the size of the memory to be requested; wherein the second memory request includes the size of the memory to be requested.

[0120] Clause A2. According to the method described in Clause A1, determining the memory requirement level of the target task based on the current memory size required for the target task to run includes:

[0121] Determine whether the memory size required for the current execution of the target task is less than a preset first threshold; wherein, the preset first threshold is determined by the distribution of the expected memory size required by the target task throughout its entire execution process;

[0122] If the memory required for the current operation of the target task is less than a preset first threshold, then the memory requirement level of the target task is determined to be low.

[0123] Clause A3. The method described in Clause A2, wherein determining the memory requirement level of the target task based on the current memory size required for the target task to run, further includes:

[0124] If the memory size required for the target task to run is not less than a preset first threshold, then the memory manager of the device where the target task is located is located is searched, and the amount of unused memory of the device is obtained.

[0125] The memory requirement level of the target task is determined based on the memory size currently required for the target task to run and the amount of unused memory on the device.

[0126] Clause A4. According to the method described in Clause A3, determining the memory requirement level of the target task based on the current memory size required for the target task to run and the current unused memory size of the device includes:

[0127] Determine whether the ratio of the memory size currently required for the target task to the current unused memory size of the device is less than a preset second threshold.

[0128] If so, then the memory requirement level of the target task is determined to be medium.

[0129] If not, then the memory requirement level of the target task is determined to be high.

[0130] Clause A5. The method described in Clause A4, wherein determining the amount of memory to be requested based on the memory requirement level of the target task includes:

[0131] If the memory requirement level of the target task is low, then the memory size to be requested is determined to be the first memory size; wherein, the first memory size is greater than or equal to a preset first threshold.

[0132] If the memory requirement level of the target task is medium, then the memory size to be requested is determined to be the second memory size; wherein, the second memory size is greater than or equal to the product of the current unused memory size of the device and the preset second threshold;

[0133] If the memory requirement level of the target task is high, then the amount of memory to be requested is equal to the amount of memory currently required for the target task to run.

[0134] Clause A6. The method according to any one of Clauses A1-A5, before determining the memory requirement level of the target task based on the current memory size required for the target task to run, further includes:

[0135] Based on the alignment requirements of the device where the target task is located, the memory size required for the current operation of the target task is aligned.

[0136] Clause A7. According to the method described in Clause A6, determining the memory requirement level of the target task based on the current memory size required for the target task to run includes:

[0137] Based on the memory size required for the target task to run after alignment, determine the memory requirement level of the target task.

[0138] Clause A8. A memory allocation device for use in a neural network, comprising:

[0139] A receiving module is configured to receive a first memory request sent by a target task; wherein the first memory request includes the memory size required for the target task to currently run;

[0140] The judgment module is used to determine the memory requirement level of the target task based on the memory size currently required for the target task to run; wherein the memory requirement level includes low, medium and high.

[0141] The determination module is used to determine the amount of memory to be requested based on the memory requirement level of the target task; wherein, different memory requirement levels correspond to different amounts of memory to be requested.

[0142] The sending module is configured to send a second memory request to the memory manager so that the memory manager allocates memory for the target task that is consistent with the size of the memory to be requested; wherein the second memory request includes the size of the memory to be requested.

[0143] Clause A9. An electronic device comprising: a processor, and a memory communicatively connected to said processor;

[0144] The memory stores computer-executed instructions;

[0145] The processor executes computer execution instructions stored in the memory to implement the method as described in any one of clauses A1-A7.

[0146] Clause A10. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the method as described in any one of Clauses A1-A7.

[0147] Clause A11. A computer program product comprising a computer program that, when executed by a processor, implements the method as described in any one of Clauses A1-A7.

[0148] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A memory allocation method applied to neural networks, characterized in that, include: Receive a first memory request sent by the target task; wherein the first memory request includes the memory size required for the target task to run currently; Based on the memory size currently required for the target task to run, determine the memory requirement level of the target task; wherein, the memory requirement level includes low, medium and high. The amount of memory to be requested is determined based on the memory requirement level of the target task; different memory requirement levels correspond to different amounts of memory to be requested, and the memory to be requested is used for the target task to complete the entire operation process. A second memory request is sent to the memory manager so that the memory manager allocates memory for the target task that is consistent with the size of the memory to be requested; wherein the second memory request includes the size of the memory to be requested.

2. The method according to claim 1, characterized in that, The step of determining the memory requirement level of the target task based on the current memory size required for its operation includes: Determine whether the memory size required for the current execution of the target task is less than a preset first threshold; wherein, the preset first threshold is determined by the distribution of the expected memory size required by the target task throughout its entire execution process; If the memory required for the current operation of the target task is less than a preset first threshold, then the memory requirement level of the target task is determined to be low.

3. The method according to claim 2, characterized in that, The step of determining the memory requirement level of the target task based on the current memory size required for its operation further includes: If the memory size required for the target task to run is not less than a preset first threshold, then the memory manager of the device where the target task is located is located is searched, and the amount of unused memory of the device is obtained. The memory requirement level of the target task is determined based on the memory size currently required for the target task to run and the amount of unused memory on the device.

4. The method according to claim 3, characterized in that, The step of determining the memory requirement level of the target task based on the memory size currently required for the target task to run and the amount of unused memory on the device includes: Determine whether the ratio of the memory size currently required for the target task to the current unused memory size of the device is less than a preset second threshold. If so, then the memory requirement level of the target task is determined to be medium. If not, then the memory requirement level of the target task is determined to be high.

5. The method according to claim 4, characterized in that, Determining the amount of memory to be requested based on the memory requirement level of the target task includes: If the memory requirement level of the target task is low, then the memory size to be requested is determined to be the first memory size; wherein, the first memory size is greater than or equal to a preset first threshold. If the memory requirement level of the target task is medium, then the memory size to be requested is determined to be the second memory size; wherein, the second memory size is greater than or equal to the product of the current unused memory size of the device and the preset second threshold; If the memory requirement level of the target task is high, then the amount of memory to be requested is equal to the amount of memory currently required for the target task to run.

6. The method according to any one of claims 1-5, characterized in that, Before determining the memory requirement level of the target task based on the current memory size required for its operation, the method further includes: Based on the alignment requirements of the device where the target task is located, the memory size required for the current operation of the target task is aligned.

7. The method according to claim 6, characterized in that, The step of determining the memory requirement level of the target task based on the current memory size required for its operation includes: Based on the memory size required for the target task to run after alignment, determine the memory requirement level of the target task.

8. A memory allocation device for use in neural networks, characterized in that, include: A receiving module is configured to receive a first memory request sent by a target task; wherein the first memory request includes the memory size required for the target task to currently run; The judgment module is used to determine the memory requirement level of the target task based on the memory size currently required for the target task to run; wherein the memory requirement level includes low, medium and high. The determination module is used to determine the amount of memory to be requested based on the memory requirement level of the target task; wherein, different memory requirement levels correspond to different amounts of memory to be requested, and the memory to be requested is used for the target task to complete the entire operation process; The sending module is configured to send a second memory request to the memory manager so that the memory manager allocates memory for the target task that is consistent with the size of the memory to be requested; wherein the second memory request includes the size of the memory to be requested.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images