Memory allocation method, memory allocation device, electronic device and readable storage medium

Abstract

The present application discloses a memory allocation method, a memory allocation device, an electronic device and a readable storage medium, which belong to the field of communication technology. The memory allocation method comprises: obtaining global information of a neural network; obtaining a memory block reuse strategy of the neural network based on the global information; and determining a memory allocation strategy of the neural network based on the memory block reuse strategy.

Description

Technical Field

[0001] The present application belongs to the field of communication technology, and specifically relates to a memory allocation method, a memory allocation device, an electronic device and a readable storage medium. Background Art

[0002] With the booming development of artificial intelligence and big data, more and more deep neural network algorithms are deployed on electronic devices in order to balance user experience and privacy protection. The forward reasoning calculation of deep neural network algorithms requires a large amount of running memory to ensure the correctness of intermediate calculations, and the limitation of running memory on electronic devices is a major factor hindering the design of mobile neural networks. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a memory allocation method, a memory allocation device, an electronic device and a readable storage medium, which can solve the problem of limited running memory of deep neural network algorithms in related technologies.

[0004] In a first aspect, an embodiment of the present application provides a memory allocation method, the memory allocation method comprising:

[0005] Get the global information of the neural network;

[0006] Obtain memory block reuse strategy for neural network based on global information;

[0007] Determine the memory allocation strategy of the neural network based on the memory block reuse strategy.

[0008] In a second aspect, an embodiment of the present application provides a memory allocation device, the memory allocation device comprising:

[0009] The acquisition module is used to obtain the global information of the neural network;

[0010] An allocation module, used to obtain the memory block reuse strategy of the neural network based on global information;

[0011] The determination module is used to determine the memory allocation strategy of the neural network based on the memory block reuse strategy.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method of the first aspect.

[0013] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method of the first aspect are implemented.

[0014] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method of the first aspect.

[0015] In an embodiment of the present application, a scheme for implementing a neural network memory reuse strategy is proposed. Specifically, according to the global information of the neural network, a memory block reuse strategy is determined for the nodes of the neural network, so that when the neural network is used, memory is allocated to the nodes of the neural network according to the memory block reuse strategy. In the above manner, the reuse rate of the electronic device memory can be improved, and the total running memory required for the forward reasoning of the neural network can be reduced, thereby enhancing the complexity of the electronic device neural network, the diversity of the running platform, and the number of parallel neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of a memory allocation method according to an embodiment of the present application;

[0017] Figure 2 It is one of the schematic diagrams of the memory allocation method of the embodiment of the present application;

[0018] Figure 3 This is the second schematic diagram of the memory allocation method of the embodiment of the present application;

[0019] Figure 4 This is the third schematic diagram of the memory allocation method of the embodiment of the present application;

[0020] Figure 5 is a schematic block diagram of a memory allocation device according to an embodiment of the present application;

[0021] Figure 6 It is one of the schematic block diagrams of the electronic device according to the embodiment of the present application;

[0022] Figure 7 This is the second schematic block diagram of the electronic device according to the embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0024] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0025] The memory allocation method, memory allocation device, electronic device and readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0026] The present application embodiment provides a memory allocation method, such as Figure 1 As shown, the memory allocation method includes:

[0027] Step 102, obtaining global information of the neural network;

[0028] Step 104, obtaining a memory block reuse strategy of the neural network based on the global information;

[0029] Step 106, determining a memory allocation strategy for the neural network based on the memory block reuse strategy.

[0030] In this embodiment, a scheme for implementing a neural network memory reuse strategy is proposed. Specifically, a memory block reuse strategy is determined for nodes of the neural network based on global information of the neural network, so that memory is allocated to the nodes of the neural network according to the memory block reuse strategy when the neural network is used.

[0031] Through the above method, the reuse rate of the electronic device memory can be improved, and the total running memory required for the forward reasoning of the neural network can be reduced, thereby enhancing the complexity of the electronic device neural network, the diversity of the operating platform and the number of parallel neural networks.

[0032] Further, in one embodiment of the present application, the global information includes required memory blocks of the nodes of the neural network; the memory block reuse strategy of the neural network is obtained based on the global information, including: sorting the nodes of the neural network in order of the required memory blocks from large to small; allocating memory blocks to the nodes of the neural network according to the sorting order of the nodes, and generating at least one memory block reuse strategy; determining the memory allocation strategy of the neural network based on the memory block reuse strategy, including: using a first target reuse strategy as the memory allocation strategy of the neural network, wherein the first target reuse strategy is a memory block reuse strategy in which the sum of memory blocks is less than or equal to a first preset threshold.

[0033] In this embodiment, according to the memory blocks required by each node of the neural network of the electronic device, all nodes of the neural network are sorted in descending order according to the required memory blocks. Then, memory blocks are allocated to the nodes in the sorted order from front to back, that is, larger memory blocks are reused first to reduce the application of the total running memory of the neural network.

[0034] At least one memory allocation method is obtained by allocating memory blocks to the nodes, and further, a memory allocation method whose total memory is less than or equal to a first preset threshold is output. For example, 1,000 memory allocation methods are obtained, and the memory allocation methods are sorted by total memory size, and the allocation method with the smallest total memory allocation is selected as the optimal memory reuse strategy for the neural network, so that when the neural network is executed, a memory block is allocated to each node according to the optimal memory reuse strategy.

[0035] Through the implementation scheme of the above-mentioned neural network memory reuse strategy in the embodiment of the present application, the reuse rate of the electronic device memory can be improved and the total running memory required for the forward reasoning of the neural network can be reduced.

[0036] It should be noted that the memory reuse of neural network operation on the central processing unit (CPU) and digital signal processing (DSP) hardware can also further compress the memory through pointer offset.

[0037] Furthermore, in one embodiment of the present application, memory blocks are allocated to nodes of a neural network according to the sorting order of the nodes, and at least one memory block reuse strategy is generated, including: dividing the nodes of the neural network into N node groups according to the sorting order of the nodes; allocating memory blocks to nodes of each node group, and generating at least one memory block reuse strategy; wherein N is a positive integer greater than or equal to 4.

[0038] In this embodiment, all nodes of the neural network are divided into N node groups in the order of the required memory blocks from large to small, each node group includes at least one node, and the number of nodes in each node group can be equal or unequal. For example, the neural network includes 20 nodes, the first 6 nodes are divided into the first node group, the 7th to 10th nodes are divided into the second node group, the 11th to 14th nodes are divided into the third node group, and the remaining nodes are divided into the fourth node group. Then, memory blocks are allocated to the nodes of each node group, at least one memory block reuse strategy of the neural network is generated, and finally the reuse strategy with the smallest total memory allocation is selected as the optimal memory allocation strategy of the neural network.

[0039] In the embodiment of the present application, a truncated (i.e., grouped) deep search is performed to determine the possibility of reusing various memory blocks, thereby selecting a reuse strategy with the smallest total running memory. The implementation of the present application can achieve compression optimization of the running memory for neural networks of any structure.

[0040] Furthermore, in one embodiment of the present application, memory blocks are allocated to nodes of each node group, and at least one memory block reuse strategy is generated, including: determining at least one first reused memory block of the i-th node in the first node group, and allocating the first reused memory block to the i-th node, and generating at least one first memory block reuse strategy; determining the first non-reuseable memory block of the i-th node, and allocating the first non-reuseable memory block to the i-th node, and generating at least one second memory block reuse strategy; using the second target reuse strategy as the memory block reuse strategy; wherein, i is greater than or equal to 1 and less than or equal to O, O is the total number of nodes in the first node group, and the second target reuse strategy is the first memory block reuse strategy or the second memory block reuse strategy in which the sum of memory blocks is less than or equal to a second preset threshold.

[0041] In this embodiment, a memory block reuse strategy is determined using permutation and combination search, which is similar to a tree structure search and explores all reuse possibilities.

[0042] Specifically, for any node in the first node group, at least one first multiplexed memory block of the node is determined, and the first multiplexed memory block is allocated to the node, generating at least one first memory block multiplexing strategy, and the first non-multiplexed memory block of the node is determined, and the first non-multiplexed memory block is allocated to the node, generating at least one second memory block multiplexing strategy. The at least one first memory block multiplexing strategy and the at least one second memory block multiplexing strategy are arranged in descending order according to the total memory block, and the memory block multiplexing strategy whose total memory (i.e., the total memory block) is less than or equal to the second preset threshold is taken as the memory block multiplexing strategy of the node in the first node group of the neural network.

[0043] For example, the first node group includes the nodes of the first 6 largest memory blocks in the neural network. When traversing the nodes of the first 6 largest memory blocks and searching for memory blocks with no connection conflicts and reusable sizes, non-reuse situations are considered. Specifically, Figure 2 As shown, the first node 202 and the second node 204 represent the traversed nodes, the number 36 in the first node 202 represents its memory size of 36 units, the number 2 represents its position in the neural network, and the number 20 in the second node 204 represents its memory size of 20 units, and the number 0 represents its position in the neural network. When traversing to the first node 202, the reusable memory block 206 (the memory size of the memory block 206 is 40 units) will be screened out first, and a branch will be split to consider the reuse of the subsequent memory.

[0044] At the same time, non-reuse situations are also considered, that is, instead of directly reusing the memory block 206, a memory block 208 is allocated to the first node 202 alone (the memory size of the memory block 208 is 36 units), and a branch is split to consider the memory reuse strategy of other nodes.

[0045] In addition to non-reuse, when the current node faces multiple reusable memory blocks, different branches will be split to consider the subsequent memory reuse strategy. Specifically, Figure 2 As shown, the second node 204 has three reusable memory blocks in the left branch: memory block 210, memory block 212, and memory block 210 (memory sizes are 20 units, 28 units, and 36 units, respectively). At this time, the selection of all memory blocks will be traversed.

[0046] As the number of traversed nodes increases, more branches will be generated, and the branches will grow exponentially. However, since only the nodes with the first 6 largest memory blocks are searched, the search volume at this stage will not exceed 720 times. After obtaining all memory reuse strategies for the nodes with the first 6 largest memory blocks, these candidate strategies will be sorted by total memory size, and the top 10 strategies with the smallest total memory will be selected for the next round of search.

[0047] Through the above method, the memory block reuse strategy is determined by using the permutation and combination search method, memory blocks are preferentially allocated to nodes of the neural network with larger memory block requirements, and the total running memory application of the neural network is reduced.

[0048] Furthermore, in one embodiment of the present application, memory blocks are allocated to nodes of each node group, and at least one memory block reuse strategy is generated, including: based on the memory block reuse strategy determined by the M-1th node group, determining at least one second reused memory block of the jth node in the Mth node group, and allocating the second reused memory block to the jth node, and generating at least one third memory block reuse strategy; determining the second non-reuseable memory block of the jth node, and allocating the second non-reuseable memory block to the i-th node, and generating at least one fourth memory block reuse strategy; using the third target reuse strategy as the memory block reuse strategy; wherein, M is greater than 1 and less than N, j is greater than or equal to 1 and less than or equal to P, P is the total number of nodes in the Mth node group, and the third target reuse strategy is a third memory block reuse strategy or a fourth memory block reuse strategy in which the sum of memory blocks is less than or equal to a third preset threshold.

[0049] In this embodiment, after determining the memory block reuse strategy of the M-1th node group, for any node in the Mth node group, determine at least one second reused memory block of the node, allocate the second reused memory block to the node, generate at least one third memory block reuse strategy, and determine the second non-reuse memory block of the node, allocate the second non-reuse memory block to the node, generate at least one fourth memory block reuse strategy. Arrange at least one third memory allocation method and at least one fourth memory allocation method in descending order of the total memory block, and take the memory block reuse strategy with a total memory (i.e., the total memory block) less than or equal to the third preset threshold as the memory block reuse strategy of the nodes in the first M node groups of the neural network.

[0050] Exemplarily, the neural network includes 20 nodes, the first 6 nodes are divided into the first node group, the 7th to 10th nodes are divided into the second node group, the 11th to 14th nodes are divided into the third node group, and the remaining nodes are divided into the fourth node group. After determining the first 10 best reuse strategies for the nodes of the first 6 largest memory blocks, the nodes of the 7th to 10th largest memory blocks will be searched for reuse memory based on these 10 best reuse strategies. Specifically, the search will still be carried out using the permutation and combination search strategy, and different branches will be split as different local memory reuse strategies. The number of branches at this stage will not exceed 2401. Finally, the 10 reuse strategies obtained from the first node group can all obtain corresponding expansion branches, and then the first 100 reuse strategies with the smallest total memory allocation are selected as the input for the next step. And continue to perform the same reuse search on the nodes of the 11th to 14th largest memory blocks, and finally select the first 1000 reuse strategies with the smallest total memory allocation from the output reuse strategies as the final output.

[0051] Through the above method, the memory block reuse strategy is determined by using the permutation and combination search method, and memory blocks are allocated to nodes in the order of the required memory block sizes, thereby reducing the total running memory application of the neural network and enabling the neural network to obtain compression optimization in terms of running memory.

[0052] Furthermore, in one embodiment of the present application, the global information includes topological structure information of the nodes of the neural network; memory blocks are allocated to the nodes of each node group, and at least one memory block reuse strategy is generated, including: based on the memory block reuse strategy determined by the N-1th node group, according to the memory blocks of the first two nodes of the tth node in the topological structure of the tth node in the Nth node group, a memory block is allocated to the tth node, and at least one fifth memory block reuse strategy is generated; the fifth memory block reuse strategy is used as the memory block reuse strategy; wherein t is greater than or equal to 1 and less than or equal to Q, and Q is the total number of nodes in the Nth node group.

[0053] In this embodiment, after traversing the first N-1 node groups, due to the characteristic of the deep neural network that gradually compresses features, the reuse allocation of the nodes that need to be traversed later will not have much impact on the total memory size. Therefore, the traversal of the subsequent nodes (that is, the nodes in the Nth node group) will no longer perform permutation and combination search, but instead use a direct importance judgment method to select reusable memory blocks. Specifically, after determining the memory block reuse strategy of the N-1th node group, in order to avoid deadlock of local nodes, for any node in the last node group, a memory block is allocated to the node according to the memory blocks of the first two nodes of the node in the topological structure, thereby generating at least one fifth memory block reuse strategy. At least one fifth memory block reuse strategy is used as the memory block reuse strategy of the nodes of the N node groups of the neural network.

[0054] For example, as in the above example, the first 1000 reuse strategies with the smallest total memory allocation are obtained. When selecting the first 1000 reuse strategies with the smallest total memory allocation to continue strategy allocation, multiple reuse strategies with the same total memory size are often encountered. At this time, these reuse strategies need to be sorted.

[0055] like Figure 3 As shown, the node 302 at position 0 and the node 304 at position 3 share the same memory block, the node 306 at position 1 and the node 308 at position 4 share the same memory block, and the node 310 at position 2 uses one memory block. It should be noted that the large number in the node represents the memory unit size required by the node, and the small number represents the topological position of the node in the neural network. If we take Figure 3 The strategy shown as the reuse strategy of the current neural network needs to allocate 36+20+16=72 units of memory. This is because the topological positions of nodes 302 and 304 cause local nodes to deadlock, and the number of reusable memory blocks has decreased. Node 302 at position 0 and node 304 at position 3 can only allocate memory separately. In fact, the best strategy is for node 302 at position 0, node 310 at position 2, and node 308 at position 4 to share the same memory block, so that the total memory allocation will be compressed to 36+20=56 units.

[0056] Therefore, when selecting the reuse strategy with the smallest total memory allocation among the first 1000 to continue strategy allocation, if a reuse strategy with the same total memory size is encountered, the topological position of each node will be judged, and the strategy of cross-node memory reuse will be selected as much as possible to reduce the occurrence of deadlock and provide more and more suitable reuse options for traversing other new nodes later.

[0057] It should be noted that the actual topological position of the node is determined by recording the previous and next topological relationships of each network node, and the network reasoning calculation errors caused by reusing the wrong memory block are avoided by recording the previous and next topological relationships of each network node, so as to prepare for the subsequent reuse of non-conflicting memory blocks. The cross node indicates two nodes separated by a node. For example, for the sth node, its cross nodes include the s-2th node and the s+2th node. Figure 4 As shown, node 304 and node 306 are cross nodes, and node 308 and node 310 are cross nodes.

[0058] In addition, more cross-node memory allocations mean that the current reuse strategy is more important. Among the reuse strategies with the same total memory size, the reuse strategy with a higher importance will be selected as the final output reuse strategy.

[0059] Through the above method, for nodes with a later required memory block size, a reusable memory block is selected by importance judgment to avoid local node deadlock and improve the accuracy of memory block reuse.

[0060] It should be noted that the memory allocation method provided in the embodiment of the present application can be executed by a memory allocation device, or a control module in the memory allocation device for executing the memory allocation method. In the embodiment of the present application, the memory allocation device provided in the embodiment of the present application is described by taking the memory allocation device executing the memory allocation method as an example.

[0061] The present application embodiment provides a memory allocation device, such as Figure 5 As shown, the memory allocation device 500 includes:

[0062] An acquisition module 502, used to acquire global information of the neural network;

[0063] An allocation module 504, used to obtain a memory block reuse strategy of a neural network based on global information;

[0064] The determination module 506 is used to determine the memory allocation strategy of the neural network based on the memory block reuse strategy.

[0065] In this embodiment, a scheme for implementing a neural network memory reuse strategy is proposed. Specifically, according to the global information of the neural network, a memory block reuse strategy is determined for the nodes of the neural network, so that when the neural network is used, memory is allocated to the nodes of the neural network according to the memory block reuse strategy. In this way, the reuse rate of the electronic device memory can be improved, and the total running memory required for the forward reasoning of the neural network can be reduced, thereby enhancing the complexity of the electronic device neural network, the diversity of the running platform, and the number of parallel neural networks.

[0066] Further, in one embodiment of the present application, the global information includes required memory blocks of the nodes of the neural network; the memory allocation device includes: a sorting module, used to sort the nodes of the neural network in order of the required memory blocks from large to small; an allocation module 504, specifically used to allocate memory blocks to the nodes of the neural network according to the sorting order of the nodes, and generate at least one memory block reuse strategy; a determination module 506, specifically used to use a first target reuse strategy as the memory allocation strategy of the neural network, wherein the first target reuse strategy is a memory block reuse strategy in which the sum of memory blocks is less than or equal to a first preset threshold.

[0067] In this embodiment, according to the required memory blocks of each node of the neural network of the electronic device, all nodes of the neural network are sorted in the order of the required memory blocks from large to small. Then, memory blocks are allocated to the nodes in the order from front to back in the sorting order, that is, larger memory blocks are reused preferentially to reduce the total running memory application of the neural network. At least one memory allocation method is obtained by allocating memory blocks to the nodes, and further, a memory allocation method in which the total memory is less than or equal to the first preset threshold is output. Through the implementation scheme of the above-mentioned neural network memory reuse strategy of the embodiment of the present application, the reuse rate of the electronic device memory can be improved, and the total running memory required for the forward reasoning of the neural network can be reduced, thereby enhancing the complexity of the electronic device neural network, the diversity of the operating platform and the number of parallel neural networks.

[0068] Furthermore, in one embodiment of the present application, the memory allocation device 500 also includes: a division module, used to divide the nodes of the neural network into N node groups according to the sorting order of the nodes; an allocation module 504, specifically used to allocate memory blocks to the nodes of each node group and generate at least one memory block reuse strategy; wherein N is a positive integer greater than or equal to 4.

[0069] Further, in one embodiment of the present application, the allocation module 504 is specifically used to: determine at least one first multiplexed memory block of the i-th node in the first node group, and allocate a first multiplexed memory block to the i-th node, generating at least one first memory block reuse strategy; determine the first non-multiplexed memory block of the i-th node, and allocate a first non-multiplexed memory block to the i-th node, generating at least one second memory block reuse strategy; use the second target reuse strategy as the memory block reuse strategy; wherein, i is greater than or equal to 1 and less than or equal to O, O is the total number of nodes in the first node group, and the second target reuse strategy is the first memory block reuse strategy or the second memory block reuse strategy in which the sum of memory blocks is less than or equal to a second preset threshold.

[0070] Further, in one embodiment of the present application, the allocation module 504 is specifically used to: determine at least one second multiplexed memory block of the jth node in the Mth node group based on the memory block reuse strategy determined by the M-1th node group, and allocate a second multiplexed memory block to the jth node to generate at least one third memory block reuse strategy; determine the second non-multiplexed memory block of the jth node, and allocate a second non-multiplexed memory block to the ith node to generate at least one fourth memory block reuse strategy; use the third target reuse strategy as the memory block reuse strategy; wherein, M is greater than 1 and less than N, j is greater than or equal to 1 and less than or equal to P, P is the total number of nodes in the Mth node group, and the third target reuse strategy is a third memory block reuse strategy or a fourth memory block reuse strategy in which the sum of memory blocks is less than or equal to a third preset threshold.

[0071] Further, in one embodiment of the present application, the allocation module 504 is specifically used to: allocate a memory block to the tth node based on the memory block reuse strategy determined by the N-1th node group and according to the memory blocks of the first two nodes of the tth node in the topological structure of the tth node in the Nth node group, and generate at least one fifth memory block reuse strategy; use the fifth memory block reuse strategy as the memory block reuse strategy; wherein t is greater than or equal to 1 and less than or equal to Q, and Q is the total number of nodes in the Nth node group.

[0072] The memory allocation device 500 in the embodiment of the present application can be a device, or a component, integrated circuit or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a palm computer, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (Ultra-Mobile Personal Computer, UMPC), a netbook or a personal digital assistant (Personal Digital Assistant, PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (Personal Computer, PC), a television (Television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0073] The memory allocation device 500 in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0074] The memory allocation device 500 provided in the embodiment of the present application can realize Figure 1To avoid repetition, the various processes implemented in the method embodiment are not described here.

[0075] Optional, such as Figure 6 As shown, an embodiment of the present application also provides an electronic device 600, including a processor 602, a memory 604, and a program or instruction stored in the memory 604 and executable on the processor 602. When the program or instruction is executed by the processor 602, each process of the above-mentioned memory allocation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0076] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0077] Figure 7 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0078] The electronic device 700 includes but is not limited to components such as a radio frequency unit 702 , a network module 704 , an audio output unit 706 , an input unit 708 , a sensor 710 , a display unit 712 , a user input unit 714 , an interface unit 716 , a memory 718 , and a processor 720 .

[0079] Those skilled in the art will appreciate that the electronic device 700 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 720 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0080] Among them, processor 720 is used to: obtain global information of the neural network; obtain a memory block reuse strategy of the neural network based on the global information; and determine a memory allocation strategy of the neural network based on the memory block reuse strategy.

[0081] In this embodiment, a scheme for implementing a neural network memory reuse strategy is proposed. Specifically, according to the global information of the neural network, a memory block reuse strategy is determined for the nodes of the neural network, so that when the neural network is used, memory is allocated to the nodes of the neural network according to the memory block reuse strategy. In this way, the reuse rate of the electronic device memory can be improved, and the total running memory required for the forward reasoning of the neural network can be reduced, thereby enhancing the complexity of the electronic device neural network, the diversity of the running platform, and the number of parallel neural networks.

[0082] Further, in one embodiment of the present application, the global information includes required memory blocks of the nodes of the neural network; the processor 720 is specifically used to: sort the nodes of the neural network in order of the required memory blocks from large to small; allocate memory blocks to the nodes of the neural network according to the sorting order of the nodes, and generate at least one memory block reuse strategy; use the first target reuse strategy as the memory allocation strategy of the neural network, wherein the first target reuse strategy is a memory block reuse strategy in which the sum of memory blocks is less than or equal to a first preset threshold.

[0083] Furthermore, in one embodiment of the present application, the processor 720 is specifically used to: divide the nodes of the neural network into N node groups according to the sorting order of the nodes; allocate memory blocks to the nodes of each node group and generate at least one memory block reuse strategy; wherein N is a positive integer greater than or equal to 4.

[0084] Further, in one embodiment of the present application, the processor 720 is specifically used to: determine at least one first multiplexed memory block of the i-th node in the first node group, and allocate a first multiplexed memory block to the i-th node, generating at least one first memory block reuse strategy; determine the first non-multiplexed memory block of the i-th node, and allocate a first non-multiplexed memory block to the i-th node, generating at least one second memory block reuse strategy; use the second target reuse strategy as the memory block reuse strategy; wherein, i is greater than or equal to 1 and less than or equal to O, O is the total number of nodes in the first node group, and the second target reuse strategy is the first memory block reuse strategy or the second memory block reuse strategy in which the sum of memory blocks is less than or equal to a second preset threshold.

[0085] Further, in one embodiment of the present application, the processor 720 is specifically used to: determine at least one second multiplexed memory block of the jth node in the Mth node group based on the memory block reuse strategy determined by the M-1th node group, and allocate a second multiplexed memory block to the jth node, generating at least one third memory block reuse strategy; determine the second non-multiplexed memory block of the jth node, and allocate a second non-multiplexed memory block to the i-th node, generating at least one fourth memory block reuse strategy; use the third target reuse strategy as the memory block reuse strategy; wherein, M is greater than 1 and less than N, j is greater than or equal to 1 and less than or equal to P, P is the total number of nodes in the Mth node group, and the third target reuse strategy is a third memory block reuse strategy or a fourth memory block reuse strategy in which the sum of memory blocks is less than or equal to a third preset threshold.

[0086] Furthermore, in one embodiment of the present application, the global information also includes topological structure information of the nodes of the neural network; the processor 720 is specifically used to: allocate memory blocks to the tth node based on the memory blocks of the first two nodes of the tth node in the topological structure in the Nth node group based on the memory block reuse strategy determined by the N-1th node group, and generate at least one fifth memory block reuse strategy; use the fifth memory block reuse strategy as the memory block reuse strategy; wherein t is greater than or equal to 1 and less than or equal to Q, and Q is the total number of nodes in the Nth node group.

[0087] It should be understood that in the embodiment of the present application, the radio frequency unit 702 can be used to send and receive information or send and receive signals during a call, specifically, receive downlink data from a base station or send uplink data to a base station. The radio frequency unit 702 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.

[0088] The network module 704 provides users with wireless broadband Internet access, such as helping users to send and receive emails, browse web pages, and access streaming media.

[0089] The audio output unit 706 can convert the audio data received by the RF unit 702 or the network module 704 or stored in the memory 718 into an audio signal and output it as sound. In addition, the audio output unit 706 can also provide audio output related to a specific function performed by the electronic device 700 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 706 includes a speaker, a buzzer, a receiver, etc.

[0090] The input unit 708 is used to receive audio or video signals. The input unit 708 may include a graphics processing unit (GPU) 7082 and a microphone 7084. The graphics processor 7082 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 712, or stored in the memory 718 (or other storage medium), or sent via the radio frequency unit 702 or the network module 704. The microphone 7084 can receive sound and can process the sound into audio data. The processed audio data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 702 in the case of a telephone call mode.

[0091] The electronic device 700 also includes at least one sensor 710, such as a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, a motion sensor, and other sensors.

[0092] The display unit 712 is used to display information input by the user or information provided to the user. The display unit 712 may include a display panel 7122, which may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like.

[0093] The user input unit 714 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the electronic device. Specifically, the user input unit 714 includes a touch panel 7142 and other input devices 7144. The touch panel 7142 is also called a touch screen, which can collect the user's touch operations on or near it. The touch panel 7142 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, and converts it into contact coordinates, and then sends it to the processor 720, receives the command sent by the processor 720 and executes it. Other input devices 7144 may include but are not limited to physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which will not be repeated here.

[0094] Furthermore, the touch panel 7142 may be covered on the display panel 7122. When the touch panel 7142 detects a touch operation on or near it, it is transmitted to the processor 720 to determine the type of the touch event, and then the processor 720 provides a corresponding visual output on the display panel 7122 according to the type of the touch event. The touch panel 7142 and the display panel 7122 may be two independent components or integrated into one component.

[0095] The interface unit 716 is an interface for connecting an external device to the electronic device 700. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 716 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the electronic device 700 or may be used to transmit data between the electronic device 700 and an external device.

[0096] The memory 718 can be used to store software programs and various data. The memory 718 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile terminal (such as audio data, a phone book, etc.), etc. In addition, the memory 718 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0097] The processor 720 performs various functions of the electronic device 700 and processes data by running or executing software programs and / or modules stored in the memory 718 and calling data stored in the memory 718, thereby monitoring the electronic device 700 as a whole. The processor 720 may include one or more processing units; preferably, the processor 720 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications.

[0098] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned memory allocation method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0099] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0100] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned memory allocation method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0101] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0102] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0103] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0104] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A memory allocation method, characterized in that: include: Get the global information of the neural network; Acquire a memory block reuse strategy of the neural network based on the global information; Determining a memory allocation strategy for the neural network based on the memory block reuse strategy; The global information includes required memory blocks of nodes of the neural network; The acquiring of the memory block reuse strategy of the neural network based on the global information includes: Sorting the nodes of the neural network in descending order of the required memory blocks; Allocating memory blocks to nodes of the neural network according to the sorting order of the nodes, and generating at least one memory block reuse strategy; The determining of the memory allocation strategy of the neural network based on the memory block reuse strategy includes: Using a first target reuse strategy as the memory allocation strategy of the neural network, wherein the first target reuse strategy is a memory block reuse strategy in which the sum of memory blocks is less than or equal to a first preset threshold; The allocating memory blocks to the nodes of the neural network according to the sorting order of the nodes and generating at least one memory block reuse strategy comprises: Dividing the nodes of the neural network into N node groups according to the sorting order of the nodes; Allocate a memory block to each node of the node group and generate at least one memory block reuse strategy; Wherein, N is a positive integer greater than or equal to 4; The global information also includes topological structure information of the nodes of the neural network; The allocating a memory block to each node of the node group and generating at least one memory block reuse strategy comprises: Based on the memory block reuse strategy determined by the N-1th node group, a memory block is allocated to the tth node in the Nth node group according to the memory blocks of the first two nodes in the topology structure, and at least one fifth memory block reuse strategy is generated; Using the fifth memory block reuse strategy as the memory block reuse strategy; Wherein, t is greater than or equal to 1 and less than or equal to Q, and Q is the total number of nodes in the Nth node group.

2. The memory allocation method according to claim 1, characterized in that: The allocating a memory block to each node of the node group and generating at least one memory block reuse strategy comprises: Determine at least one first reused memory block of the i-th node in the first node group, allocate the first reused memory block to the i-th node, and generate at least one first memory block reuse strategy; Determine a first non-multiplexed memory block of the i-th node, allocate the first non-multiplexed memory block to the i-th node, and generate at least one second memory block reuse strategy; Using the second target reuse strategy as the memory block reuse strategy; Among them, i is greater than or equal to 1 and less than or equal to O, O is the total number of nodes in the first node group, and the second target reuse strategy is the first memory block reuse strategy or the second memory block reuse strategy in which the sum of memory blocks is less than or equal to a second preset threshold.

3. The memory allocation method according to claim 2, characterized in that: The allocating a memory block to each node of the node group and generating at least one memory block reuse strategy comprises: Based on the memory block reuse strategy determined by the M-1th node group, determine at least one second reused memory block of the jth node in the Mth node group, allocate the second reused memory block to the jth node, and generate at least one third memory block reuse strategy; Determine a second non-multiplexed memory block of the j-th node, allocate the second non-multiplexed memory block to the i-th node, and generate at least one fourth memory block reuse strategy; Using the third target reuse strategy as the memory block reuse strategy; Among them, M is greater than 1 and less than N, j is greater than or equal to 1 and less than or equal to P, P is the total number of nodes in the Mth node group, and the third target reuse strategy is the third memory block reuse strategy or the fourth memory block reuse strategy in which the sum of memory blocks is less than or equal to the third preset threshold.

4. A memory allocation device, characterized in that: include: The acquisition module is used to obtain the global information of the neural network; An allocation module, configured to obtain a memory block reuse strategy of the neural network based on the global information; A determination module, configured to determine a memory allocation strategy of the neural network based on the memory block reuse strategy; The global information includes the required memory blocks of the nodes of the neural network; the memory allocation device includes: A sorting module, used to sort the nodes of the neural network in descending order of the required memory blocks; The allocation module is specifically used to allocate memory blocks to the nodes of the neural network according to the sorting order of the nodes, and generate at least one memory block reuse strategy; The determination module is specifically configured to use a first target reuse strategy as a memory allocation strategy for the neural network, wherein the first target reuse strategy is a memory block reuse strategy in which the sum of memory blocks is less than or equal to a first preset threshold; The memory allocation device further includes: a division module, for dividing the nodes of the neural network into N node groups according to the sorting order of the nodes; The allocation module is specifically used to allocate a memory block to each node of the node group and generate at least one memory block reuse strategy; Wherein, N is a positive integer greater than or equal to 4; The global information also includes topological structure information of the nodes of the neural network; the allocation module is specifically used to: Based on the memory block reuse strategy determined by the N-1th node group, a memory block is allocated to the tth node in the Nth node group according to the memory blocks of the first two nodes in the topology structure, and at least one fifth memory block reuse strategy is generated; Using the fifth memory block reuse strategy as the memory block reuse strategy; Wherein, t is greater than or equal to 1 and less than or equal to Q, and Q is the total number of nodes in the Nth node group.

5. The memory allocation device according to claim 4, characterized in that: The allocation module is specifically used for: Determine at least one first reused memory block of the i-th node in the first node group, allocate the first reused memory block to the i-th node, and generate at least one first memory block reuse strategy; Determine a first non-multiplexed memory block of the i-th node, allocate the first non-multiplexed memory block to the i-th node, and generate at least one second memory block reuse strategy; Using the second target reuse strategy as the memory block reuse strategy; Among them, i is greater than or equal to 1 and less than or equal to O, O is the total number of nodes in the first node group, and the second target reuse strategy is the first memory block reuse strategy or the second memory block reuse strategy in which the sum of memory blocks is less than or equal to a second preset threshold.

6. The memory allocation device according to claim 5, characterized in that: The allocation module is specifically used for: Based on the memory block reuse strategy determined by the M-1th node group, determine at least one second reused memory block of the jth node in the Mth node group, allocate the second reused memory block to the jth node, and generate at least one third memory block reuse strategy; Determine a second non-multiplexed memory block of the j-th node, allocate the second non-multiplexed memory block to the i-th node, and generate at least one fourth memory block reuse strategy; Using the third target reuse strategy as the memory block reuse strategy; Among them, M is greater than 1 and less than N, j is greater than or equal to 1 and less than or equal to P, P is the total number of nodes in the Mth node group, and the third target reuse strategy is the third memory block reuse strategy or the fourth memory block reuse strategy in which the sum of memory blocks is less than or equal to the third preset threshold.

7. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the memory allocation method as claimed in any one of claims 1 to 3.

8. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps of the memory allocation method according to any one of claims 1 to 3 are implemented.

Images