Memory allocation method and memory allocation apparatus based on many-core chip

Abstract

The disclosure provides a memory allocation method based on a many-core chip, which comprises: obtaining operator attribute information of at least one neural network operator to be allocated to a target computing core; inputting the operator attribute information of each neural network operator into a pre-obtained memory usage prediction model to predict the maximum memory usage of the at least one neural network operator in the memory corresponding to the target computing core; in the case that the maximum memory usage is less than or equal to the available memory amount of the memory corresponding to the target computing core, determining that the at least one neural network operator is a target allocation operator corresponding to the target computing core; and allocating the target allocation operator to the memory corresponding to the target computing core. The disclosure can reduce the computing power consumption of allocating the memory of the many-core chip for the neural network operator and shorten the time of allocating the memory of the many-core chip for the neural network operator. The disclosure also provides a memory allocation device, an electronic device and a computer readable medium.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a memory allocation method, memory allocation device, electronic device, and computer-readable medium based on many-core chips. Background Technology

[0002] The static computation graph of a deep learning framework can be compiled by a compiler to generate an instruction stream that can run on hardware. During the compilation phase of the static computation graph, after the static computation graph enters the compiler, tasks are allocated. Based on the computational load and memory access requirements of different operators (ops), as well as the synchronization information between operators, different operators are assigned to different computing cores of the many-core chip for execution.

[0003] If the memory size of a computing core is insufficient to support the operators allocated to that core, some of the operators allocated to that core need to be replaced with other operators, and the memory usage of the operators allocated to that core needs to be reassessed until all operators can be allocated to suitable computing cores. When there are many operators, this operator allocation process is time-consuming and computationally expensive. Summary of the Invention

[0004] This disclosure provides a memory allocation method, memory allocation device, electronic device, and computer-readable medium based on many-core chips.

[0005] In a first aspect, this disclosure provides a memory allocation method based on a many-core chip, wherein the many-core chip includes multiple computing cores, each of which is configured with independent memory, and the memory allocation method includes:

[0006] Obtain operator attribute information for at least one neural network operator to be assigned to the target computation kernel;

[0007] The operator attribute information of each neural network operator is input into a pre-acquired memory usage prediction model to predict the maximum memory usage of the at least one neural network operator in the memory corresponding to the target computing core.

[0008] If the maximum memory usage is less than or equal to the available memory corresponding to the target computing core, then the at least one neural network operator is determined to be the target allocation operator corresponding to the target computing core;

[0009] The target allocation operator is allocated to the memory corresponding to the target computing core.

[0010] Optionally, before inputting the operator attribute information of each of the neural network operators into a pre-acquired memory usage prediction model to predict the maximum memory usage of the at least one neural network operator in the memory corresponding to the target computing core, the memory allocation method further includes:

[0011] A memory usage prediction model is trained based on training data corresponding to multiple computing cores to obtain the trained memory usage prediction model.

[0012] The training data corresponding to the computing core includes: operator attribute information of at least one training neural network operator allocated to the computing core, and the maximum memory usage of the at least one training neural network operator in the memory corresponding to the computing core.

[0013] Optionally, before training the memory usage prediction model based on training data corresponding to multiple computing cores to obtain the trained memory usage prediction model, the memory allocation method further includes:

[0014] Perform a preset processing procedure on each of the multiple neural network computation graphs to obtain training data corresponding to multiple computation kernels;

[0015] The preset processing procedure includes:

[0016] The neural network computation graph is compiled to allocate the training neural network operators corresponding to the neural network computation graph to the multiple computation cores included in the many-core chip;

[0017] For each computational core, obtain the training data corresponding to that computational core.

[0018] Optionally, the step of training a memory usage prediction model based on training data corresponding to multiple computing cores to obtain the trained memory usage prediction model includes:

[0019] The training data corresponding to each of the computing kernels is preprocessed, and the preprocessing includes encoding processing;

[0020] The operator attribute information of at least one trained neural network operator corresponding to the preprocessed computing kernel is used as the input of the memory usage prediction model. The maximum memory usage of at least one trained neural network operator corresponding to the preprocessed computing kernel in the memory corresponding to the computing kernel is used as the learning target to train the memory usage prediction model, so as to obtain the trained memory usage prediction model.

[0021] Optionally, the memory usage prediction model includes a recurrent neural network model.

[0022] Optionally, the operator attribute information includes at least one of the following: operator identifier, operator size, operator type, operator memory allocation time, and operator memory release time.

[0023] Optionally, if the maximum memory usage is greater than the available memory corresponding to the target computing core, at least one neural network operator to be allocated to the target computing core is reconfigured;

[0024] For at least one neural network operator to be reconfigured and assigned to the target computing core, return to the step of obtaining operator attribute information of at least one neural network operator to be assigned to the target computing core.

[0025] Secondly, this disclosure provides a memory allocation device based on a many-core chip, the many-core chip including multiple computing cores, each computing core being configured with independent memory, and the memory allocation method including:

[0026] The acquisition module is used to acquire operator attribute information of at least one neural network operator to be assigned to the target computing kernel;

[0027] The prediction module is used to input the operator attribute information of each of the neural network operators into a pre-acquired memory usage prediction model to predict the maximum memory usage of the at least one neural network operator in the memory corresponding to the target computing core.

[0028] The decision module is used to determine the at least one neural network operator as the target allocation operator corresponding to the target computing core when the maximum memory usage is less than or equal to the available memory amount corresponding to the target computing core.

[0029] The allocation module is used to allocate the target allocation operator to the memory corresponding to the target computing core.

[0030] Optionally, the memory allocation device further includes:

[0031] The training module is used to train a memory usage prediction model based on training data corresponding to multiple computing cores, so as to obtain the trained memory usage prediction model.

[0032] The training data corresponding to the computing core includes: operator attribute information of at least one training neural network operator allocated to the computing core, and the maximum memory usage of the at least one training neural network operator in the memory corresponding to the computing core.

[0033] Thirdly, this disclosure provides an electronic device, including:

[0034] One or more processors;

[0035] A memory on which one or more programs are stored;

[0036] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the memory allocation methods provided in this disclosure.

[0037] Fourthly, this disclosure provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the memory allocation methods provided in this disclosure.

[0038] The memory allocation method provided in this disclosure simplifies the process of allocating many-core chip memory for neural network operators, reduces the computational power consumption of allocating many-core chip memory for neural network operators, improves the memory allocation efficiency of neural network operators, and shortens the memory allocation time.

[0039] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0040] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0041] Figure 1 A flowchart illustrating a memory allocation method based on a many-core chip, provided as an embodiment of this disclosure;

[0042] Figure 2 A flowchart illustrating another memory allocation method provided in this disclosure embodiment;

[0043] Figure 3 A flowchart illustrating a specific implementation of step S201 provided in this disclosure embodiment;

[0044] Figure 4 A flowchart illustrating yet another memory allocation method provided in this disclosure embodiment;

[0045] Figure 5 A schematic diagram of a memory allocation device based on a many-core chip provided in an embodiment of this disclosure;

[0046] Figure 6 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0047] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0048] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0049] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0050] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0051] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0052] In related technologies, after the neural network computation graph enters the compiler, the compiler first allocates each neural network operator included in the neural network computation graph to the memory corresponding to each computation core. Then, based on the computational amount and memory access requirements of the neural network operators allocated to the computation core, as well as the synchronization information between the neural network operators, it determines whether the memory corresponding to the computation core can support the allocated neural network operators, and then optimizes the neural network operators allocated on each computation core.

[0053] However, when the number of neural network operators is large, the memory allocation process in related technologies is time-consuming and computationally expensive. To address these challenges, this disclosure provides the following memory allocation method based on many-core chips.

[0054] Figure 1 This flowchart illustrates a memory allocation method based on a many-core chip, as provided in an embodiment of this disclosure. The many-core chip includes multiple computing cores, each configured with independent memory. This memory allocation method is applied to a memory allocation device, which may be part of a compiler used to compile neural network computation graphs. Figure 1 As shown, the memory allocation method includes the following steps S101-S104.

[0055] Step S101: Obtain operator attribute information for at least one neural network operator to be assigned to the target computation kernel.

[0056] The target computational core is any one of the multiple computational cores included in a many-core chip. A neural network operator is the basic computational unit that constitutes a neural network, represented by nodes in the neural network computation graph. Examples of neural network operators include convolution and pooling. Operator attribute information is the characteristic information of a neural network operator, which can influence the process of allocating memory for that operator to the computational core; this operator attribute information includes at least one of the following: operator identifier, operator size, operator type, operator memory allocation time, and operator memory release time.

[0057] Among them, the operator identifier is used to uniquely identify the identity (ID) of the operator; the operator size can represent the size of the tensor corresponding to the operator. An operator includes at least one tensor, and the tensor corresponding to the operator can include, but is not limited to, the input tensor and output tensor corresponding to the operator; the operator type can represent the operation type of the operator, such as addition operation type, multiplication operation type, convolution operation type, etc.; the memory allocation time of the operator refers to the time to allocate the memory corresponding to the operator; the memory release time of the operator refers to the time to release the memory corresponding to the operator.

[0058] In one implementation, the compiler compiles the target neural network computation graph before obtaining operator attribute information for at least one neural network operator to be assigned to the target computation kernel.

[0059] The target neural network computation graph is a neural network computation graph that needs to be compiled into an instruction stream that can run on hardware. The nodes of the target neural network computation graph represent neural network operators.

[0060] During the compilation of the target neural network computation graph, the compiler configures the neural network operators to be allocated to each computation core of the many-core chip. In one embodiment, obtaining the operator attribute information of at least one neural network operator to be allocated to the target computation core includes: determining the operator attribute information of any one of the at least one neural network operators to be allocated to the target computation core based on the target neural network computation graph in which the neural network operator is located.

[0061] In one embodiment, the step of determining the operator attribute information of the neural network operator based on the target neural network computation graph in which the neural network operator is located includes: extracting the operator identifier, operator size, and operator type of the neural network operator from the target neural network computation graph; and determining the memory allocation time and memory release time of the neural network operator based on the processing order of the neural network operator in the target neural network computation graph.

[0062] Step S102: Input the operator attribute information of each neural network operator into the pre-acquired memory usage prediction model to predict the maximum memory usage of the at least one neural network operator in the memory corresponding to the target computing core.

[0063] The memory usage prediction model is used to predict the maximum memory usage of a neural network operator within the memory corresponding to its computational kernel. In this embodiment, the specific method for obtaining the memory usage prediction model in advance is not limited; for example, a memory usage prediction model that meets the prediction conditions can be obtained by training a deep neural network with training samples in advance.

[0064] Step S103: If the maximum memory usage is less than or equal to the available memory of the target computing core, determine that the at least one neural network operator is the target allocation operator corresponding to the target computing core.

[0065] The available memory amount corresponding to the target computing core refers to the amount of memory available for allocating neural network operators when memory allocation is required. If the maximum memory usage is less than or equal to the available memory amount corresponding to the target computing core, it means that the memory corresponding to the target computing core can support the storage and loading of at least one neural network operator, and can normally process the computational tasks corresponding to at least one neural network operator. The target allocation operator corresponding to the target computing core is the neural network operator that can be allocated to the target computing core for execution.

[0066] In one implementation, if the maximum memory usage is greater than the available memory corresponding to the target computing core, at least one neural network operator to be allocated to the target computing core is reconfigured. For the reconfigured at least one neural network operator to be allocated to the target computing core, the step of obtaining operator attribute information of at least one neural network operator to be allocated to the target computing core is returned (step S101 above).

[0067] The step of reconfiguring at least one neural network operator to be assigned to the target computing core includes: replacing some or all of the neural network operators currently to be assigned to the target computing core with a corresponding number of other neural network operators to obtain a reconfigured at least one neural network operator to be assigned to the target computing core. These other neural network operators refer to all neural network operators included in the target computing graph, excluding the neural network operators currently to be assigned to the target computing core.

[0068] In this embodiment of the disclosure, the target computing core is any one of the multiple computing cores included in the many-core chip. For each computing core included in the many-core chip, the above steps S101-S103 are executed to determine the allocation operator corresponding to each computing core. The allocation operator corresponding to the computing core refers to the neural network operator that can be allocated to the computing core for execution. It should be noted that the allocation operators corresponding to all computing cores include all neural network operators included in the target neural network.

[0069] Step S104: Allocate the target allocation operator to the memory corresponding to the target computing core.

[0070] The memory allocation method based on many-core chips provided in this disclosure, compared with the method for allocating neural network operators in related technologies, does not require determining whether the memory corresponding to the computing core can support the allocated neural network operators based on the memory access, computational load and synchronization information of the neural network operators allocated to the computing core after each neural network operator is allocated to the memory corresponding to each computing core. Instead, it can predict the maximum memory usage of the at least one neural network operator to be allocated to the target computing core in the memory corresponding to the target computing core by obtaining the operator attribute information of the neural network operator to be allocated to the target computing core and the memory usage prediction model.

[0071] In this embodiment of the disclosure, a pre-trained memory usage prediction model can be used to predict or evaluate whether the memory of the target computing core meets the memory requirements of the operator to be allocated to the target computing core, by utilizing the operator attribute information of the operator to be allocated to the target computing core. Only if the memory requirements are met can the operator to be allocated be allocated to the memory corresponding to the target computing core. For example, if the maximum memory usage is less than or equal to the available memory of the memory corresponding to the target computing core, the at least one neural network operator is determined to be the target allocation operator corresponding to the target computing core, and the target allocation operator is allocated to the memory corresponding to the target computing core.

[0072] The memory allocation method provided in this disclosure simplifies the process of allocating many-core chip memory to neural network operators, reduces the computational power consumption of allocating many-core chip memory to neural network operators, improves the memory allocation efficiency of neural network operators, and shortens the memory allocation time.

[0073] Figure 2 A flowchart of another memory allocation method provided in an embodiment of this disclosure. In one embodiment, before inputting the operator attribute information of each neural network operator into a pre-acquired memory usage prediction model to predict the maximum memory usage of at least one neural network operator in the memory corresponding to the target computing core, as follows... Figure 2 As shown, the memory allocation method further includes the following step S201.

[0074] Step S201: Train a memory usage prediction model based on the training data corresponding to multiple computing cores to obtain the trained memory usage prediction model.

[0075] The training data corresponding to the computational kernel includes: operator attribute information of at least one training neural network operator allocated to the computational kernel, and the maximum memory usage of at least one training neural network operator in the memory corresponding to the computational kernel. The training neural network operator is the neural network operator included in the neural network computation graph used for training when the memory of the computational kernel is allocated in advance through a preset processing procedure. This preset processing procedure can be the memory allocation process in the aforementioned related technologies. The operator attribute information includes at least one of the following: operator identifier, operator size, operator type, operator memory allocation time, and operator memory release time. In one embodiment, the memory usage prediction model includes a Recurrent Neural Network (RNN) model. A Recurrent Neural Network is a type of recurrent neural network that takes sequence data as input, recursively moves in the direction of sequence evolution, and all nodes (recurrent units) are connected in a chain-like manner.

[0076] In one implementation, if the prediction accuracy of the trained memory usage prediction model does not exceed a preset accuracy, the training data corresponding to the computational kernel is added to train the memory usage prediction model again, so as to obtain a memory usage prediction model with a prediction accuracy not lower than the preset accuracy. The preset accuracy can be set according to the specific implementation scenario; for example, the preset accuracy can be set to 99.9%.

[0077] During training, the model parameters can be updated based on the error between the predicted results (the predicted maximum memory usage) and the actual maximum memory usage, until the model converges, thus obtaining the trained memory usage prediction model.

[0078] Figure 3 A flowchart illustrating a specific implementation of step S201 provided in this disclosure. In one implementation, as... Figure 3 As shown, the steps for training a memory usage prediction model based on training data corresponding to multiple computing cores to obtain the trained memory usage prediction model include the following steps S2011-S2012.

[0079] Step S2011: Preprocess the training data corresponding to each computing core.

[0080] Preprocessing includes encoding, which refers to mapping data of variable length to vectors of fixed length. This encoding is used to standardize the training data so that the encoded data can meet the input and output requirements of the memory allocation model.

[0081] The training data corresponding to the computing core includes: operator attribute information of at least one training neural network operator allocated to the computing core, and the maximum memory usage of at least one training neural network operator in the memory corresponding to the computing core.

[0082] In one embodiment, the preprocessing step for the training data corresponding to each computational core includes: for the training data corresponding to any computational core, encoding each data item in the operator attribute information of each training neural network operator, combining the encoded data items into a vector, and using the maximum memory usage of the encoded training neural network operator in the memory corresponding to the computational core as the labeling result of the vector. The data items in the operator attribute information include at least one of the following: operator identifier, operator size, operator type, operator memory allocation time, and operator memory release time.

[0083] It should be noted that preprocessing may also include other data processing methods, such as data cleaning to remove outlier or duplicate data from the training data. These other data processing methods can be configured according to the specific application scenario requirements, and are not specifically limited in this embodiment.

[0084] Step S2012: Take the operator attribute information of at least one trained neural network operator corresponding to the preprocessed computing kernel as the input of the memory usage prediction model, take the maximum memory usage of at least one trained neural network operator corresponding to the preprocessed computing kernel in the memory corresponding to the computing kernel as the learning target, and train the memory usage prediction model to obtain the trained memory usage prediction model.

[0085] Among them, the trained memory usage prediction model is used to predict the maximum memory usage of the neural network operator in the memory corresponding to the computation kernel.

[0086] Figure 4 A flowchart illustrating yet another memory allocation method provided in this disclosure. In one embodiment, as... Figure 4 As shown, before training a memory usage prediction model based on training data corresponding to multiple computing cores to obtain the trained memory usage prediction model (step S201), the memory allocation method further includes step S200.

[0087] Step S200: Perform a preset processing procedure on each of the multiple neural network computation graphs to obtain training data corresponding to multiple computation kernels.

[0088] Among them, the multiple neural network computation graphs are computation graphs representing different neural network structures. All of the multiple neural network computation graphs are neural network computation graphs used for training, and the neural network operators included in them are called training neural network operators.

[0089] In this embodiment, by performing a preset processing procedure on all multiple neural network computation graphs, training data corresponding to these multiple computation kernels for different neural network structures can be obtained. This results in training data with rich neural network structures and high training data quality. Furthermore, training a memory usage prediction model using this training data enables the trained memory usage prediction model to adapt to different scenarios.

[0090] In one embodiment, the aforementioned preset processing procedure may be a memory allocation process in the related art. This preset processing procedure includes the following steps one and two.

[0091] Step 1: Compile the neural network computation graph to allocate the training neural network operators corresponding to the neural network computation graph to the multiple computation cores included in the many-core chip.

[0092] The steps of compiling the neural network computation graph include: initially allocating each training neural network operator corresponding to the neural network computation graph to the memory corresponding to each computing core; optimizing the allocation of the training neural network operators allocated to each computing core based on the computational load and memory access requirements of the training neural network operators allocated to the computing cores and the synchronization information between the training neural network operators; and finally allocating the training neural network operators corresponding to the neural network computation graph to the multiple computing cores included in the many-core chip.

[0093] Step 2: For each computational core, obtain the training data corresponding to that core.

[0094] The training data corresponding to the computing core includes: operator attribute information of at least one training neural network operator allocated to the computing core, and the maximum memory usage of the at least one training neural network operator in the memory corresponding to the computing core.

[0095] The operator attribute information for training neural network operators includes at least one of the following: operator identifier, operator size, operator type, operator memory allocation time, and operator memory release time.

[0096] It should be noted that the memory allocation and deallocation times of each training neural network operator are inconsistent. Therefore, the memory usage of at least one training neural network operator in the memory corresponding to the computing core changes dynamically with the allocation and deallocation process of the neural network operator. The maximum memory usage mentioned above is the maximum memory usage of at least one training neural network operator in the memory corresponding to the computing core during the operator memory allocation and deallocation process.

[0097] In one implementation, for each computing core, obtaining the training data corresponding to that computing core includes the following first and second steps.

[0098] The first step is to determine the operator attribute information of any one of the at least one neural network operators assigned to the computation core, based on the neural network computation graph in which the neural network operator is located.

[0099] In some embodiments, determining the operator attribute information of the neural network operator based on the neural network computation graph in which the neural network operator is located includes: extracting the operator identifier, operator size, and operator type of the neural network operator from the neural network computation graph; and determining the memory allocation time and memory release time of the neural network operator based on the processing order of the neural network operator in the neural network computation graph.

[0100] The second step is to extract the maximum memory usage of the at least one training neural network operator in the memory corresponding to the computing core from the memory usage data of the computing core.

[0101] The memory usage data of the computing core refers to the memory usage data of the computing core during the storage and loading of the at least one neural network operator, and during the processing of the computing tasks corresponding to the at least one neural network operator.

[0102] Figure 5 This is a schematic diagram of a memory allocation device based on a many-core chip, provided as an embodiment of this disclosure. The many-core chip includes multiple computing cores, each of which is configured with independent memory. Figure 5 As shown, the memory allocation device includes: an acquisition module 51, a prediction module 52, a decision module 53, and an allocation module 54.

[0103] The acquisition module 51 is used to acquire operator attribute information of at least one neural network operator to be assigned to the target computing kernel.

[0104] Here, the neural network operator corresponding to the computation kernel refers to the neural network operator that is expected to be allocated to the computation kernel. The operator attribute information includes at least one of the following: operator identifier, operator size, operator type, operator memory allocation time, and operator memory release time.

[0105] The prediction module 52 is used to input the operator attribute information of each neural network operator into the pre-acquired memory usage prediction model to predict the maximum memory usage of the at least one neural network operator in the memory corresponding to the target computing core.

[0106] Among them, the memory usage prediction model is a model used to predict the maximum amount of memory used by a neural network operator in the memory corresponding to the computation kernel.

[0107] In one implementation, the memory usage prediction model includes a recurrent neural network model.

[0108] The decision module 53 is used to determine that the at least one neural network operator is the target allocation operator corresponding to the target computing core when the maximum memory usage is less than or equal to the available memory of the memory corresponding to the target computing core.

[0109] The allocation module 54 is used to allocate the target allocation operator to the memory corresponding to the target computing core.

[0110] In one embodiment, the memory allocation device further includes a training module and a data collection module.

[0111] This training module is used to train a memory usage prediction model based on training data corresponding to multiple computing cores, so as to obtain the trained memory usage prediction model.

[0112] The training data corresponding to the computing core includes: operator attribute information of at least one training neural network operator allocated to the computing core, and the maximum memory usage of the at least one training neural network operator in the memory corresponding to the computing core.

[0113] In some embodiments, the training module trains a memory usage prediction model based on training data corresponding to multiple computing cores to obtain a trained memory usage prediction model, including: preprocessing the training data corresponding to each computing core, the preprocessing including encoding processing; using the operator attribute information of at least one trained neural network operator corresponding to the preprocessed computing core as the input of the memory usage prediction model, using the maximum memory usage of at least one trained neural network operator corresponding to the preprocessed computing core in the memory corresponding to the computing core as the learning target, and training the memory usage prediction model to obtain a trained memory usage prediction model.

[0114] The model collection module is used to perform a preset processing procedure on each of the multiple neural network computation graphs to obtain training data corresponding to multiple computation kernels.

[0115] The preset processing procedure includes the following steps one and two.

[0116] Step 1: Compile the neural network computation graph to allocate the training neural network operators corresponding to the neural network computation graph to the multiple computation cores included in the many-core chip.

[0117] Step 2: For each computational core, obtain the training data corresponding to that core.

[0118] The training data corresponding to the computing core includes: operator attribute information of at least one training neural network operator allocated to the computing core, and the maximum memory usage of at least one training neural network operator in the memory corresponding to the computing core.

[0119] It should be clarified that this disclosure is not limited to the specific configurations and processes described in the foregoing embodiments and shown in the figures. For the sake of convenience and brevity, detailed descriptions of known methods are omitted here, and the specific working processes of the devices and modules described above can be referred to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0120] This disclosure provides a memory allocation device, comprising: an acquisition module for acquiring operator attribute information of at least one neural network operator to be allocated to a target computing core; a prediction module for inputting the operator attribute information of each neural network operator into a pre-acquired memory usage prediction model to predict the maximum memory usage of at least one neural network operator in the memory corresponding to the target computing core; a decision module for determining at least one neural network operator as the target allocation operator corresponding to the target computing core when the maximum memory usage is less than or equal to the available memory of the memory corresponding to the target computing core; and an allocation module for allocating the target allocation operator to the memory corresponding to the target computing core. This reduces the computational power consumption of allocating many-core chip memory to neural network operators, improves the memory allocation efficiency of neural network operators, and shortens the time for allocating many-core chip memory to neural network operators.

[0121] Figure 6 This is a block diagram illustrating the composition of an electronic device according to an embodiment of this disclosure. Figure 6 As shown, the electronic device includes:

[0122] One or more processors 601;

[0123] The memory 602 stores one or more programs that, when executed by one or more processors, cause the one or more processors to implement any of the memory allocation methods described above.

[0124] One or more I / O interfaces 603 are connected between the processor and the memory and configured to enable information exchange between the processor and the memory.

[0125] The processor 601 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 602 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 603 is connected between the processor 601 and the memory 602, enabling information exchange between the processor 601 and the memory 602, including but not limited to a data bus (Bus).

[0126] In some embodiments, the processor 601, memory 602, and I / O interface 603 are interconnected via a bus, and thus connected to other components of the computing device.

[0127] Furthermore, this disclosure also provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the memory allocation method described above.

[0128] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0129] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A memory allocation method based on many-core chips, wherein, The many-core chip includes multiple computing cores, each computing core is configured with independent memory, and the memory allocation method includes: Obtain operator attribute information for at least one neural network operator to be assigned to the target computation kernel; The operator attribute information of each neural network operator is input into a pre-acquired memory usage prediction model to predict the maximum memory usage of the at least one neural network operator in the memory corresponding to the target computing core. If the maximum memory usage is less than or equal to the available memory corresponding to the target computing core, then the at least one neural network operator is determined to be the target allocation operator corresponding to the target computing core; The target allocation operator is allocated to the memory corresponding to the target computing core; The step of obtaining operator attribute information for at least one neural network operator to be assigned to the target computation kernel includes: For the target neural network computation graph containing any one of the neural network operators to be allocated to the target computation core, extract the operator identifier, operator size, and operator type of the neural network operator from the target neural network computation graph; The memory allocation time and memory release time of the neural network operator are determined based on the processing order of the neural network operator in the target neural network computation graph.

2. The memory allocation method according to claim 1, wherein, Before inputting the operator attribute information of each neural network operator into a pre-acquired memory usage prediction model to predict the maximum memory usage of the at least one neural network operator in the memory corresponding to the target computing core, the memory allocation method further includes: A memory usage prediction model is trained based on training data corresponding to multiple computing cores to obtain the trained memory usage prediction model. The training data corresponding to the computing core includes: operator attribute information of at least one training neural network operator allocated to the computing core, and the maximum memory usage of the at least one training neural network operator in the memory corresponding to the computing core.

3. The memory allocation method according to claim 2, wherein, Before training the memory usage prediction model based on training data corresponding to multiple computing cores to obtain the trained memory usage prediction model, the memory allocation method further includes: Perform a preset processing procedure on each of the multiple neural network computation graphs to obtain training data corresponding to multiple computation kernels; The preset processing procedure includes: The neural network computation graph is compiled to allocate the training neural network operators corresponding to the neural network computation graph to the multiple computing cores included in the many-core chip; For each computational core, obtain the training data corresponding to that computational core.

4. The memory allocation method according to claim 2, wherein, The step of training a memory usage prediction model based on training data corresponding to multiple computing cores to obtain the trained memory usage prediction model includes: The training data corresponding to each of the computing kernels is preprocessed, and the preprocessing includes encoding processing; The operator attribute information of at least one trained neural network operator corresponding to the preprocessed computing kernel is used as the input of the memory usage prediction model. The maximum memory usage of at least one trained neural network operator corresponding to the preprocessed computing kernel in the memory corresponding to the computing kernel is used as the learning target to train the memory usage prediction model, so as to obtain the trained memory usage prediction model.

5. The memory allocation method according to claim 4, wherein, The memory usage prediction model includes a recurrent neural network model.

6. The memory allocation method according to claim 1, wherein, The operator attribute information includes at least one of the following: Operator identifier, operator size, operator type, operator memory allocation time, and operator memory release time.

7. The memory allocation method according to claim 1, wherein, The memory allocation method further includes: If the maximum memory usage exceeds the available memory for the target computing core, at least one neural network operator to be allocated to the target computing core is reconfigured. For at least one neural network operator to be reconfigured and assigned to the target computing core, return to the step of obtaining operator attribute information of at least one neural network operator to be assigned to the target computing core.

8. A memory allocation device based on a many-core chip, wherein, The many-core chip includes multiple computing cores, each computing core is configured with independent memory, and the memory allocation method includes: The acquisition module is used to acquire operator attribute information of at least one neural network operator to be assigned to the target computing kernel; The prediction module is used to input the operator attribute information of each of the neural network operators into a pre-acquired memory usage prediction model to predict the maximum memory usage of the at least one neural network operator in the memory corresponding to the target computing core. The decision module is used to determine the at least one neural network operator as the target allocation operator corresponding to the target computing core when the maximum memory usage is less than or equal to the available memory amount corresponding to the target computing core. The allocation module is used to allocate the target allocation operator to the memory corresponding to the target computing core; The acquisition module is specifically used to: extract the operator identifier, operator size, and operator type of any one of the neural network operators to be allocated to the target computing core from the target neural network computing graph; and determine the memory allocation time and memory release time of the neural network operator according to the processing order of the neural network operator in the target neural network computing graph.

9. An electronic device, comprising: One or more processors; A memory on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the memory allocation method as described in any one of claims 1-7.

10. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the memory allocation method as described in any one of claims 1-7.

Images