A memory management method and device, electronic equipment and storage medium

Abstract

The embodiment of the application provides a memory management method and device, electronic equipment and storage medium, which are applied to the technical field of intelligent algorithms, and include: acquiring the size of each type of memory required by each network model of a plurality of network models to be allocated in a running process; for each type of memory, calculating the size of the shared memory corresponding to the type of memory according to the size of each type of memory required by the plurality of network models to be allocated in the running process; and creating the shared memory corresponding to each type according to the size of the shared memory corresponding to each type of memory calculated. Through the method of the embodiment of the application, the size of a plurality of types of memories required by a plurality of network models to be allocated can be acquired and calculated, and the shared memory can be created, so that the model is trained through the shared memory, and the pressure of the memory in the model training process is reduced.

Description

Technical Field

[0001] This invention relates to the technical field of intelligent algorithms, and in particular to a memory management method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, the applications of artificial intelligence are becoming increasingly widespread. In the field of artificial intelligence, well-trained network models can perform tasks such as facial recognition, data prediction, and object classification, thereby greatly facilitating people's daily lives.

[0003] However, current network models often require large training datasets and tens or even millions of iterations to obtain an ideal neural network model during training. The large amount of data required during training leads to significant memory consumption, placing immense pressure on hardware. Summary of the Invention

[0004] The purpose of this invention is to provide a memory management method, apparatus, electronic device, and storage medium to reduce memory pressure during model training. The specific technical solution is as follows:

[0005] A first aspect of this application provides a memory management method, including:

[0006] For each of the multiple network models with memory to be allocated, obtain the size of each type of memory required by that network model during its operation;

[0007] For each type of memory, the size of the shared memory corresponding to that type of memory is calculated based on the size of each type of memory required by the network model during operation of the multiple memory to be allocated.

[0008] Based on the calculated size of the shared memory for each type of memory, create the shared memory corresponding to each type.

[0009] Optionally, after creating the shared memory corresponding to each type of memory based on the calculated size of the shared memory for each type, the method further includes:

[0010] The network model for the multiple unallocated memory is run using the shared memory corresponding to each type.

[0011] Optionally, for each of the multiple network models to be allocated memory, obtaining the size of each type of memory required by that network model during operation includes:

[0012] For each of the multiple network models to be allocated memory, calculate the size of at least one of the following: model weight memory, inference computation memory, model input memory, model output memory, and computation instruction memory required for the computation process of each network model to be allocated memory. The model weight memory represents the memory required to store the trained model weights, the model input memory represents the memory required to store the external input data of the model, the model output memory represents the memory required to store the output data of the model, and the computation instruction memory represents the memory required to store the computation instructions of the model.

[0013] Optionally, for each type of memory, calculating the size of the shared memory corresponding to that type of memory based on the size of each type of memory required by the network model during operation of the plurality of memory to be allocated includes:

[0014] For the inference computation memory, the maximum value of the memory of that type corresponding to each network model to be allocated memory is selected as the size of the shared memory corresponding to that type of memory.

[0015] Optionally, for each type of memory, calculating the size of the shared memory corresponding to that type of memory based on the size of each type of memory required by the network model during operation of the plurality of memory to be allocated includes:

[0016] For the model input memory and the model output memory, calculate the sum of the model input memory and the model output memory corresponding to each network model whose memory to be allocated, and then calculate the size of the shared memory corresponding to the calculation result.

[0017] Optionally, for each type of memory, calculating the size of the shared memory corresponding to that type of memory based on the size of each type of memory required by the network model during operation of the plurality of memory to be allocated includes:

[0018] Identify the same network model among the multiple network models to be allocated memory;

[0019] For the model weight memory or the computation instruction memory respectively, the size of the model weight memory or the computation instruction memory corresponding to any network model in the same network model is used as the size of the shared memory corresponding to the model weight memory or the computation instruction memory.

[0020] Optionally, for each type of memory, calculating the size of the shared memory corresponding to that type of memory based on the size of each type of memory required by the network model during operation of the plurality of memory to be allocated includes:

[0021] Identify whether the multiple network models to be allocated memory are basic process type network models or multi-path concurrent type network models;

[0022] Based on the identification results and the size of each type of memory required by the network model of the multiple memory to be allocated during operation, the size of the shared memory corresponding to that type of memory is calculated.

[0023] A second aspect of this application provides a memory management device, including:

[0024] The memory acquisition module is used to acquire the size of each type of memory required by each of the multiple network models to be allocated memory during the operation of the network model.

[0025] The shared memory calculation module is used to calculate the size of the shared memory corresponding to each type of memory based on the size of each type of memory required by the network model of the multiple memory to be allocated during operation.

[0026] The shared memory creation module is used to create shared memory for each type of memory based on the calculated size of the shared memory for each type.

[0027] Optionally, the device further includes:

[0028] The model execution module is used to run the multiple network models to be allocated memory through the shared memory corresponding to each type.

[0029] Optionally, the memory acquisition module is specifically used to calculate, for each of the multiple network models to be allocated memory, the size of at least one of the following: model weight memory, inference computation memory, model input memory, model output memory, and computation instruction memory required for the computation process of each network model to be allocated memory. The model weight memory represents the memory required to store the trained model weights, the model input memory represents the memory required to store the external input data of the model, the model output memory represents the memory required to store the output data of the model, and the computation instruction memory represents the memory required to store the computation instructions of the model.

[0030] Optionally, the shared memory computing module is specifically used to select the maximum value of the type of memory corresponding to each network model to be allocated memory as the size of the shared memory corresponding to that type of memory for the inference computing memory.

[0031] Optionally, the shared memory calculation module is specifically used to calculate the sum of the model input memory and the model output memory corresponding to each network model whose memory to be allocated is to the model input memory and the model output memory, and to calculate the size of the shared memory corresponding to the calculation result.

[0032] Optionally, the shared memory computing module includes:

[0033] The model identification submodule is used to identify the same network model among the multiple network models to be allocated memory.

[0034] The shared memory computing submodule is used to take the size of the model weight memory or the computing instruction memory corresponding to any network model in the same network model as the size of the shared memory corresponding to the model weight memory or the computing instruction memory.

[0035] Optionally, the shared memory computing module includes:

[0036] The type identification module is used to identify whether the multiple network models to be allocated memory are basic process type network models or multi-path concurrent type network models;

[0037] The type memory calculation module is used to calculate the size of the shared memory corresponding to the type of memory based on the identification results and the size of each type of memory required by the network model of the multiple memory to be allocated during operation.

[0038] Another aspect of the embodiments of this application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0039] Memory, used to store computer programs;

[0040] When a processor executes a program stored in memory, it implements any of the memory management method steps described above.

[0041] In another aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements any of the memory management method steps described above.

[0042] In another aspect of the embodiments of this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the memory management method steps described above.

[0043] Beneficial effects of the embodiments of the present invention:

[0044] This invention provides a memory management method, apparatus, electronic device, and storage medium, comprising: for each network model in a plurality of network models to be allocated memory, obtaining the size of each type of memory required by the network model during operation; for each type of memory, calculating the size of shared memory corresponding to that type of memory based on the size of each type of memory required by the plurality of network models to be allocated memory during operation; and creating shared memory corresponding to each type of memory based on the calculated size of shared memory corresponding to each type of memory. Through the method of this application embodiment, shared memory can be obtained and calculated and created based on the size of various types of memory required by multiple network models to be allocated memory, thereby reducing memory pressure during model training by using shared memory.

[0045] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0047] Figure 1 A flowchart illustrating a memory management method provided in an embodiment of this application;

[0048] Figure 2 A schematic diagram illustrating the process of obtaining the size of various types of memory provided in an embodiment of this application;

[0049] Figure 3 A flowchart illustrating the calculation of the size of shared memory provided in an embodiment of this application;

[0050] Figure 4 A schematic diagram of a network model concatenation provided in an embodiment of this application;

[0051] Figure 5 A schematic diagram of a multi-path concurrency model provided in an embodiment of this application;

[0052] Figure 6 A schematic diagram of a memory management device provided in an embodiment of this application;

[0053] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of the present invention.

[0055] First, the technical terms that may be used in the embodiments of this application will be explained:

[0056] Artificial Intelligence (AI) is a new technical science that studies, develops, and applies theories, methods, technologies, and application systems to simulate, extend, and expand human intelligence.

[0057] Artificial Neural Networks (ANNs) are mathematical models that mimic the behavioral characteristics of animal neural networks to perform distributed parallel information processing. They achieve information processing by adjusting the connections between a large number of internal nodes, depending on the system's complexity.

[0058] Deep learning: Artificial neural networks with multiple hidden layers are a type of deep learning network structure. Deep learning discovers distributed feature representations of data by combining low-level features to form more abstract high-level representations of attribute categories or features. Deep learning methods are divided into supervised learning and unsupervised learning. The learning models built under different learning frameworks are also different. For example, Convolutional Neural Networks (CNNs) are a type of deep supervised learning machine learning model, while Deep BeliefNets (DBNs) are a type of unsupervised learning neural network model.

[0059] Deep learning training: The deep learning training process involves feeding a set of training datasets into a neural network, adjusting the connection weights of each layer of the network based on the difference between the actual output and the expected output, and obtaining an ideal neural network model through a large-scale training dataset and tens or even millions of iterations.

[0060] Deep learning inference, also known as deduction, involves a well-trained neural network that can easily perform a wide variety of tasks in the digital world based on its learning. These tasks include image recognition, speech recognition, blood disease detection, and recommending shoes that match someone's style. Through proper training, a well-trained neural network can quickly and efficiently "infer" various conclusions from new data.

[0061] A first aspect of this application provides a memory management method, including:

[0062] For each of the multiple network models with memory to be allocated, obtain the size of each type of memory required by that network model during its operation;

[0063] For each type of memory, calculate the size of the shared memory corresponding to that type of memory based on the size of each type of memory required by the network model with multiple unallocated memory during operation;

[0064] Based on the calculated size of the shared memory for each type of memory, create the shared memory corresponding to each type.

[0065] The method described in this application embodiment can obtain and calculate the size of various types of memory required by multiple network models to be allocated memory, and create shared memory, thereby training the model through shared memory and reducing the memory pressure during the model training process.

[0066] See Figure 1 , Figure 1 A flowchart illustrating a memory management method provided in an embodiment of this application includes:

[0067] Step S11: For each of the multiple network models to be allocated memory, obtain the size of each type of memory required by the network model during operation.

[0068] The network models for which memory is to be allocated in this application embodiment can be of various types, such as feature recognition models, object classification models, feature extraction models, etc. The method in this application embodiment can be applied to the model deployment process. The aforementioned multiple network models for which memory is to be allocated can be pre-trained network models. During model deployment, a certain amount of memory can be allocated to each model for data storage. For example, during the operation of the model for which memory is to be allocated, the model parameters, input data, output data, etc., of the model for which memory is to be allocated can be stored.

[0069] The method described in this application is applied to a server or a smart terminal, and the model can be run through the server or smart terminal. Specifically, the smart terminal can be a computer or the like.

[0070] Step S12: For each type of memory, calculate the size of the shared memory corresponding to that type of memory based on the size of each type of memory required by the network model with multiple unallocated memory during operation.

[0071] In this embodiment, obtaining the size of various types of memory required by the network model during operation can be achieved by obtaining the size of multiple preset types of memory required by the network model during operation. For example, the memory required by the network model during operation can be classified according to preset memory types, and then the size of memory required for each network model for each type of memory can be calculated. For example, the size of memory required to store model parameters and the size of memory required to store data to be computed can be obtained during the operation of the network model.

[0072] Specifically, the size of the shared memory corresponding to each type of memory is calculated based on the size of each type of memory required by the network models to be allocated memory during operation. For each type of memory, the size of the memory required by the network models to be allocated memory during operation can be calculated separately, and then the size of the shared memory corresponding to that type of memory can be calculated.

[0073] Step S13: Based on the calculated size of the shared memory for each type of memory, create the shared memory corresponding to each type.

[0074] Based on the calculated size of the shared memory for each memory type, a shared memory space is created for each type. This can be achieved by allocating a memory region within the hardware memory as the shared memory for that type. After creating the shared memory, the network model can store or retrieve data of the corresponding type within it. For example, regarding model input memory, the memory required by multiple network models to store their output data can be obtained. Then, the sum of the memory required by each network model to store its input data is calculated to obtain the size of the shared memory corresponding to the model input memory. If, for inference computation memory, three network models require 100MB, 185MB, and 112MB of memory for inference computation respectively, the maximum value of 185MB can be selected as the shared memory size for the inference computation of these three network models.

[0075] Optionally, after creating shared memory for each type based on the calculated size of the shared memory for each type, the method further includes: running multiple network models with unallocated memory using the shared memory for each type. Running multiple network models with unallocated memory using the shared memory for each type can mean that during the execution of multiple network models with unallocated memory, data corresponding to each type is stored in the shared memory corresponding to that type for model execution. For example, for inference computation memory, during model execution, the model's input data is stored in the shared memory corresponding to the model input memory.

[0076] As can be seen, the method of this application embodiment can obtain and calculate and create shared memory according to the size of various types of memory required by multiple network models to be allocated memory, thereby training the model through shared memory and reducing the memory pressure during the model training process.

[0077] Optional, see Figure 2 Step S11, for each of the multiple network models to be allocated memory, obtains the size of each type of memory required by that network model during operation, including:

[0078] Step S111: For each of the multiple network models to be allocated memory, calculate the size of at least one of the following: model weight memory, inference computation memory, model input memory, model output memory, and computation instruction memory required for the computation process of each network model to be allocated memory.

[0079] Among them, model weight memory represents the memory required to store the model weights obtained from training, inference computation memory represents the memory required when the model performs computation, model input memory represents the memory required to store the external input data of the model, model output memory represents the memory required to store the output data of the model, and computation instruction memory represents the memory required to store the computation instructions of the model.

[0080] Optionally, for each type of memory, the size of the shared memory corresponding to that type of memory is calculated based on the size of each type of memory required by the network models to be allocated memory during operation. This includes: for inference computing memory, selecting the maximum value of the type of memory corresponding to each network model to be allocated memory as the size of the shared memory corresponding to that type of memory.

[0081] Optionally, for each type of memory, the size of the shared memory corresponding to that type of memory is calculated based on the size of each type of memory required by the network models to be allocated memory during operation. This includes: for model input memory and model output memory, calculating the sum of the model input memory and model output memory corresponding to each network model to be allocated memory, and setting the size of the shared memory corresponding to the calculation result.

[0082] Optionally, for each type of memory, the size of the shared memory corresponding to that type of memory is calculated based on the size of each type of memory required by the network models to be allocated memory during operation. This includes: identifying the same network models among the multiple network models to be allocated memory; and for model weight memory or computation instruction memory respectively, taking the size of the model weight memory or computation instruction memory corresponding to any network model in the same network model as the size of the shared memory corresponding to the model weight memory or computation instruction memory.

[0083] Optional, see Figure 3Step S12 calculates the size of the shared memory corresponding to each type of memory based on the size of each type of memory required by the network model during operation, including:

[0084] Step S121: Identify whether the multiple network models to be allocated memory are basic process type network models or multi-path concurrent type network models;

[0085] Step S122: Based on the identification results and the size of each type of memory required by the network models to be allocated memory during operation, calculate the size of the shared memory corresponding to that type of memory.

[0086] In practical use, for each type of memory, the size of the shared memory corresponding to that type is calculated based on the size of each type of memory required by the network models to be allocated memory during operation. It can also identify whether the network models to be allocated memory are multiple models of a basic process type or multiple models of a multi-path concurrency type. For examples of multiple models of a basic process type, please refer to... Figure 4 Multiple network models are connected in series to complete the process. For example, a typical face recognition process includes face detection, analysis to determine if the face is real, and feature extraction. For examples of multi-path concurrent models, please refer to [link to relevant documentation]. Figure 5 If the same algorithm process needs to be executed simultaneously on multiple paths, it can be done through multithreading or multiprocessing.

[0087] As can be seen, the method of this application embodiment can obtain and calculate and create shared memory according to the size of various types of memory required by multiple network models to be allocated memory, thereby training the model through shared memory and reducing the memory pressure during the model training process.

[0088] A second aspect of this application provides a memory management device, see [link to previous document]. Figure 6 ,include:

[0089] The memory acquisition module 601 is used to acquire the size of each type of memory required by each of the multiple network models to be allocated memory during the operation of the network model.

[0090] The shared memory calculation module 602 is used to calculate the size of the shared memory corresponding to each type of memory based on the size of each type of memory required by the network model with multiple memory to be allocated during operation.

[0091] The shared memory creation module 603 is used to create shared memory for each type of memory based on the calculated size of the shared memory for each type of memory.

[0092] Optionally, the above-mentioned device further includes:

[0093] The model running module is used to run multiple network models with allocated memory through shared memory corresponding to each type.

[0094] Optionally, the memory acquisition module 601 is specifically used to calculate the size of at least one of the following: model weight memory, inference computation memory, model input memory, model output memory, and computation instruction memory required for the computation process of each of the multiple network models to be allocated memory. Here, model weight memory represents the memory required to store the trained model weights, inference computation memory represents the memory required when the model performs computation, model input memory represents the memory required to store the model's external input data, model output memory represents the memory required to store the model's output data, and computation instruction memory represents the memory required to store the model's computation instructions.

[0095] Optionally, the shared memory computing module 602 is specifically used to select the maximum value of the type of memory corresponding to each network model to be allocated memory as the size of the shared memory corresponding to that type of memory for inference computing memory.

[0096] Optionally, the shared memory calculation module 602 is specifically used to calculate the sum of the model input memory and model output memory corresponding to each network model to be allocated memory for the model input memory and model output memory, and to calculate the size of the shared memory corresponding to the calculation result.

[0097] Optionally, the shared-memory computing module 602 includes:

[0098] The model identification submodule is used to identify the same network model among multiple network models to be allocated memory.

[0099] The shared memory computation submodule is used to take the size of the model weight memory or computation instruction memory of any network model in the same network model as the size of the shared memory corresponding to the model weight memory or computation instruction memory.

[0100] Optionally, the shared-memory computing module 602 includes:

[0101] The type identification module is used to identify whether multiple network models to be allocated memory are basic process type network models or multi-concurrency type network models.

[0102] The type memory calculation module is used to calculate the size of the shared memory corresponding to the type of memory based on the identification results and the size of each type of memory required by multiple network models to be allocated memory during operation.

[0103] The apparatus of this application embodiment can acquire and calculate and create shared memory based on the size of various types of memory required by multiple network models to be allocated memory, thereby training the model through shared memory and reducing the memory pressure during the model training process.

[0104] This invention also provides an electronic device, such as... Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703, and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0105] Memory 703 is used to store computer programs;

[0106] When processor 701 executes a program stored in memory 703, it performs the following steps:

[0107] For each of the multiple network models with memory to be allocated, obtain the size of each type of memory required by that network model during its operation;

[0108] For each type of memory, the size of the shared memory corresponding to that type of memory is calculated based on the size of each type of memory required by the network model during operation of the multiple memory to be allocated.

[0109] Based on the calculated size of the shared memory for each type of memory, create the shared memory corresponding to each type.

[0110] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0111] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0112] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0113] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0114] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of any of the above memory management methods.

[0115] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the memory management methods described above.

[0116] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0117] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0118] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, storage media, and computer program products are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0119] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A memory management method characterized by comprising: The method comprises the following steps: For each network model of the plurality of network models to be allocated memory, the size of each type of memory required by the network model during operation is obtained, including: model weight memory required for saving model weights obtained by training, inference calculation memory required during model execution calculation, model input memory required for saving external input data of the model, model output memory required for saving output data of the model, and calculation instruction memory required for saving calculation instructions of the model. For each type of memory, the size of the shared memory corresponding to the type of memory is calculated according to the size of each type of memory required by the plurality of network models to be allocated memory during operation, including: for the inference calculation memory, the maximum value of the type of memory corresponding to each of the network models to be allocated memory is selected as the size of the shared memory corresponding to the type of memory; for the model input memory and the model output memory, the sum of the model input memory and the model output memory corresponding to each of the network models to be allocated memory is calculated, and the calculation result is taken as the size of the corresponding shared memory; the same network models in the plurality of network models to be allocated memory are identified, and for the model weight memory or the calculation instruction memory, the size of the model weight memory or the calculation instruction memory corresponding to any network model in the same network models is taken as the size of the shared memory corresponding to the model weight memory or the calculation instruction memory; or, it is identified whether the plurality of network models to be allocated memory are network models of a basic flow type or network models of a multi-path concurrent type; according to the identification result and the size of each type of memory required by the plurality of network models to be allocated memory during operation, the size of the shared memory corresponding to the type of memory is calculated. According to the size of the shared memory corresponding to each type of memory calculated, the shared memory corresponding to each type is created.

2. The method of claim 1, wherein, After the shared memory corresponding to each type is created according to the size of the shared memory corresponding to each type of memory calculated, the method further comprises the following steps: The plurality of network models to be allocated memory are run through the shared memory corresponding to each type.

3. The method of claim 1, wherein, The method comprises the following steps: For each network model of the plurality of network models to be allocated memory, the size of each type of memory required by the network model during operation is obtained, including:

4. A memory management device, characterized by, For each network model of the plurality of network models to be allocated memory, the size of each type of memory required by the network model during operation is obtained, including: model weight memory required for saving model weights obtained by training, inference calculation memory required during model execution calculation, model input memory required for saving external input data of the model, model output memory required for saving output data of the model, and calculation instruction memory required for saving calculation instructions of the model. The method comprises the following steps: The memory obtaining module is configured to obtain sizes of each type of memory required by each network model in the plurality of network models during running, the each type of memory including a model weight memory required for saving a model weight obtained through training, an inference calculation memory required for model execution calculation, a model input memory required for saving external input data of the model, a model output memory required for saving output data of the model, and a calculation instruction memory required for saving a calculation instruction of the model. The shared memory calculation module is configured to calculate, for each type of memory, a size of shared memory corresponding to the type of memory according to the sizes of each type of memory required by the plurality of network models during running. The shared memory creation module is configured to create the shared memory corresponding to each type according to the size of shared memory corresponding to each type of memory calculated. The shared memory calculation module is specifically configured to, for the inference calculation memory, select a maximum value of the type of memory corresponding to each network model of the plurality of network models as the size of shared memory corresponding to the type of memory; for the model input memory and the model output memory, calculate a sum of the model input memory and the model output memory corresponding to each network model of the plurality of network models, and take the calculation result as the size of corresponding shared memory; identify identical network models in the plurality of network models, and for the model weight memory or the calculation instruction memory, take the size of the model weight memory or the calculation instruction memory corresponding to any network model in the identical network models as the size of shared memory corresponding to the type of memory; or, identify whether the plurality of network models are network models of a basic flow type or network models of a multi-path concurrent type; and according to the identification result and the sizes of each type of memory required by the plurality of network models during running, calculate the size of shared memory corresponding to the type of memory.

5. An electronic device, comprising: The memory is configured to store a computer program. The processor is configured to execute the program stored on the memory to implement the method steps in any one of claims 1-3. The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method steps in any one of claims 1-3.

6. A computer readable storage medium characterized by, ​

Images