Memory management method, device and system

Abstract

The invention discloses a memory management method, which is applied to the technical field of artificial intelligence, and comprises the following steps: obtaining a computational graph of an artificial intelligence model, the computational graph comprising a plurality of operators and at least one edge, each edge being used for representing an input relationship or an output relationship between two of the plurality of operators; the memory is sequentially allocated for each operator according to the sequence of the requirements of each operator in the plurality of operators for the memory from large to small, the memory address of the first operator is overlapped with the memory address of the second operator, the second operator is not a mutual exclusion operator of the first operator, and the memory of the mutual exclusion operator of the first operator cannot be reused by the first operator. According to the technical scheme, the memory of the non-mutual exclusion operator can be reused, so that the utilization rate of the allocated memory can be improved, and the peak demand of the memory occupied by the computational graph can be reduced.

Description

technical field [0001] The present application relates to the field of computer technology, and in particular, to a method, device and system for memory management. Background technique [0002] With the development of the Internet, artificial intelligence (AI) has developed rapidly, and AI models have been applied to various products and services. Generally, an AI model is constructed by using a deep neural network or a convolutional neural network, and then using training samples to train the constructed model to obtain an AI model that can be reasoned in application scenarios. [0003] The AI ​​model can be converted into a computational graph. A computational graph includes multiple operators and multiple edges. Each edge is associated with two operators. The edge is also used to represent the input between the two associated operators. / output relation. In the AI ​​inference process, the terminal side needs to calculate one by one according to the execution order of e...

AI Technical Summary

Problems solved by technology

If the memory of these operators is not managed, the executing operator cannot know where to schedule the input data, nor can it determine where the output data should be stored in the memory, resulting in data confusion and inability to complete reasoning. Currently, for the calculation graph Each operator of the algorithm allocates a block of memory, resulting in a waste of memory resources

Images