Memory allocation optimization method and device, electronic equipment, medium and program product

Abstract

Embodiments of the invention provide a memory allocation optimization method and apparatus, an electronic device, a medium and a program product. The method comprises the steps of initializing a memory for each time step of a target program; a time step corresponds to an operator, and the number, the size and the survival time period of data blocks are known; traversing time steps; traversing the data blocks of the time steps; when the data block of the time step is traversed, executing a preset memory adjustment action by traversing a preset action set, if the preset memory adjustment action is successfully executed, re-initializing the memory for the subsequent time step, if the current time step is a preset state, updating the time step, recording the currently updated memory state, and calculating the total consumed time of the read-write task; and after the data block traversal of the current time step is completed, selecting the memory state with reduced total time consumption and minimum time consumption to update the current memory state, and obtaining the memory state of each time step of the target program after the traversal of all the time steps is completed. According to the embodiment of the invention, adaptive memory allocation optimization of different programs can be realized.

Description

technical field [0001] The embodiments of the present invention relate to the technical field of computers, and in particular to a memory allocation optimization method, device, electronic equipment, media, and program product. Background technique [0002] Allocating and releasing memory for data is essential in computing processes such as AI network deduction. Since there are multiple levels of memory in the GPU or CPU, and each level of memory has different bandwidth, latency, and access limitations (the higher the limitation, the lower the probability of memory access conflicts), so the memory usage strategy when performing calculations such as AI network deduction Can have a huge impact on computing performance. [0003] In order to improve computing performance, an existing memory usage strategy is through natural data hit, that is, the memory requested by the user is unified in the main memory (lower level, larger capacity). After the CPU or GPU accesses, by leaving...

AI Technical Summary

Problems solved by technology

For example, in calculations such as AI network deduction, the AI ​​network can be very large and of various types. Manually planning the memory layout for each AI network is very inefficient, and the final effect greatly depends on the individual ability of the engineer; It is difficult for the program to allocate memory to cover the variables of various AI networks, and it is difficult to achieve the global optimum or even the local optimum

Images