Tensor management method, electronic device, storage medium and program product

By introducing tensor pools and intelligent tensor data structures in deep learning model training, we ensure that the tensors used in each iteration have the same memory address, solving the performance optimization problem caused by inconsistent memory address, and achieving resource multiplexing and performance acceleration.

CN119718693AActive Publication Date: 2025-03-28SHANGHAI BIREN TECH CO LTD
10 Cites 4 Cited by

Patent Information

Application Number
CN202510239452.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-03-03
Publication Date
2025-03-28
Estimated Expiration
2045-03-03

AI Technical Summary

Technical Problem

During the training of deep learning model, the memory address allocated to the tensor each time is different, which makes it impossible to meet the performance optimization requirements, especially in high-performance communication scenarios.

Method used

By introducing tensor pools and intelligent tensor data structures, based on the attribute information of the tensors required by the current operator, we look up or create corresponding tensor queues from the tensor pool to ensure that the tensors used in each iteration are consistent in the memory address.

Benefits of technology

The reuse of tensor memory resources is realized, reducing the complexity of data transmission and synchronization, and reducing memory access latency, thereby accelerating the performance of model training.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention relates to the technical field of artificial intelligence, and provides a tensor management method, electronic equipment, a storage medium and a program product.The method comprises the steps that based on attribute information of tensors needed by a current operator, whether a corresponding tensor queue exists or not is searched from a tensor pool, and instance objects of all the tensors are stored in the tensor queue; under the condition that the corresponding tensor queue exists in the tensor pool, based on the storage sequence of each tensor in the tensor queue, extracting a target tensor from the tensor queue; and based on the current operator, using the target tensor to participate in model training, and automatically putting the target tensor back to the tensor queue after the use is completed. According to the method, the tensor pool and the tensor queue are introduced, so that the tensor can be taken out from the queue when the tensor is used and can be automatically put back into the queue after the tensor is used, an operator can obtain the tensor of the same memory address in each iteration of model training, and the performance of model training is improved.
Need to check novelty before this filing date? Find Prior Art