Tensor management method, electronic device, storage medium and program product

Abstract

The invention relates to the technical field of artificial intelligence, and provides a tensor management method, electronic equipment, a storage medium and a program product.The method comprises the steps that based on attribute information of tensors needed by a current operator, whether a corresponding tensor queue exists or not is searched from a tensor pool, and instance objects of all the tensors are stored in the tensor queue; under the condition that the corresponding tensor queue exists in the tensor pool, based on the storage sequence of each tensor in the tensor queue, extracting a target tensor from the tensor queue; and based on the current operator, using the target tensor to participate in model training, and automatically putting the target tensor back to the tensor queue after the use is completed. According to the method, the tensor pool and the tensor queue are introduced, so that the tensor can be taken out from the queue when the tensor is used and can be automatically put back into the queue after the tensor is used, an operator can obtain the tensor of the same memory address in each iteration of model training, and the performance of model training is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a tensor management method, electronic equipment, storage medium and program product. Background Art

[0002] In the process of deep learning model training, tensors are the basic units of data, and their efficient management is crucial to improving training performance and resource utilization. Deep learning frameworks (such as PyTorch) provide a resource management pool called Allocator, which is responsible for the memory allocation and recycling of tensors. During the model training process, the allocation of each tensor needs to be requested from Allocator. If there are enough memory resources inside Allocator, it will directly allocate tensors; if there are insufficient resources, Allocator will request memory allocation from the driver, and then allocate tensors.

[0003] During model training, Allocator manages most of the memory resources. This management mechanism can effectively reduce the number of times memory is requested directly from the driver, avoids memory requests blocking calculations during training, and significantly saves time. However, the memory address allocated by Allocator to the tensor each time is not guaranteed to be the same, which has become a major obstacle to performance optimization during model training. In scenarios that require high-performance communication (such as distributed training), if the communication operator cannot obtain the tensor object with the same memory address in each iterative training, the communication efficiency will be reduced and performance acceleration will not be achieved. Summary of the invention

[0004] The present invention provides a tensor management method, electronic device, storage medium and program product, which are used to solve the defect that the memory address allocated to the tensor each time during the model training process is different, resulting in the inability to meet the performance optimization requirements.

[0005] The present invention provides a tensor management method, comprising: Based on the attribute information of the tensor required by the current operator, searching from the tensor pool whether there is a corresponding tensor queue, the attribute information of each tensor in the tensor queue is the same as the attribute information of the tensor required by the current operator, and the tensor queue stores instance objects of each tensor; When a corresponding tensor queue is found in the tensor pool, a target tensor is taken out from the tensor queue based on the storage order of the tensors in the tensor queue; Based on the current operator, the target tensor is used to participate in model training, and after use, the target tensor is automatically put back into the tensor queue.

[0006] According to a tensor management method provided by the present invention, the method of using the target tensor to participate in model training based on the current operator also includes: If no corresponding tensor queue is found in the tensor pool, a target tensor is created based on the attribute information of the tensor required by the current operator.

[0007] According to a tensor management method provided by the present invention, based on the current operator, using the target tensor to participate in model training includes: Create a weak reference object of the target tensor, register the weak reference object in the tensor pool, and set the target tensor as the original tensor attribute of the weak reference object; Based on the current operator, the weak reference object is used to replace the target tensor to participate in model training.

[0008] According to a tensor management method provided by the present invention, after use, the target tensor is automatically put back into the tensor queue, comprising: When it is detected that the life cycle of the weak reference object has ended, the target tensor is automatically put back into the tensor queue, and the weak reference object is recycled.

[0009] According to a tensor management method provided by the present invention, the automatically placing the target tensor back into the tensor queue includes: Based on the original tensor attribute of the weak reference object, obtain the target tensor; Based on the attribute information of the target tensor, searching from the tensor pool whether there is a corresponding tensor queue; When a corresponding tensor queue is found in the tensor pool, the target tensor is added to the tail of the tensor queue; If no corresponding tensor queue is found in the tensor pool, a new tensor queue is created in the tensor pool based on the attribute information of the target tensor, and the target tensor is added to the new tensor queue.

[0010] According to a tensor management method provided by the present invention, the detecting that the life cycle of the weak reference object ends includes: When it is detected that the reference count of the weak reference object returns to zero, it is determined that the life cycle of the weak reference object ends.

[0011] According to a tensor management method provided by the present invention, the automatically placing the target tensor back into the tensor queue also includes: Check whether the weak reference object has been registered in the tensor pool; If the weak reference object has been registered in the tensor pool, the weak reference object is deregistered from the tensor pool.

[0012] According to a tensor management method provided by the present invention, the tensor pool is a data structure for storing and retrieving tensors that need to be reused during model training. Each element in the tensor pool is stored in the form of key-value pairs. The key of any element is determined based on the attribute information of the tensor. The value of any element is a first-in-first-out queue, and the queue stores at least one tensor instance object.

[0013] The present invention also provides a tensor management device, comprising: A tensor search unit, used to search from the tensor pool whether there is a corresponding tensor queue based on the attribute information of the tensor required by the current operator, the attribute information of each tensor in the tensor queue is the same as the attribute information of the tensor required by the current operator, and the tensor queue stores instance objects of each tensor; A tensor fetching unit, configured to fetch a target tensor from the tensor queue based on the storage order of the tensors in the tensor queue when a corresponding tensor queue is found in the tensor pool; The tensor using unit is used to use the target tensor to participate in model training based on the current operator, and automatically put the target tensor back into the tensor queue after the use is completed.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, any of the tensor management methods described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the tensor management method as described above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the tensor management method described above is implemented.

[0017] The tensor management method, electronic device, storage medium and program product provided by the present invention search whether there is a corresponding tensor queue from the tensor pool based on the attribute information of the tensor required by the current operator. When the corresponding tensor queue is found, the target tensor can be directly taken out from the tensor queue according to the storage order of the tensors in the tensor queue without creating a new tensor, which can reduce the overhead of memory allocation and release. After taking out the target tensor, the current operator can use the target tensor to participate in model training, and automatically put the target tensor back into the tensor queue after use. Since the tensors in the tensor queue are arranged in a certain storage order, such as first-in-first-out order, and the tensor queue stores instance objects of each tensor, the memory address of the tensor can be obtained through the instance object. Therefore, for the current operator, the position order of the target tensor taken out in each iteration of model training is consistent in the tensor queue. As a result, the operator using the target tensor will obtain a tensor with the same memory address in each iteration, which not only realizes the reuse of tensor memory resources, but also reduces the complexity of data transmission and synchronization, and reduces memory access latency, thereby accelerating performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flowchart of the tensor management method provided by the present invention; Figure 2 It is a flowchart of a tensor data structure and a resource management method thereof provided by the present invention; Figure 3 is a schematic diagram of the structure of the tensor management device provided by the present invention; Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Performance optimization is crucial in model training, which is directly related to training speed and resource utilization efficiency. In distributed training or model parallelization scenarios, communication operations (such as gradient synchronization, parameter updates, etc.) occupy a large part of the training time. In order to ensure the efficient execution of these communication operations, it is usually necessary to ensure that the tensor objects operated by the communication operator have the same memory address during each iteration of model training. Here, the communication operator is a set of global communication operations responsible for synchronizing data and parameters between different computing nodes to ensure the consistency and accuracy of model training. Among them, global communication operations refer to those operations involving data exchange between multiple computing nodes or processors. These operations can be used to synchronize data, aggregate results, or distribute information. It should be understood that in distributed training or model parallelization scenarios, communication operators need to transmit data between different computing nodes, and these data usually exist in the form of tensors. In order to ensure the efficiency of communication, the communication operator needs to be able to directly access and operate these tensor objects, and in order to access and operate these tensor objects, the communication operator needs to know the location of these tensor objects in memory, that is, their memory addresses.

[0022] In model training, especially in distributed training scenarios, if the communicating operators can obtain tensors with the same memory address in each iteration, the operators can efficiently transfer and synchronize data during training without frequently remapping memory addresses in each iteration, reducing unnecessary memory mapping overhead. Moreover, by reducing the change of memory addresses, communication efficiency can be significantly improved, thereby speeding up the overall training process. In addition, tensors with the same memory address also mean better cache hit rates, which can further reduce memory access latency and improve data access efficiency.

[0023] In the process of model training, tensors are the basic units of data, and their efficient management is crucial to improving training performance and resource utilization. The PyTorch framework provides a resource management pool (i.e., Allocator) that is responsible for the memory allocation and recycling of tensors. In the process of model training, whenever a new tensor needs to be created, PyTorch will issue a memory resource request to Allocator. Allocator first checks whether the memory resource pool it manages has enough resources to meet the request. If there are enough resources, Allocator will directly allocate memory to the new tensor from the resource pool; if the resources are insufficient, Allocator will request more memory resources from the graphics driver and allocate them to the new tensor after obtaining the resources. The allocated memory resources are used to store the data of the tensor. When the life cycle of a tensor ends (for example, the tensor is deleted or out of scope), Allocator will automatically reclaim the memory resources it occupies and put them back into the resource pool for subsequent use.

[0024] Allocator manages most of the memory resources, reducing the number of times memory is requested directly from the graphics driver, thus avoiding the blocking of calculations by requesting graphics memory during model training, which helps maintain the stability and efficiency of training. In addition, Allocator reduces the number of interactions with the graphics driver by pre-managing and recycling memory resources, thus saving a lot of time for driver requests.

[0025] Although PyTorch's Allocator performs well in memory resource management, the memory address it allocates to tensors each time is not guaranteed to be the same, mainly for efficiency and flexibility considerations. When Allocator manages memory resources, it dynamically allocates addresses based on current memory usage and application requirements to maximize memory utilization and reduce fragmentation. However, this feature of Allocator has become a major obstacle to performance optimization. Allocator may return a different address each time it allocates memory to a tensor, which violates the performance optimization requirement for memory address stability. Moreover, due to the instability of memory addresses, communication operators have to frequently remap memory, increasing communication latency and overhead.

[0026] In order to make the communication operator obtain the tensor with the same memory address in each iteration, the related art proposes to manually create a tensor object before the communication operator and perform the communication operator operation based on this object. This is because the manually created tensor object is persistent during its life cycle, and its memory address is fixed after creation. Therefore, in each iteration, as long as the same tensor object is used for communication operations, it can be ensured that the tensor with the same address is obtained in each iteration.

[0027] In PyTorch, memory resource management is usually done automatically by Allocator. When the life cycle of a tensor ends, Allocator will automatically recycle the memory resources it occupies so that other tensors can use them. However, when tensor objects are created manually, this automatic recycling mechanism is no longer applicable. Instead, users are required to manually recycle tensor resources and manually manage tensor resources. In order to manually recycle and manage tensor resources, users need to read the code carefully, find the place where the tensor is used based on professional experience, and manually modify the code for recycling. This operation not only increases the difficulty and complexity of code writing, but also causes the program to crash if the recycling location is wrong or missed, making it difficult to troubleshoot and locate.

[0028] In this regard, the present invention proposes a tensor management method, which can effectively solve the above problems by introducing a tensor pool (i.e., TensorPool) and an intelligent tensor data structure (i.e., SmartTensor). Among them, the tensor pool is used to solve the problem that the memory address allocated by PyTorch's Allocator is different each time, and the tensor data structure is used to solve the problem that tensor resources cannot be automatically recovered and managed. The technical solution provided by the present invention will be introduced in detail below.

[0029] Figure 1 is a flow chart of the tensor management method provided by the present invention, such as Figure 1 As shown, the method includes: Step 110, based on the attribute information of the tensor required by the current operator, search the tensor pool for a corresponding tensor queue, the attribute information of each tensor in the tensor queue is the same as the attribute information of the tensor required by the current operator, and the tensor queue stores instance objects of each tensor.

[0030] Specifically, the current operator refers to the operation or function that is currently being called or executed in model training. In an embodiment of the present invention, the current operator mainly refers to a communication operator, that is, an operation involving data transmission between different devices. The tensor required by the current operator refers to the tensor required when the current operator is executed. Here, the tensor is a commonly used data structure in deep learning models, which can be regarded as a multidimensional array or matrix for storing the input, output or intermediate results of the operator.

[0031] The attribute information of the tensor required by the current operator may include the data type (expressed as type), device type (expressed as device), data shape (expressed as shape), etc. of the tensor, where the data type refers to the data type of the elements in the tensor, such as float32, float64, int64, etc.; the device type refers to the type of hardware device where the tensor is located, such as CPU (Central Processing Unit, central processing unit), GPU (Graphics Processing Unit, graphics processing unit), etc.; the data shape refers to the size of the tensor in each dimension.

[0032] For the current operator, you can first look for the corresponding tensor queue in the tensor pool based on the attribute information of the tensor it needs. Here, the tensor pool is a pre-created data structure for storing, managing, and reclaiming tensors that all communication operators need to reuse during model training. Specifically, in order to find out whether there is a corresponding tensor queue, you can use the attribute information of the tensor required by the current operator as a query condition to search for a matching tensor queue in the tensor pool.

[0033] It is understandable that during the model training process, there may be multiple different communication operators calling tensors with the same attribute information. Therefore, many tensors with the same attribute information may be created during the model training process. In order to manage these tensors, tensor queues are introduced in the tensor pool. Tensor queues refer to the collection of tensors with the same attribute information stored in the tensor pool. All tensors in each tensor queue have the same data type, device type, and data shape, but the instance objects corresponding to each tensor are different, and the data elements stored are also different.

[0034] It should be noted that in actual operation, the memory address of the tensor is usually not directly stored in the tensor queue, but the memory address is encapsulated into the instance object of the tensor, and the instance object is stored in the tensor queue. When a tensor instance object is obtained from the tensor queue, the corresponding memory address can be obtained through the instance object. In other words, obtaining the instance object of the tensor is equivalent to obtaining the memory address of the tensor, so that the corresponding tensor data can be accessed and operated.

[0035] Step 120, when it is found that there is a corresponding tensor queue in the tensor pool, the target tensor is taken out from the tensor queue based on the storage order of the tensors in the tensor queue.

[0036] Specifically, if a tensor queue that matches the attribute information of the tensor required by the current operator is found in the tensor pool, it means that in the previous iteration of model training, the corresponding tensor has been created for the current operator and stored in the tensor queue. In this case, the tensor previously created for the operator can be directly taken out according to the storage order of the tensors in the tensor queue, that is, the target tensor required by the current operator. This allows the current operator to reuse the tensors that have been created before, without having to create new tensors for them, thereby avoiding repeated application and destruction of memory resources. It should be understood that the current operator reuses the tensors that have been created before, which means that the data structure and memory resources of the tensors are reused, thereby reducing the overhead of memory allocation and release.

[0037] The storage order of tensors in a tensor queue refers to the arrangement of tensors in the queue. In a tensor queue, each tensor is arranged in the order in which they are added to the queue. In other words, the tensors in a tensor queue can be arranged in an order-preserving manner to ensure that the communication operator can obtain tensors with the same memory address in each iteration of model training. For example, a tensor queue can be a first-in-first-out queue, which means that the first tensor in the queue (i.e., the head of the queue) is the first to be added, and the last tensor (i.e., the tail of the queue) is the most recently added.

[0038] Specifically, for each communication operator, when the corresponding tensor queue is matched from the tensor pool according to the attribute information of the tensor required by the operator, the target tensor of the operator can be taken out from the head of the tensor queue according to the storage order of the tensors in the queue (such as first-in-first-out order), and after the operator uses the target tensor, it is automatically put back to the tail of the queue so that the operator can be reused in the next iteration. It should be understood that since the tensor pool is searched based on the attribute information of the tensor, in multiple iterations of model training, as long as the tensor attribute information required by the current operator remains unchanged, it will always search for tensors from the same tensor queue. The tensors in the tensor queue are arranged in a certain storage order (such as first-in-first-out order). In each iteration, when the operator takes out the target tensor from the tensor queue, it is based on this stable storage order. Due to the stability of the tensor queue and storage order, the position of the target tensor taken out by the operator from the tensor queue in each iteration is consistent, which means that in multiple iterations, the operator will reuse the tensor at the same position in the queue, that is, for a certain operator, it will reuse the same tensor in each iteration. Since the instance object of the tensor obtained by the operator in each iteration remains unchanged, it can be ensured that the operator can obtain the tensor with the same memory address in each iteration, thereby accelerating performance.

[0039] It can be understood that the target tensor refers to the tensor taken from the tensor queue for the current operator operation, which is determined based on the current operator's requirements and the attribute information of the tensor in the tensor queue. In each iteration, the target tensor is taken from the queue for model training and is put back into the queue after use for subsequent use.

[0040] Step 130: Based on the current operator, use the target tensor to participate in model training, and automatically put the target tensor back into the tensor queue after use.

[0041] Specifically, for the current operator, when the target tensor is taken out and passed to the current operator, the operator will process the tensor according to its specific operation logic. For example, the current operator can access the memory address of the target tensor, read the tensor data from it, and perform operations such as calculation, transformation, or transmission on the tensor data. During the model training process, these operations will be continuously iterated to gradually optimize the parameters and performance of the model. When the target tensor is used, it will be automatically placed back to the end of the tensor queue to ensure the recycling and orderly management of the tensor.

[0042] In addition, if a tensor queue matching the attribute information of the tensor required by the current operator is not found in the tensor pool, it indicates that the current may be the first iteration of model training, the tensor required by the current operator is used for the first time, and the corresponding tensor has not been created for the current operator before. In this case, a target tensor can be created for the current operator based on the attribute information of the tensor required by the current operator, so that the current operator can use the target tensor to participate in model training. Here, creating a target tensor means allocating corresponding memory resources to the tensor based on the attribute information of the tensor required by the current operator (such as data type, device type, and data shape, etc.). Subsequently, after the target tensor is used, the target tensor can be placed in the tensor pool. Here, when the target tensor is placed in the tensor pool, a new tensor queue can be created in the tensor pool based on the attribute information of the target tensor, and the target tensor can be added to the new tensor queue for subsequent use.

[0043] The method provided by the embodiment of the present invention searches for the existence of a corresponding tensor queue from the tensor pool based on the attribute information of the tensor required by the current operator. When the corresponding tensor queue is found, the target tensor can be directly taken out from the tensor queue according to the storage order of each tensor in the tensor queue without creating a new tensor, which can reduce the overhead of memory allocation and release. After taking out the target tensor, the current operator can use the target tensor to participate in model training, and automatically put the target tensor back into the tensor queue after use. Since the tensors in the tensor queue are arranged in a certain storage order, such as in a first-in-first-out order, and the tensor queue stores the instance objects of each tensor, the memory address of the tensor can be obtained through the instance object. Therefore, for the current operator, the position order of the target tensor taken out in each iteration of the model training is consistent in the tensor queue, so that the operator using the target tensor can obtain a tensor with the same memory address in each iteration, which not only realizes the reuse of tensor memory resources, but also reduces the complexity of data transmission and synchronization, and reduces memory access delay, thereby playing a role in performance acceleration.

[0044] Based on the above embodiment, the tensor pool is a data structure for storing and retrieving tensors that need to be reused during model training. Each element in the tensor pool is stored in the form of key-value pairs. The key of any element is determined based on the attribute information of the tensor. The value of any element is a first-in-first-out queue, and the queue stores at least one tensor instance object.

[0045] Specifically, in order to quickly find out whether there is a tensor queue in the tensor pool that matches the attribute information of the tensor required by the operator, the tensor pool can be implemented using a hash table, that is, by using a specific key-value pair to store and retrieve the tensor queue. Each element in the tensor pool corresponds to a unique key, which is a unique identifier composed of the attribute information of the tensor, that is, the type, device, and shape of the tensor. The value can be a first-in-first-out queue. The queue can store one or more tensor instance objects, and the memory address of the tensor can be obtained through the instance object. It should be understood that the queue structure makes the allocation and recovery of tensors more orderly, and can be implemented as a communication operator to provide tensors with the same memory address in each iteration, thereby improving the performance of model training.

[0046] For each operator, when it is necessary to find the corresponding target tensor, the corresponding key can be first generated according to the attribute information of the tensor required by the operator, and then the key can be found in the hash table. If the key to be found does not exist in the tensor pool, it means that the tensor required by the operator is used for the first time. In this case, a new tensor can be created according to the attribute information of the tensor required by the operator, and after the tensor is used, it can be placed in the tensor pool. If the key to be found exists in the tensor pool, it means that the tensor queue to be found currently exists in the tensor pool. At this time, the required target tensor can be directly taken out from the head of the tensor queue, and after the target tensor is used, it will be automatically put back to the tail of the tensor queue.

[0047] It is understandable that during the entire model training process, the tensors in the tensor pool are dynamically transformed. This is because the currently required tensors are taken out of the queue when used and put back into the queue after use. Since model training is an iterative process, the order of the target tensors obtained by the operator from the tensor queue is consistent in each iteration, so that the operator that calls the target tensor will obtain the tensor with the same memory address, thereby accelerating performance.

[0048] Based on any of the above embodiments, before step 130, the method further includes: If no corresponding tensor queue is found in the tensor pool, a target tensor is created based on the attribute information of the tensor required by the current operator.

[0049] Specifically, according to the attribute information of the tensor required by the current operator, if a matching tensor queue is not found in the tensor pool, it means that the tensor required by the current operator is used for the first time, and the corresponding tensor has not been created for the operator and stored in the tensor pool before. In this case, a target tensor can be created for it based on the attribute information of the tensor required by the current operator. Here, when creating a target tensor, you can call the torch.empty interface provided by the PyTorch framework to create it. Subsequently, the current operator can use the target tensor to participate in model training. After the target tensor is used, it can be placed in the tensor pool for subsequent reuse.

[0050] For example, assuming that the operations of communication operator A, communication operator B, and communication operator C are involved in each iteration of model training, an empty tensor pool can be initialized before model training. In the first iteration of model training, for communication operator A, assuming that the attribute information of the tensor required is type1+device1+shape1, since the tensor pool is empty at this time, a target tensor TensorA can be created for it according to the attribute information of the tensor required by communication operator A, so that communication operator A can use TensorA to participate in model training. Similarly, assuming that the attribute information of the tensor required by communication operator B is also type1+device1+shape1, and the attribute information of the tensor required by communication operator C is type2+device2+shape2, corresponding target tensors TensorB and TensorC can be created for communication operator B and communication operator C respectively, so that they can use their corresponding target tensors to participate in model training.

[0051] When the target tensor TensorA is used, it is automatically placed in the tensor pool. At this time, according to the attribute information of TensorA (i.e. type1+device1+shape1), a tensor queue Q1 can be created in the tensor pool, and TensorA can be added to the tensor queue Q1. When the target tensor TensorB is used, a key can be generated according to its attribute information (i.e. type1+device1+shape1), and the tensor pool can be searched according to the key. At this time, the tensor queue Q1 can be retrieved, so TensorB can be directly added to the tail of the tensor queue Q1. Similarly, when the target tensor TensorC is used, a key can be generated according to the attribute information of TensorC (i.e. type2+device2+shape2), and the tensor pool can be searched according to the key. At this time, the matching tensor queue cannot be retrieved. Therefore, a new tensor queue Q2 can be created in the tensor pool according to the attribute information of TensorC, and TensorC can be added to the tensor queue Q2.

[0052] In the second iteration of model training, when communication operator A is executed, the corresponding key can be generated according to the attribute information of the required tensor (i.e. type1+device1+shape1), and the tensor pool can be retrieved according to the key. At this time, the matching tensor queue Q1 can be retrieved. Therefore, the target tensor TensorA can be directly taken out from the head of tensor queue Q1, and subsequent communication operator A can use TensorA to participate in model training. After TensorA is used, it can be automatically put back to the tail of tensor queue Q1. Similarly, when communication operator B is executed, the tensor queue Q1 can be retrieved from the tensor pool according to the attribute information of the required tensor, and the target tensor TensorB can be taken out from the head of tensor queue Q1 to participate in model training.

[0053] It should be understood that for the tensors required by each communication operator, it is usually only necessary to create the tensor when the tensor is used for the first time. After one iteration of model training is completed, the tensor pool already contains the tensors required by all communication operators. Therefore, in subsequent iterations, there is no need to recreate new tensors. Each communication operator can take out the corresponding target tensor from the tensor pool for reuse, thereby achieving the purpose of performance acceleration.

[0054] Based on any of the above embodiments, in step 130, based on the current operator, using the target tensor to participate in model training includes: Step 131, create a weak reference object of the target tensor, register the weak reference object in the tensor pool, and set the target tensor as the original tensor attribute of the weak reference object.

[0055] It should be noted that in order to realize the automatic recycling of tensor resources, the embodiment of the present invention introduces an intelligent tensor data structure SmartTensor, which is used to construct a weak reference object of the tensor. This data structure inherits from torch.Tensor, so SmartTensor contains all the properties and methods of torch.Tensor, and its usage is the same as torch.Tensor. Here, torch.Tensor is a tensor data structure defined in the PyTorch framework.

[0056] It is understood that a weak reference is a reference that does not increase the reference count of a tensor. Here, the reference count refers to the number of times an object (such as a tensor) is referenced. In programming languages ​​such as Python, a normal reference (i.e., a strong reference) increases the reference count of an object, thereby preventing the object from being reclaimed by the garbage collector. Weak references do not increase the reference count of an object, and they cannot guarantee the survival of the object. Therefore, when only weak references remain to a tensor, the garbage collector can reclaim the tensor and reuse its memory resources for other purposes. It should be understood that the garbage collector is part of the runtime environment of a programming language, responsible for automatically managing memory, especially automatically reclaiming memory space that is no longer used during program execution.

[0057] Specifically, for the current operator, after obtaining its corresponding target tensor, the corresponding function can be called to create a weak reference object of the target tensor. For example, in the Python programming language, the weakref module provides the function of creating weak references, and the weakref.ref() function can be used to create a weak reference object. For example, assuming that tensor is the target tensor, then smart_tensor = weakref.ref(tensor) will create a weak reference object smart_tensor pointing to tensor. Here, tensor is the tensor object created by torch.Tensor, and smart_tensor is the created SmartTensor object.

[0058] After creating a weak reference object of the target tensor, you can register it with the tensor pool. For example, you can register the id of the weak reference object smart_tensor with the tensor pool so that the tensor pool can manage and track the life cycle of the target tensor. Since weak references do not increase the reference count of the tensor, the tensor pool can be notified when there are no other strong references to the tensor, so that these tensors can be safely recycled.

[0059] Furthermore, since weak references cannot guarantee the survival of tensor objects, in order to prevent the target tensor from being recycled by PyTorch's Allocator due to scope detachment, the target tensor can be set to the original tensor attribute of the weak reference object, which means saving a strong reference to the target tensor in the weak reference object. For example, the target tensor tensor can be set to an original tensor attribute original_tensor of the weak reference object smart_tensor, that is, smart_tensor.original_tensor = tensor. It should be understood that since weak references do not increase the reference count of the target tensor, the survival of the tensor object cannot be guaranteed. If the target tensor has no other strong references, it may be recycled by the garbage collector. By setting the target tensor as an attribute of the weak reference object, it can be ensured that the target tensor has at least one strong reference, thereby preventing it from being accidentally recycled.

[0060] In an embodiment of the present invention, a weak reference object of the target tensor is created, the weak reference object is registered in the tensor pool, and the target tensor is set to the original tensor attribute of the weak reference object. The weak reference object thus constructed not only includes all the functions of the target tensor, but also isolates the Allocator from managing the target tensor, so that it will not be recycled by the Allocator, but will be recycled by the tensor pool after the life cycle ends, so as to create a new weak reference when it is reused next time.

[0061] Step 132: Based on the current operator, use the weak reference object to replace the target tensor to participate in model training.

[0062] Specifically, since the weak reference object is only a reference to the target tensor, it contains all the functions of the target tensor. Therefore, the current operator can directly use the weak reference object to replace the target tensor to participate in model training. In other words, the weak reference object can directly participate in model training in the form of a target tensor. For example, the weak reference object can be used to participate in the calculation of related operators, or data can be transmitted and exchanged between communication operators. The embodiment of the present invention does not specifically limit this.

[0063] Based on any of the above embodiments, in step 130, automatically putting the target tensor back into the tensor queue after use includes: Step 133, when it is detected that the life cycle of the weak reference object has ended, the target tensor is automatically put back into the tensor queue, and the weak reference object is recycled.

[0064] Specifically, when the end of the life cycle of a weak reference object is detected, it means that the weak reference object is no longer held by any strong reference, and the garbage collector can freely recycle the object. Here, detecting whether the life cycle of a weak reference object has ended usually depends on the behavior of the garbage collector. For example, in the Python programming language, the garbage collector automatically tracks the references of weak reference objects. When it detects that the reference count of a weak reference object has returned to zero, it can be determined that the life cycle of the weak reference object has ended.

[0065] Furthermore, in step 133, the step of automatically putting the target tensor back into the tensor queue also includes: Step 1331, checking whether the weak reference object has been registered in the tensor pool; Step 1332: If the weak reference object has been registered in the tensor pool, the weak reference object is deregistered from the tensor pool.

[0066] Specifically, when the life cycle of a weak reference object ends (for example, when no variable references it anymore), Python's garbage collector triggers the destruction of the weak reference object. Here, destruction refers to the destruction process of the object. Before destroying a weak reference object, it first checks whether the weak reference object has been registered with the tensor pool. If it has been registered, it will be unregistered, otherwise it can be skipped.

[0067] It is understood that a list or hash table for registering weak reference objects can be maintained in the tensor pool to track which objects have been registered. When each weak reference object is created, the id of the weak reference object can be registered in the corresponding list or hash table. When the end of the life cycle of a weak reference object is detected, the list or hash table can be queried to determine whether the weak reference object has been registered in the tensor pool.

[0068] When it is found that the weak reference object has been registered in the tensor pool, the weak reference object can be deregistered from the tensor pool. Here, deregistration means removing the weak reference object from the registration list maintained by the tensor pool. It should be understood that in the tensor pool, each weak reference object will be registered so that the tensor pool can track and manage its life cycle; when the weak reference object is no longer needed or has been used up, it needs to be deregistered. After deregistration, the tensor pool can reclaim the memory resources occupied by the weak reference object so that these resources can be allocated to other required tensors, thereby improving the utilization and performance of memory resources.

[0069] After completing the deregistration of the weak reference object, the corresponding target tensor can be obtained according to the original tensor attribute of the weak reference object, and the target tensor is separated from the calculation graph, and then the target tensor is recycled by the tensor pool. It should be understood that in the model training process based on the PyTorch framework, the tensors are in its entire calculation graph. If a tensor is to be recycled, it must be ensured that the tensor is separated from the calculation graph before it can be recycled, otherwise an exception will be introduced.

[0070] Finally, after completing the recycling of the target tensor, the weak reference object can be recycled. Here, the recycling of weak reference objects is automatically completed by the garbage collector. When the garbage collector detects that an object has no strong references, it marks the object as recyclable and recycles its memory space when appropriate. For weak reference objects, since they do not increase the reference count of the object, the garbage collector can recycle the object immediately when all strong references are removed.

[0071] Based on any of the above embodiments, in step 133, automatically putting the target tensor back into the tensor queue includes: Step 1333, based on the original tensor attribute of the weak reference object, obtain the target tensor; Step 1334: Based on the attribute information of the target tensor, search the tensor pool to see whether there is a corresponding tensor queue.

[0072] Specifically, after completing the deregistration of the weak reference object, the target tensor can be recycled. Since the target tensor has been set as the original tensor attribute of the weak reference object before, the target tensor can be obtained by obtaining the original tensor attribute of the weak reference object, and then the target tensor is automatically put back into the tensor pool for subsequent reuse. Here, when the target tensor is automatically put back into the tensor pool, the corresponding key can be generated according to the attribute information of the target tensor, and the key can be used to search the tensor pool to see if a matching tensor queue can be found.

[0073] Step 1335, when it is found that there is a corresponding tensor queue in the tensor pool, the target tensor is added to the tail of the tensor queue.

[0074] Specifically, if a corresponding tensor queue is found in the tensor pool, it means that a tensor with the same attributes has been added to the queue before. In this case, the target tensor can be automatically added to the tail of the tensor queue, so that in the next iteration, the communication operator calling the same position of the tensor queue will obtain the target tensor with the same memory address, thereby accelerating performance.

[0075] Step 1336, if no corresponding tensor queue is found in the tensor pool, a new tensor queue is created in the tensor pool based on the attribute information of the target tensor, and the target tensor is added to the new tensor queue.

[0076] Specifically, if there is no corresponding tensor queue in the tensor pool, it indicates that the target tensor is used for the first time, so a new tensor queue needs to be created to store the target tensor. Here, a new tensor queue can be created and added to the tensor pool based on the attribute information of the target tensor (i.e., data type, device type, data shape, etc.). Specifically, a unique key (calculated based on the attribute information of the target tensor) can be assigned to the new tensor queue in the tensor pool, and the key can be associated with the new tensor queue.

[0077] After creating a new tensor queue, the target tensor can be automatically added to the tensor queue so that it can be reused later.

[0078] Based on any of the above embodiments, Figure 2 is a flow chart of the tensor data structure and resource management method provided by the present invention, such as Figure 2 As shown, the embodiment of the present invention introduces a new tensor data structure SmartTensor and a tensor pool TensorPool, wherein TensorPool solves the problem that the Allocator of PyTorch allocates different memory addresses each time, and SmartTensor solves the problem that tensor resources cannot be automatically recycled.

[0079] It should be noted that the tensor data structure SmartTensor inherits from torch.Tensor, so SmartTensor contains all the properties and methods of torch.Tensor, and its usage is the same as torch.Tensor. When creating a SmartTensor, first create a tensor object, then return a weak reference to the tensor object as a SmartTensor object, and use the tensor as an attribute of the SmartTensor object to ensure that the tensor will not be recycled by PyTorch's Allocator. In addition, this tensor object will not be provided for external use, and external calculations use SmartTensor objects. When the SmartTensor object is used, it will be destructed. During the destructor, its corresponding tensor object will be recycled by TensorPool to create a new SmartTensor object for the next reuse. SmartTensor isolates Allocator from the management of tensors and can achieve self-management of tensors.

[0080] TensorPool is a data structure that inherits Dict (i.e. dictionary). It is a container for storing tensors that need to be reused. Its key is composed of the type+device+shape of the tensor (type refers to the data type, device refers to the device type, and shape refers to the data shape), and the value corresponds to a first-in-first-out queue data structure to ensure that the tensors reused in each iteration are ordered.

[0081] The most important thing about TensorPool is to implement the pop and append functions. The pop function refers to finding the corresponding queue according to the key and returning the head tensor, and the append function refers to generating a key according to the input tensor attributes (i.e. type+device+shape), and then appending the tensor to the tail of the queue corresponding to the key. During the entire training process, the tensors in TensorPool are dynamically changed. They are popped out of the queue when used and put back into the queue after use. Because model training is an iterative process, the order of tensors obtained from the tensor pool is consistent in each iteration, which will make the communication operator obtain tensors with the same memory address in each iteration, thereby accelerating performance.

[0082] The process of this method is as follows: Step S1, find the location of the communication operator in the code, and create a tensor by calling the SmartTensor.empty interface before the communication operator.

[0083] Specifically, at the required position before the communication operator, developers can manually call the SmartTensor.empty interface to create a tensor according to the needs of model training. Specifically, when calling the SmartTensor.empty interface to create a tensor, the type, device, and shape of the tensor required by the communication operator are first transmitted to the interface as parameters. The interface generates the corresponding key based on these attribute information of the tensor and queries whether there is a matching tensor queue in the tensor pool. If it exists, you can call the pop() function to take out the target tensor from the head of the matched tensor queue; if it does not exist, you can call the torch.empty interface to create a new tensor as the target tensor.

[0084] It can be understood that by providing the SmartTensor.empty interface, it is possible to manage tensor resources. During the model training process, the communication operator at the same code position can obtain the target tensor from the tensor pool, so that the tensor can be reused and the memory address of the tensor obtained by the same communication operator in each iteration is the same.

[0085] Step S2, for the target tensor obtained in step S1, call the corresponding constructor to create a weak reference object smart_tensor of the target tensor, and register the id of smart_tensor to the tensor pool. The purpose of this step is not to increase the reference count of the original target tensor, and to ensure that the tensor pool manages the life cycle of the target tensor.

[0086] Step S3, sets the target tensor as an attribute of smart_tensor to ensure that the target tensor has a strong reference, thereby preventing the target tensor from being recycled by PyTorch's Allocator.

[0087] In step S4, the weak reference object smart_tensor can participate in model training in the form of a target tensor. For example, smart_tensor can participate in the calculation of the computing operator, and can also perform data transmission and exchange between communication operators.

[0088] Step S5, when the weak reference object smart_tensor is used up, that is, when its life cycle ends (the reference count returns to zero), Python's garbage collector will actively start recycling, and smart_tensor will be destructed. Before destructing, it will first determine whether smart_tensor has been registered in the tensor pool. If it has been registered, smart_tensor will be deregistered, otherwise it will be skipped. After deregistration, the properties of smart_tensor are obtained to obtain the target tensor, and the target tensor is separated from the calculation graph and recycled by the tensor pool. Here, when the tensor pool recycles the target tensor, it can automatically put the target tensor into the tensor pool through the append function of the tensor pool. Finally, the related resources of the weak reference object smart_tensor are automatically recycled by Python's garbage collector.

[0089] It is understandable that SmartTensor provides a tensor constructor, that is, when creating a tensor, it can be obtained from the tensor pool. When there is no matching tensor, the torch.empty interface is called to create it, and then a weak reference object of the tensor is returned. If the reference does not increase the reference count of the original tensor, the function of the original tensor can be used, and the life cycle of the tensor can be rewritten. When the weak reference object is destroyed, the tensor will be automatically put back to the tensor pool for subsequent reuse.

[0090] The method provided in the embodiment of the present invention, by introducing SmartTensor, isolates the life cycle management of tensors by Allocator, and realizes custom management of the life cycle of tensors. By introducing the tensor pool, the function of providing tensors with the same memory address to the communication operator each time is realized, which improves the performance of model training. At the same time, the automatic recycling of weak reference objects and tensor resources is realized without manual recycling.

[0091] The tensor management device provided by the present invention is described below. The tensor management device described below and the tensor management method described above can be referenced to each other.

[0092] Based on any of the above embodiments, Figure 3 is a schematic diagram of the structure of the tensor management device provided by the present invention, such as Figure 3 As shown, the device comprises: A tensor search unit 310 is used to search whether there is a corresponding tensor queue in the tensor pool based on the attribute information of the tensor required by the current operator, the attribute information of each tensor in the tensor queue is the same as the attribute information of the tensor required by the current operator, and the tensor queue stores instance objects of each tensor; A tensor fetching unit 320 is configured to fetch a target tensor from the tensor queue based on the storage order of the tensors in the tensor queue when a corresponding tensor queue is found in the tensor pool; The tensor using unit 330 is used to use the target tensor to participate in model training based on the current operator, and automatically put the target tensor back into the tensor queue after use.

[0093] The device provided by the embodiment of the present invention searches for the existence of a corresponding tensor queue from the tensor pool based on the attribute information of the tensor required by the current operator. When the corresponding tensor queue is found, the target tensor can be directly taken out from the tensor queue according to the storage order of each tensor in the tensor queue, without creating a new tensor, so that the overhead of memory allocation and release can be reduced. After taking out the target tensor, the current operator can use the target tensor to participate in model training, and automatically put the target tensor back into the tensor queue after use. Since the tensors in the tensor queue are arranged in a certain storage order, such as in a first-in-first-out order, and the tensor queue stores the instance objects of each tensor, the memory address of the tensor can be obtained through the instance object. Therefore, for the current operator, the position order of the target tensor taken out in each iteration of the model training is consistent in the tensor queue, so that the operator using the target tensor can obtain the tensor with the same memory address in each iteration, which not only realizes the reuse of tensor memory resources, but also reduces the complexity of data transmission and synchronization, and reduces memory access delay, thereby playing a role in performance acceleration.

[0094] Based on any of the above embodiments, the device further includes a tensor creation unit, and the tensor creation unit is used to: If no corresponding tensor queue is found in the tensor pool, a target tensor is created based on the attribute information of the tensor required by the current operator.

[0095] Based on any of the above embodiments, the tensor using unit 330 is specifically used for: Create a weak reference object of the target tensor, register the weak reference object in the tensor pool, and set the target tensor as the original tensor attribute of the weak reference object; Based on the current operator, the weak reference object is used to replace the target tensor to participate in model training.

[0096] Based on any of the above embodiments, the device further includes a tensor recovery unit, and the tensor recovery unit is used to: When it is detected that the life cycle of the weak reference object has ended, the target tensor is automatically put back into the tensor queue, and the weak reference object is recycled.

[0097] Based on any of the above embodiments, the tensor recovery unit is used to: Check whether the weak reference object has been registered in the tensor pool; If the weak reference object has been registered in the tensor pool, the weak reference object is deregistered from the tensor pool.

[0098] Based on any of the above embodiments, the tensor recovery unit is specifically used for: Based on the original tensor attribute of the weak reference object, obtain the target tensor; Based on the attribute information of the target tensor, searching from the tensor pool whether there is a corresponding tensor queue; When a corresponding tensor queue is found in the tensor pool, the target tensor is added to the tail of the tensor queue; If no corresponding tensor queue is found in the tensor pool, a new tensor queue is created in the tensor pool based on the attribute information of the target tensor, and the target tensor is added to the new tensor queue.

[0099] Based on any of the above embodiments, the tensor pool is a data structure for storing and retrieving tensors that need to be reused during model training. Each element in the tensor pool is stored in the form of key-value pairs. The key of any element is determined based on the attribute information of the tensor. The value of any element is a first-in-first-out queue, and the queue stores at least one tensor instance object.

[0100] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the tensor management method, which includes: based on the attribute information of the tensor required by the current operator, searching from the tensor pool whether there is a corresponding tensor queue, the attribute information of each tensor in the tensor queue is the same as the attribute information of the tensor required by the current operator, and the tensor queue stores the instance objects of each tensor; in the case of finding that there is a corresponding tensor queue in the tensor pool, based on the storage order of each tensor in the tensor queue, taking out the target tensor from the tensor queue; based on the current operator, using the target tensor to participate in model training, and automatically putting the target tensor back into the tensor queue after use.

[0101] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0102] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the tensor management method provided by the above methods, which method includes: based on the attribute information of the tensor required by the current operator, searching from the tensor pool whether there is a corresponding tensor queue, the attribute information of each tensor in the tensor queue is the same as the attribute information of the tensor required by the current operator, and the tensor queue stores instance objects of each tensor; when it is found that there is a corresponding tensor queue in the tensor pool, based on the storage order of each tensor in the tensor queue, take out the target tensor from the tensor queue; based on the current operator, use the target tensor to participate in model training, and after use, automatically put the target tensor back to the tensor queue.

[0103] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the tensor management method provided by the above-mentioned methods, the method comprising: based on the attribute information of the tensor required by the current operator, searching from the tensor pool whether there is a corresponding tensor queue, the attribute information of each tensor in the tensor queue is the same as the attribute information of the tensor required by the current operator, and the tensor queue stores instance objects of each tensor; when it is found that there is a corresponding tensor queue in the tensor pool, based on the storage order of each tensor in the tensor queue, taking out a target tensor from the tensor queue; based on the current operator, using the target tensor to participate in model training, and automatically putting the target tensor back into the tensor queue after use.

[0104] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0105] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tensor management method, characterized in that: include: Based on the attribute information of the tensor required by the current operator, searching from the tensor pool whether there is a corresponding tensor queue, the attribute information of each tensor in the tensor queue is the same as the attribute information of the tensor required by the current operator, and the tensor queue stores instance objects of each tensor; When a corresponding tensor queue is found in the tensor pool, a target tensor is taken out from the tensor queue based on the storage order of the tensors in the tensor queue; Based on the current operator, the target tensor is used to participate in model training, and after use, the target tensor is automatically put back into the tensor queue.

2. The tensor management method according to claim 1, characterized in that: The method of using the target tensor to participate in model training based on the current operator also includes: If no corresponding tensor queue is found in the tensor pool, a target tensor is created based on the attribute information of the tensor required by the current operator.

3. The tensor management method according to claim 1, characterized in that: The using the target tensor to participate in model training based on the current operator includes: Create a weak reference object of the target tensor, register the weak reference object in the tensor pool, and set the target tensor as the original tensor attribute of the weak reference object; Based on the current operator, the weak reference object is used to replace the target tensor to participate in model training.

4. The tensor management method according to claim 3, characterized in that: After the use is completed, automatically putting the target tensor back into the tensor queue includes: When it is detected that the life cycle of the weak reference object has ended, the target tensor is automatically put back into the tensor queue, and the weak reference object is recycled.

5. The tensor management method according to claim 4, characterized in that: The automatically placing the target tensor back into the tensor queue also includes: Check whether the weak reference object has been registered in the tensor pool; If the weak reference object has been registered in the tensor pool, the weak reference object is deregistered from the tensor pool.

6. The tensor management method according to claim 4, characterized in that: The automatically placing the target tensor back into the tensor queue includes: Based on the original tensor attribute of the weak reference object, obtain the target tensor; Based on the attribute information of the target tensor, searching from the tensor pool whether there is a corresponding tensor queue; When a corresponding tensor queue is found in the tensor pool, the target tensor is added to the tail of the tensor queue; If no corresponding tensor queue is found in the tensor pool, a new tensor queue is created in the tensor pool based on the attribute information of the target tensor, and the target tensor is added to the new tensor queue.

7. The tensor management method according to any one of claims 1 to 6, characterized in that: The tensor pool is a data structure used to store and recycle tensors that need to be reused during model training. Each element in the tensor pool is stored in the form of key-value pairs. The key of any element is determined based on the attribute information of the tensor. The value of any element is a first-in-first-out queue, and the queue stores at least one tensor instance object.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the tensor management method according to any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the tensor management method according to any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the tensor management method according to any one of claims 1 to 7 is implemented.

Images