Tensor processing method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to the technical field of artificial intelligence, and provides a tensor processing method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: determining the shape of a first physical tensor based on the shape of a logic tensor; under the condition that the first dimension of the first physical tensor exceeds the dimension limit, splitting the first physical tensor into a plurality of second physical tensors with the same size, and determining a tensor distance based on the shapes of the second physical tensors; traversing all the second physical tensors, and executing the following operations in the traversing process: obtaining tensor data corresponding to the currently traversed second physical tensor based on the index and tensor distance of the currently traversed second physical tensor, and calling the operator library interface to process the tensor data to obtain a processing result. According to the method, the operator library interface is cyclically called, so that a plurality of physical tensors are processed, the operator library interface does not need to be modified on a large scale, and the modification workload is greatly reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a tensor processing method, device, electronic device, storage medium and program product. Background Art

[0002] In the wide application scenarios of artificial intelligence (AI), in order to improve computing efficiency and resource utilization, it is often necessary to process a large amount of data in parallel. To this end, the industry has widely adopted batch processing technology, which introduces the Batch dimension in the tensor to achieve simultaneous processing of multiple data samples, thereby significantly improving computing efficiency. However, with the continuous expansion of the scale and complexity of AI models, the size of each dimension of the tensor, especially the Height and Width dimensions that represent the spatial characteristics of the data, has also increased significantly.

[0003] When processing these high-dimensional tensors, due to the hardware driver's restrictions on addressable space, there are upper limits on the memory size allocation of a single tensor and the size of each dimension. When the Height and Width dimensions exceed the upper limit, a common solution is to fold these two dimensions into the Batch dimension to reduce the size of each dimension to meet the hardware limitations. However, folding these dimensions into the Batch dimension will cause the Batch dimension to increase. Once the folded Batch dimension exceeds the upper limit supported by the hardware driver, a single logical tensor cannot be allocated as a physical tensor, that is, the logical tensor cannot be stored in a single physical memory block, but needs to be allocated as multiple physical tensors for storage and processing.

[0004] At present, in order to solve the problem of tensor hyperdimensional folding and process multiple physical tensors, the solution usually adopted is to modify the interface inside the operator library to identify multiple physical tensors as a logical tensor for processing. However, the amount of modification is huge, because the interface of the existing operator library is standardized and unified, and these interfaces are developed for a single physical tensor. In order to adapt to the processing of multiple physical tensors, these interfaces need to be modified accordingly and the interface description needs to be increased, resulting in a huge amount of modification work. Summary of the invention

[0005] The present invention provides a tensor processing method, device, electronic device, storage medium and program product, which are used to solve the defect in the related art that a single logical tensor cannot be efficiently calculated and processed when it is folded into multiple physical tensors for storage.

[0006] The present invention provides a tensor processing method, comprising: Determine, based on the shape of the logical tensor, a shape of a first physical tensor, where the first physical tensor is obtained after dimension folding and physical transformation of the logical tensor; When the first dimension of the first physical tensor exceeds the dimension limit, split the first physical tensor into multiple second physical tensors of the same size, and determine the tensor distance based on the shapes of the second physical tensors, where the tensor distance is the distance between the start addresses of any two adjacent second physical tensors; Traverse all second physical tensors, and perform the following operations during the traversal process: based on the index of the second physical tensor currently traversed and the tensor distance, obtain the tensor data corresponding to the second physical tensor currently traversed, and call the operator library interface to process the tensor data to obtain the processing result.

[0007] According to a tensor processing method provided by the present invention, determining the shape of the first physical tensor based on the shape of the logical tensor includes: Based on the shape of the logical tensor, determining whether a specified dimension of the logical tensor exceeds a dimension limit; When the specified dimension of the logical tensor exceeds the dimension limit, split the specified dimension, fold the split sub-dimensions onto the first dimension of the logical tensor, and physically transform the folded tensor to obtain a first physical tensor; Based on the dimensions and sizes of the dimensions of the first physical tensor, the shape of the first physical tensor is determined.

[0008] A tensor processing method provided by the present invention further includes: When the specified dimension of the logical tensor does not exceed the dimension limit, the tensor data corresponding to the logical tensor is obtained, and the operator library interface is called to process the tensor data to obtain a processing result.

[0009] According to a tensor processing method provided by the present invention, obtaining tensor data corresponding to the second physical tensor currently traversed based on the index of the second physical tensor currently traversed and the tensor distance includes: Determine an address offset based on an index of a second physical tensor currently traversed and the tensor distance; Determine, based on the starting address of the logical tensor and the address offset, the starting address of the second physical tensor currently traversed; Based on the starting address and shape of the second physical tensor currently traversed, tensor data corresponding to the second physical tensor currently traversed is obtained.

[0010] According to a tensor processing method provided by the present invention, the number of the second physical tensors is the same as the size of the first dimension of the logical tensor.

[0011] A tensor processing method provided by the present invention further includes: When the first dimension of the first physical tensor does not exceed the dimension limit, tensor data corresponding to the first physical tensor is obtained, and an operator library interface is called to process the tensor data to obtain a processing result.

[0012] According to a tensor processing method provided by the present invention, the multiple second physical tensors are stored continuously and closely, and the tensor distance is the same as the size of each second physical tensor.

[0013] The present invention also provides a tensor processing device, comprising: A determining unit, configured to determine a shape of a first physical tensor based on a shape of the logical tensor, wherein the first physical tensor is obtained after dimension folding and physical conversion of the logical tensor; a splitting unit, configured to split the first physical tensor into a plurality of second physical tensors of the same size when a first dimension of the first physical tensor exceeds a dimension limit, and determine a tensor distance based on shapes of the second physical tensors, wherein the tensor distance is a distance between start addresses of any two adjacent second physical tensors; A processing unit is used to traverse all second physical tensors and perform the following operations during the traversal process: based on the index of the second physical tensor currently traversed and the tensor distance, obtain the tensor data corresponding to the second physical tensor currently traversed, and call the operator library interface to process the tensor data to obtain a processing result.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the tensor processing methods described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the tensor processing methods described above.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the tensor processing method described above is implemented.

[0017] The tensor processing method, device, electronic device, storage medium and program product provided by the present invention can dynamically determine the shape of the first physical tensor based on the shape of the logical tensor, and when the first dimension of the first physical tensor exceeds the dimension limit, the first physical tensor can be split into multiple second physical tensors of the same size, and can determine the distance between the starting addresses of any two adjacent second physical tensors based on the shape of the second physical tensor, which enables the present invention to adapt to tensor processing requirements of different scales and dimensions. By traversing all second physical tensors, it can ensure that each second physical tensor is properly processed, and according to the index and tensor distance of the second physical tensor currently traversed, the tensor data corresponding to the second physical tensor currently traversed can be obtained, and the operator library interface is called to process the tensor data, which not only realizes that multiple physical tensors can be processed by cyclically calling the operator library interface, but also does not require large-scale modification of the existing operator library interface, greatly reducing the workload of modification and reducing the difficulty of implementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flowchart of the tensor processing method provided by the present invention; Figure 2 is a schematic diagram of a logical tensor and a plurality of second physical tensors provided by the present invention; Figure 3 It is a schematic diagram of the process of calling the library interface provided by the present invention to perform tensor processing; Figure 4 is a structural schematic diagram of a tensor processing device provided by the present invention; Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] In deep learning, models are usually built through multi-layer neural networks, where each layer processes input data and produces output. These data (including input, weights, activation values, etc.) are usually represented as multi-dimensional arrays, i.e. tensors. As the model size continues to expand, for example, more layers, more neurons in each layer, etc., the amount of data processed will also increase, resulting in a significant increase in the size of the various dimensions of the tensor, especially the Height and Width dimensions that describe the spatial characteristics of the data.

[0022] In order to improve computing efficiency and resource utilization, batch processing is usually used to process tensors, that is, the batch dimension is introduced into the tensor to process multiple data samples in parallel. However, due to the limitation of the addressable space by the hardware driver, there is an upper limit on the memory size allocation of a single tensor and the size of each dimension. This is because when allocating memory for each tensor, it is necessary to ensure that the allocated memory address is within the addressable space of the hardware. If the size of the tensor exceeds the maximum memory address range (that is, the addressable space) supported by the hardware driver, then the memory allocation will fail. Similarly, the size of each dimension of the tensor is also limited by the addressable space, because each dimension index requires a memory address to represent it, so the size of the dimension cannot exceed the maximum index value supported by the hardware driver.

[0023] When the Height and Width dimensions exceed the upper limit, a common solution is to fold these two dimensions toward the Batch dimension to reduce the size of each dimension to meet the hardware limit. For example, a three-dimensional tensor can have a shape of [Batch, Height, Width] (abbreviated as [B, H, W]). When H and W exceed the upper limit, they can be folded toward the Batch dimension. The folded tensor shape becomes , subH and subW are the granularity after folding the H and W dimensions, respectively, and num_subH and num_subW are the number of foldings of the H and W dimensions, respectively. Here, granularity is a term used to describe the size of each small block (or sub-block) during the folding process. Where subH<=H, It should be understood that in the folding operation, subH represents the height of each small block after folding, and num_subH represents the number of small blocks folded along the height direction. In order to ensure that the folded tensor can fully represent the information of the original tensor, the total height after folding (i.e. ) must be greater than or equal to the original height H. For example, when subH is exactly divisible by H, num_subH is the result of dividing H by subH. Equal to H, taking H=12 as an example, you can choose subH=3, then num_subH=4, For another example, when subH cannot be divided by H, in order to completely cover the original height H, it is necessary to round up to determine the value of num_subH. At this time, will be greater than H. For example, if H=10, and subH=3 is selected, then num_subH needs to be rounded up to 4. , in which case the last folding block contains some extra padding values ​​(usually zero or other default values) that are not part of the original tensor to ensure that the folded tensor has consistent dimensions. Similarly, subW<=W, .

[0024] The above folding method actually recombines the original Batch dimension with part of the Height and Width dimensions to adapt to hardware limitations. The product of ) exceeds the upper limit supported by the hardware driver (for example, 1024), a single logical tensor cannot be allocated as a physical tensor. This means that even if the Height and Width dimensions are reduced by folding, there is still the problem of not being able to store the entire logical tensor in a single physical memory block.

[0025] At present, in order to realize the processing of multiple physical tensors in order to solve the problem of tensor hyperdimensional folding, the solution usually adopted is to modify the interface of the operator library to identify multiple physical tensors as a logical tensor for processing. Here, the operator library refers to a set of functions containing various mathematical operations and logical operations. These functions or operators are usually used to process data. Especially in deep learning models, the operator library is the basis for model training and reasoning. The functions in the operator library can perform operations such as addition, multiplication, convolution, and activation on data. The interface of the operator library refers to the application interface provided by the operator library, which allows developers to call functions or methods in the operator library through this set of interfaces to implement specific mathematical operations or logical operations.

[0026] The solution to modify the operator library interface requires adding a start pointer inside the operator to distinguish whether multiple continuously stored physical tensors belong to the same logical tensor. However, the amount of modification is huge, because the existing operator library is a standardized unified application program interface, which is developed for a single physical tensor, and there is no situation where an address pointer can point to multiple continuous tensors. Therefore, in order to support the need to process multiple physical tensors, developers need to modify all interfaces of the operator library and add corresponding interface descriptions, which is a huge workload and a waste of manpower.

[0027] In this regard, the present invention provides a tensor processing method, which can support the processing requirements of tensor hyperdimensional scenarios by making full use of the existing operator library and cyclically calling the operator library interface. It not only realizes the processing of multiple physical tensors, but also does not need to modify the operator library interface, greatly reducing the workload, thereby overcoming the above-mentioned defects.

[0028] Figure 1 It is a flowchart of the tensor processing method provided by the present invention, such as Figure 1 As shown, the method includes: Step 110, based on the shape of the logical tensor, determine the shape of the first physical tensor, where the first physical tensor is obtained after dimension folding and physical transformation of the logical tensor.

[0029] It should be noted that logical tensor refers to an abstract concept used to represent data structure in deep learning or numerical computing. It usually does not directly correspond to physical storage, but describes the attributes of data such as dimension, data type and size. Physical tensor refers to the tensor actually stored in memory. Unlike logical tensor, physical tensor directly corresponds to a memory area for storing data. The shape of a tensor refers to the dimension of the tensor and the size of each dimension. For example, a tensor with a shape of (4,1024, 4096) means it is a three-dimensional tensor, where the first dimension size is 4, the second dimension size is 1024, and the third dimension size is 4096.

[0030] Specifically, when some dimensions of the logical tensor exceed the dimension limit, these dimensions can be folded, and the folded tensor can be physically transformed to obtain the first physical tensor. This folding process can be completed through the API (Application Programming Interface) provided by the deep learning framework. For example, the deep learning framework provides an interface function GetReshaped(). Developers call this interface function and pass in the shape of the logical tensor as a parameter. The framework will automatically calculate and perform dimension folding and physical transformation on the logical tensor according to the hardware constraints to return the first physical tensor. It should be understood that physical transformation refers to operations such as memory allocation and data copying on tensors to achieve the conversion of logical tensors into corresponding physical tensors.

[0031] For example, assuming that the shape of the logical tensor is [B, H, W], when the H dimension and W dimension exceed the upper limit, the H dimension can be split into , split the W dimension into , then fold the sub-dimensions num_subH and num_subW obtained by splitting into the Batch dimension, and perform physical transformation on the tensor after dimension folding to obtain the first physical tensor. The shape of the first physical tensor can be expressed as .

[0032] Step 120, when the first dimension of the first physical tensor exceeds the dimension limit, split the first physical tensor into multiple second physical tensors of the same size, and determine the tensor distance based on the shape of the second physical tensors, where the tensor distance is the distance between the starting addresses of any two adjacent second physical tensors.

[0033] Specifically, the first dimension of the first physical tensor usually refers to the batch dimension (i.e., the Batch dimension). In deep learning, the batch dimension is used to represent the size of the data batch, that is, the number of data samples processed at one time. When the first dimension of the first physical tensor exceeds the dimension limit, it indicates that the current hardware environment cannot process so many data samples at one time, which may be due to insufficient memory, computing resource limitations, or specific hardware limitations (such as the maximum batch size of the GPU). In this case, the first physical tensor can be split into multiple second physical tensors of the same size to reduce the size of the batch dimension. Here, the second physical tensor refers to the data representation of the smaller batch obtained after splitting the first physical tensor. Compared with the first physical tensor, the batch dimension size of the second physical tensor is reduced, but the size of other dimensions remains unchanged. The second physical tensor is a data structure actually stored in memory and is used to process data in subsequent calculations.

[0034] Specifically, when splitting the first physical tensor into multiple second physical tensors, the data samples in the original batch can be divided into multiple smaller batches, each of which contains the same number of data samples. In this way, each second physical tensor corresponds to a small batch of data, and its shape is similar to that of the first physical tensor, but the batch dimension is reduced. For example, assuming that the shape of the logical tensor is [B, H, W], the shape of the first physical tensor after folding is , if the first dimension of the first physical tensor If the upper limit is exceeded, the first physical tensor can be split into B second physical tensors for storage. In this case, the shape of each second physical tensor is It should be understood that, although from the user's perspective, this logical tensor is still a single one (because the tensor is identified by a starting pointer and size), from the computer and developer's perspective, it is actually B physical tensors. These B physical tensors (i.e., the second physical tensor) are usually allocated in memory as continuous and dense arrangements to avoid unnecessary memory waste.

[0035] In order to make full use of the existing developed operator library to process multiple physical tensors, it is necessary to first determine the distance between the starting addresses of each second physical tensor after folding, and use this as the address pointer offset. By cyclically calling the operator library interface developed for a single tensor, each physical tensor can be processed. Since multiple second physical tensors have the same size and are stored continuously and tightly in memory, the distance between the starting addresses of any two adjacent second physical tensors (i.e., the tensor distance) is actually the memory size occupied by each second physical tensor. In other words, the starting address of the previous second physical tensor is exactly the same as the starting address of the next second physical tensor. The size of the second physical tensor. Here, the starting address of the physical tensor refers to the starting position of the tensor in the memory. The data of the tensor can be correctly accessed and operated through the starting address of each tensor.

[0036] According to the shape of the second physical tensor, the size of the physical tensor can be calculated, that is, the tensor distance. For example, assuming the shape of the second physical tensor is , then the formula , to calculate the tensor distance stride.

[0037] Step 130, traverse all second physical tensors, and perform the following operations during the traversal process: based on the index of the second physical tensor currently traversed and the tensor distance, obtain the tensor data corresponding to the second physical tensor currently traversed, and call the operator library interface to process the tensor data to obtain the processing result.

[0038] Specifically, after calculating the tensor distance, all the second physical tensors can be looped through to realize the processing of each physical tensor. Here, traversing all the second physical tensors means accessing these tensors one by one in a certain order. Since the second physical tensor is obtained by splitting the first physical tensor, and they are stored continuously and tightly, the traversal process can be regarded as a sequential access to a continuous memory area. It should be understood that the traversal can be implemented using a loop structure, such as a for loop. In the loop, an index variable can be used to track the second physical tensor currently being accessed. Each time the loop iterates, the index variable is incremented until all the second physical tensors are accessed.

[0039] During each loop, the starting address of the second physical tensor currently traversed can be calculated based on the index of the second physical tensor currently traversed, the tensor distance, and the starting address corresponding to the logical tensor, so that according to the starting address, the tensor data corresponding to the physical tensor can be read from the memory for processing. Here, the second physical tensor currently traversed refers to the tensor currently being accessed and processed during the traversal process, and the index of the second physical tensor currently traversed refers to the digital identifier used to identify the tensor currently being accessed during the traversal process. The index usually starts counting from 0 and increases as the traversal proceeds. The index is used to calculate the starting position of the current tensor in memory so that the data of the tensor can be correctly accessed and operated.

[0040] Specifically, in order to obtain the tensor data corresponding to the second physical tensor currently traversed, we first need to know the starting address and shape of the tensor in memory. The starting address can be calculated by the index and tensor distance (starting address = starting address base value + index Tensor distance), where the starting address base value refers to the starting address of the first physical tensor traversed, that is, the starting pointer used to identify the logical tensor. After determining the starting address and shape of the second physical tensor, you can use the API provided by the deep learning framework to access and operate the tensor data. Here, tensor data refers to the specific data of the tensor in memory.

[0041] For the second physical tensor currently traversed, after obtaining the tensor data corresponding to the tensor, you can call the appropriate operator library interface function to process the tensor data according to the calculation task to be performed (such as matrix multiplication, convolution, etc.), and receive the returned processing result. Here, the processing result refers to the result obtained after calculating the tensor data.

[0042] For example, when a tensor is used to perform a matrix multiplication (e.g. ), you can perform matrix multiplication by calling the GEMM interface function provided by the operator library. The specific code example is as follows: Among them, i is the index of the currently traversed physical tensor, B represents the number of physical tensors actually allocated (i.e., the second physical tensor), that is, the size of the logical tensor batch dimension, E and F represent the starting address base value of the input tensor, and D represents the starting address base value of the output tensor. The shape of the input tensor E is , the shape of the input tensor F is , the shape of the output tensor D is .

[0043] The method provided by the embodiment of the present invention can dynamically determine the shape of the first physical tensor based on the shape of the logical tensor, and when the first dimension of the first physical tensor exceeds the dimension limit, the first physical tensor can be split into multiple second physical tensors of the same size, and can determine the distance between the starting addresses of any two adjacent second physical tensors based on the shape of the second physical tensor, which enables the present invention to adapt to tensor processing requirements of different scales and dimensions. By traversing all second physical tensors, it can ensure that each second physical tensor is properly processed, and according to the index and tensor distance of the second physical tensor currently traversed, the tensor data corresponding to the second physical tensor currently traversed can be obtained, and the operator library interface is called to process the tensor data, which not only realizes that multiple physical tensors can be processed by cyclically calling the operator library interface, but also does not require large-scale modification of the existing operator library interface, greatly reducing the workload of modification and the difficulty of implementation.

[0044] Based on any of the above embodiments, the method further includes: When the first dimension of the first physical tensor does not exceed the dimension limit, tensor data corresponding to the first physical tensor is obtained, and the operator library interface is called to process the tensor data to obtain a processing result.

[0045] Specifically, when the first dimension (i.e., batch dimension) of the first physical tensor does not exceed the dimension limit, it indicates that the current hardware environment can support processing so many data samples at one time. Therefore, the logical tensor can still be allocated as a physical tensor and stored in a single physical memory block. In this case, since the tensor is still a single physical tensor and the operator library interface is originally developed for a single physical tensor, there is no need to call the operator library interface in a loop. Instead, the corresponding tensor data can be read directly from the memory according to the starting address of the first physical tensor and the operator library interface can be called for processing.

[0046] Based on the above embodiment, step 110 specifically includes: Step 111, based on the shape of the logical tensor, determining whether a specified dimension of the logical tensor exceeds a dimension limit; Step 112: if the specified dimension of the logical tensor exceeds the dimension limit, split the specified dimension, fold the split sub-dimensions onto the first dimension of the logical tensor, and perform physical conversion on the folded tensor to obtain a first physical tensor; Step 113: determine the shape of the first physical tensor based on the dimensions and sizes of the dimensions of the first physical tensor.

[0047] Specifically, the specified dimension of the logical tensor refers to one or more dimensions that have dimensional restrictions when processing the tensor. For example, when the shape of the logical tensor is [B, H, W], the specified dimensions can be H dimensions and W dimensions. According to the shape of the logical tensor, the size of each dimension can be determined, and according to the dimension size, it can be determined whether the specified dimension exceeds the dimensional restriction. Here, the dimensional restriction refers to the upper limit threshold of each dimension pre-set according to hardware restrictions and software restrictions. By comparing the size of the specified dimension of the logical tensor with the pre-set threshold, if it exceeds the threshold, it is judged to exceed the dimensional restriction.

[0048] When the specified dimensions of the logical tensor (such as the H dimension and the W dimension) exceed the dimension limit, these dimensions can be split for operation on the hardware and software platforms. Specifically, the dimensions that exceed the limit can be split into smaller sub-dimensions, and the split sub-dimensions can be folded onto the batch dimension of the logical tensor, and then the folded tensors can be physically transformed to form new physical tensors.

[0049] For example, assuming that the shape of the logical tensor is [B, H, W], when the H dimension and W dimension exceed the dimension limit, the H dimension can be split into , split the W dimension into , then fold the sub-dimensions num_subH and num_subW obtained by splitting into the Batch dimension, and perform physical transformation on the folded tensors to obtain the first physical tensor. The shape of the first physical tensor can be expressed as .

[0050] Based on any of the above embodiments, the method further includes: Step 114, when the specified dimension of the logical tensor does not exceed the dimension limit, obtain the tensor data corresponding to the logical tensor, and call the operator library interface to process the tensor data to obtain the processing result.

[0051] Specifically, when the specified dimension of the logical tensor does not exceed the dimension limit, the logical tensor can be directly stored as a single physical tensor in a single physical memory block. Since it is a single physical tensor, the corresponding tensor data can be directly obtained from the memory according to the starting pointer of the logical tensor, and the operator library interface can be called to process it to obtain the processing result.

[0052] Based on any of the above embodiments, the number of the second physical tensors is the same as the size of the first dimension of the logical tensor, the multiple second physical tensors are stored continuously and closely, and the tensor distance is the same as the size of each second physical tensor.

[0053] Specifically, when the specified dimension of the logical tensor exceeds the dimension limit, these dimensions need to be split and folded onto the first dimension to obtain the first physical tensor. After dimension folding, if the first dimension of the first physical tensor exceeds the upper limit, the first physical tensor needs to be split. At this time, the simplest way to split is to split the first physical tensor according to the size of the first dimension of the logical tensor, so that the number of the second physical tensors is consistent with the size of the first dimension of the logical tensor.

[0054] If the number of allocated second physical tensors is different from the size of the first dimension of the logical tensor, although it is feasible from a memory perspective, it will make subsequent logical calculations very complicated. This is because deep learning models usually assume that input data is provided in batch form. If the number of physical tensors does not match the batch dimension size of the logical tensor, the model needs to handle these mismatches during calculation, which will make the calculation logic complicated and difficult to maintain.

[0055] In addition, to avoid unnecessary memory waste, when a logical tensor is allocated to multiple physical tensors for storage, these physical tensors are allocated in memory to be arranged continuously and tightly.

[0056] Figure 2 is a schematic diagram of a logical tensor and a plurality of second physical tensors provided by the present invention, such as Figure 2 As shown, the shape of the logical tensor is [B, H, W]. When the H and W dimensions exceed the dimension limit, the H dimension can be split into , split the W dimension into , then fold the sub-dimensions num_subH and num_subW obtained by splitting into the Batch dimension, and perform physical transformation on the folded tensors to obtain the first physical tensor. The shape of the first physical tensor is If the size of the batch dimension of the first physical tensor is If the upper limit is exceeded, the first physical tensor can be allocated as B second physical tensors for storage, and the shape of each second physical tensor is From the user's perspective, the logical tensor is identified by a starting pointer and size and is still a single tensor. However, for computers and developers, it is actually B physical tensors. Figure 2 In the example of B=3, the dividing line between the physical tensors indicates the starting address of each physical tensor. These three physical tensors are arranged continuously and closely in the memory. It should be understood that due to the folding and alignment requirements of each dimension, the actual logic occupied by the physical tensor may increase.

[0057] Based on any of the above embodiments, in step 130, obtaining tensor data corresponding to the currently traversed second physical tensor based on the index of the currently traversed second physical tensor and the tensor distance includes: Step 131, determining an address offset based on the index of the second physical tensor currently traversed and the tensor distance.

[0058] Specifically, the address offset refers to the additional address amount that needs to be calculated in order to locate another specific data or data structure starting from a known base address (or starting address) in the memory. In the embodiment of the present invention, the address offset is used to determine the starting position of each second physical tensor in the memory.

[0059] In this step, the address offset is determined based on the index of the second physical tensor currently traversed and the tensor distance. The tensor distance is the distance between the starting addresses of any two adjacent second physical tensors, which is equal to the size of each second physical tensor. Therefore, for the i-th second physical tensor, its address offset can be calculated by the following formula: Address offset = i Tensor distance, where i refers to the index of the second physical tensor currently traversed (counting starts from 0).

[0060] Step 132: Determine the starting address of the second physical tensor currently traversed based on the starting address of the logical tensor and the address offset.

[0061] Step 133: Based on the starting address and shape of the second physical tensor currently traversed, obtain tensor data corresponding to the second physical tensor currently traversed.

[0062] Specifically, the starting address of the logical tensor is known (it may be assigned when the program starts), and the starting address of the logical tensor is added to the address offset calculated in step 131 to calculate the starting address of the second physical tensor currently traversed.

[0063] After determining the starting address of the second physical tensor currently traversed, starting from the starting address, data is read according to the shape information of the physical tensor to obtain the tensor data corresponding to the physical tensor.

[0064] Based on any of the above embodiments, Figure 3 It is a flowchart of calling the library interface provided by the present invention to perform tensor processing, such as Figure 3 As shown, the method includes: Step S1, according to the shape of the logical tensor (for example, shape[B, H, W]), determine whether the H dimension and the W dimension need to be folded, that is, determine whether the H dimension and the W dimension exceed the upper limit; Step S2: If both the H dimension and the W dimension do not exceed the upper limit, the standard interface of the operator library can be directly called to process the data of the logical tensor; Step S3: If any dimension of the H dimension or the W dimension exceeds the upper limit, the dimension that exceeds the limit can be split, and the sub-dimensions obtained by the split can be folded into the Batch dimension. For example, if both the H dimension and the W dimension exceed the upper limit, the H dimension can be split into , split the W dimension into , then fold the sub-dimensions num_subH and num_subW obtained by splitting into the Batch dimension, and perform physical transformation on the folded tensor to obtain a physical tensor with the shape of ; Step S4, determining whether the batch dimension of the physical tensor obtained in step S3 exceeds the dimension limit. If it does not exceed the dimension limit, the standard interface of the operator library can be directly called to process the data of the physical tensor; Step S4: If the batch dimension of the physical tensor obtained in step S3 exceeds the limit, it can be reallocated into B physical tensors of the same size for storage. At this time, the shape of each physical tensor is , these B physical tensors are stored continuously and densely in memory; Step S5, cyclically calling the standard interface of the B-time operator library to process each physical tensor in step S4.

[0065] Specifically, for each physical tensor in step S4, since they are of the same size and stored continuously and closely, the starting address distance between each physical tensor can be calculated according to the shape of the physical tensor. By using the distance as a pointer offset and cyclically calling the operator library interface, the processing of each physical tensor can be achieved.

[0066] The method provided by the embodiment of the present invention achieves the effect of quickly utilizing the developed operator library to process a single logical tensor but actually multiple physical tensors by cyclically calling the operator library interface. Users can make full use of the developed operator library to quickly support the implementation of applications without having to understand very complex folding rules, and do not need to worry about the implementation in the specific library. In addition, the developers and users of the operator library do not have to modify the operator design and interface to support scenarios with multiple physical tensors, thereby saving a lot of manpower.

[0067] Based on any of the above embodiments, Figure 4 is a schematic diagram of the structure of the tensor processing device provided by the present invention, such as Figure 4 As shown, the device comprises: A determining unit 410 is configured to determine a shape of a first physical tensor based on a shape of the logical tensor, where the first physical tensor is obtained after dimension folding and physical conversion of the logical tensor; a splitting unit 420, configured to split the first physical tensor into a plurality of second physical tensors of the same size when the first dimension of the first physical tensor exceeds a dimension limit, and determine a tensor distance based on shapes of the second physical tensors, where the tensor distance is a distance between start addresses of any two adjacent second physical tensors; The processing unit 430 is used to traverse all second physical tensors and perform the following operations during the traversal process: based on the index of the second physical tensor currently traversed and the tensor distance, obtain the tensor data corresponding to the second physical tensor currently traversed, and call the operator library interface to process the tensor data to obtain the processing result.

[0068] The device provided by the embodiment of the present invention can dynamically determine the shape of the first physical tensor based on the shape of the logical tensor, and when the first dimension of the first physical tensor exceeds the dimension limit, the first physical tensor can be split into multiple second physical tensors of the same size, and can determine the distance between the starting addresses of any two adjacent second physical tensors based on the shape of the second physical tensor, which enables the present invention to adapt to tensor processing requirements of different scales and dimensions. By traversing all second physical tensors, it can ensure that each second physical tensor is properly processed, and according to the index and tensor distance of the second physical tensor currently traversed, the tensor data corresponding to the second physical tensor currently traversed can be obtained, and the operator library interface is called to process the tensor data, which not only realizes that multiple physical tensors can be processed by cyclically calling the operator library interface, but also does not require large-scale modification of the existing operator library interface, greatly reducing the workload of modification and the difficulty of implementation.

[0069] Based on any of the foregoing embodiments, the determining unit 410 is specifically configured to: Based on the shape of the logical tensor, determining whether a specified dimension of the logical tensor exceeds a dimension limit; When the specified dimension of the logical tensor exceeds the dimension limit, split the specified dimension, fold the split sub-dimensions onto the first dimension of the logical tensor, and physically transform the folded tensor to obtain a first physical tensor; Based on the dimensions and sizes of the dimensions of the first physical tensor, the shape of the first physical tensor is determined.

[0070] Based on any of the foregoing embodiments, the determining unit 410 is further configured to: When the specified dimension of the logical tensor does not exceed the dimension limit, the tensor data corresponding to the logical tensor is obtained, and the operator library interface is called to process the tensor data to obtain a processing result.

[0071] Based on any of the above embodiments, the processing unit 430 includes a tensor data acquisition subunit, and the tensor data acquisition subunit is used to: Determine an address offset based on an index of a second physical tensor currently traversed and the tensor distance; Determine, based on the starting address of the logical tensor and the address offset, the starting address of the second physical tensor currently traversed; Based on the starting address and shape of the second physical tensor currently traversed, tensor data corresponding to the second physical tensor currently traversed is obtained.

[0072] Based on any of the foregoing embodiments, the number of the second physical tensors is the same as the size of the first dimension of the logical tensor.

[0073] Based on any of the above embodiments, the device further includes a calling unit, and the calling unit is used to: When the first dimension of the first physical tensor does not exceed the dimension limit, tensor data corresponding to the first physical tensor is obtained, and an operator library interface is called to process the tensor data to obtain a processing result.

[0074] Based on any of the above embodiments, the multiple second physical tensors are stored continuously and densely, and the tensor distance is the same as the size of each second physical tensor.

[0075] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the tensor processing method, which includes: determining the shape of the first physical tensor based on the shape of the logical tensor, wherein the first physical tensor is obtained after dimension folding and physical conversion of the logical tensor; when the first dimension of the first physical tensor exceeds the dimension limit, splitting the first physical tensor into a plurality of second physical tensors of the same size, and determining the tensor distance based on the shape of the second physical tensor, wherein the tensor distance is the distance between the starting addresses of any two adjacent second physical tensors; traversing all second physical tensors, and performing the following operations during the traversal process: obtaining the tensor data corresponding to the second physical tensor currently traversed based on the index of the second physical tensor currently traversed and the tensor distance, and calling the operator library interface to process the tensor data to obtain the processing result.

[0076] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0077] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the tensor processing method provided by the above methods, the method including: determining the shape of a first physical tensor based on the shape of a logical tensor, the first physical tensor being obtained after dimension folding and physical conversion of the logical tensor; when the first dimension of the first physical tensor exceeds the dimension limit, splitting the first physical tensor into multiple second physical tensors of the same size, and determining the tensor distance based on the shape of the second physical tensor, the tensor distance being the distance between the starting addresses of any two adjacent second physical tensors; traversing all second physical tensors, and performing the following operations during the traversal process: based on the index of the second physical tensor currently traversed and the tensor distance, obtaining the tensor data corresponding to the second physical tensor currently traversed, and calling the operator library interface to process the tensor data to obtain the processing result.

[0078] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the tensor processing method provided by the above methods, the method comprising: determining the shape of a first physical tensor based on the shape of a logical tensor, the first physical tensor being obtained based on dimension folding and physical conversion of the logical tensor; when the first dimension of the first physical tensor exceeds the dimension limit, splitting the first physical tensor into multiple second physical tensors of the same size, and determining the tensor distance based on the shape of the second physical tensor, the tensor distance being the distance between the starting addresses of any two adjacent second physical tensors; traversing all second physical tensors, and performing the following operations during the traversal process: obtaining the tensor data corresponding to the second physical tensor currently traversed based on the index of the second physical tensor currently traversed and the tensor distance, and calling the operator library interface to process the tensor data to obtain a processing result.

[0079] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0080] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tensor processing method, characterized in that: include: Determine, based on the shape of the logical tensor, a shape of a first physical tensor, where the first physical tensor is obtained after dimension folding and physical transformation of the logical tensor; When the first dimension of the first physical tensor exceeds the dimension limit, split the first physical tensor into multiple second physical tensors of the same size, and determine the tensor distance based on the shapes of the second physical tensors, where the tensor distance is the distance between the start addresses of any two adjacent second physical tensors; Traverse all second physical tensors, and perform the following operations during the traversal process: based on the index of the second physical tensor currently traversed and the tensor distance, obtain the tensor data corresponding to the second physical tensor currently traversed, and call the operator library interface to process the tensor data to obtain the processing result.

2. The tensor processing method according to claim 1, characterized in that: The determining the shape of the first physical tensor based on the shape of the logical tensor includes: Based on the shape of the logical tensor, determining whether a specified dimension of the logical tensor exceeds a dimension limit; When the specified dimension of the logical tensor exceeds the dimension limit, split the specified dimension, fold the split sub-dimensions onto the first dimension of the logical tensor, and physically transform the folded tensor to obtain a first physical tensor; Based on the dimensions and sizes of the dimensions of the first physical tensor, the shape of the first physical tensor is determined.

3. The tensor processing method according to claim 2, characterized in that: Also includes: When the specified dimension of the logical tensor does not exceed the dimension limit, the tensor data corresponding to the logical tensor is obtained, and the operator library interface is called to process the tensor data to obtain a processing result.

4. The tensor processing method according to claim 1, characterized in that: The acquiring, based on the index of the second physical tensor currently traversed and the tensor distance, tensor data corresponding to the second physical tensor currently traversed includes: Determine an address offset based on an index of a second physical tensor currently traversed and the tensor distance; Determine, based on the starting address of the logical tensor and the address offset, the starting address of the second physical tensor currently traversed; Based on the starting address and shape of the second physical tensor currently traversed, tensor data corresponding to the second physical tensor currently traversed is obtained.

5. The tensor processing method according to claim 1, characterized in that: The number of the second physical tensors is the same as the size of the first dimension of the logical tensor.

6. The tensor processing method according to any one of claims 1 to 5, characterized in that: Also includes: When the first dimension of the first physical tensor does not exceed the dimension limit, tensor data corresponding to the first physical tensor is obtained, and an operator library interface is called to process the tensor data to obtain a processing result.

7. The tensor processing method according to any one of claims 1 to 5, characterized in that: The multiple second physical tensors are stored continuously and densely, and the tensor distance is the same as the size of each second physical tensor.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the tensor processing method according to any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the tensor processing method according to any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the tensor processing method according to any one of claims 1 to 7 is implemented.

Images