Data processing apparatus, method of determining tensor storage address, device and medium

Abstract

The present disclosure provides a data processing apparatus, relates to the technical field of artificial intelligence, and particularly relates to the technical field of chips. The specific implementation scheme is: a storage unit; a processor configured to: perform a target operation according to at least one continuous address space in a target storage area of the storage unit, so as to determine at least one current address space of a target operator in a plurality of operators, wherein the at least one current address space respectively corresponds to at least one first to-be-addressed tensor of the target operator; determine at least one current target remaining space in the target storage area according to a life cycle of the at least one first to-be-addressed tensor and the at least one current address space; and in response to determining that a sum of capacities of the at least one current target remaining space does not satisfy a preset condition, return to the target operation. The present disclosure also provides a method for determining a tensor storage address, an electronic device and a storage medium.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the field of chip technology. More specifically, this disclosure provides a data processing apparatus, a method for determining tensor storage addresses, an electronic device, and a storage medium. Background Technology

[0002] With the development of artificial intelligence technology, AI chips can be applied to intensive computing or diverse computing based on preset rules. Summary of the Invention

[0003] This disclosure provides a data processing apparatus, a method for determining tensor storage addresses, an electronic device, and a storage medium.

[0004] According to one aspect of this disclosure, a data processing apparatus is provided, comprising: a storage unit; and a processor configured to: perform a target operation based on at least one contiguous address space in a target storage region of the storage unit to determine at least one current address space of a target operator among a plurality of operators, wherein the at least one current address space corresponds to at least one first addressable tensor of the target operator; determine at least one current target remaining space in the target storage region based on the lifetime of the at least one first addressable tensor and the at least one current address space, wherein the at least one current target remaining space corresponds to at least one associated operator associated with the target operator; and return to the target operation in response to the determination that the sum of the capacities of the at least one current target remaining space does not meet a preset condition.

[0005] According to another aspect of this disclosure, a method for determining a tensor storage address is provided. The method includes: performing a target operation based on at least one contiguous address space in a target storage region of a storage unit to determine at least one current address space of a target operator among a plurality of operators, wherein the at least one current address space corresponds to at least one first addressable tensor of the target operator; determining at least one current target remaining space in the target storage region based on the lifetime of the at least one first addressable tensor and the at least one current address space, wherein the at least one current target remaining space corresponds to at least one associated operator associated with the target operator; and returning to the target operation in response to the determination that the sum of the capacities of the at least one current target remaining space does not meet a preset condition.

[0006] According to another aspect of this disclosure, an electronic device is provided, including the data processing apparatus provided in this disclosure.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method provided according to this disclosure.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the methods provided according to this disclosure.

[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided according to this disclosure.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1A This is a schematic computation diagram of a neural network model according to an embodiment of the present disclosure;

[0013] Figure 1B This is a schematic diagram of multiple address spaces in a three-level cache according to an embodiment of the present disclosure;

[0014] Figure 2 This is a schematic block diagram of a data processing apparatus according to an embodiment of the present disclosure;

[0015] Figure 3A This is a schematic diagram of multiple address spaces in a three-level cache according to another embodiment of the present disclosure;

[0016] Figure 3B This is a schematic diagram of multiple address spaces in a target storage region according to another embodiment of the present disclosure;

[0017] Figure 3C and Figure 3D This is a schematic diagram of multiple address spaces in a target storage region according to another embodiment of the present disclosure;

[0018] Figure 4 This is a schematic diagram of a data processing apparatus according to an embodiment of the present disclosure;

[0019] Figure 5This is a flowchart of a method for determining a tensor storage address according to an embodiment of the present disclosure;

[0020] Figure 6 This is a schematic block diagram of an electronic device according to an embodiment of the present disclosure; and

[0021] Figure 7 This is a block diagram of an electronic device to which a method for determining tensor storage addresses can be applied, according to an embodiment of the present disclosure. Detailed Implementation

[0022] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0023] Taking Kunlun Core's AI chip (XPU) as an example, the architecture of AI chips is more suitable for intensive computing or diverse computing based on preset rules. In the inference engine, a neural network (NN) model can be represented as a computational graph including operators and tensors. During the compilation of the neural network model, the storage space of the tensors can be pre-allocated to reduce memory access latency.

[0024] Artificial intelligence chips can include multi-level storage. For example, multi-level storage can include a Level 1 cache (L1 cache), a Level 2 cache (L2 cache), a Level 3 cache (L3 cache), and a global memory unit. If these tensors are properly allocated within the Level 3 cache, the computational performance of the artificial intelligence chip can be improved. The following will combine... Figure 1A and Figure 1B Please provide an explanation.

[0025] Figure 1A This is a schematic computation diagram of a neural network model according to an embodiment of the present disclosure.

[0026] like Figure 1AAs shown, the neural network model can include operators OP101 to OP114. Inputting the input tensor of the neural network model into operator OP101 yields tensor T101. Inputting tensor T101 into operator OP102 yields tensor T102. Inputting tensor T102 into operator OP103 yields tensor T103. Inputting tensor T103 into operator OP104 yields tensor T104. Inputting tensor T104 into operator OP105 yields tensor T105. Inputting tensor T105 into operator OP106 yields tensor T106. Inputting tensor T106 and tensor T103 into operator OP107 yields tensor T107. Inputting tensor T107 into operator OP108 yields tensor T108. Inputting tensors T108 and T107 into operator OP109 yields tensor T109. Inputting tensor T109 into operator OP110 yields tensor T110. Inputting tensor T110 into operator OP111 yields tensor T111. Inputting tensors T111 and T110 into operator OP112 yields tensor T112. Inputting tensor T112 into operator OP113 yields tensor T113. Inputting tensor T113 into operator OP114 yields tensor T114.

[0027] like Figure 1A As shown, tensor T103 is the output of operator OP103 and serves as the input to operators OP104 and OP107, respectively. Therefore, the lifetime of tensor T103 must last at least until operator OP107 is executed. It can be understood that a time period can be determined as the lifetime of the tensor based on the time from when the tensor is written to the memory location to when its address space is released. Furthermore, when tensor T103 serves as the input to operators OP104 and OP107, there is a dependency between operators OP104 and OP103, and also a dependency between operators OP107 and OP103. After the tensor is generated, it can be written to the L3 cache. The address spaces of multiple tensors in the L3 cache are as follows... Figure 1B As shown.

[0028] Figure 1B This is a schematic diagram of multiple address spaces in a three-level cache according to an embodiment of the present disclosure.

[0029] In some embodiments, the address space of tensors in the L3 cache can be determined manually. Alternatively, the address space of each operator can be determined based on the topological relationships between operators.

[0030] However, for complex and diverse neural network models, the time cost required to manually determine the address space is high, resulting in low model development efficiency.

[0031] Furthermore, the address space determined by topological relationships is difficult to reuse for high performance. For example, after tensor T101 is generated, it can be written to the L3 cache. After inputting tensor T101 into operator OP102, its address space can be freed. Other tensors are still written to or freed according to predetermined addresses, making it difficult to reuse the address space of tensor T101.

[0032] Therefore, in order to improve the performance of artificial intelligence chips, this disclosure provides a data processing device, which will be described below.

[0033] Figure 2 This is a schematic block diagram of a data processing apparatus according to an embodiment of the present disclosure.

[0034] like Figure 2 As shown, the data processing device 200 may include a storage unit 210 and a processor 220.

[0035] Storage unit 210 may include the aforementioned three-level cache. For example, the storage unit may include a target storage area for storing multiple tensors.

[0036] The processor 220 can be configured to perform a target operation based on at least one contiguous address space in the target storage region of the storage unit, in order to determine at least one current address space of the target operator among a plurality of operators. In embodiments of this disclosure, at least one current address space corresponds to at least one first addressable tensor of the target operator. For example, the target operator can be any one of a plurality of operators. The plurality of operators can, for example, perform convolution processing, fusion processing based on an attention mechanism, etc. The data size of the first addressable tensor can be less than the capacity of a contiguous address space. The capacity of the current address space can be less than or equal to the capacity of the contiguous address space. The current address space can store one first addressable tensor. Taking the target operator as the aforementioned operator OP103 as an example, the tensor T103 output by operator OP103 and the tensor T102 input to operator OP103 can both be used as the first addressable tensor.

[0037] The processor 220 can also be configured to: determine at least one current target remaining space in the target storage region based on the lifetime of at least one first addressable tensor and at least one current address space.

[0038] Taking the target operator OP103 as an example, the tensor T103 output by operator OP103 can be used as the input to operators OP104 and OP107. After being processed by operator OP107, the address space of tensor T103 can be released. During the execution time of operators OP104 to OP107, tensor T103 can be stored in a memory location. Furthermore, the tensor T102 output by operator OP102 can be used as the input to tensor OP103. Therefore, at least one operator associated with operator OP103 can include: operator OP102, operator OP104, operator OP105, operator OP106, and operator OP107.

[0039] The starting address of the current address space of tensor T102 can be the same as the starting address of the target storage region, and the starting address of the current address space of tensor T103 can be determined by the ending address of the current address space of tensor T102. In this case, the unoccupied contiguous address space in the target storage region can be used as the current target remaining space corresponding to operator OP104.

[0040] The processor 220 can also be configured to return to the target operation in response to determining that the sum of the capacities of at least one current target remaining space does not meet a preset condition. In this embodiment of the present disclosure, the preset condition may be that the sum of the capacities of at least one current target remaining space is greater than or equal to a preset capacity threshold. The sum of the capacities of the five current target remaining spaces corresponding to operators OP102, OP104, OP105, OP106, and OP107 can be determined respectively. If the sum of the capacities is less than the preset capacity threshold, the iteration of the current round can be terminated. Next, the process can return to the target operation to perform iterations in subsequent rounds in order to determine at least one subsequent address space of operator OP103, and then redetermine the subsequent target remaining spaces corresponding to the associated operators. It is understood that the starting address of the subsequent address space of tensor T102 may not be consistent with the starting address of the target storage region.

[0041] Using the embodiments of this disclosure, each of the multiple operators can serve as a target operator. For each operator, at least one target operation is performed, and at least one round of iterations is conducted. This allows the sum of the remaining capacity of the current target space to approach a preset capacity threshold, thereby maximizing the available storage space for each associated operator. This effectively avoids a large amount of free space within the target storage area and also helps improve the address reuse rate within the target storage area.

[0042] In this embodiment of the disclosure, the processor 220 may further be configured to: in response to determining that the sum of the capacities of at least one current target remaining space satisfies a preset condition, use at least one current address space as at least one first address space. After repeatedly executing the target operation multiple times, if it is determined that the sum of the capacities of at least one current target remaining space is greater than a preset capacity threshold, the current address space can be used as the first address space. Next, the operator OP104 can be used as the target operator for at least one round of iteration.

[0043] It is understood that the apparatus of this disclosure has been described above, and some methods for determining the target storage area will be described below.

[0044] In some embodiments, the processor may further be configured to determine the occupied space corresponding to the operator in the storage unit based on the sum of the data amounts of at least one tensor associated with the operator. For example, the tensors associated with the operator may include the operator's input tensor and output tensor. Taking the operator OP103 described above as an example, the tensors associated with operator OP103 may include tensor T102 and tensor T103. The occupied space corresponding to operator OP103 can be determined based on the sum of the data amounts of tensor T102 and tensor T103.

[0045] In some embodiments, the processor may also be configured to determine a target storage region in a storage unit based on the occupied space corresponding to a first operator among a plurality of operators.

[0046] In this embodiment, the capacity of the space corresponding to the first operator is greater than or equal to the capacity of the space corresponding to any one of the multiple operators. For example, among operators OP101 to OP114, if operator OP103 has the largest space, operator OP103 can be used as the first operator. The space corresponding to operator OP103 can be used as the target storage area. Through this embodiment, the operator with the largest space is determined, and the target storage area is determined based on the space occupied by this operator. Therefore, the capacity of the target storage area is more reasonable, which helps to improve the robustness of multiple operations executed by the processor and also helps to save space as much as possible.

[0047] As can be understood, the above text has described some methods for determining the target storage area, and the processor disclosed herein will be further described below.

[0048] Figure 3A This is a schematic diagram of multiple address spaces in a three-level cache according to another embodiment of the present disclosure.

[0049] like Figure 3AAs shown, the neural network model may include operators OP30 to OP39. Taking operator OP33 as the first operator as an example, at least one tensor associated with operator OP33 may include tensor T30, tensor T31, tensor T32, tensor T33, and tensor T34. Tensor T30 and tensor T32 can serve as input tensors of operator OP33. Tensor T33 can be the output tensor of operator OP33. Tensor T31 and tensor T34 can be the output tensors of operator OP31. The lifetimes of tensors T31 and T34 overlap with the runtime of operator OP33. Therefore, during the runtime of operator OP33, tensors T31 and T34 can be stored in the memory location.

[0050] like Figure 3A As shown, the width of a tensor represents the amount of data in the tensor. The height of a tensor represents its lifetime. The amount of data in tensor T31 can be less than the amount of data in tensor T30. The lifetime of tensor T33 can be longer than the lifetime of tensor T30.

[0051] When the target operator is the first operator, tensors T30, T31, T32, T33, and T34 can all be used as the first tensor to be addressed.

[0052] In this embodiment of the disclosure, the processor can be configured to perform a target operation based on at least one contiguous address space in the target storage region of the storage unit, so as to determine at least one current address space of the target operator among a plurality of operators. As shown in FIG3, when the target operator is the first operator, the entire address space of the target storage region A30 can be considered as a contiguous address space. Multiple current address spaces can be determined from this contiguous address space. The current address space can store a first addressable tensor. As shown in FIG3, the starting address of the current address space of tensor T30 can be consistent with the starting address of the target storage region, and the ending address of the current address space of tensor T34 can be consistent with the ending address of the target storage region. The current address spaces of tensor T30, tensor T31, tensor T32, tensor T33, and tensor T34 can be located sequentially within the target storage region A30.

[0053] In this embodiment of the disclosure, the processor may further be configured to: determine at least one current target remaining space in the target storage region based on the lifetime of at least one first addressable tensor and at least one current address space. The at least one current target remaining space corresponds to at least one associated operator associated with the target operator. For example, at least one associated operator associated with the target operator can be determined based on the lifetime of the first addressable tensor. Figure 3AAs shown, the lifetime of tensor T30 overlaps with the runtime segments of operators OP30, OP31, OP32, and OP33. The lifetime of tensor T33 overlaps with the runtime segments of operators OP33, OP34, OP35, OP36, OP37, and OP38. Therefore, operators OP30, OP31, OP32, OP34, OP35, OP36, OP37, and OP38 can each be considered as associated operators of operator OP33.

[0054] In embodiments of this disclosure, the processor may further be configured to: determine at least one current remaining address space of the associated operator within the target storage region based on the current address space of at least one first addressable tensor associated with the associated operator. For example, the tensor associated with the associated operator may be a tensor whose lifetime overlaps with the operator's runtime segment. Figure 3A As shown, taking operator OP34 as an example, tensors T31, T33, and T34 can be associated with operator OP34. The current remaining address space of operator OP34 can be determined in the target storage area to be two spaces: current remaining address space A341 and current remaining address space A342. The capacity of current remaining address space A341 can be greater than the capacity of current remaining address space A342.

[0055] In this embodiment of the disclosure, the processor can also be configured to use the current remaining address space with the largest capacity as the current target remaining space of the association operator. For example, the current remaining address space A341 can be used as the current target remaining space of operator OP34. By using the current remaining address space with the largest capacity as the current target remaining space of the association operator through this embodiment of the disclosure, iterative convergence can be accelerated and the efficiency of determining the first address space can be improved.

[0056] In some embodiments, the processor may further be configured to determine whether the sum of the capacities of at least one current target remaining space satisfies a preset condition. For example, the preset condition may include: the sum of the capacities of at least one current target remaining space is greater than or equal to a preset capacity threshold.

[0057] In this embodiment of the disclosure, in response to determining that the sum of the capacities of at least one current target remaining space does not meet a preset condition, the process returns to the target operation. For example, the sum of the capacities of the current target remaining spaces of operators OP30, OP31, OP32, OP34, OP35, OP36, OP37, and OP38 can be determined. If this sum of capacities is less than a preset capacity threshold, the iteration of the current round can be terminated. The process returns to the target operation, and the iteration in the next round is executed to redetermine the current address spaces of the multiple first addressable tensors. The following will be combined with... Figure 3B Further explanation is needed.

[0058] Figure 3B This is a schematic diagram of multiple address spaces in a target storage region according to another embodiment of the present disclosure.

[0059] like Figure 3B As shown, after redetermining the address spaces of the multiple first addressable tensors, the starting address of the current address space of tensor T34 can be the same as the starting address of the target storage region, and the ending address of the current address space of tensor T31 can be the same as the ending address of the target storage region. The current address spaces of tensors T34, T33, T30, T32, and T31 can be located sequentially within the target storage region A30.

[0060] Next, at least one current target remaining space can be determined in the target storage area. For example... Figure 3B As shown, taking operator OP34 as an example, tensors T31, T33, and T34 can be associated with operator OP34. The current remaining address space of operator OP34 can be determined to be one space in the target storage region, designated as current remaining address space A343. Current remaining address space A343 can be used as the current target remaining space corresponding to operator OP34.

[0061] Next, it can be determined whether the sum of the capacities of at least one current target remaining space satisfies the preset conditions.

[0062] In some embodiments, the processor may further be configured to: in response to determining that the sum of the capacities of at least one current target remaining space satisfies a preset condition, to respectively use at least one current address space as at least one first address space, wherein the at least one first address space corresponds to at least one first addressable tensor. For example, in this iteration process, the sum of the capacities of the current target remaining space of operator OP30, operator OP31, operator OP32, operator OP34, operator OP35, operator OP36, operator OP37, and operator OP38 may be determined. If the sum of the capacities is greater than the preset capacity threshold, the current address space of tensor T30 can be used as the first address space of tensor T30, the current address space of tensor T31 can be used as the first address space of tensor T31, the current address space of tensor T32 can be used as the first address space of tensor T32, the current address space of tensor T33 can be used as the first address space of tensor T33, and the current address space of tensor T34 can be used as the first address space of tensor T34.

[0063] It is understood that the processor of this disclosure has been described above using the example of the target operator being the first operator. However, this disclosure is not limited to this, and the target operator can also be an operator other than the first operator. Further explanation will follow.

[0064] In this embodiment of the disclosure, when the target operator is an operator other than the first operator among multiple operators, an addressed tensor exists in the target storage region. For example... Figure 3B As shown, operator OP34 can be used as the target operator. For operator OP34, there are three addressed tensors in the target storage area, namely tensor T34, tensor T33 and tensor T31.

[0065] In embodiments of this disclosure, the processor may be configured to determine at least one contiguous address range based on the target memory region and the addressed tensor. For example... Figure 3B As shown, based on the target storage region A30, the first address interval of tensor T34, the first address interval of tensor T33, and the first address interval of tensor T31, a contiguous address space corresponding to operator OP34 can be determined. This contiguous address space can, for example, be the aforementioned current remaining address interval A343. Through this embodiment, by determining a contiguous address space based on the address space of the addressed tensors, the addresses of the addressed tensors can be effectively maintained, reducing the number of tensors to be addressed, thereby reducing processor resource consumption and contributing to improved processor performance.

[0066] It is understood that the above explanation uses the example of a contiguous address space having a capacity greater than the data size of the tensor to be addressed. However, this disclosure is not limited to this, and the following explanation will use the example of a contiguous address space having a capacity smaller than the tensor to be addressed.

[0067] In this embodiment of the disclosure, after determining the contiguous address space corresponding to the target operator, it can be determined whether the capacity of the contiguous address space is greater than the data size of the tensor to be addressed. For example... Figure 3B As shown, the tensor output by operator OP34 can be used as a tensor to be addressed. If the data size of the tensor to be addressed is greater than the capacity of the contiguous address space, the tensor to be addressed can be used as a second tensor to be addressed.

[0068] In this embodiment of the disclosure, the processor may further be configured to: determine a second address space of the second addressable tensor of the target operator in a storage unit. The second address space is located outside the target storage region, and the data size of the second addressable tensor may be greater than the capacity of any contiguous address space. For example, the second address space may be determined outside the target storage region A30.

[0069] It is understandable that the remaining space of the current target was determined above based on the lifetime of the first addressable tensor and the current address space, but this disclosure is not limited to this. The following will combine... Figure 3C Further explanation is needed.

[0070] Figure 3C and Figure 3D This is a schematic diagram of multiple address spaces in a target storage region according to another embodiment of the present disclosure.

[0071] After determining the second address space of the tensor output by operator OP34, operator OP35 can be used as the target operator. The processor can determine at least one contiguous address range based on the target memory region and the addressed tensor. For example... Figure 3C As shown, based on the target storage area A30, the first address space of tensor T33, and the first address space of tensor T34, a contiguous address space A351 can be determined. Operator OP35 can output tensor T36. The data size of tensor T36 can, for example, be smaller than the capacity of the contiguous address space A351. Tensor T36 can serve as the first addressable tensor of operator OP35. Based on the contiguous address space A351, a target operation can be performed to determine the current address space of operator OP35 from the contiguous address space A351. This current address space can store the tensor T36 output by operator OP35. The starting address of this current address space can, for example, be the same as the starting address of the contiguous address space A351.

[0072] Next, in this embodiment of the disclosure, the processor may further be configured to: determine at least one current target remaining space in the target storage region based on the lifetime of at least one first addressable tensor, at least one current address space, and the address space of the addressed tensor. For example... Figure 3D As shown, for operator OP36 associated with operator OP35, tensors T33 and T34 can be used as addressed tensors. Based on the first address space of tensor T33, the first address space of tensor T34, and the current address space of tensor T36, the current remaining address space A361 of operator OP36 can be determined in the target storage region. The current remaining address space A361 can be used as the current target remaining space of operator OP36. Through the embodiments of this disclosure, the first addressable tensors of multiple operators can efficiently reuse the address space in the target storage region. This fully utilizes the bandwidth advantage of the L3 cache unit, improves the processor's memory access efficiency, and effectively enhances the performance of the artificial intelligence chip.

[0073] In other embodiments of this disclosure, the preset condition may further include: the sum of the capacities of at least one current target remaining space is maximized. The preset capacity threshold may be a value smaller than the maximum sum of capacities. For example, the number of iterations or the duration may be preset; after reaching a predetermined number of iterations, the iteration round with the maximum sum of capacities of the current target remaining space can be determined as the target iteration round. The current address space determined in the target iteration round is used as the first address space of the tensor.

[0074] It is understood that the processor of this disclosure has been described above, and the memory unit of this disclosure will be further described below.

[0075] In some embodiments, the storage unit may store layer connection information related to the neural network model. This layer connection information may indicate the relationships between different operators. For example, the layer connection information may indicate that operator OP33 is associated with operators OP30, OP31, OP32, OP34, OP35, OP36, OP37, and OP38.

[0076] In some embodiments, the storage unit may store layer size information associated with the operator. This layer size information may indicate the sum of the layer sizes of at least one tensor associated with the operator. For example, the layer size information for operator OP33 may indicate the sum of the layer sizes of tensors T30, T31, T32, T33, and T34.

[0077] In some embodiments, the storage unit may store operator sorting index information. The operator sorting index information (Layer sort idx) may correspond to the operator index sorted according to the data volume information. Based on this operator sorting index information, it can be determined that among multiple operators, the sum of the data volumes of the multiple tensors related to operator OP33 is the largest.

[0078] In some embodiments, the storage unit may store tensor sorting index information (datasort idx) for the operator. For example, tensor sorting index information may be generated after a target operation. The tensor sorting index information may indicate at least one address space. The tensor sorting index information may include at least one tensor location. The tensor location information may indicate the tail address of the tensor. This tail address may be the tail address of the current address space. The tensor sorting index information is adjusted multiple times during multiple iterations to determine the first address space.

[0079] In some embodiments, the storage unit may store tensor allocation information for the operator. The tensor allocation information may indicate the tail address of the tensor. This tail address may be the tail address of the tensor's first address space.

[0080] It is understood that the data processing apparatus of this disclosure has been described above in conjunction with operators OP30 to OP39. The effects of the data processing apparatus of this disclosure will now be described in conjunction with the aforementioned operators OP101 to OP114.

[0081] Figure 4 This is a schematic diagram of a data processing apparatus according to an embodiment of the present disclosure.

[0082] like Figure 4 As shown, after the address space of the tensor to be addressed for each operator is determined using the device 200, the address space of the first tensor to be addressed can be located in the target storage area A40. The address spaces of the multiple second tensors to be addressed can be located in the first storage area A41 and the second storage area A42.

[0083] like Figure 4 As shown, the addresses within the target storage area A40 can be efficiently reused, thereby significantly reducing the storage space required for the operation of multiple operators in the neural network model, which is beneficial to improving the utilization rate of the L3 cache.

[0084] As can be understood, the processing apparatus of this disclosure has been described above, and the method for determining the tensor storage address of this disclosure will be described below.

[0085] Figure 5This is a flowchart of a method for determining a tensor storage address according to an embodiment of the present disclosure.

[0086] like Figure 5 As shown, the method 500 may include operations S510 to S530.

[0087] In operation S510, a target operation is performed based on at least one contiguous address space in the target storage region of the storage cell in order to determine at least one current address space of the target operator among a plurality of operators.

[0088] In this embodiment of the disclosure, at least one current address space corresponds to at least one first addressable tensor of the target operator.

[0089] In operation S520, at least one current target remaining space is determined in the target storage region based on the lifetime of at least one first addressable tensor and at least one current address space.

[0090] In this embodiment of the disclosure, at least one current target remaining space corresponds to at least one associated operator associated with the target operator.

[0091] In operation S530, in response to determining that the sum of the capacities of at least one current target remaining space does not meet a preset condition, the process returns to the target operation.

[0092] It is understandable that the processor described above can be used to implement method 500.

[0093] In some embodiments, method 500 may further include: in response to determining that the sum of the capacities of at least one current target remaining space satisfies a preset condition, using at least one current address space as at least one first address space. For example, at least one first address space corresponds to at least one first addressable tensor.

[0094] In some embodiments, the preset conditions include at least one of the following: the sum of the capacities of at least one current target remaining space is maximum; or the sum of the capacities of at least one current target remaining space is greater than or equal to a preset capacity threshold.

[0095] In some embodiments, determining at least one current target remaining space in the target storage region may include: determining at least one current remaining address space of the associated operator within the target storage region based on the current address space of at least one first addressable tensor associated with the associated operator. The current remaining address space with the largest capacity is selected as the current target remaining space of the associated operator.

[0096] In some embodiments, performing a target operation based on at least one contiguous address space in the target storage region of the storage unit may include: determining, in the storage unit, an occupied space corresponding to an operator based on the sum of the data amounts of at least one tensor associated with the operator; and determining, in the storage unit, a target storage region based on the occupied space corresponding to a first operator among a plurality of operators. For example, the capacity of the occupied space corresponding to the first operator is greater than or equal to the capacity of the occupied space corresponding to any one of the plurality of operators.

[0097] In some embodiments, when the target operator is an operator other than the first operator among a plurality of operators, there are addressed tensors in the target storage region. Determining at least one current target remaining space in the target storage region may include: determining at least one current target remaining space in the target storage region based on the lifetime of at least one first addressable tensor, at least one current address space, and the address space of the addressed tensors.

[0098] In some embodiments, method 500 may further include: determining a second address space of the second addressable tensor of the target operator in a storage unit. For example, the second address space is located outside the target storage region, and the data size of the second addressable tensor is greater than the capacity of any contiguous address space.

[0099] In some embodiments, performing a target operation based on at least one contiguous address space in the target storage region of the storage unit may further include: determining at least one contiguous address interval based on the target storage region and the addressed tensor.

[0100] It is understood that the method for determining the tensor storage address of this disclosure has been described above, and the electronic device including the data processing device will be described below.

[0101] Figure 6 This is a block diagram of an electronic device according to another embodiment of the present disclosure.

[0102] like Figure 6 As shown, the electronic device 60 may include the data processing apparatus 600 provided in this disclosure. The data processing apparatus 600 may, for example, be the apparatus 200 described above.

[0103] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0104] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0105] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0106] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0107] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0108] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the method for determining a tensor memory address. For example, in some embodiments, the method for determining a tensor memory address may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the method for determining a tensor memory address described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the method for determining a tensor memory address by any other suitable means (e.g., by means of firmware).

[0109] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0110] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0111] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0112] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) monitor or an LCD (liquid crystal display)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0113] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0114] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0115] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0116] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A data processing apparatus, comprising: Storage unit; The processor is configured as follows: Based on at least one contiguous address space in the target storage region of the storage unit, a target operation is performed to determine at least one current address space of the target operator among a plurality of operators, wherein at least one of the current address spaces corresponds to at least one first addressable tensor of the target operator; Based on the lifetime of at least one of the first addressable tensors and at least one of the current address spaces, at least one current target remaining space is determined in the target storage region, wherein each of the at least one current target remaining space corresponds to at least one associated operator associated with the target operator; and In response to determining that the sum of the capacities of at least one of the remaining spaces of the current target does not meet a preset condition, the process returns to the target operation.

2. The apparatus according to claim 1, wherein, The processor is also configured to: In response to determining that the sum of the capacities of at least one of the remaining spaces of the current target satisfies a preset condition, at least one of the current address spaces is respectively designated as at least one first address space, wherein at least one first address space corresponds to at least one first addressable tensor.

3. The apparatus according to claim 1, wherein, The preset conditions include at least one of the following: At least one of the remaining spaces of the current target has a maximum sum of its capacity; and The sum of the capacities of at least one of the remaining spaces of the current target is greater than or equal to a preset capacity threshold.

4. The apparatus according to claim 1, wherein, The processor is also configured to: Based on the current address space of at least one first addressable tensor associated with the association operator, determine at least one current remaining address space of the association operator within the target storage region; as well as The current remaining address space with the largest capacity is taken as the current target remaining space of the association operator.

5. The apparatus according to claim 1, wherein, The processor is also configured to: Based on the sum of the data amounts of at least one tensor associated with the operator, the storage space corresponding to the operator is determined in the storage unit; and The target storage region is determined in the storage unit based on the occupied space corresponding to the first operator among the plurality of operators, wherein the capacity of the occupied space corresponding to the first operator is greater than or equal to the capacity of the occupied space corresponding to any one of the plurality of operators.

6. The apparatus according to claim 5, wherein, If the target operator is one of the multiple operators other than the first operator, and there is an addressed tensor in the target storage region, The processor is also configured to: Based on the lifetime of at least one of the first addressable tensors, at least one of the current address spaces, and the address spaces of the addressed tensors, at least one of the current target remaining spaces is determined in the target storage region.

7. The apparatus according to claim 4, wherein, The processor is also configured to: A second address space of the second addressable tensor of the target operator is determined in the storage unit, wherein the second address space is located outside the target storage area, and the data size of the second addressable tensor is greater than the capacity of any of the contiguous address spaces.

8. The apparatus according to claim 6, wherein, The processor is also configured to: Based on the target storage region and the addressed tensor, at least one continuous address range is determined.

9. A method for determining the storage address of a tensor, comprising: Based on at least one contiguous address space in the target storage region of the storage unit, a target operation is performed to determine at least one current address space of the target operator among a plurality of operators, wherein at least one of the current address spaces corresponds to at least one first addressable tensor of the target operator; Based on the lifetime of at least one of the first addressable tensors and at least one of the current address spaces, at least one current target remaining space is determined in the target storage region, wherein each of the at least one current target remaining space corresponds to at least one associated operator associated with the target operator; and In response to determining that the sum of the capacities of at least one of the remaining spaces of the current target does not meet a preset condition, the process returns to the target operation.

10. The method of claim 9, further comprising: In response to determining that the sum of the capacities of at least one of the remaining spaces of the current target satisfies a preset condition, at least one of the current address spaces is respectively designated as at least one first address space, wherein at least one first address space corresponds to at least one first addressable tensor.

11. The method according to claim 9, wherein, The preset conditions include at least one of the following: At least one of the remaining spaces of the current target has a maximum sum of its capacity; and The sum of the capacities of at least one of the remaining spaces of the current target is greater than or equal to a preset capacity threshold.

12. The method according to claim 9, wherein, Determining at least one current target remaining space in the target storage area includes: Based on the current address space of at least one first addressable tensor associated with the association operator, determine at least one current remaining address space of the association operator within the target storage region; and The current remaining address space with the largest capacity is taken as the current target remaining space of the association operator.

13. The method according to claim 9, wherein, The step of performing the target operation based on at least one contiguous address space in the target storage region of the storage unit includes: Based on the sum of the data amounts of at least one tensor associated with the operator, the storage space corresponding to the operator is determined in the storage unit; and The target storage region is determined in the storage unit based on the occupied space corresponding to the first operator among the plurality of operators, wherein the capacity of the occupied space corresponding to the first operator is greater than or equal to the capacity of the occupied space corresponding to any one of the plurality of operators.

14. The method according to claim 13, wherein, If the target operator is one of the multiple operators other than the first operator, and there is an addressed tensor in the target storage region, Determining at least one current target remaining space in the target storage area includes: Based on the lifetime of at least one of the first addressable tensors, at least one of the current address spaces, and the address spaces of the addressed tensors, at least one of the current target remaining spaces is determined in the target storage region.

15. The method of claim 12, further comprising: A second address space of the second addressable tensor of the target operator is determined in the storage unit, wherein the second address space is located outside the target storage area, and the data size of the second addressable tensor is greater than the capacity of any of the contiguous address spaces.

16. The method of claim 14, wherein, The step of performing the target operation based on at least one contiguous address space in the target storage region of the storage unit further includes: Based on the target storage region and the addressed tensor, at least one continuous address range is determined.

17. An electronic device comprising the data processing apparatus according to any one of claims 1 to 8.

18. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 9 to 16.

19. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 9 to 16.

20. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 9 to 16.

Images