Method, computing apparatus, medium, and program product for moving tensor data

Abstract

This invention relates to a method, computing device, medium, and program product for moving tensor data. The method includes: in response to determining that a source object is multidimensional tensor data, initializing parameters of the source object with respect to a minimum repeating unit to obtain a repeating layout pattern of the minimum repeating unit of the source object, the minimum repeating unit describing at least how the multidimensional tensor data is organized in registers; traversing the registers based on the repeating layout pattern of the minimum repeating unit to generate an access sequence of registers for the source object and an access sequence of registers for the target object; and performing a tensor data movement operation from the source object to the target object, at least based on the generated access sequence of registers for the source object and the target object. This invention is adaptable to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms, and is beneficial for parallel computing efficiency.

Description

Technical Field

[0001] The embodiments of the present invention generally relate to the field of artificial intelligence, and more specifically to a method, computing device, computer-readable storage medium, and computer program product for moving tensor data. Background Technology

[0002] Traditional methods for moving tensor data often rely on tensor layout characteristics. However, these methods have limitations in describing tensor layouts, only capable of depicting simple row / column order or basic block partitioning patterns. They struggle to indicate complex or irregular tensor layouts. Furthermore, tensor layouts are typically designed for specific hardware architectures, making them difficult to adapt to different hardware platforms and hindering cross-platform performance optimization. Moreover, the inability to accurately describe the mapping between computational units and data blocks limits parallel computing efficiency.

[0003] In summary, traditional methods for moving tensor data have the following shortcomings: they are difficult to adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms, and they are not conducive to improving parallel computing efficiency. Summary of the Invention

[0004] This invention provides a method, computing device, computer-readable storage medium, and computer program product for moving tensor data, which can adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms, and is beneficial to parallel computing efficiency.

[0005] According to a first aspect of the present invention, a method for moving tensor data is provided. The method includes: in response to determining that a source object is multidimensional tensor data, initializing parameters of the source object with respect to a minimum repeating unit to obtain a repeating layout pattern of the minimum repeating unit of the source object, the minimum repeating unit describing at least how the multidimensional tensor data is organized in registers; traversing the registers based on the repeating layout pattern of the minimum repeating unit to generate an access sequence of registers for the source object and an access sequence of registers for a target object; and performing a tensor data movement operation from the source object to the target object, at least based on the generated access sequence of registers for the source object and the access sequence of registers for the target object.

[0006] In some embodiments, initializing parameters for the minimum repeating unit of the source object includes: in response to determining that the source object is source tensor data whose attribute is multidimensional tensor data, obtaining the repeating layout pattern of the source tensor data; obtaining the offset of the minimum repeating unit of the source tensor data in the height direction and the offset in the width direction; determining whether the source tensor data is a discontinuous minimum repeating unit; obtaining the size and layout information of the registers inside the minimum repeating unit of the source tensor data; obtaining the offset of the registers inside the minimum repeating unit of the source tensor data; obtaining the number of minimum repeating units of the source tensor data; and obtaining thread granularity information.

[0007] In some embodiments, traversing registers based on a repeating layout pattern of minimum repeating units to generate access sequences for registers of a source object and for registers of a target object includes: traversing the minimum repeating units of the source tensor data based on the index of the minimum repeating unit; traversing registers based on register indices for each traversal of the minimum repeating unit; calculating a global register index based on the offsets of the traversed minimum repeating unit index from the minimum repeating unit in the row and column directions, and the offsets of the traversed register index from the registers within the minimum repeating unit in the row and column directions; and generating access sequences for registers of the source object and for registers of the target object based on the global register index calculated during the traversal.

[0008] In some embodiments, calculating the register index includes: multiplying the index of the smallest repeating unit in the column direction of the current traversal by the offset of the smallest repeating unit in the column direction to generate a column-direction smallest repeating unit index value; multiplying the index of the smallest repeating unit in the row direction of the current traversal by the offset of the smallest repeating unit in the row direction to generate a row-direction smallest repeating unit index value; multiplying the register index in the column direction within the smallest repeating unit of the current traversal by the offset of the register in the column direction within the smallest repeating unit to generate a register index value in the column direction within the smallest repeating unit; multiplying the register index in the row direction within the smallest repeating unit of the current traversal by the offset of the register in the row direction within the smallest repeating unit to generate a register index value in the row direction within the smallest repeating unit; and adding the column-direction smallest repeating unit index value, the row-direction smallest repeating unit index value, the register index value in the column direction within the smallest repeating unit, and the register index value in the row direction within the smallest repeating unit to generate a global register index corresponding to the current traversal based on the addition result.

[0009] In some embodiments, obtaining the repeating layout pattern of the minimum repeating unit of the source object includes: automatically processing the boundaries of the multidimensional tensor data if it is determined that the shape of the multidimensional tensor data is not an integer multiple of the minimum repeating unit.

[0010] In some embodiments, obtaining the repeating layout pattern of the minimum repeating unit of the source object includes: if the dimension of the multidimensional tensor data is greater than or equal to three dimensions, reducing the multidimensional tensor data to two-dimensional tensor data; and converting the two-dimensional tensor data into an organization of multiple repetitions of the minimum repeating unit in order to obtain the repeating layout pattern of the minimum repeating unit of the source object.

[0011] In some embodiments, the layout information of the registers within the minimum repeating unit of the source tensor data includes: the layout information of the registers within the minimum repeating unit of the source tensor data in the height direction and the layout information of the registers in the width direction; the offset of the registers within the minimum repeating unit of the source tensor data includes: the offset of the registers within the minimum repeating unit of the source tensor data in the height direction and the offset of the registers in the width direction; and the number of the minimum repeating units of the source tensor data includes: the number of the minimum repeating units of the source tensor data in the height direction and the number of the minimum repeating units of the source tensor data in the width direction.

[0012] In some embodiments, the minimum repeating unit is associated with multiple layout information, including: the name of the minimum repeating unit, the main direction, the layout of threads and values, the shape and step size of the threads inside the register, the number and arrangement of values ​​processed by each thread, and the size of the minimum repeating unit.

[0013] According to a second aspect of the present invention, a computing device is also provided. The computing device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the computing device to perform the method of the first aspect of the present invention.

[0014] According to a third aspect of the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program that, when executed by a machine, performs the method of the first aspect of the present invention.

[0015] According to a fourth aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a machine, performs the method of the first aspect of the present invention.

[0016] This invention transforms complex and variable multidimensional tensor data into multiple repetitions of the smallest repeating unit. Furthermore, by using a unified layout rule for the smallest repeating unit, it flexibly describes different multidimensional tensor data layout patterns. Therefore, this invention can adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms. In addition, based on a unified abstract layout rule that accurately describes the mapping relationship between multidimensional tensor data and thread registers, this invention can generate access sequences for the registers of the source object and the registers of the target object, thereby realizing the movement of multidimensional tensors. This invention not only significantly reduces the programming complexity and maintenance cost of tensor data movement but also improves parallel computing efficiency. Therefore, this invention can adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms, and it also improves parallel computing efficiency.

[0017] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0018] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements.

[0019] Figure 1 The diagram illustrates a computing apparatus for implementing a method for moving tensor data according to an embodiment of the present invention.

[0020] Figure 2 A flowchart of a method for moving tensor data according to an embodiment of the present invention is shown.

[0021] Figure 3 A schematic diagram showing the arrangement of the smallest repeating units according to an embodiment of the present invention is shown.

[0022] Figure 4 A schematic diagram of the main direction of the smallest repeating unit according to some embodiments of the present invention is shown.

[0023] Figure 5 A schematic diagram showing the shape and step size of the internal threads of a TLR according to an embodiment of the present invention is provided.

[0024] Figure 6 A schematic diagram illustrating the number and arrangement of values ​​processed by each thread according to some embodiments of the present invention is shown.

[0025] Figure 7A flowchart of a method for initializing parameters of a source object with respect to a minimum repeating unit, according to an embodiment of the present invention, is shown.

[0026] Figure 8 A flowchart of a method for generating access sequences of registers for a source object and access sequences of registers for a target object, according to an embodiment of the present invention, is shown.

[0027] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0028] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0029] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects.

[0030] As described above, traditional methods for moving tensor data have the following shortcomings: they are difficult to adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms, and they are not conducive to improving parallel computing efficiency.

[0031] To at least partially address one or more of the aforementioned problems and other potential issues, exemplary embodiments of the present invention propose a scheme for moving tensor data. In this scheme, when the source object is determined to be multidimensional tensor data, parameters regarding the minimum repeating unit are initialized for the source object. This yields a repeating layout pattern of the minimum repeating unit (which at least describes the organization of the multidimensional tensor data in the thread register). The present invention can transform complex and varied multidimensional tensor data into multiple repetitions of the minimum repeating unit. Furthermore, through a unified layout rule regarding the minimum repeating unit, different multidimensional tensor data layout patterns can be flexibly described. Therefore, the present invention can adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms. Furthermore, by using a repeating layout pattern based on the minimum repeating unit to traverse registers, this invention generates access sequences for registers of the source object and the target object; and, based at least on the generated access sequences for registers of the source and target objects, performs tensor data movement operations from the source object to the target object. This invention, based on a unified abstract layout rule of a repeating layout pattern with the minimum repeating unit that accurately describes the mapping relationship between multidimensional tensor data and thread registers, generates access sequences for registers of the source and target objects and realizes multidimensional tensor movement. This invention not only significantly reduces the programming complexity and maintenance cost of tensor data movement but also improves parallel computing efficiency. Therefore, this invention can adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms, and is beneficial for parallel computing efficiency.

[0032] Furthermore, this invention significantly simplifies the management overhead of data layout for multidimensional tensors, optimizes data layout, and reduces bandwidth requirements and energy consumption.

[0033] Figure 1 A schematic diagram of a computing device 100 implementing a method for moving tensor data according to an embodiment of the present invention is shown. Figure 1As shown, the computing device 100 may have one or more processing units, including dedicated processing units such as a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), or a General-purpose computing on graphics processing units (GPGPU), as well as general-purpose processing units such as a CPU. The computing device 100 also includes at least: a repeating layout pattern acquisition unit 102 for the minimum repeating unit of the source object, a register access sequence generation unit 104 for the source and target objects, and a tensor data move operation execution unit 106. It should be understood that the repeating layout pattern acquisition unit 102 for the minimum repeating unit of the source object, the register access sequence generation unit 104 for the source and target objects, and the tensor data move operation execution unit 106 can be software modules, such as those running within one or more processing units configured in the computing device 100.

[0034] The repeating layout pattern acquisition unit 102 for the minimum repeating unit of the source object is used to initialize the parameters of the minimum repeating unit for the source object in response to determining that the source object is multidimensional tensor data, thereby obtaining the repeating layout pattern of the minimum repeating unit of the source object. The repeating layout pattern of the minimum repeating unit is used to describe the organization of multiple repetitions of the minimum repeating unit.

[0035] The register access sequence generation unit 104 for the source and target objects is used to traverse the registers based on a repeating layout pattern of the minimum repeating unit in order to generate an access sequence for the registers of the source object and an access sequence for the registers of the target object.

[0036] The tensor data move operation execution unit 106 is used to perform tensor data move operations from the source object to the destination object based at least on the generated access sequence of registers of the source object and the access sequence of registers of the destination object.

[0037] The following will combine Figure 2 A method 200 for moving tensor data, according to an embodiment of the present invention, is described. Figure 2 A flowchart of a method 200 for moving tensor data according to an embodiment of the present invention is shown. It should be understood that method 200 can, for example, be implemented in... Figure 1The described computing device 100 performs the operation. Method 200 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0038] At step 202, if the computing device 100 determines that the source object is multidimensional tensor data, it initializes the parameters of the minimum repeating unit for the source object to obtain the repeating layout pattern of the minimum repeating unit of the source object, wherein the minimum repeating unit at least describes the organization of the multidimensional tensor data in the thread register.

[0039] Regarding the method for initializing parameters of the minimum repeating unit for a source object, in some embodiments, it includes, for example, the following: in response to determining that the source object is source tensor data with the attribute of multidimensional tensor data, obtaining the repeating layout pattern of the source tensor data; obtaining the offset of the minimum repeating unit of the source tensor data in the height direction and the offset in the width direction; determining whether the source tensor data is a discontinuous minimum repeating unit; obtaining the size and layout information of the registers inside the minimum repeating unit of the source tensor data; obtaining the offset of the registers inside the minimum repeating unit of the source tensor data; obtaining the number of minimum repeating units of the source tensor data; and obtaining thread granularity information. The following will combine... Figure 7 A specific description is provided of method 700 for initializing parameters of the minimum repeating unit for the multidimensional tensor data. Further details will not be provided here.

[0040] In some embodiments, the method for obtaining a repeating layout pattern of the minimum repeating unit of a source object further includes, for example, automatically handling the boundaries of the multidimensional tensor data if it is determined that the shape of the multidimensional tensor data is not an integer multiple of the minimum repeating unit.

[0041] A minimal repeating unit (Trait) is the smallest repeating unit used to describe tensor data. A Trait at least describes how multidimensional tensor data is organized in a thread register. For example, it indicates which data within the tensor data is processed by a specific register of a particular thread. A Trait can also indicate the mapping relationship between tensor data and threads. In some embodiments, a Trait is a structure stored in the header file of computing device 100. It should be understood that computing device 100 can be configured with various Traits. The Traits of this invention are adaptable to various hardware platforms, thus facilitating migration.

[0042] The repeating layout pattern of the minimum repeating unit (Trait) describes how multiple repetitions of the minimum repeating unit are organized. As mentioned earlier, a Trait reflects the layout of the minimum repeating units of tensor data; therefore, different tensor data can be viewed as multiple repetitions of the minimum repeating unit in different ways. Thus, by describing the repeating layout pattern of the minimum unit, the layout of the entire tensor data can be derived. For example, the layout of the entire source tensor data can be derived from the repeating layout pattern of the minimum repeating units of the source tensor data.

[0043] In some embodiments, the method for obtaining the repeating layout pattern of the minimum repeating unit for a source object includes, for example, configuring the corresponding minimum repeating unit repeating layout pattern for the source object based on a base minimum repeating unit (BaseTrait). In some implementations, the method for configuring the corresponding minimum repeating unit for the source object includes, for example, configuring the name of the source object's Trait, main direction, thread and value layout, shape and stride of the threads within the register (TLR), the number and arrangement of values ​​processed by each thread, and the size of the Trait.

[0044] In some embodiments, a method for obtaining a repeating layout pattern of the minimum repeating unit of a source object includes, for example: reducing the multidimensional tensor data to two-dimensional tensor data if the dimension of the multidimensional tensor data is greater than or equal to three dimensions; and converting the two-dimensional tensor data into an organization of multiple repetitions of the minimum repeating unit in order to obtain a repeating layout pattern of the minimum repeating unit of the source object.

[0045] Table 1 below illustrates instances of Traits configured in some embodiments.

[0046] Table 1

[0047]

[0048] As shown in Table 1, “matrixTlr64x8Fp32Col” is the name of the configured Trait. “H_MAJOR” (i.e., column major order) is the major direction of the configured Trait. “Layout(Shape(Shape(4, 8, 4), Shape(1, 2, 2)),Stride(Stride(64, 1, 16), Stride(1, 256, 8)))” is the layout of the threads and values ​​of the configured Trait. “Layout(Shape(Shape(4), Shape(8), Shape(4)), Stride(Stride(1), Stride(4),Stride(32)))” is the shape and stride of the internal threads of the TLR of the configured Trait. "Layout(Shape(1, 2, 2),Stride(0, 1, 2)" specifies the number and arrangement of values ​​processed by each thread. "Layout(Shape(64, 8),Stride(1, 64)))" specifies the size of the configured Trait. Table 1 describes a 64×8 matrix layout of Traits. Different Traits are arranged in column-major order (H_MAJOR). Each Trait contains 4×8×4=128 threads. Each thread processes 1×2×2=4 values. A Trait can represent a total of 64×8=512 FP32 data points.

[0049] Therefore, it is possible to use rich features to construct different Trait layout features for different types of tensor data.

[0050] Methods for constructing Traits include, for example, abstracting recurring units in tensor data. For instance, a thread might have 128 or 256 registers. In some embodiments, registers are grouped into sets of four, corresponding to one Trait of the first type. For example, if the current tensor data requires 96 registers per thread, the layout of the tensor data can be viewed as a repeating layout of 24 Traits of the first type. As another example, registers are grouped into sets of two, corresponding to one Trait of the second type, then the layout of the current tensor data can be viewed as a repeating layout of 48 Traits of the second type.

[0051] It should be understood that Traits can describe multi-level layout characteristics. For example, they can describe the thread layout, value layout, and overall shape of tensor data. Therefore, based on Traits, this invention can precisely control how tensor data is organized at different levels of abstraction.

[0052] The smallest repeating unit (Trait) includes several layout information items. These include: name, major direction, layout of threads and values ​​(layoutTv), shape and stride of threads within the TLR (layoutTvThread), number and arrangement of values ​​processed by each thread (layoutValuesPerThread), and size (hw).

[0053] Regarding the name, it is the identifier of the smallest repeating unit. The trait name is used for identification and debugging.

[0054] The major direction describes the arrangement of traits. This arrangement can be, for example, row-major (W_MAJOR, or "width-major") or column-major (H_MAJOR, or "height-major"). Figure 3 A schematic diagram illustrating the arrangement of the smallest repeating units according to an embodiment of the present invention is shown. Figure 3 As shown, the diagram on the left, indicated by label 310, illustrates the arrangement of Trains in row-major order, i.e., majorDirection=W_MAJOR. (As...) Figure 3 As shown in the left-hand portion, one Trait corresponds to one Computation Unit (CU). One CU comprises four Sub-Computation Units (EUs). Trait 302, for example, corresponds to one CU, and the four color blocks included in Trait 302, for example, correspond to four EUs. Marker 330 indicates r0 in the first color block included in Trait 302. r0 corresponds to 32 threads (these 32 threads are, for example, ... Figure 4 The zeroth register in (as shown in T0 to T31) Figure 4 V0 (as shown). Figure 4 A schematic diagram of the main direction of the minimum repeating unit according to some embodiments of the present invention is shown. Similarly, denoted by 332 indicates r1 in the first color block included in Trait 302. r1 corresponds to 32 threads (the 32 threads are, for example, Figure 4 The first register in T0 to T31 (as shown) Figure 4 (See V1 shown). The specifications of a Trait are, for example, 64*8. The arrow indicated by mark 312 indicates that the next Trait 304 is arranged in the direction of the row extension, that is, the next Trait 304 is placed to the right of the current Trait 302.

[0055] exist Figure 3In the diagram, the right-hand portion indicated by marker 320 shows the arrangement of the smallest repeating units in column-major order, i.e., majorDirection=H_MAJOR. For example... Figure 3 As shown on the right side, the Trait specification is also, for example, 64*8. The arrow indicated by marker 322 indicates that the next Trait 304 is arranged according to the column extension direction, that is, the next Trait 304 is placed below the current Trait 302. It should be understood that the present invention can guide the arrangement of different Traits by controlling the major direction attribute, thereby achieving a more flexible organization of large-scale tensor data.

[0056] The following combination Figure 4 This describes the layout of threads and values. Figure 4 A schematic diagram of the layout of threads and values ​​according to an embodiment of the present invention is shown. The layout of threads and values ​​indicates the correspondence between tensor data and the threads and registers that process the tensor data. In some embodiments, the shape of the thread dimension is configured, for example, as " / / T (Thread) dimension Shape(4,8,4): Shape(64,1,16)". Wherein "Shape(4,8,4)" indicates that there are 32 threads within each EU, with a shape of 4 columns and 8 rows. Each Trait corresponds to 4 EUs. The shape of the value dimension is configured, for example, as " / / V (Value) dimension Shape(1,2,2): Shape(1,256,8)". Figure 4 As shown, the left-hand portion 402 indicates multiple Traits. The right-hand portion indicates the correspondence between the threads and registers used to process the tensor data of the corresponding Traits. Specifically, the threads and registers used to process the tensor data of the Trait corresponding to r0 are shown within the dashed box indicated by arrow 422. In other words, the tensor data of the Trait corresponding to r0 is processed by register V0 of the 32 threads T0 to T31. Similarly, the tensor data of the Trait corresponding to r1 is processed by register V1 of the 32 threads T0 to T31. The tensor data of the Trait corresponding to r2 is processed by register V2 of the 32 threads T0 to T31. The tensor data of the Trait corresponding to r3 is processed by register V3 of the 32 threads T0 to T31. Figure 4As shown on the right side, T represents a thread, for example, T0 represents thread T0 (i.e., thread zero). V represents a register, for example, V0 represents register V0 (i.e., register zero). T0 to T31 represent 32 threads, i.e., threads zero to thirty-first. Each of these 32 threads is configured with register V0, which represents a trait that can process 8*4 (i.e., 8 rows and 4 columns) of data. If 8*8 data needs to be processed, each of these 32 threads needs to be configured with both register V0 and register V1. For example, for FP32 format data, one thread stores one data item. In this case, the stride of the padding data corresponding to the arrow indicated by marker 410 is, for example, the default value (the default value is, for example, "0", i.e., stride=0). The stride of the padding data corresponding to the arrow indicated by marker 412 is "1" (i.e., stride=1). The stride of the padding data corresponding to the arrow indicated by marker 414 is "8" (i.e., stride=8). Since a column contains 64 data points, the stride of the fill data corresponding to the arrow indicated by marker 416 is "64" (i.e., stride=64). The stride of the fill data corresponding to the arrow indicated by marker 418 is "16" (i.e., stride=16), which actually represents the stride of the fill data at the corresponding positions of two adjacent EUs. Because a column contains 64 data points, and moving from r0 to r1 requires traversing 4 columns, the stride of the fill data corresponding to the arrow indicated by marker 420 is "256" (i.e., stride=256).

[0057] The following combination Figure 5 Explain the shape and step size of the threads inside the TLR. Figure 5 A schematic diagram illustrating the shape and step size of the internal threads of the TLR according to an embodiment of the present invention is shown. Table 2 below illustrates... Figure 5 The diagram shows the configuration of the shape and step size of the internal threads of the TLR.

[0058] Table 2

[0059]

[0060] As shown in Table 2, the shape “Shape(Shape(4), Shape(8), Shape(4))” represents that an EU is configured with 4*8=32 threads, and a CU is configured with 4 EUs, thus having 128 threads.

[0061] like Figure 5As shown, the TLR internal thread identifier (ID) corresponding to the arrow indicated by marker 510 has a step size of "1" from "0" to "1" (i.e., from thread T0 to thread T1) (i.e., "Stride(1)"). The TLR internal thread identifier (ID) corresponding to the arrow indicated by marker 512 has a step size of "4" from "0" to "4" (i.e., from thread T0 to thread T4) (i.e., "Stride(4)"). The TLR internal thread identifier (ID) corresponding to the arrow indicated by marker 514 is from "0" of one EU to "0" of the adjacent EU (i.e., from thread T0 of the current EU to the next thread T0). Given that an EU has 32 threads, the internal thread corresponding to the arrow indicated by marker 514 has a step size of "32" (i.e., "Stride(32)"). The identifier (ID) of the internal thread of the TLR corresponding to the arrow indicated by marker 516 is from thread T0 of a CU to thread T0 of the adjacent CU, and the step size of the corresponding internal thread is "32" (i.e., "Stride(32)").

[0062] The following table 3 shows an example. Figure 6 This shows the configuration of the number of values ​​processed by each thread and their arrangement (layoutValuesPerThread).

[0063] Table 3

[0064]

[0065] As shown in Table 3, "Shape(1,2,2)" represents the shape of the Trait as viewed from the register dimension. Figure 6 As shown, an EU is configured with registers in a 2x2 grid. Figure 6 For example, the first row contains V0 and V1, and the step size between adjacent registers in the same row is "1"; the first column contains V0 and V2, and the step size between adjacent registers in the same column is "2".

[0066] The following combination Figure 6 This describes the number and arrangement of values ​​processed by each thread (layoutValuesPerThread). Figure 6 A schematic diagram illustrating the number and arrangement of values ​​processed by each thread according to some embodiments of the present invention is shown. Figure 6As shown, the 32 threads in the range indicated by label 610 correspond to the 32 threads of the tensor data corresponding to r0, which are processed by register V0 (i.e., the zeroth register). The 32 threads in the range indicated by label 612 correspond to the 32 threads of the tensor data corresponding to r1, which are processed by register 1 (i.e., the first register) represented by V1. Similarly, the 32 threads in the range indicated by label 614 correspond to the 32 threads of the tensor data of the Trait corresponding to r2. These 32 threads are processed by register V2 (i.e., the second register). The 32 threads in the range indicated by label 616 correspond to the 32 threads of the tensor data corresponding to r3. These 32 threads are processed by register V3 (i.e., the third register). Label 620 indicates the step size from register V0 corresponding to the range indicated by label 610 to register V1 corresponding to the range indicated by label 612, which is "1", that is, the step size between adjacent registers in the same row is "1". Marker 622 indicates the step size from register V0 corresponding to the range indicated by mark 610 to register V2 corresponding to the range indicated by mark 614. The step size is "2", that is, the step size between adjacent registers in the same column is "1".

[0067] Regarding size (hw), it indicates the dimensions of the Trait, such as... Figures 3 to 6 As shown, each Trait is 64*8 in size.

[0068] At step 204, the computing device 100 traverses the registers based on a repeating layout pattern of minimum repeating units in order to generate an access sequence of registers for the source object and an access sequence of registers for the target object.

[0069] A method for generating access sequences of registers for a target object includes, for example, the following: a computing device 100 traverses the smallest repeating unit of source tensor data based on the index of the smallest repeating unit; for each traversal of the smallest repeating unit, traversing registers based on register indices; calculating a global register index based on the offsets of the traversed smallest repeating unit's index from the smallest repeating unit in the row and column directions, and the offsets of the traversed register's index from registers within the smallest repeating unit in the row and column directions; and generating access sequences of registers for the source object and access sequences of registers for the target object based on the global register index calculated during the traversal. The following will combine... Figure 8 The method 800 for generating the access sequence of registers for the target object is described in detail here. Further details will not be provided.

[0070] At step 206, the computing device 100 performs a tensor data movement operation from the source object to the destination object based at least on the generated access sequence of registers about the source object and the access sequence of registers about the target object.

[0071] Table 4 below schematically illustrates exemplary code for performing a tensor data movement operation from a source object to a destination object according to some embodiments of the present invention.

[0072] Table 4

[0073]

[0074] As shown in Table 4, a method for performing a tensor data move operation from a source object to a destination object includes, for example,: in response to determining that the source object and the destination object are data of the same type, confirming whether the source tensor data is a non-contiguous minimum repeating unit; if the source tensor data is determined to be a non-contiguous minimum repeating unit, sequentially calculating the register index of the source object (e.g., "src_reg") and the register index of the destination object (e.g., "dst_reg") for each TLR within the total number of TLRs. This is to perform the tensor data move operation based on the calculated register indices of the source object and the destination object.

[0075] In some embodiments, the method for calculating the register index of the target object includes, for example, selecting a target register from a set of closed segment registers (e.g., x_registers) if the computing device 100 determines that "dst_is_x_reg" is true. Here, "dst_is_x_reg" is an identifier indicating whether the target tensor data (dst tensor) is associated with closed segment registers (x0-x3). Closed segment registers are used to alleviate TLR pressure. It should be understood that closed segment registers x0-x3 provide four 32-bit temporary registers for each thread. This reduces TLR port conflicts, lowers register pressure, and saves power. For example, by using the code "dst_reg = x_registers[dst_x_base + dst_tlr_seq[i]] if dst_is_x_reg \ else r_registers[dst_r_base + dst_tlr_seq[i]]" in Table 4, the target base index of the closed segment register (e.g., "dst_x_base") is added to the offset in the access sequence of the registers of the target object (e.g., "dst_tlr_seq[i]"), thereby obtaining the register index of the target object; and if the computing device 100 determines that "dst_is_x_reg" is false, the target register is calculated from the r register group (r_registers). For example, the register index of the target object is obtained by adding the target base index of the r register (e.g., "dst_r_base") to the offset in the access sequence of the registers of the target object (e.g., "dst_tlr_seq[i]") using the code "r_registers[dst_r_base + dst_tlr_seq[i]" in Table 4.

[0076] In some embodiments, the method for calculating the register index of the source object includes, for example, selecting a target register from a group of closed-segment registers (e.g., x_registers) if the computing device 100 determines that "src_is_x_reg" is true. Here, "src_is_x_reg" is an identifier used to indicate whether the source tensor data (src tensor) is associated with a closed-segment register. For example, by using the code "src_reg = x_registers[src_x_base + src_tlr_seq[i]] ifsrc_is_x_reg \ else r_registers[src_r_base + src_tlr_seq[i]]" in Table 4, the source base index of the closed segment register (e.g., "src_x_base") is added to the offset in the access sequence of the registers of the source object (e.g., "src_tlr_seq[i]"), thereby obtaining the register index of the source object; and if the computing device 100 determines that "src_is_x_reg" is false, the source register is calculated from the r register group (r_registers). For example, the register index of the source object is obtained by adding the source base index of the r register (e.g., "src_r_base") to the offset in the access sequence of the registers of the source object (e.g., "src_tlr_seq[i]]") using the code "r_registers[src_r_base + src_tlr_seq[i]]" in Table 4.

[0077] In some embodiments, the method for moving tensor data from a source object to a destination object includes, for example, the following: the computing device 100 uses the code “mih.mov(dst_reg, src_reg, mov_mod)” in Table 4 to perform a tensor data movement operation based on the calculated register index of the destination object and the register index of the source object.

[0078] In some embodiments, a method for performing a tensor data movement operation from a source object to a destination object further includes, for example, moving the tensor data in batches in response to the computing device 100 determining that the source object and the destination object are data of the same type and that the source object is not a discontinuous minimum repeating unit. Specifically, the method for moving the tensor data in batches includes, for example, the computing device 100 calculating a size for moving the tensor data in batches based on the total number of TLRs; looping within a range from "0" to the number of registers to be processed in the access sequence of registers, with the calculated size of the tensor data to be moved in steps; and calculating the register index of the destination object and the register index of the source object in each loop to move the tensor data in batches.

[0079] A method for calculating the size of tensor data for bulk shifting includes, for example, the computing device 100 calculating the size of the tensor data for bulk shifting using the exemplary code "burst_size = calculate_max_burst(tlr_num)". Here, "burst_size" represents the size of the tensor data for bulk shifting. "calculate_max_burst()" represents a function. "tlr_num" represents the number of registers to be processed in the register access sequence.

[0080] A method for batch-shifting tensor data by looping with the size of the calculated batch shift tensor data as the step size includes, for example, the computing device 100 starting from "0" and ending at "tlr_num", stepping "burst_size" each time. For example, if "tlr_num" = 100 and "burst_size" = 16, then the offset i takes the value "0, 16, 32, ..., 96". In some embodiments, the computing device 100 calculates the register index "dst_reg" of the target object, for example, by the code "dst_reg = r_registers[dst_r_base + i]". It calculates the register index "src_reg" of the source object, by the code "src_reg = r_registers[src_r_base + i]". And it batch-shifts the tensor data by the code "mih.mov(dst_reg, src_reg, mov_mod)".

[0081] Table 5 below schematically illustrates exemplary code for performing a tensor data movement operation from a source object to a destination object according to some embodiments of the present invention.

[0082] Table 5

[0083]

[0084] As shown in Table 5, the method for performing a tensor data movement operation from a source object to a destination object further includes, for example, the following: if the computing device 100 determines that the data type of the source object is different from the data type of the destination object. For example, if the source object has a 32-bit data type (e.g., as indicated by the code "src_data_size == 4") and the target object has a 16-bit data type (e.g., as indicated by the code "dst_data_size == 2"), then within the range of the number of registers to be processed in the register access sequence, the target register index is obtained by adding the r register target base index "dst_r_base" to the offset "dst_tlr_seq[i]" in the access sequence of the target object's registers (e.g., as indicated by the code "dst_reg = r_registers[dst_r_base + dst_tlr_seq[i]]"); the target object's register index is obtained by adding twice the offset "src_tlr_seq[2*i]" to the r register source base index "src_r_base" in the access sequence of the source object's registers (as indicated by the code "src_reg = r_registers[src_r_base]" in Table 5). +src_tlr_seq[2*i]]” indicates); and the movement of tensor data is performed based on the obtained register index of the target object and the register index of the source object (e.g., the movement of tensor data from 32 bits to 16 bits is achieved by the code “mih.mov(dst_reg, src_reg, mov_mod)”).

[0085] In the above-described scheme, this invention can transform complex and variable multidimensional tensor data into multiple repetitions of the smallest repeating unit. Furthermore, through a unified layout rule regarding the smallest repeating unit, it can flexibly describe different multidimensional tensor data layout patterns. Therefore, this invention can adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms. In addition, based on a unified abstract layout rule that accurately describes the mapping relationship between multidimensional tensor data and thread registers, this invention can generate access sequences for the registers of the source object and the registers of the target object, thereby realizing the movement of multidimensional tensors. This invention not only significantly reduces the programming complexity and maintenance cost of tensor data movement but also improves parallel computing efficiency. Therefore, this invention can adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms, and it also improves parallel computing efficiency.

[0086] The following will combine Figure 7 The present invention describes a method 700 for initializing parameters of a source object with respect to a minimum repeating unit. Figure 7A flowchart of a method 700 for initializing parameters of a source object with respect to a minimum repeating unit, according to an embodiment of the present invention, is shown. It should be understood that method 700 can, for example, be implemented in... Figure 1 The described computing device 100 performs the operation. Method 700 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0087] In some embodiments, a function "initSDLayoutParam" for Trait initialization is first defined. This function belongs to the "MovCvt" class and returns a value of "status_t".

[0088] At step 702, in response to determining that the source object is source tensor data with the attribute of multidimensional tensor data, the computing device 100 obtains a repeating layout pattern for the source tensor data.

[0089] In some embodiments, if the computing device 100 determines that the source object (the source object is referred to as "src" for example) is source tensor data (or referred to as "srcTensor" or "src tensor"), it can do so, for example, by using the code "TensorPtr srcTensor = std::get" as exemplified in Table 6 below. <tensorptr>The tensor pointer (src) is obtained as the source object (e.g., "TensorPtr") and assigned to the tensor pointer of the source tensor data (srcTensor). Further, the computing device 100 calls the function "getRepeatTraitLayout()" via the code "srcRepeat = getRepeatTraitLayout(srcTensor)" to extract the repeating layout information of the Trait from the tensor data (srcTensor), thereby obtaining the repeating layout pattern of the source tensor data.

[0090] The repeated layout pattern of the source tensor data refers to the repeated layout information of Traits in the source tensor data. The repeated layout information of Traits includes, for example, at least the Trait name, major direction, thread and value layout (layoutTv), the shape and stride of the TLR internal thread (layoutTvThread), the number and arrangement of values ​​processed by each thread (layoutValuesPerThread), and size (hw).

[0091] At step 704, the computing device 100 obtains the offset of the smallest repeating unit in the height direction and the offset in the width direction for the source tensor data.

[0092] The offset of the smallest repeating unit in the source tensor data in the height direction (e.g., "srcTraitOffsetH" in Table 6) and the offset in the width direction (e.g., "srcTraitOffsetW" in Table 6) are determined, for example, based on the stride of the Trait in the column direction and the stride of the Trait in the row direction.

[0093] For example, the computing device 100 obtains the offset of the Trait in the height direction (srcTraitOffsetH) of the source tensor data through the code "srcTraitOffsetH = srcRepeat.getStride().get(0,0)->get().value()"; and obtains the offset of the Trait in the width direction (srcTraitOffsetW) of the source tensor data through the code "srcTraitOffsetW=srcRepeat.getStride().get(0,1)->get().value()".

[0094] At step 706, the computing device 100 determines whether the source tensor data is a discontinuous minimum repeating unit.

[0095] For example, computing device 100 determines whether the source tensor data is a discontinuous minimum repeating unit (or "Gap Trait"). If the source tensor data is determined to be a discontinuous minimum repeating unit, the determination result is stored in a Boolean variable (e.g., "srcIsGapTrait" in Table 6). In some embodiments, computing device 100 determines whether the source tensor data is a discontinuous minimum repeating unit, for example, via the code "srcIsGapTrait = isGapTrait(srcTensor->getTrait())".

[0096] At step 708, the computing device 100 obtains the size and layout information of the registers inside the smallest repeating unit of the source tensor data.

[0097] The layout information of the registers inside the smallest repeating unit of the source tensor data includes, for example, the layout information of the registers inside the Trait of the source tensor data in the height direction and the layout information of the registers inside the Trait in the width direction.

[0098] For example, computing device 100 obtains the size of the registers inside the Trait of the source tensor data (e.g., srcTraitTLRSize) via the code "srcTraitTLRSize = getTraitTLRSize(srcTensor->getTrait())". Computing device 100 obtains the layout information of the registers inside the Trait of the source tensor data in the height direction (e.g., srcTraitTLRSizeInH) via the code "srcTraitTLRSizeInH = getTraitTLRSizeInH(srcTensor->getTrait())". Computing device 100 obtains the layout information of the registers inside the Trait of the source tensor data in the width direction (e.g., srcTraitTLRSizeInM) via the code "srcTraitTLRSizeInM = getTraitTLRSizeInM(srcTensor->getTrait())".

[0099] At step 710, the computing device 100 obtains the offset of the register inside the smallest repeating unit of the source tensor data.

[0100] The offsets of registers within the smallest repeating unit of the source tensor data include, for example, the offsets of registers within the Trait of the source tensor data in the height direction and the offsets of registers within the Trait in the width direction. In some embodiments, the offsets of registers within the smallest repeating unit of the source tensor data are obtained based on the step size of the thread layout within the Trait. For example, computing device 100 obtains the offsets of registers within the Trait of the source tensor data in the height direction (e.g., srcTraitTLROffsetH) via the code "srcTensor->getTrait()->layoutValuesPerThread.getStride().get(2, 0)". Furthermore, computing device 100 obtains the offsets of registers within the Trait of the source tensor data in the width direction (e.g., srcTraitTLROffsetH) via the code "srcTraitTLROffsetM =srcTensor->getTrait()->layoutValuesPerThread.getStride().get(1, 0)".

[0101] At step 712, the computing device 100 obtains the number of minimum repeating units of the source tensor data.

[0102] The number of minimum repeating units in the source tensor data includes, for example, the number of traits in the height direction and the number of traits in the width direction. In some embodiments, the number of minimum repeating units in the source tensor data is the number of traits obtained from the shape of the repeating layout pattern (e.g., srcRepeat) of the source tensor data. For example, computing device 100 obtains the number of traits in the height direction (e.g., srcTraitNumH) of the source tensor data via the code "srcTraitNumH = srcRepeat.getShape().get(0, 0)->get().value()". Also, for example, computing device 100 obtains the number of traits in the width direction (e.g., srcTraitNumW) of the source tensor data via the code "srcTraitNumW = srcRepeat.getShape().get(0, 1)->get().value()".

[0103] In step 714, the computing device 100 acquires thread granularity information.

[0104] The thread granularity information of the source tensor data includes, for example, thread granularity information in the height direction and thread granularity information in the width direction. In some embodiments, the computing device 100 obtains the thread granularity information from the shape of the "shape and stride of the thread inside the TLR" (e.g., layoutTVThread). For example, the computing device 100 obtains the thread granularity information of the source tensor data in the height direction (e.g., srcTLRGranH) via the code "srcTLRGranH = srcTensor->getTrait()->layoutTVThread.getShape().get(1, 0)->get().value()". Also, for example, the computing device 100 obtains the thread granularity information of the source tensor data in the width direction (e.g., srcTLRGranW) via the code "srcTLRGranW = srcTensor->getTrait()->layoutTVThread.getShape().get(0, 0)->get().value()".

[0105] Table 6 below schematically illustrates exemplary code for initializing parameters of the minimum repeating unit for the multidimensional tensor data according to some embodiments of the present invention. It should be understood that the code exemplified in Table 6 is merely exemplary, and method 700 can also be implemented by other code.

[0106] Table 6

[0107]

[0108] By employing the above methods, the present invention can effectively obtain Trait information of source tensor data.

[0109] The following will combine Figure 8 A method 800 for generating access sequences of registers of a source object and access sequences of registers of a target object, according to an embodiment of the present invention, is described. Figure 8 A flowchart of a method 800 for generating access sequences of registers with respect to a source object and access sequences of registers with respect to a target object, according to an embodiment of the present invention, is shown. It should be understood that method 800 may, for example, be used in... Figure 1 The described computing device 100 performs the operation. Method 800 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0110] At step 802, the computing device 100 traverses the smallest repeating unit of the source tensor data based on the index of the smallest repeating unit.

[0111] In some embodiments, prior to step 802, method 800 further includes: initializing a register access sequence. The register access sequence includes, for example, a source tensor data register access sequence (such as "srcTlrSequence" in Table 7) and a target object register access sequence (such as "dstTlrSequence" in Table 7).

[0112] In some embodiments, the method for traversing the minimum repeating unit of the source tensor data includes, for example: starting the current traversal along the column direction (or "height direction"), such that the index of the minimum repeating unit in the column direction of the current traversal is "0" (e.g., "hTraitIdx = 0" in Table 7); confirming whether the index of the minimum repeating unit in the column direction of the current traversal is less than the number of minimum repeating units in the column direction (e.g., "dstTraitNumH" in Table 7); if it is confirmed that the index of the minimum repeating unit in the column direction is less than the number of minimum repeating units in the column direction (e.g., "hTraitIdx < dstTraitNumH" in Table 7); incrementing the index of the minimum repeating unit in the column direction of the current traversal (e.g., "++hTraitIdx" in Table 7). Furthermore, for each column-direction traversal, the current traversal is performed along the row direction (or "width direction"), ensuring that the index of the smallest repeating unit in the current row direction is "0" (e.g., "wTraitIdx = 0" in Table 7); it is then checked whether the index of the smallest repeating unit in the current row direction is less than the number of smallest repeating units in the row direction (e.g., "dstTraitNumW" in Table 7); if it is confirmed that the index of the smallest repeating unit in the current row direction is less than the number of smallest repeating units in the row direction (e.g., "wTraitIdx < dstTraitNumW" in Table 7); the index of the smallest repeating unit in the current row direction is incremented (e.g., "++wTraitIdx" in Table 7) until it is confirmed that the index of the smallest repeating unit in the current row direction is equal to the number of smallest repeating units in the row direction.

[0113] At step 804, the computing device 100 performs a traversal of the registers based on the register index for each traversal of the minimum repeating unit.

[0114] In some embodiments, the method for traversing registers includes, for example,: for each traversal of the minimum repeating unit in the row direction, performing the current traversal of registers within the minimum repeating unit in the column direction (or "height direction"), such that the index of the register in the current traversed column direction is "0" (e.g., "hTlrIdx = 0" in Table 7); confirming whether the index of the register in the current traversed column direction is less than the number of registers in the column direction (e.g., "dstTraitTlrSizeInH" in Table 7); if it is confirmed that the index of the register in the current column direction is less than the number of registers in the column direction (e.g., "hTlrIdx < dstTraitTlrSizeInH" in Table 7); incrementing the index of the register in the current traversed column direction (e.g., "++hTlrIdx" in Table 7). Furthermore, for each column-direction register traversal, the current traversal is performed along the row direction (or "width direction"), ensuring that the index of the register in the current row direction is "0" (e.g., "wTlrIdx = 0" in Table 7). It is then checked whether the index of the register in the current row direction is less than the number of registers in the row direction (e.g., "dstTraitTlrSizeInW" in Table 7). If it is confirmed that the index of the register in the current row direction is less than the number of registers in the row direction (e.g., "wTlrIdx < dstTraitTlrSizeInW" in Table 7), the index of the register in the current row direction is incremented (e.g., "++wTlrIdx" in Table 7) until it is confirmed that the index of the register in the current row direction equals the number of registers in the row direction. Thus, this invention can achieve register traversal for the entire tensor data using a four-level loop approach.

[0115] At step 806, the computing device 100 calculates the global register index based on the index of the traversed minimum repeating unit and its offset in the row and column directions, and the index of the traversed register and its offset in the row and column directions from the registers within the minimum repeating unit. By employing the above methods, the present invention can abstract the layout of tensor data from coordinate space to index space, and the mapping process from coordinate space to index space can be precisely described by parameters relating to traits.

[0116] The following example code, which generates an access sequence of registers for a target object, illustrates the method for calculating the global index of a register. Table 7 below schematically shows example code for generating an access sequence of registers for a target object. It should be understood that since this invention is used to move multidimensional tensor data from a source object to a target object, there is a correspondence between the data layouts of the source object and the target object. Therefore, the method for generating an access sequence of registers for the source object is similar to the method for generating an access sequence of registers for the target object.

[0117] Table 7

[0118]

[0119] The method for calculating the register index includes: multiplying the index of the smallest repeating unit in the column direction of the current traversal (e.g., "hTraitIdx" in Table 7) by the column-direction offset of the smallest repeating unit (e.g., "dstTraitOffsetH" in Table 7) to generate the column-direction smallest repeating unit index value; multiplying the index of the smallest repeating unit in the row direction of the current traversal (e.g., "wTraitIdx" in Table 7) by the row-direction offset of the smallest repeating unit (e.g., "dstTraitOffsetW" in Table 7) to generate the row-direction smallest repeating unit index value; and multiplying the register index in the column direction within the currently traversed smallest repeating unit (e.g., "hTlrIdx" in Table 7) by the register index in the column direction within the smallest repeating unit. The register offsets (e.g., "dstTraitTlrOffsetH" in Table 7) are multiplied to generate the register index value in the column direction within the minimum repeating unit; the register index in the row direction within the currently traversed minimum repeating unit (e.g., "wTlrIdx" in Table 7) is multiplied by the register offset in the row direction within the minimum repeating unit (e.g., "dstTraitTlrOffsetW" in Table 7) to generate the register index value in the row direction within the minimum repeating unit; and the minimum repeating unit index values ​​in the column direction, row direction, column direction, and row direction are added together to generate the global register index corresponding to the current traversal based on the addition result.

[0120] The following formula (1) illustrates the algorithm for calculating the global index of a register.

[0121] tlrIdx = hTraitIdx * dstTraitOffsetH +wTraitIdx * dstTraitOffsetW +hTlrIdx * dstTraitTlrOffsetH +wTlrIdx * dstTraitTlrOffsetW (1)

[0122] In formula (1) above, "hTraitIdx" represents the minimum repeating unit index value in the column direction. "wTraitIdx" represents the minimum repeating unit index value in the row direction. "hTlrIdx" represents the register index value in the column direction within the minimum repeating unit. "wTlrIdx" represents the register index value in the row direction within the minimum repeating unit. It should be understood that "hTlrIdx" and "wTlrIdx" can indicate the two-dimensional index of the registers inside the Trait. dstTraitTlrOffsetH represents the offset of the register in the column direction inside the minimum repeating unit. dstTraitTlrOffsetW represents the offset of the register in the row direction inside the minimum repeating unit.

[0123] At step 808, the computing device 100 generates an access sequence for the registers of the source object and an access sequence for the registers of the target object based on the global register indexes calculated during the traversal.

[0124] For example, using the code "dstTlrSequence.push_back(tlrIdx)" shown in Table 7, an access sequence (dstTlrSequence) of the target object's registers is generated based on the register index (tlrIdx) generated in each traversal.

[0125] By adopting the above scheme, the present invention can determine the access sequence of the registers of the source object and the access sequence of the registers of the target object based on the layout of the minimum repeating unit of multidimensional tensor data.

[0126] The various processes and procedures described above, such as methods 200 to 500, can be executed at a computing device. This computing device may include, for example, at least one processor (at least one graphics processor and at least one central processing unit); and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor. In some embodiments, methods 200 to 500 may be implemented as a computer software program or program product tangibly contained in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed on the computing device via read-only memory (ROM) and / or a communication unit. When the computer program is loaded into random-access memory (RAM) and executed by the GPU and CPU, one or more actions of methods 200 to 500 described above can be performed.

[0127] This invention can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention. The computer-readable storage medium may be a tangible device capable of holding and storing instructions used by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0128] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network, to an external computer or external storage device. Various aspects of the invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0129] These computer-readable program instructions can be provided to the central processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the central processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0131] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0132] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors.< / tensorptr>

Claims

1. A method for moving tensor data, characterized in that, include: In response to determining that the source object is multidimensional tensor data, the parameters of the minimum repeating unit of the source object are initialized to obtain the repeating layout pattern of the minimum repeating unit of the source object. The minimum repeating unit describes at least how the multidimensional tensor data is organized in the register. The initialization of the parameters of the minimum repeating unit of the source object includes: in response to determining that the source object is source tensor data with the attribute of multidimensional tensor data, obtaining the repeating layout pattern of the source tensor data; obtaining the offset of the minimum repeating unit of the source tensor data in the height direction and the offset in the width direction. Based on a repeating layout pattern using the minimum repeating unit, registers are traversed to generate access sequences for registers of the source object and for registers of the target object; and Based at least on the generated access sequence of registers about the source object and the access sequence of registers about the target object, perform a tensor data move operation from the source object to the target object.

2. The method according to claim 1, characterized in that, Initializing the parameters of the source object with respect to the smallest repeating unit also includes: Determine whether the source tensor data is a discontinuous minimum repeating unit; Obtain the size and layout information of the registers inside the smallest repeating unit of the source tensor data; Get the offset of the register inside the smallest repeating unit of the source tensor data; Obtain the minimum number of repeating units in the source tensor data; and Get thread granularity information.

3. The method according to claim 2, characterized in that, Based on a repeating layout pattern using the minimum repeating unit, register traversal is performed to generate access sequences for registers of the source object and for registers of the target object, including: Based on the index of the smallest repeating unit, traverse the smallest repeating unit of the source tensor data; For each traversal of the smallest repeating unit, the registers are traversed based on their indices. Calculate the global register index based on the index of the traversed minimum repeating unit and its offsets in the row and column directions, and the index of the traversed registers and their offsets in the row and column directions from the registers within the minimum repeating unit; and Based on the global register indices calculated during the traversal, an access sequence for the registers of the source object and an access sequence for the registers of the target object are generated.

4. The method according to claim 3, characterized in that, Calculating register indices includes: Multiply the index of the smallest repeating cell in the current column direction by the offset of the smallest repeating cell in the column direction to generate the index value of the smallest repeating cell in the column direction. Multiply the index of the smallest repeating unit in the current row direction by the offset of the smallest repeating unit in the row direction to generate the index value of the smallest repeating unit in the row direction. Multiply the register index in the column direction inside the currently traversed minimum repeating unit by the offset of the register in the column direction inside the minimum repeating unit to generate the register index value in the column direction inside the minimum repeating unit. Multiply the register index in the row direction within the currently traversed minimum repeating unit by the offset of the register in the row direction within the minimum repeating unit to generate the register index value in the row direction within the minimum repeating unit; and The minimum repeating unit index in the column direction, the minimum repeating unit index in the row direction, the register index in the column direction inside the minimum repeating unit, and the register index in the row direction inside the minimum repeating unit are added together to generate a global register index corresponding to the current traversal based on the sum.

5. The method according to claim 1, characterized in that, Repeating layout patterns that obtain the minimum repeating unit of the source object include: If it is confirmed that the shape of the multidimensional tensor data is not an integer multiple of the smallest repeating unit, the boundary of the multidimensional tensor data is automatically processed.

6. The method according to claim 2, characterized in that, Repeating layout patterns that obtain the minimum repeating unit of the source object include: If the dimension of the multidimensional tensor data is greater than or equal to three dimensions, reduce the multidimensional tensor data to two-dimensional tensor data; and The two-dimensional tensor data is converted into an organization of multiple repetitions of the smallest repeating unit in order to obtain a repeating layout pattern of the smallest repeating unit of the source object.

7. The method according to claim 2, characterized in that, The layout information of the registers inside the smallest repeating unit of the source tensor data includes: the layout information of the registers in the height direction and the layout information of the registers in the width direction. The offset of the register within the smallest repeating unit of the source tensor data includes: the register's offset in the height direction and the register's offset in the width direction; and The number of minimum repeating units in the source tensor data includes: the number of minimum repeating units in the height direction of the source tensor data, and the number of minimum repeating units in the width direction of the source tensor data.

8. The method according to claim 1, characterized in that, The minimum repeating unit is associated with multiple layout information, including: the name of the minimum repeating unit, the main direction, the layout of threads and values, the shape and step size of the threads inside the register, the number and arrangement of values ​​processed by each thread, and the size of the minimum repeating unit.

9. A computing device, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a machine, performs the method according to any one of claims 1-8.

11. A computer program product, characterized in that, Includes a computer program, which, when executed by a machine, performs the method according to any one of claims 1-8.

Images