Method, computing device, medium and program product for moving tensor data

Abstract

The invention relates to a method, a computing device, a medium and a program product for moving tensor data. The method comprises the steps that in response to the fact that a source object is determined to be multi-dimensional tensor data, initialization about parameters of a minimum repetitive unit is conducted on the source object, so that a repetitive layout mode about the minimum repetitive unit of the source object is obtained, and the minimum repetitive unit at least describes the organization mode of the multi-dimensional tensor data in a register; traversing the registers based on the repetition layout mode of the minimum repetition unit so as to generate an access sequence of the registers related to the source object and an access sequence of the registers related to the target object; and executing a moving operation of tensor data from the source object to the target object at least based on the generated access sequence of the register related to the source object and the access sequence of the register related to the target object. The method can adapt to complex and irregular tensor data layout modes and characteristics of different hardware platforms, and is beneficial to parallel computing efficiency.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to the field of artificial intelligence, and more particularly to a method, a computing device, a computer-readable storage medium, and a computer program product for moving tensor data. Background Art

[0002] Traditional methods for moving tensor data, for example, move tensor data based on the characteristics of the tensor data layout. In this traditional approach, due to the limited ability to describe tensor data layout, it can only describe simple row / column priority or simple blocking patterns, and it is difficult to indicate complex, irregular tensor data layout patterns. In addition, tensor data layout is usually designed for a specific hardware architecture and is difficult to adapt to the characteristics of different hardware platforms, resulting in difficulties in cross-platform performance optimization. Furthermore, it is impossible to accurately describe the mapping relationship between computing units and data blocks, which limits the efficiency of parallel computing.

[0003] In summary, the traditional methods for moving tensor data have the following shortcomings: it is difficult to adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms, and is not conducive to improving the efficiency of parallel computing. Summary of the Invention

[0004] The present invention provides a method, computing device, computer-readable storage medium and computer program product for moving tensor data, which can adapt to complex and irregular tensor data layout patterns and characteristics of different hardware platforms, and are conducive to parallel computing efficiency.

[0005] According to a first aspect of the present invention, a method for moving tensor data is provided. The method includes: in response to determining that a source object is multidimensional tensor data, initializing parameters of a minimum repeating unit for the source object, thereby obtaining a repeating layout pattern of the minimum repeating unit for the source object, wherein the minimum repeating unit at least describes how the multidimensional tensor data is organized in a register; based on the repeating layout pattern of the minimum repeating unit, traversing registers to generate an access sequence for the registers of the source object and an access sequence for the registers of a target object; and performing a move operation of the tensor data from the source object to a destination object based at least on the generated access sequence for the registers of the source object and the access sequence for the registers of the target object.

[0006] In some embodiments, initializing parameters regarding a minimal repeating unit for a source object includes: in response to determining that the source object is source tensor data whose attribute is multidimensional tensor data, obtaining a repeating layout pattern regarding the source tensor data; obtaining an offset in a height direction and an offset in a width direction regarding the minimal repeating unit of the source tensor data; determining whether the source tensor data is a discontinuous minimal repeating unit; obtaining size and layout information regarding registers inside the minimal repeating unit of the source tensor data; obtaining an offset regarding registers inside the minimal repeating unit of the source tensor data; obtaining the number of minimal repeating units regarding the source tensor data; and obtaining thread granularity information.

[0007] In some embodiments, traversing registers based on a repeating layout pattern of minimal repeating units to generate an access sequence for registers of a source object and an access sequence for registers of a target object includes: traversing the minimal repeating unit of source tensor data based on an index of the minimal repeating unit; traversing registers based on a register index for each traversal of the minimal repeating unit; calculating a register global index based on an index of the traversed minimal repeating unit and an offset of the minimal repeating unit in row and column directions, and an index of the traversed register and an offset of the register inside the minimal repeating unit in row and column directions; and generating an access sequence for registers of the source object and an access sequence for registers of the target object based on the register global index calculated during the traversal process.

[0008] In some embodiments, calculating the register index includes: multiplying the index of the minimum repeating unit in the column direction of the current traversal by the offset of the minimum repeating unit in the column direction to generate a minimum repeating unit index value in the column direction; multiplying the index of the minimum repeating unit in the row direction of the current traversal by the offset of the minimum repeating unit in the row direction to generate a minimum repeating unit index value in the row direction; multiplying the register index in the column direction inside the minimum repeating unit of the current traversal by the offset of the register in the column direction inside the minimum repeating unit to generate a register index value in the column direction inside the minimum repeating unit; multiplying the register index in the row direction inside the minimum repeating unit of the current traversal by the offset of the register in the row direction inside the minimum repeating unit to generate a register index value in the row direction inside the minimum repeating unit; and adding the minimum repeating unit index value in the column direction, the minimum repeating unit index value in the row direction, the register index value in the column direction inside the minimum repeating unit, and the register index value in the row direction inside the minimum repeating unit to generate a register global index corresponding to the current traversal based on the addition result.

[0009] In some embodiments, obtaining a repeating layout pattern of a minimal repeating unit of the source object includes: if it is determined that the shape of the multidimensional tensor data is not an integer multiple of the minimal repeating unit, automatically processing a boundary of the multidimensional tensor data.

[0010] In some embodiments, obtaining a repeating layout pattern of a minimum repeating unit with respect to a source object includes: if the dimension of the multidimensional tensor data is greater than or equal to three dimensions, reducing the multidimensional tensor data to two-dimensional tensor data; and converting the two-dimensional tensor data into an organization method of multiple repetitions of the minimum repeating unit so as to obtain a repeating layout pattern of the minimum repeating unit with respect to the source object.

[0011] In some embodiments, the layout information of the registers inside the minimum repeating unit of the source tensor data includes: the layout information of the registers inside the minimum repeating unit of the source tensor data in the height direction and the layout information of the registers in the width direction; the offset of the registers inside the minimum repeating unit of the source tensor data includes: the offset of the registers inside the minimum repeating unit of the source tensor data in the height direction and the offset of the registers in the width direction; and the number of minimum repeating units of the source tensor data includes: the number of minimum repeating units of the source tensor data in the height direction and the number of minimum repeating units of the source tensor data in the width direction.

[0012] In some embodiments, the minimal repeating unit is associated with multiple layout information, including: the name of the minimal repeating unit, the main direction, the layout of threads and values, the shape and stride of the threads inside the register, the number and arrangement of values ​​processed by each thread, and the size of the minimal repeating unit.

[0013] According to a second aspect of the present invention, a computing device is further provided. The computing device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the computing device to perform the method of the first aspect of the present invention.

[0014] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a machine, the method according to the first aspect of the present invention is executed.

[0015] According to a fourth aspect of the present invention, there is further provided a computer program product, comprising a computer program, wherein when the computer program is executed by a machine, the method of the first aspect of the present invention is performed.

[0016] The present invention can convert complex and changeable multidimensional tensor data into multiple repetitions of the minimum repeating unit, and then can flexibly describe different multidimensional tensor data layout patterns through a unified layout rule about the minimum repeating unit, so the present invention can adapt to complex, irregular tensor data layout patterns and the characteristics of different hardware platforms. In addition, the present invention can generate an access sequence for the registers of the source object and the registers of the target object based on the unified abstract layout rule of the repeated layout pattern of the minimum repeating unit that can accurately describe the mapping relationship between multidimensional tensor data and thread registers, thereby realizing the movement of multidimensional tensors. The present invention can not only significantly reduce the programming complexity and maintenance cost of tensor data movement, but also be beneficial to parallel computing efficiency. Therefore, the present invention can adapt to complex, irregular tensor data layout patterns and the characteristics of different hardware platforms, and be beneficial to parallel computing efficiency.

[0017] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages and aspects of the embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements.

[0019] Figure 1 A schematic diagram of a computing device implementing a method for moving tensor data according to an embodiment of the present invention is schematically shown.

[0020] Figure 2 A flowchart of a method for moving tensor data according to an embodiment of the present invention is shown.

[0021] Figure 3 A schematic diagram showing the arrangement of minimal repeating units according to an embodiment of the present invention is shown.

[0022] Figure 4 Schematic diagram showing the main directions of a minimal repeating unit according to some embodiments of the present invention.

[0023] Figure 5 A schematic diagram showing the shape and step size of a TLR internal thread according to an embodiment of the present invention.

[0024] Figure 6 A schematic diagram showing the number and arrangement of values ​​processed by each thread according to some embodiments of the present invention.

[0025] Figure 7A flowchart of a method for initializing parameters related to a minimal repeating unit for a source object according to an embodiment of the present invention is shown.

[0026] Figure 8 A flowchart of a method for generating an access sequence for registers of a source object and an access sequence for registers of a target object according to an embodiment of the present invention is shown.

[0027] In the various drawings, the same or corresponding reference numerals denote the same or corresponding parts. DETAILED DESCRIPTION

[0028] The preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to make the present invention more thorough and complete and to fully convey the scope of the present invention to those skilled in the art.

[0029] As used herein, the term "including" and its variations represent open inclusion, i.e., "including but not limited to." The term "based on" means "based at least in part on." The terms "one example embodiment" and "an embodiment" mean "at least one example embodiment." The term "another embodiment" means "at least one additional embodiment." The terms "first," "second," and so on may refer to different or identical objects.

[0030] As described above, the traditional method for moving tensor data has the following shortcomings: it is difficult to adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms, and is not conducive to improving the efficiency of parallel computing.

[0031] In order to at least partially solve one or more of the above-mentioned problems and other potential problems, an exemplary embodiment of the present invention proposes a scheme for moving tensor data. In this scheme, when determining that the source object is multidimensional tensor data, the parameters of the minimum repeating unit are initialized for the source object, thereby obtaining a repeating layout pattern of the minimum repeating unit of the source object (the minimum repeating unit at least describes the organization of the multidimensional tensor data in the thread register). The present invention can convert complex and changeable multidimensional tensor data into multiple repetitions of the minimum repeating unit, and then can flexibly describe different multidimensional tensor data layout patterns through a unified layout rule for the minimum repeating unit. Therefore, the present invention can adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms. In addition, by traversing the registers based on the repetitive layout pattern of the minimum repetitive unit, an access sequence for the registers of the source object and an access sequence for the registers of the target object are generated; and based on at least the generated access sequence for the registers of the source object and the access sequence for the registers of the target object, a tensor data movement operation from the source object to the destination object is performed. The present invention can generate an access sequence for the registers of the source object and the registers of the target object and realize the movement of multi-dimensional tensors based on the unified abstract layout rule of the repetitive layout pattern of the minimum repetitive unit that can accurately describe the mapping relationship between multi-dimensional tensor data and thread registers. The present invention can not only significantly reduce the programming complexity and maintenance cost of tensor data movement, but also facilitate parallel computing efficiency. Therefore, the present invention can adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms, and facilitate parallel computing efficiency.

[0032] Furthermore, the present invention significantly simplifies the management overhead of the data layout of multi-dimensional tensors, optimizes the data layout, and reduces bandwidth requirements and energy consumption.

[0033] Figure 1 Schematically shows a schematic diagram of a computing device 100 for implementing a method for moving tensor data according to an embodiment of the present invention. Figure 1As shown, the computing device 100 may have one or more processing units, including specialized processing units such as a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a general-purpose graphics processing unit (GPGPU), as well as general-purpose processing units such as a CPU. The computing device 100 also includes at least: a source object's minimal repeating unit layout pattern obtaining unit 102, a source object and a target object register access sequence generating unit 104, and a tensor data move operation executing unit 106. It should be understood that the source object's minimal repeating unit layout pattern obtaining unit 102, the source object and the target object register access sequence generating unit 104, and the tensor data move operation executing unit 106 may be software modules, which, for example, run on one or more processing units configured in the computing device 100.

[0034] A repetitive layout pattern obtaining unit 102 for the minimum repetitive unit of the source object is used to initialize the parameters of the minimum repetitive unit for the source object in response to determining that the source object is multidimensional tensor data, thereby obtaining the repetitive layout pattern of the minimum repetitive unit of the source object, and the repetitive layout pattern of the minimum repetitive unit is used to describe the organization method of multiple repetitions of the minimum repetitive unit.

[0035] The register access sequence generating unit 104 for the source object and the target object is configured to traverse the registers based on the repetitive layout pattern of the minimum repetitive unit to generate an access sequence for the registers of the source object and an access sequence for the registers of the target object.

[0036] The tensor data move operation execution unit 106 is configured to execute a move operation of the tensor data from a source object to a destination object based on at least the generated register access sequence of the source object and the register access sequence of the destination object.

[0037] The following will be combined Figure 2 A method 200 for moving tensor data according to an embodiment of the present invention is described. Figure 2 FIG. 2 is a flow chart of a method 200 for moving tensor data according to an embodiment of the present invention. It should be understood that the method 200 may be used, for example, in Figure 1The method 200 is executed at the described computing device 100. The method 200 may also include additional actions not shown and / or may omit actions shown, and the scope of the present invention is not limited in this respect.

[0038] At step 202, if the computing device 100 determines that the source object is multidimensional tensor data, the parameters of the minimum repeating unit of the source object are initialized to obtain a repeating layout pattern of the minimum repeating unit of the source object, where the minimum repeating unit at least describes how the multidimensional tensor data is organized in the thread register.

[0039] Regarding the method for initializing parameters of a minimum repeating unit for a source object, in some embodiments, it includes, for example: in response to determining that the source object is source tensor data with an attribute of multidimensional tensor data, obtaining a repeating layout pattern of the source tensor data; obtaining an offset in the height direction and an offset in the width direction of the minimum repeating unit of the source tensor data; determining whether the source tensor data is a discontinuous minimum repeating unit; obtaining size and layout information of registers inside the minimum repeating unit of the source tensor data; obtaining an offset of registers inside the minimum repeating unit of the source tensor data; obtaining the number of minimum repeating units of the source tensor data; and obtaining thread granularity information. The following will be combined with Figure 7 The method 700 for initializing the parameters of the minimum repeating unit for the multi-dimensional tensor data is specifically described and will not be repeated here.

[0040] In some embodiments, the method for obtaining a repeating layout pattern of a minimum repeating unit of a source object further includes: if it is confirmed that the shape of the multidimensional tensor data is not an integer multiple of the minimum repeating unit, automatically processing the boundary of the multidimensional tensor data.

[0041] Regarding the minimal repeating unit (Trait), it is the minimal repeating unit used to describe tensor data. Trait at least describes how multi-dimensional tensor data is organized in thread registers. For example, it indicates which data in the tensor data is processed by a register of a certain thread. Trait can also indicate the mapping relationship between tensor data and threads. In some embodiments, Trait is a structure that is stored in the header file of the computing device 100. It should be understood that the computing device 100 can be configured with multiple traits. The Trait of the present invention can be adapted to multiple hardware platforms, and therefore facilitates migration.

[0042] Regarding the repetitive layout pattern of the minimal repeating unit (Trait), it is used to describe the organization of multiple repetitions of the minimal repeating unit. As mentioned earlier, the trait reflects the layout of the minimal repeating unit of tensor data, so different tensor data can be regarded as multiple repetitions of the minimal repeating unit in different ways. Therefore, by describing the repetitive layout pattern of the minimal unit, the layout of the entire tensor data can be inferred. For example, the layout of the entire source tensor data can be inferred from the repetitive layout pattern of the minimal repeating unit of the source tensor data.

[0043] Regarding the method for obtaining a repeating layout pattern for a minimal repeating unit of a source object, in some embodiments, for example, the method includes configuring a repeating layout pattern for the corresponding minimal repeating unit for the source object based on a base minimal repeating unit (BaseTrait). In some implementations, the method for configuring the corresponding minimal repeating unit for the source object includes, for example, configuring the source object's trait name, main direction, thread and value layout, the shape and stride of threads within registers (TLRs), the number and arrangement of values ​​processed by each thread, and the size of the trait.

[0044] Regarding the method of obtaining a repeating layout pattern of a minimum repeating unit regarding a source object, in some embodiments, it includes, for example: if the dimension of the multidimensional tensor data is greater than or equal to three dimensions, reducing the multidimensional tensor data to two-dimensional tensor data; and converting the two-dimensional tensor data into an organization method of multiple repetitions of the minimum repeating unit, so as to obtain a repeating layout pattern of the minimum repeating unit regarding the source object.

[0045] Table 1 below illustrates examples of traits configured in some embodiments.

[0046] Table 1

[0047] As described in Table 1, "matrixTlr64x8Fp32Col" is the name of the configured trait. "H_MAJOR" (i.e., column-major) is the major direction of the configured trait. "Layout(Shape(Shape(4, 8, 4), Shape(1, 2, 2)),Stride(Stride(64, 1, 16), Stride(1, 256, 8)))" is the layout of the threads and values ​​of the configured trait. "Layout(Shape(Shape(4), Shape(8), Shape(4)), Stride(Stride(1), Stride(4),Stride(32)))" is the shape and stride of the threads within the TLR of the configured trait. "Layout(Shape(1, 2, 2),Stride(0, 1, 2)" indicates the number and arrangement of values ​​processed by each configured thread. "Layout(Shape(64, 8),Stride(1, 64)))" indicates the size of the configured trait. The trait configured in Table 1 describes a 64×8 matrix layout. It arranges different traits using column-major order (H_MAJOR). Each trait contains 4×8×4=128 threads. Each thread processes 1×2×2=4 values. A trait can represent a total of 64×8=512 FP32 data.

[0048] In this way, it is possible to use rich features to build layout features of different traits for different types of tensor data.

[0049] Regarding the method of constructing a trait, it includes, for example: abstracting the repeated units in the tensor data. For example, a thread has 128 or 256 registers. In some embodiments, 4 registers are grouped together, corresponding to 1 trait in the first type of trait. For example, if the current tensor data requires 96 registers for each thread, the layout of the tensor data can be regarded as a repeated layout of 24 first type traits. For another example, if 2 registers are grouped together, corresponding to 1 trait in the second type of trait, the layout of the current tensor data can be regarded as a repeated layout of 48 second type traits.

[0050] It should be understood that traits can describe multiple levels of layout features. For example, they can describe the thread layout, value layout, and overall shape of tensor data. Therefore, the present invention, based on traits, can precisely control how tensor data is organized at different levels of abstraction.

[0051] The minimal repeating unit (Trait) includes multiple layout information, including the name (name), major direction (majorDirection), layout of threads and values ​​(layoutTv), shape and stride of threads within the TLR (layoutTvThread), number and arrangement of values ​​processed by each thread (layoutValuesPerThread), and size (hw).

[0052] Regarding the name, it is the identifier of the smallest repeating unit. The name of the trait is used for identification and debugging.

[0053] The major direction (majorDirection) is used to describe the arrangement of traits, for example, row-major order (W_MAJOR, also known as "width-major order") or column-major order (H_MAJOR, also known as "height-major order"). Figure 3 Schematic diagram showing the arrangement of the minimum repeating units according to an embodiment of the present invention. Figure 3 As shown, the left part of the figure indicated by mark 310 shows the situation where the arrangement between traits is row-major order, that is, majorDirection=W_MAJOR. Figure 3 As shown in the left part of the diagram, one trait corresponds to one computing unit (CU). One CU includes four sub-computing units (EUs). Trait 302 corresponds to one CU, and the four color blocks included in Trait 302 correspond to four EUs. Mark 330 indicates r0 in the first color block included in Trait 302. r0 corresponds to 32 threads (the 32 threads are, for example, Figure 4 T0 to T31) in the zeroth register (such as Figure 4 V0 shown). Figure 4 Schematic diagram of the main direction of the minimum repeating unit according to some embodiments of the present invention is shown. Similarly, the mark 332 indicates r1 in the first color block included in Trait 302. r1 corresponds to 32 threads (the 32 threads are, for example, Figure 4 T0 to T31) in the first register (as shown Figure 4 The size of a trait is, for example, 64*8. The arrow direction indicated by mark 312 indicates that the next trait 304 is arranged in the direction of the row extension, that is, the next trait 304 is arranged to the right of the current trait 302.

[0054] exist Figure 3In FIG. 3 , the right portion of the diagram indicated by mark 320 shows the case where the arrangement between the minimum repeating units is column-major order, that is, majorDirection=H_MAJOR. Figure 3 As shown in the right portion of the diagram, the trait size is also, for example, 64*8. The arrow indicated by marker 322 indicates that the next trait 304 is arranged in the direction of the column extension, that is, the next trait 304 is arranged below the current trait 302. It should be understood that the present invention can guide the arrangement of different traits by controlling the major direction (majorDirection) attribute, thereby achieving more flexible organization of large-scale tensor data.

[0055] The following combination Figure 4 Describes the layout of threads and values. Figure 4 A schematic diagram of the layout of threads and values ​​according to an embodiment of the present invention is shown. Regarding the layout of threads (Thread) and values ​​(Value), it indicates the correspondence between tensor data and the threads and registers that process the tensor data. In some embodiments, the shape of the thread (Thread) dimension is configured as " / / T (Thread) dimension Shape (4,8,4): Shape (64,1,16)". "Shape (4,8,4)" indicates that there are 32 threads inside each EU, and its shape is: 4 columns and 8 rows. Each Trait corresponds to 4 EUs. The shape of the value (Thread) dimension is configured as " / / V (Value) dimension Shape (1,2,2): Shape (1,256,8)". As shown Figure 4 As shown, the left part 402 indicates multiple traits. The right part indicates the correspondence between the threads and registers used to process the tensor data of the corresponding traits. Among them, the threads and registers used to process the tensor data of the trait corresponding to r0 are shown in the dotted box indicated by the arrow 422. In other words, the tensor data of the trait corresponding to r0 is processed by the register V0 of the 32 threads T0 to T31. Similarly, the tensor data of the trait corresponding to r1 is processed by the register V1 of the 32 threads T0 to T31. The tensor data of the trait corresponding to r2 is processed by the register V2 of the 32 threads T0 to T31. The tensor data of the trait corresponding to r3 is processed by the register V3 of the 32 threads T0 to T31. As shown Figure 4As shown on the right side of the , T represents a thread, for example, T0 represents thread T0 (i.e., the zeroth thread). V represents a register, for example, V0 represents register V0 (i.e., the zeroth register). T0 to T31 represent 32 threads, i.e., threads 0 to 31. Each of these 32 threads is configured with register V0, which means that a trait can process 8*4 (i.e., 8 rows and 4 columns) data. If 8*8 data needs to be processed, each of these 32 threads T0 to T31 must be configured with registers V0 and V1. For example, for data in FP32 format, one thread stores one piece of data. The stride of the padding data corresponding to the arrow indicated by marker 410 is, for example, the default value (e.g., "0"). The stride of the padding data corresponding to the arrow indicated by marker 412 is "1" (i.e., stride = 1). The stride of the padding data corresponding to the arrow indicated by marker 414 is "8" (i.e., stride = 8). Because the number of data in a column is 64, the step length of the padding data corresponding to the arrow indicated by mark 416 is "64" (i.e., stride=64). The step length of the padding data corresponding to the arrow indicated by mark 418 is "16" (i.e., stride=16), which actually represents the step length of the padding data at the corresponding position of two adjacent EUs. Since the number of data in a column is 64, from r0 to r1, it needs to span 4 columns. Therefore, the step length of the padding data corresponding to the arrow indicated by mark 420 is "256" (i.e., stride=256).

[0056] The following combination Figure 5 Describes the shape and stride of the TLR internal threads. Figure 5 Schematic diagram showing the shape and step length of the TLR internal thread according to an embodiment of the present invention. Figure 5 Configuration of the shape and stride of the TLR internal threads shown.

[0057] Table 2

[0058] As shown in Table 2, the shape "Shape(Shape(4), Shape(8), Shape(4))" represents that 4*8=32 threads are configured in one EU, and 4 EUs are configured in one CU, thus having 128 threads.

[0059] like Figure 5As shown, the ID of the TLR internal thread corresponding to the arrow indicated by mark 510 goes from "0" to "1" (i.e., from thread T0 to thread T1) with a step length of "1" (i.e., "Stride(1)"). The ID of the TLR internal thread corresponding to the arrow indicated by mark 512 goes from "0" to "4" (i.e., from thread T0 to thread T4) with a step length of "4" (i.e., "Stride(4)"). The ID of the TLR internal thread corresponding to the arrow indicated by mark 514 goes from "0" of one EU to "0" of the adjacent EU (i.e., from thread T0 of the current EU to the next thread T0). Given that one EU has 32 threads, the step length of the internal thread corresponding to the arrow indicated by mark 514 is "32" (i.e., "Stride(32)"). The TLR internal thread ID corresponding to the arrow indicated by mark 516 is from thread T0 of one CU to thread T0 of an adjacent CU, and the corresponding internal thread stride is “32” (ie, “Stride(32)”).

[0060] The following table 3 shows an example Figure 6 The number of values ​​processed by each thread and the configuration of the layout (layoutValuesPerThread) are shown.

[0061] Table 3

[0062] As shown in Table 3, “Shape(1,2,2)” represents the shape of the trait in terms of register dimensions. Figure 6 As shown in FIG, a EU is configured with a register shape of 2 rows and 2 columns. Figure 6 For example, the first row is V0 and V1, and the step length between adjacent registers in the same row is "1"; the first column is V0 and V2, and the step length between adjacent registers in the same column is "2".

[0063] The following combination Figure 6 Specifies the number and layout of values ​​processed by each thread (layoutValuesPerThread). Figure 6 Schematic diagram showing the number and arrangement of values ​​processed by each thread according to some embodiments of the present invention. Figure 6As shown, the 32 threads involved in the range indicated by marker 610 correspond to the 32 threads of tensor data corresponding to r0, and these 32 threads are processed by register V0 (i.e., register 0). The 32 threads involved in the range indicated by marker 612 correspond to the 32 threads of tensor data corresponding to r1, and these 32 threads are processed by register 1 (i.e., register 1) represented by V1. Similarly, the 32 threads involved in the range indicated by marker 614 correspond to the 32 threads of tensor data corresponding to the trait of r2. These 32 threads are processed by register V2 (i.e., register 2). The 32 threads involved in the range indicated by marker 616 correspond to the 32 threads of tensor data corresponding to r3. These 32 threads are processed by register V3 (i.e., register 3). Marker 620 indicates the step length from register V0 corresponding to the range indicated by marker 610 to register V1 corresponding to the range indicated by marker 612. This step length is "1," meaning that the step length between adjacent registers in the same row is "1." Mark 622 indicates a step length from register V0 corresponding to the range indicated by mark 610 to register V2 corresponding to the range indicated by mark 614 , which is “2”, that is, a step length between adjacent registers in the same column is “1”.

[0064] Regarding size (hw), it indicates the size of the Trait, such as Figures 3 to 6 As shown, the size of each Trait is 64*8.

[0065] At step 204 , the computing device 100 traverses the registers based on the repetitive layout pattern of the minimal repetitive unit to generate an access sequence for the registers of the source object and an access sequence for the registers of the target object.

[0066] Regarding the method for generating an access sequence for registers of a target object, it includes, for example: the computing device 100 traverses the minimal repeating unit of the source tensor data based on the index of the minimal repeating unit; for each traversal of the minimal repeating unit, traverses the register based on the register index; and calculates the register global index based on the index of the traversed minimal repeating unit and the offset of the minimal repeating unit in the row direction and the column direction, and the index of the traversed register and the offset of the register inside the minimal repeating unit in the row direction and the column direction; and based on the register global index calculated during the traversal, generates an access sequence for the registers of the source object and an access sequence for the registers of the target object. The following will be combined with Figure 8 The method 800 for generating an access sequence for a register of a target object is described in detail, and will not be repeated here.

[0067] At step 206 , the computing device 100 performs a move operation of tensor data from the source object to the destination object based on at least the generated access sequence of registers of the source object and the generated access sequence of registers of the target object.

[0068] Table 4 below schematically illustrates exemplary code for performing a move operation of tensor data from a source object to a destination object according to some embodiments of the present invention.

[0069] Table 4

[0070] As shown in Table 4, a method for performing a move operation of tensor data from a source object to a destination object includes, for example, determining, in response to determining that the source object and the destination object are of the same type of data, whether the source tensor data is a discontinuous minimal repeating unit; if the source tensor data is determined to be a discontinuous minimal repeating unit, calculating, for each TLR within a total number of TLRs, a register index of the source object (e.g., "src_reg") and a register index of the destination object (e.g., "dst_reg"), and performing the move tensor data operation based on the calculated register index of the source object and the register index of the destination object.

[0071] Regarding the method for calculating the register index of a target object, in some embodiments, for example, the method includes: if the computing device 100 determines that "dst_is_x_reg" is true, selecting a target register from a closed segment register set (e.g., x_registers). "dst_is_x_reg" is an indicator indicating whether the target tensor data (dst tensor) is associated with a closed segment register (x0-x3). Closed segment registers are used to alleviate TLR pressure. It should be understood that closed segment registers x0-x3 provide four 32-bit temporary registers for each thread. This can reduce TLR port conflicts, lower register pressure, and save power. For example, through the code “dst_reg = x_registers[dst_x_base + dst_tlr_seq[i]] if dst_is_x_reg \ else r_registers[dst_r_base + dst_tlr_seq[i]]” in Table 4, the target base index of the closed segment register (e.g., “dst_x_base”) is added to the offset in the access sequence of the register of the target object (e.g., “dst_tlr_seq[i]”) to obtain the register index of the target object; and if the computing device 100 determines that “dst_is_x_reg” is false, the target register is calculated from the r register group (r_registers). For example, through the code “r_registers[dst_r_base + dst_tlr_seq[i]]” in Table 4, the target base index of the r register (e.g., “dst_r_base”) is added to the offset in the access sequence of the register of the target object (e.g., “dst_tlr_seq[i]”), thereby obtaining the register index of the target object.

[0072] Regarding the method for calculating the register index of the source object, in some embodiments, for example, the method includes: if the computing device 100 determines that "src_is_x_reg" is true, selecting a target register from a closed segment register group (e.g., x_registers). "src_is_x_reg" is a flag indicating whether the source tensor data (src tensor) is associated with a closed segment register. For example, through the code “src_reg = x_registers[src_x_base + src_tlr_seq[i]] if src_is_x_reg \ else r_registers[src_r_base + src_tlr_seq[i]]” in Table 4, the closed segment register source base index (e.g., “src_x_base”) is added to the offset in the access sequence of the register of the source object (e.g., “src_tlr_seq[i]”) to obtain the register index of the source object; and if the computing device 100 determines that “src_is_x_reg” is false, the source register is calculated from the r register group (r_registers). For example, through the code “r_registers[src_r_base + src_tlr_seq[i]]” in Table 4, the r register source base index (e.g., “src_r_base”) is added to the offset in the access sequence of the register of the source object (e.g., “src_tlr_seq[i]]”), thereby obtaining the register index of the source object.

[0073] Regarding the method of moving tensor data from a source object to a destination object, in some embodiments, it includes, for example: the computing device 100 uses the code "mih.mov(dst_reg, src_reg, mov_mod)" in Table 4 to implement the movement operation of tensor data based on the calculated register index of the target object and the register index of the source object.

[0074] In some embodiments, a method for performing a move operation of tensor data from a source object to a destination object, for example, further includes: in response to the computing device 100 determining that the source object and the destination object are of the same type of data and that the source object is not a discontinuous minimal repeating unit, batch moving the tensor data. Specifically, regarding the method of moving the batch tensor data, for example, it includes: the computing device 100 calculates a size for batch moving the tensor data based on the total number of TLRs; loops within a range from "0" to the number of registers to be processed in the register access sequence, with the calculated size of the batch moving tensor data as a step size; and calculates the register index of the destination object and the register index of the source object in each loop to batch move the tensor data.

[0075] Regarding the method for calculating the size for batch moving of tensor data, for example, the method includes: the computing device 100 calculates the size for batch moving of tensor data through the exemplary code "burst_size = calculate_max_burst(tlr_num)". Wherein, "burst_size" represents the size of batch moving of tensor data. "calculate_max_burst()" represents a function. "tlr_num" represents the number of registers to be processed in the register access sequence.

[0076] Regarding the method of looping with the calculated size of the batch moving tensor data as the step size to batch move tensor data, for example, it includes: the computing device 100 starts from "0" and ends at tlr_num, stepping burst_size each time. For example, if tlr_num=100 and burst_size=16, the offset i takes the value of "0, 16, 32,..., 96". In some embodiments, the computing device 100 calculates the register index "dst_reg" of the target object by the code "dst_reg = r_registers[dst_r_base + i]". The register index "src_reg" of the source object is calculated by the code "src_reg = r_registers[src_r_base + i]". And the tensor data is batch moved by the code "mih.mov(dst_reg, src_reg, mov_mod)".

[0077] Table 5 below schematically illustrates exemplary code for performing a move operation of tensor data from a source object to a destination object according to some embodiments of the present invention.

[0078] Table 5

[0079] As shown in Table 5, the method for performing a move operation of tensor data from a source object to a destination object, for example, also includes: if the computing device 100 determines that the data type of the source object is different from the data type of the target object. For example, if the data type of the source object is 32 bits (for example, as indicated by the code “src_data_size == 4”), and the data type of the target object is 16 bits (for example, as indicated by the code “dst_data_size == 2”), then, within the range of the number of registers to be processed in the register access sequence, the r register target base index “dst_r_base” is added to the offset “dst_tlr_seq[i]” in the access sequence for the register of the target object to obtain the index of the target register (for example, as indicated by the code “dst_reg = r_registers[dst_r_base + dst_tlr_seq[i]]”); based on the r register source base index “src_r_base”, the offset “src_tlr_seq[2*i]” in the access sequence for the register of the source object is added to obtain the register index of the target object (as indicated by the code “src_reg = r_registers[src_r_base +src_tlr_seq[2*i]]”); and moving the tensor data based on the obtained register index of the target object and the register index of the source object (for example, moving the tensor data from 32 bits to 16 bits is achieved by the code “mih.mov(dst_reg, src_reg, mov_mod)”).

[0080] In the above scheme, the present invention can convert complex and changeable multi-dimensional tensor data into multiple repetitions of the minimum repeating unit, and then can flexibly describe different multi-dimensional tensor data layout patterns through a unified layout rule about the minimum repeating unit, so the present invention can adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms. In addition, the present invention can generate an access sequence for the registers of the source object and the registers of the target object based on the unified abstract layout rule of the repeated layout pattern of the minimum repeating unit that can accurately describe the mapping relationship between multi-dimensional tensor data and thread registers, thereby realizing the movement of multi-dimensional tensors. The present invention can not only significantly reduce the programming complexity and maintenance cost of tensor data movement, but also be beneficial to parallel computing efficiency. Therefore, the present invention can adapt to complex and irregular tensor data layout patterns and the characteristics of different hardware platforms, and be beneficial to parallel computing efficiency.

[0081] The following will be combined Figure 7 A method 700 for initializing parameters related to a minimal repeating unit for a source object according to an embodiment of the present invention is described. Figure 7FIG. 7 is a flow chart showing a method 700 for initializing parameters of a minimal repeating unit for a source object according to an embodiment of the present invention. It should be understood that the method 700 can be implemented in Figure 1 The method 700 is executed at the described computing device 100. The method 700 may also include additional actions not shown and / or may omit actions shown, and the scope of the present invention is not limited in this respect.

[0082] In some embodiments, a function "initSDLayoutParam" for Trait initialization is first defined, which belongs to the "MovCvt" class and has a return type of "status_t".

[0083] At step 702 , in response to determining that the source object is source tensor data whose attribute is multi-dimensional tensor data, the computing device 100 obtains a repeating layout pattern regarding the source tensor data.

[0084] In some embodiments, if the computing device 100 determines that the source object (the source object is, for example, called "src") is the source tensor data (or called "srcTensor" or "src tensor"), for example, the code "TensorPtr srcTensor = std::get <tensorptr>(src)” obtains the tensor pointer (e.g., “TensorPtr”) as the source object and assigns it to the tensor pointer of the source tensor data (srcTensor). Furthermore, the computing device 100 calls the function “getRepeatTraitLayout()” through the code “srcRepeat = getRepeatTraitLayout(srcTensor)” to extract the repeated layout information of the Trait from the tensor data (srcTensor), thereby obtaining the repeated layout pattern of the source tensor data.

[0085] The repeat layout pattern of the source tensor data refers to the repeat layout information of traits in the source tensor data. Trait repeat layout information includes at least the trait name (name), major direction (majorDirection), layout of threads and values ​​(layoutTv), shape and stride of threads within the TLR (layoutTvThread), number and arrangement of values ​​processed by each thread (layoutValuesPerThread), and size (hw) mentioned above.

[0086] At step 704 , the computing device 100 obtains an offset in the height direction and an offset in the width direction of the minimum repeating unit of the source tensor data.

[0087] The offset in the height direction (e.g., "srcTraitOffsetH" in Table 6) and the offset in the width direction (e.g., "srcTraitOffsetW" in Table 6) of the minimum repeating unit of the source tensor data are determined based on, for example, the stride of Trait in the column direction and the stride of Trait in the row direction.

[0088] For example, the computing device 100 obtains the offset of the Trait of the source tensor data in the height direction (srcTraitOffsetH) through the code "srcTraitOffsetH = srcRepeat.getStride().get(0,0)->get().value()"; and obtains the offset of the Trait of the source tensor data in the width direction (srcTraitOffsetW) through the code "srcTraitOffsetW=srcRepeat.getStride().get(0,1)->get().value()".

[0089] At step 706 , the computing device 100 determines whether the source tensor data is a discontinuous minimal repeating unit.

[0090] For example, the computing device 100 determines whether the source tensor data is a discontinuous minimal repeating unit (a discontinuous minimal repeating unit or a "Gap Trait"). If the source tensor data is determined to be a discontinuous minimal repeating unit, the determination result is stored in a Boolean variable (e.g., "srcIsGapTrait" in Table 6). In some embodiments, the computing device 100 determines whether the source tensor data is a discontinuous minimal repeating unit, for example, via the code "srcIsGapTrait = isGapTrait(srcTensor->getTrait())".

[0091] At step 708 , the computing device 100 obtains size and layout information of registers inside the minimal repeating unit of the source tensor data.

[0092] The layout information of the registers inside the minimum repeating unit of the source tensor data includes, for example, the layout information of the registers inside the Trait of the source tensor data in the height direction and the layout information of the registers inside the Trait in the width direction.

[0093] For example, the computing device 100 obtains the size of the registers inside the Trait of the source tensor data (for example, srcTraitILRSize) via the code "srcTraitTLRSize = getTraitTLRSize(srcTensor->getTrait())". The computing device 100 obtains the layout information of the registers inside the Trait of the source tensor data in the height direction (for example, srcTraitTLRSizeInH) via the code "srcTraitTLRSizeInH = getTraitTLRSizeInH(srcTensor->getTrait())". The computing device 100 obtains the layout information of the registers inside the Trait of the source tensor data in the width direction (for example, srcTraitTLRSizeInM) via the code "srcTraitTLRSizeInM = getTraitTLRSizeInM(srcTensor->getTrait())".

[0094] At step 710 , the computing device 100 obtains the offset of the register inside the minimal repeating unit with respect to the source tensor data.

[0095] Regarding the offset of the register inside the minimum repeating unit of the source tensor data, it includes, for example: the offset of the register inside the Trait of the source tensor data in the height direction, and the offset of the register inside the Trait in the width direction. In some embodiments, the offset of the register inside the minimum repeating unit of the source tensor data is obtained based on the stride of the thread layout inside the Trait. For example, the computing device 100 obtains the offset of the register inside the Trait of the source tensor data in the height direction (for example, srcTraitTLROffsetH) via the code "srcTraitTLROffsetH = srcTensor->getTrait()->layoutValuesPerThread.getStride().get(2, 0)". Furthermore, the computing device 100 obtains the offset of the register inside the Trait of the source tensor data in the width direction (for example, srcTraitTLROffsetH) via the code "srcTraitTLROffsetM = srcTensor->getTrait()->layoutValuesPerThread.getStride().get(1, 0)".

[0096] At step 712 , the computing device 100 obtains the number of minimum repeating units regarding the source tensor data.

[0097] Regarding the number of minimum repeating units of the source tensor data, it includes, for example: the number of traits of the source tensor data in the height direction, and the number of traits in the width direction. In some embodiments, the number of minimum repeating units of the source tensor data is the number of traits obtained from the shape (Shape) of the repeated layout pattern of the source tensor data (e.g., srcRepeat). For example, the computing device 100 obtains the number of traits of the source tensor data in the height direction (e.g., srcTraitNumH) via the code "srcTraitNumH = srcRepeat.getShape().get(0, 0)->get().value()". For another example, the computing device 100 obtains the number of traits of the source tensor data in the width direction (e.g., srcTraitNumW) via the code "srcTraitNumW = srcRepeat.getShape().get(0, 1)->get().value()".

[0098] At step 714 , the computing device 100 obtains thread granularity information.

[0099] The thread granularity information about the source tensor data includes, for example, thread granularity information of the source tensor data in the height direction and thread granularity information in the width direction. In some embodiments, the computing device 100 obtains the thread granularity information from the shape of the "shape and stride of the TLR internal thread" (for example, layoutTVThread). For example, the computing device 100 obtains the thread granularity information of the source tensor data in the height direction (for example, srcTLRGranH) via the code "srcTLRGranH = srcTensor->getTrait()->layoutTVThread.getShape().get(1, 0)->get().value()". For another example, the computing device 100 obtains the thread granularity information of the source tensor data in the width direction (for example, srcTLRGranW) via the code "srcTLRGranW = srcTensor->getTrait()->layoutTVThread.getShape().get(0, 0)->get().value()".

[0100] Table 6 below schematically illustrates exemplary code for initializing parameters of a minimal repeating unit for the multidimensional tensor data according to some embodiments of the present invention. It should be understood that the code illustrated in Table 6 is merely exemplary, and method 700 may also be implemented using other code.

[0101] Table 6

[0102] By adopting the above means, the present invention can effectively obtain the Trait information of the source tensor data.

[0103] The following will be combined Figure 8 A method 800 for generating an access sequence with respect to registers of a source object and an access sequence with respect to registers of a target object according to an embodiment of the present invention is described. Figure 8 FIG. 8 is a flow chart showing a method 800 for generating an access sequence for a register of a source object and an access sequence for a register of a target object according to an embodiment of the present invention. It should be understood that the method 800 may be performed, for example, in Figure 1 The method 800 is executed at the described computing device 100. The method 800 may also include additional actions not shown and / or may omit actions shown, and the scope of the present invention is not limited in this respect.

[0104] At step 802 , the computing device 100 traverses the minimal repeating unit of the source tensor data based on the index of the minimal repeating unit.

[0105] In some embodiments, before step 802, method 800 further includes: initializing a register access sequence. The register access sequence includes, for example: a source tensor data register access sequence (such as "srcTlrSequence" in Table 7) and a target object register access sequence (such as "dstTlrSequence" in Table 7). In some embodiments, a method for traversing the minimum repeating unit of source tensor data includes, for example: starting the current traversal in the column direction (or "height direction"), so that the index of the minimum repeating unit in the column direction of the current traversal is "0" (for example, "hTraitIdx = 0" in Table 7); confirming whether the index of the minimum repeating unit in the column direction of the current traversal is less than the number of minimum repeating units in the column direction (the number of minimum repeating units in the column direction is, for example, "dstTraitNumH" in Table 7); if it is confirmed that the index of the minimum repeating unit in the column direction is less than the number of minimum repeating units in the column direction (for example, "hTraitIdx < dstTraitNumH" in Table 7); accumulating the index of the minimum repeating unit in the column direction of the current traversal (for example, "++hTraitIdx" in Table 7). Also, for each traversal in the column direction, the current traversal is performed in the row direction (or "width direction"), so that the index of the minimum repeating unit in the row direction of the current traversal is "0" (for example, "wTraitIdx = 0" in Table 7); confirm whether the index of the minimum repeating unit in the row direction of the current traversal is less than the number of the minimum repeating units in the row direction (the number of the minimum repeating units in the row direction is, for example, "dstTraitNumW" in Table 7); if it is confirmed that the index of the minimum repeating unit in the row direction of the current traversal is less than the number of the minimum repeating units in the row direction (for example, "wTraitIdx < dstTraitNumW" in Table 7); accumulate the index of the minimum repeating unit in the row direction of the current traversal (for example, "++wTraitIdx" in Table 7) until it is confirmed that the index of the minimum repeating unit in the row direction of the current traversal is equal to the number of the minimum repeating units in the row direction.

[0106] At step 804 , the computing device 100 traverses the registers based on the register index for each traversal of the minimal repeating unit.

[0107] In some embodiments, a method for traversing registers may include, for example, the following steps: for each traversal of a minimal repeating unit in the row direction, starting a current traversal of registers within the minimal repeating unit in the column direction (or "height direction"), such that the index of the register in the column direction currently traversed is "0" (e.g., "hTlrIdx = 0" in Table 7); confirming whether the index of the register in the column direction currently traversed is less than the number of registers in the column direction (the number of registers in the column direction is, for example, "dstTraitTlrSizeInH" in Table 7); if it is confirmed that the index of the register in the column direction is less than the number of registers in the column direction (e.g., "hTlrIdx < dstTraitTlrSizeInH" in Table 7); accumulating the index of the register in the column direction currently traversed (e.g., "++hTlrIdx" in Table 7). Furthermore, for each traversal of the registers in the column direction, the current traversal is performed in the row direction (or "width direction"), so that the index of the register in the row direction currently traversed is "0" (for example, "wTlrIdx = 0" in Table 7); a check is made as to whether the index of the register in the row direction currently traversed is less than the number of registers in the row direction (for example, the number of registers in the row direction is "dstTraitTlrSizeInW" in Table 7); if it is confirmed that the index of the register in the row direction currently traversed is less than the number of registers in the row direction (for example, "wTlrIdx < dstTraitTlrSizeInW" in Table 7), the index of the register in the row direction currently traversed is accumulated (for example, "++wTlrIdx" in Table 7) until it is confirmed that the index of the register in the row direction currently traversed is equal to the number of registers in the row direction. Thus, the present invention can implement register traversal for the entire tensor data through a four-layer loop.

[0108] At step 806, the computing device 100 calculates the register global index based on the index of the traversed minimal repeating unit and the offset of the minimal repeating unit in the row and column directions, as well as the index of the traversed register and the offset of the register within the minimal repeating unit in the row and column directions. By adopting the above-mentioned means, the present invention can abstract the layout of tensor data from the coordinate space to the index space, and can accurately describe the mapping process from the coordinate space abstraction to the index space through the parameters of the trait.

[0109] The following uses exemplary code for generating an access sequence for a register of a target object as an example to illustrate a method for calculating a global register index. Table 7 below schematically illustrates exemplary code for a method for generating an access sequence for a register of a target object. It should be understood that since the present invention is used to move multidimensional tensor data from a source object to a target object, there is a corresponding relationship between the data layouts of the source object and the target object. Therefore, the method for generating an access sequence for a register of a source object is similar to the method for generating an access sequence for a register of a target object.

[0110] Table 7

[0111] Regarding the method for calculating the register index, it includes: multiplying the index of the minimum repeating unit in the column direction of the current traversal (for example, "hTraitIdx" in Table 7) by the offset of the minimum repeating unit in the column direction (for example, "dstTraitOffsetH" in Table 7) to generate the minimum repeating unit index value in the column direction; multiplying the index of the minimum repeating unit in the row direction of the current traversal (for example, "wTraitIdx" in Table 7) by the offset of the minimum repeating unit in the row direction (for example, "dstTraitOffsetW" in Table 7) to generate the minimum repeating unit index value in the row direction; multiplying the register index in the column direction inside the minimum repeating unit of the current traversal (for example, "hTlrIdx" in Table 7) by the register index in the column direction inside the minimum repeating unit The register index in the row direction of the currently traversed minimal repeating unit (e.g., "wTlrIdx" in Table 7) is multiplied by the offset of the register in the row direction of the minimal repeating unit (e.g., "dstTraitTlrOffsetW" in Table 7) to generate the register index value in the row direction of the minimal repeating unit; and the minimal repeating unit index value in the column direction, the minimal repeating unit index value in the row direction, the register index value in the column direction of the minimal repeating unit, and the register index value in the row direction of the minimal repeating unit are added to generate the register global index corresponding to the current traversal based on the addition result.

[0112] The following formula (1) illustrates the calculation algorithm of the register global index.

[0113] tlrIdx = hTraitIdx * dstTraitOffsetH +wTraitIdx * dstTraitOffsetW +hTlrIdx * dstTraitTlrOffsetH +wTlrIdx * dstTraitTlrOffsetW (1) In the above formula (1), "hTraitIdx" represents the minimum repeating unit index value in the column direction. "wTraitIdx" represents the minimum repeating unit index value in the row direction. "hTlrIdx" represents the register index value in the column direction inside the minimum repeating unit. "wTlrIdx" represents the register index value in the row direction inside the minimum repeating unit. It should be understood that "hTlrIdx" and "wTlrIdx" can indicate the two-dimensional index of the register inside the Trait. dstTraitTlrOffsetH represents the offset of the register in the column direction inside the minimum repeating unit. dstTraitTlrOffsetW represents the offset of the register in the row direction inside the minimum repeating unit.

[0114] At step 808 , the computing device 100 generates an access sequence for registers of the source object and an access sequence for registers of the target object based on the register global index calculated during the traversal process.

[0115] For example, the code “dstTlrSequence.push_back(tlrIdx)” shown in Table 7 is used to generate an access sequence (dstTlrSequence) of registers of the target object based on the register index (tlrIdx) generated in each traversal.

[0116] By adopting the above solution, the present invention can arrange the access sequence of the registers of the source object and the access sequence of the registers of the target object based on the layout of the smallest repeating unit of the multi-dimensional tensor data.

[0117] The various processes and processing described above, such as methods 200 through 500, may be performed on a computing device. The computing device may include, for example, at least one processor (at least one graphics processor and at least one central processing unit); and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor. In some embodiments, methods 200 through 500 may be implemented as a computer software program or program product tangibly embodied on a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed onto the computing device via read-only memory (ROM) and / or a communication unit. When the computer program is loaded into random-access memory (RAM) and executed by the GPU and CPU, one or more actions of methods 200 through 500 described above may be performed.

[0118] The present invention may be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions for performing various aspects of the present invention. The computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0119] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. Various aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each block in the flowcharts and / or block diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0120] These computer-readable program instructions can be provided to a central processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine such that when these instructions are executed by the central processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0121] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented using a dedicated hardware-based system that performs the prescribed function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0122] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0123] The above specific embodiments do not constitute a limitation on the scope of protection of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors.< / tensorptr>

Claims

1. A method for moving tensor data, characterized in that include: In response to determining that the source object is multidimensional tensor data, initializing parameters of a minimal repeating unit for the source object, thereby obtaining a repeating layout pattern of the minimal repeating unit of the source object, wherein the minimal repeating unit at least describes an organization manner of the multidimensional tensor data in a register; Based on the repetitive layout pattern of the minimal repetitive unit, traversing the registers to generate an access sequence for the registers of the source object and an access sequence for the registers of the target object; as well as A move operation of tensor data from the source object to the destination object is performed based on at least the generated access sequence with respect to the registers of the source object and the generated access sequence with respect to the registers of the target object.

2. The method according to claim 1, characterized in that Initializing the parameters of the minimum repeating unit for the source object includes: In response to determining that the source object is source tensor data whose attribute is multi-dimensional tensor data, obtaining a repeating layout pattern with respect to the source tensor data; Get the offset of the smallest repeating unit of the source tensor data in the height direction and the offset in the width direction; Determine whether the source tensor data is a discontinuous minimal repeating unit; Get information about the size and layout of registers inside the minimal repeating unit of the source tensor data; Get the offset of the register inside the minimal repeating unit with respect to the source tensor data; Get the number of minimal repeating units for the source tensor data; and Get thread granularity information.

3. The method according to claim 2, characterized in that Based on the repetitive layout pattern of the minimal repetitive unit, traversing the registers to generate an access sequence for the registers of the source object and an access sequence for the registers of the target object includes: Based on the index of the minimum repeating unit, traverse the minimum repeating unit of the source tensor data; For each traversal of the minimum repeating unit, traverse the register based on the register index; Calculating a register global index based on the index of the traversed minimal repeating unit and the offset of the minimal repeating unit in the row and column directions, and the index of the traversed register and the offset of the register inside the minimal repeating unit in the row and column directions; and Based on the register global index calculated during the traversal process, an access sequence for registers of the source object and an access sequence for registers of the target object are generated.

4. The method according to claim 3, characterized in that Calculating the register index involves: Multiply the index of the minimum repeating unit in the column direction currently traversed by the offset of the minimum repeating unit in the column direction to generate the minimum repeating unit index value in the column direction; Multiply the index of the minimum repeating unit in the row direction of the current traversal by the offset of the minimum repeating unit in the row direction to generate the minimum repeating unit index value in the row direction; Multiplying the register index in the column direction inside the currently traversed minimal repeating unit by the offset of the register in the column direction inside the minimal repeating unit to generate the register index value in the column direction inside the minimal repeating unit; Multiplying the row-direction register index within the currently traversed minimal repeating unit by the row-direction register offset within the minimal repeating unit to generate a row-direction register index value within the minimal repeating unit; and The minimum repeating unit index value in the column direction, the minimum repeating unit index value in the row direction, the register index value in the column direction inside the minimum repeating unit, and the register index value in the row direction inside the minimum repeating unit are added together to generate a register global index corresponding to the current traversal based on the addition result.

5. The method according to claim 1, characterized in that Obtaining a repeating layout mode based on the smallest repeating unit of a source object includes: If it is determined that the shape of the multidimensional tensor data is not an integer multiple of the minimum repeating unit, the boundary of the multidimensional tensor data is automatically processed.

6. The method according to claim 2, wherein Obtaining a repeating layout mode based on the smallest repeating unit of a source object includes: If the dimension of the multidimensional tensor data is greater than or equal to three dimensions, reducing the multidimensional tensor data to two-dimensional tensor data; and The two-dimensional tensor data is converted into an organization manner of multiple repetitions of the minimum repeating unit, so as to obtain a repeating layout pattern of the minimum repeating unit of the source object.

7. The method according to claim 2, characterized in that The layout information of the registers within the minimal repeating unit of the source tensor data includes: layout information of the registers within the minimal repeating unit of the source tensor data in a height direction and layout information of the registers in a width direction; The offset of the register within the smallest repeating unit of the source tensor data includes: the offset of the register within the smallest repeating unit of the source tensor data in the height direction, and the offset of the register in the width direction; and The number of minimum repeating units of the source tensor data includes: the number of minimum repeating units of the source tensor data in a height direction and the number of minimum repeating units of the source tensor data in a width direction.

8. The method according to claim 1, characterized in that The minimal repeating unit is associated with multiple layout information, including: the name of the minimal repeating unit, the main direction, the layout of threads and values, the shape and stride of threads within registers, the number and arrangement of values ​​processed by each thread, and the size of the minimal repeating unit.

9. A computing device, characterized in that: include: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a machine, performs the method according to any one of claims 1 to 8.

11. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a machine, performs the method according to any one of claims 1 to 8.

Images