Reformatting a tensor to provide a subtensor

Abstract

A tensor in a first selected dimension is reformatted to provide one or more subtensors in a second selected dimension. The reformatting includes determining a number of subtensors used to represent the tensor. The reformatting further includes generating a number of subtensors, the subtensors starting on boundaries of memory units. Data for the tensor is restructured to fit within the number of subtensors.

Description

Technical Field

[0001] One or more aspects generally relate to facilitating processing within a computing environment, and in particular, to improving such processing.

Background Art

[0002] To enhance processing in computing environments that are data and / or compute intensive, coprocessors such as artificial intelligence accelerators (also referred to as neural network processors or neural network accelerators) are utilized. Such accelerators provide significant computing power for use in performing complex computations, such as computations on matrices or tensors.

[0003] Tensor computations are used in complex processing, including, by way of example, deep learning, which is a subset of machine learning. Deep learning or machine learning, aspects of artificial intelligence, are used in a variety of technologies including, but not limited to, engineering, manufacturing, medical technology, automotive technology, computer processing, and the like.

[0004] Tensors and tensor computations enable large amounts of data and / or detailed data to be input into deep learning processing. However, accelerators are often limited by the data bandwidth to the accelerators. Previously, data locality and data reuse in the accelerators have been employed to address this limitation. Developments in the use of such tensors and / or processing using such tensors improve technologies that use machine learning, including computer processing.

Summary of the Invention

[0005] To facilitate processing within a computing environment, the shortcomings of prior art are overcome and additional advantages are provided through the provisioning of computer program products. A computer program product includes one or more computer-readable storage media and program instructions, which are stored together in one or more computer-readable storage media to perform a method. The method includes obtaining a tensor of a first choice dimension. The tensor is reformatted to provide one or more subtensors of a second choice dimension. The reformatting includes determining several subtensors to be used to represent the tensor, generating several subtensors (which begin at the boundaries of memory units), and reorganizing the data of the tensor so that it fits within several subtensors.

[0006] By generating subtensors that start on the memory unit boundaries, processing is facilitated by reducing the number of address translation requests and increasing the data transfer rate. Furthermore, the memory address for the subtensor is easily computable, reducing the overhead of complex address calculations.

[0007] In one embodiment, determining several subtensors involves using one or more ceil functions based on at least the number of element values ​​contained in the tensor and the size of the memory unit.

[0008] In one embodiment, reconstructing data for at least one subtensor of several subtensors involves padding at least one subtensor of several subtensors in at least one dimension to complete the at least one subtensor. Padding provides a fixed-dimension subtensor and provides efficiency in address calculation and data transfer.

[0009] In one embodiment, reconstructing data for a selected subtensor of several subtensors involves selecting data at the same position in multiple matrices representing the tensor; placing the selected data in one row of the selected subtensor; and repeating the selection and placement for multiple positions in multiple matrices.

[0010] Furthermore, in one embodiment, reconstructing the data further includes padding one or more rows of the selected subtensor on the basis that one or more rows have less data than can be accommodated by the size of the selected subtensor; and padding the selected subtensor with one or more additional padded rows on the basis that the selected subtensor has fewer rows than can be accommodated by the size of the selected subtensor. Similarly, padding provides a fixed-dimension subtensor and provides efficiency in address calculation and data transfer.

[0011] For example, a memory unit is a memory page, and the boundary of a memory unit is a page boundary. Processing is facilitated by generating subtensors that start on page boundaries, thereby reducing the number of address translation requests and increasing the data transfer rate.

[0012] In one embodiment, one or more subtensors are provided to a processor for use in tensor computation. Providing one or more subtensors includes, for example, providing the processor used to store the data of one or more subtensors in a single operation, address information and information related to a second selection dimension. This enables block storage of data in a single operation and improves system performance.

[0013] In one embodiment, providing one or more subtensors includes providing address information and information related to a second selection dimension to a processor used to load the data of one or more subtensors in a single operation. This enables block loading of data in a single operation and improves system performance.

[0014] In one example, the first chosen dimension is 4 dimensions, and the second chosen dimension is 2 dimensions, where one or more 2-dimensional subtensors represent one 4-dimensional tensor.

[0015] Computer implementation methods and systems relating to one or more embodiments are also described and claimed herein. Furthermore, services relating to one or more embodiments are also described and may be claimed herein.

[0016] Additional features and benefits are realized through the technologies described herein. Other embodiments and aspects are described in detail herein and are considered to be part of the claimed embodiments. [Brief explanation of the drawing]

[0017] One or more embodiments are specifically pointed out and claimed individually as examples in the claims at the end of this specification. The purposes, features, and advantages of the foregoing, as well as of one or more embodiments, will become apparent from the following detailed description in conjunction with the accompanying drawings. [Figure 1A] An example of a computing environment for incorporating and using one or more aspects of the present invention is illustrated. [Figure 1B] Further details of the processor shown in Figure 1A, according to one or more aspects of the present invention, are illustrated. [Figure 2A] An example of a 2D tensor according to one or more aspects of the present invention is illustrated. [Figure 2B] An example of a process used to generate a tensor subtensor according to one or more aspects of the present invention is illustrated. [Figure 2C] Illustrate an example of a process used to generate a sub-tensor of a tensor according to one or more aspects of the present invention. [Figure 3A] Illustrate an example of a format of a neural network processing support instruction according to one or more aspects of the present invention. [Figure 3B] Illustrate an example of a general-purpose register used by a neural network processing support instruction according to one or more aspects of the present invention. [Figure 3C] Illustrate an example of function code supported by a neural network processing support instruction according to one or more aspects of the present invention. [Figure 3D] Illustrate an example of another general-purpose register used by a neural network processing support instruction according to one or more aspects of the present invention. [Figure 3E] Illustrate an example of a parameter block used by a query function of a neural network processing support instruction according to one or more aspects of the present invention. [Figure 3F] Illustrate an example of a parameter block used by one or more non-query functions of a neural network processing support instruction according to one or more aspects of the present invention. [Figure 3G] Illustrate an example of a tensor descriptor used by a neural network processing support instruction according to one or more aspects of the present invention. [Figure 4] Illustrate an example of a format of a neural network processing (NNP) data type 1 data type according to one or more aspects of the present invention. [Figure 5A] Illustrate an example of an input data layout used by a neural network processing support instruction according to one or more aspects of the present invention. [Figure 5B] Illustrate an example of an input data layout used by a neural network processing support instruction according to one or more aspects of the present invention. [Figure 5C]An example of an input data layout used by a neural network processing support instruction according to one or more aspects of the present invention is illustrated. [Figure 6A] The following illustrates exemplary outputs corresponding to the input data layouts shown in Figures 5A to 5C, according to one or more aspects of the present invention. [Figure 6B] The following illustrates exemplary outputs corresponding to the input data layouts shown in Figures 5A to 5C, according to one or more aspects of the present invention. [Figure 6C] The following illustrates exemplary outputs corresponding to the input data layouts shown in Figures 5A to 5C, according to one or more aspects of the present invention. [Figure 7A] An example illustrating how processing within a computing environment is facilitated according to one or more aspects of the present invention is shown. [Figure 7B] An example illustrating how processing within a computing environment is facilitated according to one or more aspects of the present invention is shown. [Figure 8A] Another example of a computing environment for incorporating and using one or more aspects of the present invention is illustrated. [Figure 8B] An example of further details of the memory shown in Figure 8A according to one or more aspects of the present invention is illustrated below. [Figure 8C] Another example of further details of the memory shown in Figure 8A, according to one or more aspects of the present invention, is illustrated. [Figure 9A] Further examples of computing environments for incorporating and using one or more aspects of the present invention are illustrated. [Figure 9B] Further details of the memory shown in Figure 9A according to one or more aspects of the present invention are illustrated. [Figure 10] An embodiment of a cloud computing environment according to one or more aspects of the present invention is illustrated. [Figure 11] An example of an abstraction model layer according to one or more aspects of the present invention is illustrated. [Modes for carrying out the invention]

[0018] According to one or more aspects of the present invention, the ability to facilitate processing within a computing environment is provided. For example, the ability to reformat a tensor to provide subtensors, where one or more subtensors represent the original tensor. This optimizes processing including, but not limited to, memory address calculation, load / store operations, and / or prefetching. For example, a tensor is reformatted so that the subtensors representing the tensor start on the boundary of a memory unit (e.g., a memory page), and the information of the tensor is reconfigured to fit within subtensors (i.e., tiles) of one or more selected dimensions (e.g., 2D). Based on the reformatting, the subtensors have readily computable addresses and can be block loaded and / or stored (e.g., loaded / stored in a single operation), providing efficiency in the use of the subtensors.

[0019] An example of an instruction for using a generated tensor that includes subtensors is a neural network processing support instruction, which is a single instruction (e.g., a single architecture hardware machine instruction in a hardware / software interface) configured to execute multiple functions. Each of these functions is configured as part of a single instruction (e.g., a single architecture instruction), reducing system resource usage and complexity and improving system performance.

[0020] An instruction may be part of a general-purpose processor instruction set architecture (ISA) that is dispatched by a program on a processor such as a general-purpose processor. It may be executed by a general-purpose processor, and / or one or more functions of the instruction may be executed by a dedicated processor, such as a processor configured for a particular function that is coupled to or part of a general-purpose processor. Other variations are also applicable.

[0021] One embodiment of a computing environment for incorporating and using one or more aspects of the present invention is described with reference to Figure 1A. As an example, the computing environment is based on the z / Architecture® instruction set architecture provided by International Business Machines Corporation in Armonk, New York. One embodiment of the z / Architecture instruction set architecture is described in the publication entitled "z / Architecture Principles of Operation," IBM Publication No. SA22-7832-12, 13th edition, September 2019, which is incorporated herein by reference in its entirety. However, the z / Architecture instruction set architecture is only one exemplary architecture, and other architectures and / or other types of computing environments of International Business Machines Corporation and / or other entities may include and / or use one or more aspects of the present invention. z / Architecture and IBM are trademarks or registered trademarks of International Business Machines Corporation in at least one jurisdiction.

[0022] Referring to Figure 1A, for example, the computing environment 100 includes a computer system 102, which is shown, for example, in the form of a general-purpose computing device. The computer system 102 may include, but is not limited to, one or more general-purpose processors or processing units 104 (e.g., a central processing unit (CPU)), at least one dedicated processor such as a neural network processor 105, memory 106 (i.e., system memory, main memory, main storage, central storage, or storage, for example), and one or more input / output (I / O) interfaces 108 coupled to each other via one or more buses and / or other connections. For example, processors 104, 105, and memory 106 are coupled to the I / O interface 108 via one or more buses 110, and processors 104, 105 are coupled to each other via one or more buses 111.

[0023] Bus 111 is, for example, a memory or cache coherence bus, and Bus 110 represents one or more of several types of bus structures, including, for example, a memory bus or memory controller, peripheral bus, accelerated graphics port, processor or local bus, using any of various bus architectures. Such architectures include, but are not limited to, Industry Standard Architecture (ISA), Microchannel Architecture (MCA), Enhanced ISA (EISA), Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI).

[0024] For example, one or more application-specific processors (e.g., neural network processors) may be separate from or coupled to one or more general-purpose processors, and / or integrated within one or more general-purpose processors. Many variations are possible.

[0025] For example, memory 106 may include a cache 112, such as a shared cache, which can be connected to the local cache 114 of the processor 104 and / or the neural network processor 105, for example, via one or more buses 111. Furthermore, memory 106 may include one or more programs or applications 116 and at least one operating system 118. An exemplary operating system is the z / OS® operating system provided by International Business Machines Corporation in Armonk, New York. z / OS is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction. Other operating systems provided by International Business Machines Corporation and / or other entities may also be used. Memory 106 may also include one or more computer-readable program instructions 120, which can be configured to execute functions of embodiments of the present invention.

[0026] In one or more embodiments, the memory 106 also includes processor firmware 122. The processor firmware includes, for example, the processor's microcode or millicode. It includes, for example, hardware-level instructions and / or data structures used in higher-level machine code implementations. In one embodiment, it includes, for example, trusted software, microcode, or millicode specific to the underlying hardware, and proprietary code that is typically supplied as microcode or millicode that controls the operating system's access to the system hardware.

[0027] The computer system 102 may communicate with one or more external devices 130, such as a user terminal, tape drive, pointing device, display, and one or more data storage devices 134, for example, via an I / O interface 108. The data storage device 134 may store one or more programs 136, one or more computer-readable program instructions 138, and / or data. The computer-readable program instructions may be configured to execute functions according to embodiments of the invention.

[0028] The computer system 102 may also communicate with a network interface 132, for example via an I / O interface 108, which enables the computer system 102 to communicate with one or more networks, such as a local area network (LAN), a general-purpose wide area network (WAN), and / or a public network (e.g., the Internet), and provides communication with other computing devices or systems.

[0029] Computer system 102 includes and / or may be coupled with removable / non-removable, volatile / non-volatile computer system storage media. For example, it may include and / or may be coupled with a magnetic disk drive for reading from and writing to a non-removable non-volatile magnetic medium (commonly called a “hard drive”), a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a “floppy disk”), and / or an optical disk drive for reading from and writing to a removable non-volatile optical disk such as a CD-ROM, DVD-ROM, or other optical medium. It should be understood that other hardware and / or software components may be used in conjunction with computer system 102. Examples include, but are not limited to, microcode or millicode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0030] Computer system 102 may operate in many other general-purpose or application-specific computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that may be suitable for use with computer system 102 include, but are not limited to, personal computer (PC) systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments, which may include any of the systems or devices listed above and similarly.

[0031] In one example, a processor (e.g., processor 104 and / or processor 105) includes several functional components (or a subset thereof) used to execute instructions. As shown in Figure 1B, these functional components include, for example, an instruction fetch component 150 for fetching instructions to be executed; an instruction decoding unit 152 for decoding the fetched instructions and obtaining the operands of the decoded instructions; one or more instruction execution components 154 for executing the decoded instructions; a memory access component 156 for accessing memory for instruction execution if necessary; and a write-back component 158 ​​for providing the results of the executed instructions. One or more of the components may access and / or use one or more registers 160 in instruction processing. Furthermore, one or more components may include, or have access to, at least a portion of, one or more other components used in tensor processing (including, but not limited to, the generation and / or use of subtensors) and / or neural network processing support processing (or other processing using one or more aspects of the present invention) as described herein, according to one or more aspects of the present invention. One or more other components may include, for example, tensor component 170 and / or neural network processing support component 172 (and / or one or more other components).

[0032] According to one or more aspects of the present invention, processing within a computing environment is facilitated by providing an improved data format for use by a processor, such as a dedicated processor (e.g., a neural network processor 105). For example, a designed data format is provided to the processor. As an example, systolic and single-instruction multiple data (SIMD) units (2D / 3D compute arrays) are accommodated by providing subtensors of input operands in similar dimensions. In one example, a tensor (e.g., a 4D tensor) is reformatted so that the subtensors representing the tensor begin on the boundary of a memory unit (e.g., a memory page boundary), and the information of the tensor is reconstructed in the subtensors, e.g., one or more 2D subtensors (also referred to as 2D tiles or 2D tensors) of the original tensor. The dimensions of the tensor in the dimensions of the 2D tile are rounded in that dimension to the next full tile (e.g., padding is provided to generate a fixed-size 2D tensor). This allows access to each tile on the memory unit boundary (e.g., a page boundary) and facilitates computation of the address of any 2D tile. By providing alignment on memory unit boundaries, address translation requests are reduced and data transfer rates are increased. Furthermore, in one example, each 2D tile can be loaded via an operation such as direct memory access (DMA), which accesses one memory unit (e.g., a page) in accelerator memory at once. This significantly increases bandwidth.

[0033] In one example, multiple 2D tensors (e.g., nine 2D tensors in one particular example) are generated to represent a 4D tensor. Further in one example, three 2D tensors generate a 3D tensor, and three 3D tensors generate a 4D tensor. Thus, in one example, a 4D tensor is reformatted into multiple 2D tensors, each starting on the boundary of a memory unit, for example, on a memory page boundary. Tensors of various dimensions are described herein, but tensors of other dimensions may be generated, reformatted, and / or able to represent other tensors.

[0034] An example of a 2D tensor is illustrated in Figure 2A. As shown, a 2D tensor (also called a 2D subtensor, subtensor, or 2D tile) 200 starts on a memory boundary and uses multiple memory units, such as multiple 4K pages (e.g., pages 0-11 numbered in the 2D tensor). Each page contains a pre-selected number of rows 202 (e.g., 32), and each row contains a pre-selected number of elements, e.g., 64 elements. If a row has fewer data than the pre-selected number of elements, it is padded with a pre-specified value, e.g., zeros or blanks 204. Furthermore, if there is not enough data to provide the pre-selected number of rows, additional padding 206 (e.g., unpredictable data, existing values, arbitrary values, etc.) is provided, and additional padded rows are added, as shown in Figure 2A.

[0035] The designed data format for 2D tensors provides easily computable addresses and memory-adjacent tensor units, thereby reducing the overhead of multiple complex address calculations. This supports hardware-supported block load / storage operations and engine prefetching, significantly increasing the effective data bandwidth (e.g., 2x~5x) to the accelerator (e.g., neural network processor 105).

[0036] As shown, multiple 2D tensors (e.g., 9 in a particular example) are used to represent a 4D tensor, and each 2D tensor may use one or more memory units (e.g., memory pages).

[0037] Further details relating to the creation of 2D tensors according to aspects of the present invention are described with reference to Figures 2B-2C. In one example, this process is performed by a processor such as a general-purpose processor 104. This process can, but is not limited to, generating 2D, 3D, or 4D tensors, for example. In one example, the process creates a tensor based on a 4D feature data layout described herein.

[0038] Referring to Figure 2B, in one example, e2_limit is set to equal to ceil(E2 / 32)*32 (252), indicating that the resulting 2D tensor will have, for example, 32 rows, where E2 refers to the index size of dimension 2. Furthermore, e1_limit is set to equal to ceil(E1 / 64)*64 (254), indicating that the resulting 2D tensor will have, for example, 64 elements per row, where E1 refers to the index size of dimension 1. Index e4x is initialized to zero (256).

[0039] After initialization, it is determined whether e4x is less than E4 (258), where E4 refers to the index size of dimension 4. If e4x is not less than E4, processing ends 260; otherwise, processing continues and the index e3x is initialized to zero 262. It is determined whether e3x is less than E3 (264), where E3 refers to the index size of dimension 3. If e3x is not less than E3, processing is iterated so that e4x is incremented by, for example, 1 (266), and processing continues to 258. However, if e3x is less than E3, the index e2x is initialized to zero 268. It is determined whether e2x is less than e2_limit (270). If e2x is not less than e2_limit, processing is iterated so that e3x is incremented by, for example, 1 272, and processing continues to 264. If e2x is less than e2_limit, then index e1x is initialized to zero, which is 274.

[0040] Referring to Figure 2C, it is determined whether e1x is less than e1_limit (276). If e1x is not less than e1_limit, the process is repeated so that e2x is incremented by, for example, 1 (278), and the process continues to 270 (Figure 2B). If e1x is less than e1_limit, arr_pos (for example, position in the row) is (E3*e2_limit*e1_limit*e4x)+(e2_limit*e3x*64)+(e2x*64)+(

number

number

[0041] It is determined whether e2x is greater than or equal to E2 (284). If e2x is less than E2, it is further determined whether e1x is greater than or equal to E1 (286). If e1x is less than E1, the value is set to equal input_array[e4x][e3x][e2x][e1x] 288; if e1x is greater than or equal to E1, the value is E1pad 290 (a row is padded). Furthermore, if e2x is greater than or equal to E2 (284), the value is E2pad 292 (an additional row is added to the 2D tensor to be padded). After setting the value, OutputTensor[arr_pos] = value 294. Furthermore, the index e1x is incremented by, for example, 1 (296), and processing continues to 276.

[0042] As a further example, tensors can be generated based on the 4D kernel layout described herein. The process in Figures 2B-2C is used to generate 2D, 3D, 4D and / or other tensors, but 282 is kem_pos=(

number

[0043] For example, to reformat a 4D tensor, several 2D tensors are generated, the number of which depends on how many rows each 2D tensor contains and how many elements each row contains. Multiple 2D tensors can generate a 3D tensor, and multiple 3D tensors can generate a 4D tensor.

[0044] Using the above process, at least one subtensor representing the reformatted tensor is defined. For example, if the original 4D tensor has 65 elements in its rows, two 2D subtensors are generated: the first containing a row with 64 elements, and the second containing a row with a 65th element and padding. Other variations are possible.

[0045] The generated tensor (e.g., a subtensor generated from reformatting the original tensor (e.g., a 4D tensor)) can be used by one or more instructions. For example, address information (e.g., the start of a 4D or 2D tensor), tensor dimensions, etc., are transferred from a general-purpose processor to a dedicated processor (e.g., a neural network 105) for use in loading / storing data in the correct format (e.g., at the correct location in a page of memory), and for using the data (e.g., in tensor computation). In other embodiments, the general-purpose processor uses the generated subtensor. Other variations are possible.

[0046] As described herein, in one example, an instruction for using a generated tensor (also referred to herein as a subtensor) is a neural network processing support instruction. In one example, the instruction is started on a general-purpose processor (e.g., processor 104), and the function specified by the instruction is executed on either the general-purpose processor and / or a dedicated processor (e.g., neural network processor 105), depending on the function. For example, in one example, a query function is executed on the general-purpose processor, and a non-query function is executed on the dedicated processor. However, other variations are possible. When a function is executed on a dedicated processor (e.g., a non-query function, or, in another example, one or more selected functions), information such as memory address information related to the tensor data used in the neural network computation is provided to the application-specific processor for use in the execution of the function, for example, by the general-purpose processor. The dedicated processor retrieves the information and executes the function. The instruction then completes on the general-purpose processor. In other examples, the instruction is started, executed, and completed on one or more general-purpose processors or one or more application-specific processors. Other variations are possible.

[0047] In one example, referring to Figure 3A, the neural network processing support instruction 300 has an RRE format that indicates registers and register operations using extended operation codes (opcodes). As shown in Figure 3A, in one example, the neural network processing support instruction 300 includes an operation code (opcode) field 302 (e.g., bits 0-15) that indicates the neural network processing support operation. In one example, bits 16-31 of the instruction are reserved and include zero. In the description herein, instructions and / or instructions are described as having functions, specific locations, specific fields, and / or specific sizes of fields (e.g., specific bytes and / or bits). However, other locations, fields, and / or sizes may be provided. Furthermore, bits may be specified to be set to specific values, e.g., 1 or zero, but this is merely an example. Bits may be set to different values, such as the opposite value or another value, in other examples, where they are set. Many variations are possible.

[0048] In one example, an instruction uses several general-purpose registers implicitly specified by the instruction. For example, the neural network processing support instruction 300 uses the implicit register, general-purpose register 0, and general-purpose register 1, which are illustrated with reference to Figures 3B and 3D, respectively.

[0049] Referring to Figure 3B, in one example, general-purpose register 0 includes a function code field and a status field, which may be updated upon instruction completion. For example, general-purpose register 0 includes a response code field 310 (e.g., bits 0-15), an exception flag field 312 (e.g., bits 24-31), and a function code field 314 (e.g., bits 56-63). Furthermore, in one example, bits 16-23 and 32-55 of general-purpose register 0 are reserved and include zero. One or more fields are used by specific functions executed by the instruction. In one example, not all fields are used by all functions. Each of the fields is described below.

[0050] Response Code (RC) 310: This field (e.g., bit positions 0-15) contains the response code. The response code is stored when the execution of a neural network processing support instruction is completed with a condition code of, for example, 1. When an invalid input condition is encountered, a non-zero value is stored in the response code field, which indicates the cause of the invalid input condition recognized during execution and the selected condition code, for example, 1, that is set. The code stored in the response code field is defined as follows in one example:

[0051] Response code meaning

[0052] 0001 The parameter block format specified by the parameter block version number is not supported by the model.

[0053] 0002 The specified function is not defined or installed on the machine.

[0054] 0010 The specified tensor data layout format is not supported.

[0055] 0011 The specified tensor data type is not supported.

[0056] 0012 The specified single tensor dimension is larger than the maximum dimension index size.

[0057] 0013 The size of the specified tensor is greater than the maximum tensor size.

[0058] 0014 The specified tensor address is not aligned on a 4K byte boundary.

[0059] 0015 Function-specific storage area addresses are not aligned on 4K byte boundaries.

[0060] F000~FFFF: Function-specific response codes. These response codes are defined for specific functions.

[0061] Exception Flags (EF) 312: This field (e.g., bit positions 24-31) contains exception flags. If an exception condition is detected during instruction execution, the corresponding exception flag control (e.g., bit) is set to 1, otherwise the control remains immutable. The exception flags field is initialized to zero before the first call of the instruction. Reserved flags remain immutable during instruction execution. The flags stored in the exception flags field are defined as follows in one example:

[0062] EF (bit) meaning

[0063] 0 Scope violation. This flag is set either when a non-numeric value is detected in the input tensor or when it is stored in the output tensor. This flag is valid, for example, only when the instruction completes with a condition code, such as 0.

[0064] 1-7 are reserved.

[0065] Function Code (FC) 314: This field (e.g., bit positions 56-63) contains the function code. An example of an assigned function code for neural network processing support instructions is illustrated in Figure 3C. No other function codes are assigned. If an unassigned or uninstalled function code is specified, a response code of, for example, 0002 hex and a selection condition code of, for example, 1 are set. This field is not modified during execution.

[0066] As shown, in addition to general-purpose register 0, neural network processing support instructions also use general-purpose register 1, an example of which is illustrated in Figure 3D. For example, bits 40-63 in 24-bit addressing mode, bits 33-63 in 31-bit addressing mode, or bits 0-63 in 64-bit addressing mode contain the address of parameter block 320. The contents of general-purpose register 1 specify, for example, the logical address of the leftmost byte of the parameter block in storage. The parameter block is specified on a double-word boundary; otherwise, a specification exception is recognized. For all functions, the contents of general-purpose register 1 are not modified.

[0067] In access register mode, access register 1 specifies, as an example, an address space that includes a parameter block, an input tensor, an output tensor, and a function-specific storage area.

[0068] In one example, a parameter block may have a different format depending on the function specified by the instruction being executed. For instance, a query function might have a parameter block of one format, while other functions in an instruction might have a parameter block of a different format. In another example, all functions might use the same parameter block format. Other variations are also possible.

[0069] An example of a parameter block used by query functions such as NNPA-Query Available Functions (QAF) operations is illustrated with reference to Figure 3E. As shown, in one example, the NNPA-Query Available Functions parameter block 330 includes, for example, the following:

[0070] Vector of functions to be installed 332: This field of the parameter block (e.g., bytes 0-31) contains a vector of functions to be installed. In one example, bits 0-255 of the vector of functions to be installed correspond to function codes 0-255 of the neural network processing support instructions, respectively. When a bit is, for example, 1, the corresponding function is installed; otherwise, the function is not installed.

[0071] Vector of parameter block formats to be installed 334: This field of the parameter block (e.g., bytes 32-47) contains the vector of parameter block formats to be installed. In one example, bits 0-127 of the vector of parameter block formats to be installed correspond to parameter block formats 0-127 for non-query functions of neural network processing support instructions. When a bit is, for example, 1, the corresponding parameter block format is installed; otherwise, the parameter block format is not installed.

[0072] Installed Data Types 336: This field in the parameter block (e.g., bytes 48-49) contains a vector of data types to be installed. In one example, bits 0-15 of the vector of data types to be installed correspond to the data types to be installed. When a bit is, for example, 1, the corresponding data type is installed; otherwise, the data type is not installed. Exemplary data types include (more, fewer, and / or other data types are possible):

[0073] Bit data type

[0074] 0 NNP Data Type 1

[0075] 1-15 are reserved.

[0076] The data layout format to be installed (338): This field in the parameter block (e.g., bytes 52-55) contains a vector of data layout formats to be installed. In one example, bits 0-31 of the vector of data layout formats to be installed correspond to the data layout formats to be installed. When a bit is, for example, 1, the corresponding data layout format is installed; otherwise, the data layout format is not installed. Exemplary data layout formats include (more, fewer, and / or other data types are possible):

[0077] Bit Data Layout Format

[0078] 0 4D Feature Tensor

[0079] 1 4D kernel tensor

[0080] 2-31 Reserved

[0081] Maximum dimension index size 340: This field in the parameter block (e.g., bytes 60-63) contains a 32-bit unsigned binary integer that specifies, for example, the maximum number of elements in a given dimension index size for any given tensor. In another example, the maximum dimension index size specifies the maximum number of bytes in a given dimension index size for any given tensor. Other examples are also possible.

[0082] Maximum Tensor Size 342: This field in the parameter block (e.g., bytes 64-71) contains a 32-bit unsigned binary integer specifying the maximum number of bytes in any given tensor, including any padding bytes required by the tensor format. In another example, the maximum tensor size specifies the maximum total number of elements in any given tensor, including any padding required by the tensor format. Other examples are also possible.

[0083] Installed NNP Data Type 1 Conversion Vector 344: This field in the parameter block (e.g., bytes 72-73) contains the installed NNP Data Type 1 conversion vector. In one example, bits 0-15 of the installed NNP Data Type 1 conversion vector correspond to data type conversions to and from the installed NNP Data Type 1 format. When a bit is 1, the corresponding conversion is installed; otherwise, the conversion is not installed. More, fewer, and / or other conversions may be specified.

[0084] Bit data type

[0085] 0 reserved

[0086] 1 BFP Small Format

[0087] 2 BFP Short Format

[0088] 3-15 are booked.

[0089] An example of a parameter block for a query function is illustrated with reference to Figure 3E, but other formats of parameter blocks for query functions, including the NNPA-Query Available Functions operation, may be used. The format may, in one example, depend on the type of query function being executed. Furthermore, the parameter block and / or each field of the parameter block may contain more, less, and / or other information.

[0090] In addition to parameter blocks for query functions, there are parameter block formats for non-query functions, such as the non-query function of the Neural-Network Processing Assist instruction. An example of a parameter block used by non-query functions, such as the non-query function of the neural network processing assistance instruction, is illustrated with reference to Figure 3F.

[0091] As shown, in one example, the parameter block 350 adopted by, for example, a non-query function of a neural network processing support instruction includes, for example, the following:

[0092] Parameter Block Version Number 352: This field of the parameter block (e.g., bytes 0-1) specifies the version and size of the parameter block. In one example, bits 0-8 of the parameter block version number are reserved and include zero, and bits 9-15 of the parameter block version number contain an unsigned binary integer specifying the format of the parameter block. The query function provides a mechanism to indicate available parameter block formats. When the specified parameter block size or format is not supported by the model, for example, a response code of 0001 hex is stored in general register 0, and the instruction is completed by setting a condition code, for example, condition code 1. The parameter block version number is specified by the program and is not modified during instruction execution.

[0093] Model version number 354: This field in the parameter block (e.g., byte 2) is an unsigned binary integer that identifies the model executing the instruction (e.g., a specific non-query function). When the continuation flag (described below) is 1, the model version number may be an input to an operation intended to resume the operation by interpreting the contents of the continuation state buffer field (described below) of the parameter block.

[0094] Continuation flag 356: This field in the parameter block (e.g., bit 63) indicates, for example, that the operation is partially completed when it is 1, and the contents of the continuation state buffer may be used to resume the operation. The program initializes the continuation flag to zero and does not modify it when an instruction is re-executed for the purpose of resuming an operation; otherwise, the result is unpredictable.

[0095] If the continuation flag is set at the beginning of the operation and the contents of the parameter block have changed since the first call, the result is unpredictable.

[0096] Function-specific storage area address 358: This field in the parameter block (e.g., bytes 56-63) contains the logical address of the function-specific storage area. In one example, the function-specific storage area address is aligned on a 4K byte boundary; otherwise, for example, a response code of 0015 hex is set in general-purpose register 0, and the instruction completes with a condition code of 1. The address is affected by the current addressing mode. The size of the function-specific storage area depends on the function code.

[0097] When the entire function-specific storage area overlaps with a Program Event Log (PER) storage area designation, and is applicable, a PER storage change event is recognized for the function-specific storage area. When only a portion of the function-specific storage area overlaps with a PER storage area designation, which of the following occurs is model-dependent.

[0098] * When applicable, PER storage change events are recognized for the entire function-specific storage area.

[0099] * When applicable, PER storage change events are recognized for a portion of the function-specific storage areas where the event is stored.

[0100] When the entire parameter block overlaps with the PER storage area specification, and is applicable, a PER storage change event is recognized for the parameter block. When only a portion of the parameter block overlaps with the PER storage area specification, which of the following occurs is model-dependent.

[0101] * When applicable, PER storage change events are recognized for the entire parameter block.

[0102] * When applicable, PER storage change events are recognized for a portion of the stored parameter blocks.

[0103] When applicable, a PER zero-address detection event is recognized for a parameter block. Zero-address detection does not apply, for example, tensor addresses or function-specific storage area addresses.

[0104] Output tensor descriptor (e.g., 1-2) 360 / Input tensor descriptor (e.g., 1-3) 365: An example of a tensor descriptor is explained with reference to Figure 3G. In one example, tensor descriptors 360 and 365 include the following:

[0105] Data layout format 382: This field of the tensor descriptor (e.g., byte 0) specifies the data layout format. Valid data layout formats include, for example, (more, fewer, and / or other data layout formats are possible):

[0106] Format Description Alignment (bytes)

[0107] 0 4D Feature Tensor 4096

[0108] 1 4D kernel tensor 4096

[0109] 2-255 reserved

[0110] If an unsupported or reserved data layout format is specified, for example, a response code of 0010 hex is stored in general-purpose register 0, and the instruction is completed by setting a condition code, for example, 1.

[0111] Data type 384: This field (e.g., byte 1) specifies the data type of the tensor. Examples of supported data types are described below (more, fewer, and / or other data types are possible).

[0112] Value, Data Type, Data Size (bits)

[0113] 0 NNP Data Type 1 16

[0114] 1-255 Reserved

[0115] If an unsupported or reserved data type is specified, for example, a response code of 0011 hex is stored in general-purpose register 0, and the instruction is completed by setting a condition code, for example, 1.

[0116] Dimension 1-4 Index Size 386: Dimension index sizes 1-4 (e.g., E4, E3, E2, E1) collectively specify the shape of the 4D tensor. Each dimension index size is greater than zero and less than or equal to the maximum dimension index size (340, Figure 3E); otherwise, a response code, e.g., 0012 hex, is stored in general-purpose register 0, and the instruction is completed by setting a condition code, e.g., 1. The total tensor size is less than or equal to the maximum tensor size (342, Figure 3E); otherwise, a response code, e.g., 0013 hex, is stored in general-purpose register 0, and the instruction is completed by setting a condition code, e.g., 1.

[0117] In one example, to determine the number of bytes (i.e., total tensor size) in a 4D feature tensor having elements of NNPA data type 1, the following formula is used: dimension index 4 * dimension index 3 * ceil(dimension index 2 / 32) * 32 * ceil(dimension index 1 / 64) * 64 * 2.

[0118] Tensor address 388: This field of the tensor descriptor (e.g., bytes 24-31) contains the logical address of the leftmost byte of the tensor. The address is affected by the current addressing mode.

[0119] If the address is not aligned on the boundary of the associated data layout format, for example, a response code of 0014 hex is stored in general-purpose register 0, and the instruction is completed by setting a condition code, for example, 1.

[0120] In access register mode, access register 1 specifies the address space that includes all active input and output tensors in the storage.

[0121] Returning to Figure 3F, the parameter block 350 further includes, in one example, function-specific parameters 1-5 that may be used by a particular function, as described herein (370).

[0122] Furthermore, the parameter block 350 includes, in one example, a continuation state buffer field 375 containing data (or the location of data) used when the operation of this instruction is resumed.

[0123] Reserved fields in a parameter block should contain zeros as input to an operation. When the operation finishes, the reserved fields may be stored as zeros or remain unchanged.

[0124] An example of a parameter block for a non-query function is illustrated with reference to Figure 3F, but other formats of parameter blocks for non-query functions, including non-query functions for neural network processing support instructions, may be used. The format may, in one example, depend on the type of function being executed. Furthermore, an example of a tensor descriptor is illustrated with reference to Figure 3G, but other formats may be used. Additionally, different formats for input and output tensors may be used. Other variations are possible.

[0125] Further details regarding the various functions supported by one embodiment of the neural network processing support instructions are described below.

[0126] Function code 0: NNPA-QAF (Query Available Functions)

[0127] Neural Network Processing Assisted (NNPA) query functions provide a mechanism for indicating selected information, such as the availability of the function to be installed, the parameter block format to be installed, the data type to be installed, the data layout format to be installed, the maximum dimension index size, and the maximum tensor size. This information is retrieved and placed at selected locations, such as parameter blocks (e.g., parameter block 330). When the operation is completed, reserved fields in the parameter block may be stored as zero or remain immutable.

[0128] In the execution of one embodiment of the query function, a processor such as a general-purpose processor 104 retrieves information relating to a specific model of a selected processor, such as a specific model of a neural network processor such as a neural network processor 105. A specific model of a processor or machine has specific capabilities. Another model of a processor or machine may have more, less, and / or different capabilities, and / or may be of a different generation (e.g., current or future generation) with more, less, and / or different capabilities. The retrieved information is placed in a parameter block (e.g., parameter block 330), or in another structure accessible to and / or used together by one or more applications that may use this information in further processing. In one example, the parameter block and / or information is maintained in memory. In other embodiments, the parameter block and / or information may be maintained in one or more hardware registers. As another example, the query function may be a privileged operation executed by the operating system, which makes this information available to the application program interface so that it can be used by an application or an unprivileged program. In further examples, the query function is executed by a dedicated processor, such as the neural network processor 105. Other variations are possible.

[0129] The information is obtained, for example, by the firmware of the processor executing the query function. The firmware has knowledge of the attributes of a particular model of a particular processor (e.g., a neural network processor). This information may be stored, for example, in control block registers and / or memory, and / or otherwise accessible from the processor executing the query function.

[0130] The information obtained includes, for example, model-dependent details about at least one or more data attributes of a particular processor, including one or more installed or supported data types, one or more installed or supported data layout formats, and / or one or more installed or supported data sizes for a selected model of a particular processor. Condition code 0 is set as an example when the execution of a query function (e.g., the NNPA-QAF function) is complete. Condition codes 1, 2, and 3 are not applicable to the query function. Further information related to the obtained information is described below.

[0131] As shown, in one example, the information obtained includes model-dependent information about one or more data attributes of a particular model of a neural network processor. An example of a data attribute is the installed data type of the neural network processor. For example, a particular model of a neural network processor (or other processor) may support one or more data types, such as NNP data type 1 data types (also referred to as neural network processing data type 1 data types) and / or other data types. NNP data type 1 data types are a 16-bit floating-point format that provides several advantages to deep learning training and inference computations, including, for example, maintaining the accuracy of deep learning networks; eliminating unnormalized formats that simplify rounding modes and corner case handling; automatic rounding to the nearest value for arithmetic operations; and the special entities of infinity and not-number (NaN) being combined into a single value (NINF) that is accepted and processed by arithmetic operations. NINF provides better defaults for exponential overflow and invalid operations (such as division by zero). This allows many programs to continue execution without hiding such errors and without using dedicated exception handlers. Other model-dependent data types are also possible.

[0132] An example of the format for an NNP data type 1 data type is illustrated in Figure 4. As illustrated, in one example, an NNP data type 1 data may be represented in format 400, which includes, for example, a sign 402 (e.g., bit 0), an exponent +31 404 (e.g., bits 1-6), and a fraction 406 (e.g., bits 7-15).

[0133] An example of the characteristics of the NNP data type 1 format is illustrated below.

[0134] Characteristics NNP Data Type 1

[0135] Format length (bits): 16 bits

[0136] Bias exponential length (bits): 6 bits

[0137] Fractional length (bits): 9 bits

[0138] Precision (p): 10 bits

[0139] Maximum left unit exponent (Emax): 32

[0140] Minimum left unit exponent (Emin): -31

[0141] Left-hand unit display (LUV) bias 31

[0142] Nmax (1-2 -9 )×2 33 ≈ 8.6 × 10 9

[0143] Nmin (1+2 -9 )×2 -31 ≈4.6 × 10 -10

[0144] Dmin ---

[0145] Here, ≈ indicates that the values ​​are approximations, Nmax is the largest representable finite number (in magnitude), and Nmin is the smallest representable number (in magnitude).

[0146] Further details regarding NNP Data Type 1 data types are explained below.

[0147] Biased exponent: The bias used to enable the representation of the exponent as an unsigned number is shown above. The biased exponent is similar to the properties of the binary floating-point format, except that no special meaning is assigned to biased exponents that are all zero and all ones, as described below with reference to the NNP data type 1 data type class.

[0148] Mantissa: The binary point of a number of NNP data type 1 is considered to be to the left of the leftmost fractional bit. To the left of the binary point is an implicit unit bit, which is considered 1 for normal numbers and zero for zeros. The fractional part with the implicit unit bit appended to the left is the mantissa of the number.

[0149] The value of a typical NNP data type 1 is obtained by multiplying the mantissa by a base 2 powered to an unbiased exponential level.

[0150] Non-zero values: Non-zero values ​​are shown below.

[0151] Number of classes of values

[0152] Normal number ±2 e-31 ×(1.f)

[0153] Here, e is the bias exponent expressed in decimal, and f is the fractional part in binary.

[0154] In one embodiment, there are three classes of NNP data type 1 data, including numeric and associated non-numeric entities. Each data item includes a sign, exponent, and mantissa. The exponent is an unsigned number where all biased exponents are non-negative and biased such that the minimum biased exponent is zero. The mantissa includes an explicit fraction and an implicit unit bit to the left of the binary decimal point. The sign bit is zero for positive and 1 for negative.

[0155] All allowed non-zero finite numbers have their own unique NNP data type 1 representation. There are no denormalized numbers, which are numbers that can have multiple representations for the same value, and there are no denormalized arithmetic operations. The three classes include, for example:

[0156] Data class, sign, bias exponent, unit bits*, fractional bits

[0157] Zero ± 0 0 0

[0158] Normal numbers ± 0 1 Non-zero

[0159] Normal numbers ± Non-zero, all not equal to 1 1 Arbitrary

[0160] Normal numbers ± all 1s, all non-1s

[0161] NINF ± All 1 All 1

[0162] Here, - indicates not applicable, * indicates that the unit bit is implicit, and NINF means non-digital or infinite.

[0163] Further details about each class are provided below.

[0164] Zero: Zero has a zero bias exponent and zero fractional bits. The implicit unit bit is zero.

[0165] Normalized Numbers: Normalized numbers can have bias exponents of any value. When the bias exponent is 0, the fractional parts are non-zero. When the bias exponent is all 1, the fractional parts are not all 1. The values ​​of other bias exponents can have any fractional values. The implicit unit bit is 1 if all numbers are normalized.

[0166] NINF: NINF is represented by a bias exponent of all ones and a fraction of all ones. NINF represents a value that does not fall within the range of representable values ​​in NNP data type 1 (i.e., a 16-bit floating-point number designed for deep learning, with 6 exponent bits and 9 fractional bits). Typically, NINF is simply propagated during computation so that it remains visible until the end.

[0167] NNP data type 1 data types are supported in one example, but other model-dependent proprietary or non-standard data types, and one or more standard data types including, but not limited to, IEEE 754 short-precision, 16-bit binary floating-point, IEEE half-precision floating-point, 8-bit floating-point, 4-bit integer format, and / or 8-bit integer format may be supported. These data formats have different qualities for neural network processing. For example, smaller data types (e.g., fewer bits) can be processed faster and use less cache / memory, while larger data types provide results with higher accuracy in neural networks. Supported data types may have one or more allocated bits in the query parameter block (e.g., in the installed data type field 336 of parameter block 330). For example, model-dependent proprietary or non-standard data types supported by a particular processor are indicated in the installed data type field, while standard data types are not. In other embodiments, one or more standard data types are also indicated. Other variations are possible.

[0168] In one particular example, when bit 0 of the installed data type field 336 is reserved for the NNP data type 1 data type and set to, for example, 1, it indicates that the processor supports NNP data type 1. As an example, the bit vector of installed data types is configured to represent up to 16 data types, with bits assigned to each data type. However, bit vectors in other embodiments may support more or fewer data types. Furthermore, vectors may be configured in which one or more bits are assigned to data types. Many examples are possible, and / or more, fewer, and / or other data types may be supported and / or indicated in the vector.

[0169] In one example, a query function retrieves an indication of a data type installed on the model-dependent processor and places the indication in a parameter block, for example, by setting one or more bits in the installed data type field 336 of the parameter block 330. Furthermore, in one example, a query function retrieves an indication of an installed data layout format (another data attribute) and places the information in a parameter block, for example, by setting one or more bits in the installed data layout format field 338. Exemplary data layout formats include, for example, a 4D feature tensor layout and a 4D kernel tensor layout. The 4D feature tensor layout is used, in one example, by a function shown herein, and the convolution function uses the 4D kernel tensor layout. These data layout formats arrange data in storage for tensors in a manner that increases processing efficiency in the execution of functions of neural network processing support instructions. For example, to perform calculations efficiently, a neural network processing support instruction uses input tensors provided in a particular data layout format. An example layout is provided, but more, fewer, and / or other layouts may be provided for the functions described herein and / or other functions.

[0170] The use or availability of a layout for a particular processor model is provided by a vector of the installed data layout format (e.g., field 338 of parameter block 330). The vector is, for example, a bit vector of the installed data layout format that allows the CPU to tell the application which layouts are supported. For example, bit 0 is reserved for the 4D feature tensor layout and, when set to, for example, 1, indicates that the processor supports the 4D feature tensor layout; bit 1 is reserved for the 4D kernel tensor layout and, when set to, for example, 1, indicates that the processor supports the 4D kernel tensor layout. In one example, the bit vector of the installed data layout format is configured to represent up to 16 data layouts, with bits assigned to each data layout. However, bit vectors in other embodiments may support more or fewer data layouts. Furthermore, vectors may be configured with one or more bits assigned to data layouts. Many examples are possible. Further details regarding the 4D feature tensor layout and the 4D kernel tensor layout are described below. Similarly, other layouts may be used here or in the future to optimize performance.

[0171] In one example, neural network processing support instructions perform operations using 4D tensors, i.e., 4-dimensional tensors. These 4D tensors are obtained from the general-purpose input tensors described herein, for example, in row-major order, i.e., when enumerating the tensor elements in increasing order of memory address, the innermost dimension, called E1, is first stepped up through the E1 index size value, starting from 0 up to E1 index size - 1, then the index of the E2 dimension is incremented, and the stepping through the E1 dimension is repeated. The index of the outermost dimension, called E4 dimension, is incremented last.

[0172] A tensor with a lower number of dimensions (e.g., a 3D or 1D tensor) is represented as a 4D tensor, where one or more dimensions of the 4D tensor exceed the original tensor dimension, which is set to 1.

[0173] This specification describes how to convert a row-major general-purpose 4D tensor having dimensions E4, E3, E2, and E1 into a 4D feature tensor layout (also referred to herein as NNPA data layout format 0 4D feature tensor).

[0174] For example, the resulting tensor can be represented as a 4D tensor of a 5D tensor having, for example, a 64-element vector or dimensions.

[0175]

number

number

[0176] The elements [e4][e3][e2][e1] of the general-purpose tensor can be mapped to the following elements of the resulting 5D tensor.

[0177]

number

number

number

number

number

[0178] The resulting tensor may be larger than the generic tensor. Elements in the resulting tensor that do not have a corresponding element in the generic tensor are called padded elements.

[0179] The NNPA data layout format 0 for 64-element vectors considers the elements [fe4][fe1][fe3][fe2][fe0] of a 4D feature tensor, or their equivalent representation, as a 5D tensor of elements. Whether these elements are padded elements, or their corresponding elements in a generic 4D tensor of dimensions E4, E3, E2, E1, can be determined by the following formula.

[0180] If fe2 ≥ E2, then this is an E2 (or page) padded element.

[0181] Otherwise, if fe1*64+fe0≧E1, then this is an E1 (or row) padded element.

[0182] Otherwise, the corresponding elements in the general-purpose 4D tensor are:

[0183] [fe4][fe3][fe2][fe1*64+fe0]

[0184] For convolutional neural network-based artificial intelligence models, the meanings of the four dimensions of a feature tensor can generally be mapped as follows:

[0185] E4: N-Mini Batch Size

[0186] E3: H-3D tensor / image height

[0187] E2:W-3D tensor / image width

[0188] E1: Channel or class of a C-3D tensor

[0189] For machine learning or recurrent neural network-based artificial intelligence models, the meanings of the four dimensions of a 4D feature tensor can generally be mapped as follows:

[0190] E4: T - Number of timesteps or models

[0191] E3: Reserved, generally set to 1

[0192] E2:N mb - Mini badge size

[0193] E1:L-Feature

[0194] The NNPA Data Layout Format 0 provides, for example, two-dimensional data locality with 4K byte blocks (pages) of data, and 4K byte block data alignment for the outer dimensions of the generated tensor.

[0195] Pad element bytes are ignored in the input tensor and are unpredictable in the output tensor. PER storage changes on pad bytes are unpredictable.

[0196] An example of an input data layout for a 4D feature tensor layout with dimensions E1, E2, E3, and E4 is shown in Figures 5A to 5C, and an exemplary output for a 4D feature tensor layout is illustrated in Figures 6A to 6C. Referring to Figure 5A, a 3D tensor 500 with dimensions E1, E2, and E3 is shown. In this example, each 3D tensor contains multiple 2D tensors 502. The number in each 2D tensor 502 describes the memory offset where each of its elements is located in memory. The input is used to lay out the data of the original tensor (e.g., the original 4D tensor in Figures 5A to 5C) in memory, as shown in Figures 6A to 6C, which correspond to Figures 5A to 5C.

[0197] In Figure 6A, for example, a unit of memory 600 (e.g., a memory page) contains a pre-selected number (e.g., 32) of rows 602, each identified by, for example, e2_page_idx, and each row has elements 604 of a pre-selected number (e.g., 64), each identified by, for example, e1_page_idx. If a row does not contain the pre-selected number of elements, it is padded 606 (referred to as row padding or E1 padding); if a memory unit does not have the pre-selected number of rows, it is padded (referred to as page padding or E2 padding) 608. For example, row padding is, for example, zero or other values, and page padding is, for example, an existing value, zero, or other values.

[0198] In one example, the output elements of a row are provided in memory (e.g., on a page) based on the element position in the E1 direction of its corresponding input. For example, referring to Figure 5A, the element positions 0, 1, and 2 of the three matrices shown (e.g., the same element position in each matrix) are shown on page 0, row 0, as in Figure 6A. In this example, the 4D tensor is small, and all the elements of each 2D tensor representing the 4D tensor fit on one page. However, this is just one example. A 2D tensor can contain one or more pages. As shown in Figure 2A, the 2D tensor in that example contains 12 pages. However, this is just one example. Again, a 2D tensor can contain one or more pages. If a 2D tensor is generated based on a reformatting of a 4D tensor, the number of pages in the 2D tensor is based on the size of the 4D tensor. In one example, one or more ceil functions are used to determine the number of rows in the 2D tensor and the number of elements in each row, indicating how many pages are used. Other transformations are possible.

[0199] In addition to 4D feature tensor layouts, in one example, a neural network processor may support 4D kernel tensors that reconstruct the elements of a 4D tensor, reducing the number of memory accesses and data acquisition steps when performing certain artificial intelligence (e.g., neural network processing assistance) operations such as convolution. For example, a row-first general-purpose 4D tensor using dimensions E4, E3, E2, and E1 is converted to the NNPA data layout format 1 4D kernel tensor described herein.

[0200] The resulting tensor can be represented, for example, as a 4D tensor with 64 elements, or as a 5D tensor with the following dimensions.

[0201]

number

number

[0202] The elements [e4][e3][e2][e1] of the general-purpose tensor can be mapped to the following elements of the resulting 5D tensor.

[0203]

number

number

number

number

number

[0204] The resulting tensor may be larger than the generic tensor. Elements in the resulting tensor that do not have a corresponding element in the generic tensor are called padded elements.

[0205] The NNPA data layout format 1 for 64-element vectors considers the elements [fe1][fe4][fe3][fe2][fe0] of a 4D feature tensor, or an equivalent representation, as a 5D tensor of elements. Whether these elements are padded elements, or their corresponding elements in a generic 4D tensor with dimensions E4, E3, E2, E1, can be determined by the following formula.

[0206] If fe2 ≥ E2, then this is an E2 (or page) padded element.

[0207] Otherwise, if fe1*64+fe0≧E1, then this is an E1 (or row) padded element.

[0208] Otherwise, the corresponding elements in the general-purpose 4D tensor are:

[0209] [fe4][fe3][fe2][fe1*64+fe0]

[0210] For convolutional neural network-based artificial intelligence models, the meanings of the four dimensions of the kernel tensor can generally be mapped as follows:

[0211] E4: H-3D tensor / image height

[0212] E3:W-3D Tensor / Image Width

[0213] E2: Number of channels in a C-3D tensor

[0214] E1: Number of K kernels

[0215] The NNPA Data Layout Format 1 provides, for example, two-dimensional kernel parallelism within a 4KB block of data (page), and 4KB block data alignment, to the outer dimensions of the generated tensor for efficient processing.

[0216] Pad bytes are ignored in the case of input tensors. PER storage changes on pad bytes are unpredictable.

[0217] Similarly, exemplary data layout formats include 4D feature tensor layouts and 4D kernel tensor layouts, but other data layout formats may be supported by the processor (e.g., neural network processor 105). An indication of the supported data layout is obtained and placed in the query parameter block, for example, by setting one or more bits in field 338.

[0218] The query parameter block also includes other data attribute information, including supported size information about the data, according to one or more aspects of the present invention. Processors such as neural network processors typically have limitations based on internal buffer size, processing units, data bus structure, firmware limitations, etc., which can limit the maximum size of tensor dimensions and / or the overall size of the tensor. Therefore, the query function provides fields to communicate these limitations to the application. For example, based on the execution of the query function, the processor obtains various data sizes, such as the maximum dimension index size (e.g., 65,536 elements) and the maximum tensor size (e.g., 8GB), and includes this information in fields 340 and 342 of the parameter block (e.g., parameter block 330), respectively. More, less, and / or other size information may also be supported by the processor (e.g., neural network processor 105) and therefore can be obtained and placed in the parameter block, e.g., fields 340, 342 and / or other fields. In other embodiments, the limits may be smaller or larger, and / or the size may be in other units such as bytes instead of elements, or elements instead of bytes, etc. Furthermore, other embodiments allow for different maximum sizes for each dimension, rather than the same maximum size for all dimensions. Many modifications are possible.

[0219] According to one or more aspects of the present invention, a query function is provided that conveys detailed information relating to a particular model of a selected processor (e.g., a neural network processor 105). The detailed information includes, for example, model-dependent information relating to a particular processor. (The processor may also support standard data attributes, such as standard data types and standard data layouts, which are implicit and not necessarily presented by the query function; in other embodiments, the query function may indicate all or various selected subsets of data attributes, etc.) Exemplary information is provided, but other information may be provided in other embodiments. The acquired information, which may differ for different models of processors and / or different processors, is used to perform artificial intelligence and / or other processing. The artificial intelligence and / or other processing may employ, for example, one or more non-query functions of neural network processing support instructions. A particular non-query function employed in the processing is performed by executing the neural network processing support instructions one or more times and specifying a non-query specific function.

[0220] Further details of exemplary non-query functions supported by neural network processing support instructions are described below (more, fewer, and / or other functions may be supported in other embodiments).

[0221] Function code 16: NNPA-ADD (addition)

[0222] When the NNPA-ADD function is specified, each element of input tensor 1 described by tensor descriptor 1 is added to the corresponding element of input tensor 2 described by tensor descriptor 2, and the resulting sum is placed in the corresponding element of the output tensor described by the output tensor descriptor.

[0223] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type=0), a response code, e.g., 0010 hex or 0011 hex, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0224] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and output tensor are identical; otherwise, an exception for generic operand data will be recognized.

[0225] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage area address field are ignored.

[0226] Function code 17: NNPA-SUB (subtraction)

[0227] When the NNPA-SUB function is specified, each element of input tensor 2 described by tensor descriptor 2 is subtracted from the corresponding element of input tensor 1 described by tensor descriptor 1, and the resulting difference is placed in the corresponding element of the output tensor.

[0228] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type=0), a response code, e.g., 0010 hex or 0011 hex, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0229] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and output tensor are identical; otherwise, an exception for generic operand data will be recognized.

[0230] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage area address field are ignored.

[0231] Function code 18: NNPA-MUL (multiplication)

[0232] When the NNPA-MUL function is specified, the product of each element (multiplier) of input tensor 1 described by tensor descriptor 1 and the corresponding element (multiplicand) of input tensor 2 described by tensor descriptor 2 is placed in the corresponding element of the output tensor.

[0233] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type=0), a response code, e.g., 0010 hex or 0011 hex, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0234] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and output tensor are identical; otherwise, an exception for generic operand data will be recognized.

[0235] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage area address field are ignored.

[0236] Function code 19: NNPA-DIV (division)

[0237] When the NNPA-DIV function is specified, each element (dividend) of input tensor 1 described by tensor descriptor 1 is divided by the corresponding element of input tensor 2 (divisor) described by tensor descriptor 2, and the quotient is placed in the corresponding element of the output tensor.

[0238] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type=0), a response code, e.g., 0010 hex or 0011 hex, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0239] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and output tensor are identical; otherwise, an exception for generic operand data will be recognized.

[0240] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage area address field are ignored.

[0241] Function code 20: NNPA-MIN (minimum value)

[0242] When the NNPA-MIN function is specified, each element of input tensor 1, described by tensor descriptor 1, is compared with the corresponding element of input tensor 2, described by tensor descriptor 2. The smaller of the two values ​​is placed in the corresponding element of the output tensor descriptor. If both values ​​are equal, the values ​​are placed in the corresponding elements of the output tensor.

[0243] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0), or if the data type in any of the specified tensor descriptors does not specify the NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010 hex, or 0011 hex is set in general-purpose register 0 respectively, and the instruction completes with a condition code, e.g., 1.

[0244] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and the output tensor are the same; otherwise, a general operand data exception is recognized. <000089\> In one example, output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and the function-specific save area address field are ignored.

[0246] Function code 21: NNPA-MAX (maximum value)

[0247] When the NNPA-MAX function is specified, each element of input tensor 1 described by tensor descriptor 1 is compared with the corresponding element of input tensor 2 described by tensor descriptor 2. The larger of the two values is placed in the corresponding element of the output tensor descriptor. If both values are the same, the value is placed in the corresponding element of the output tensor.

[0248] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0), or if the data type in any of the specified tensor descriptors does not specify the NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010 hex, or 0011 hex is set in general-purpose register 0 respectively, and the instruction completes with a condition code, e.g., 1.

[0249] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and the output tensor are the same; otherwise, a general operand data exception is recognized.

[0250] In one example, output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and the function-specific save area address field are ignored.

[0251] Function code 32: NNPA-LOG (natural logarithm)

[0252] When the NNPA-LOG function is specified, for each element of the input tensor described by tensor descriptor 1, if the element is greater than zero, the corresponding element in the output tensor described by the output tensor descriptor is the natural logarithm of that element. Otherwise, the corresponding element in the output tensor is not numerically representable and a value related to negative infinity in the target data type is stored. <​​​​​​​​​​​​​Function code 33: NNPA-EXP (exponential)

[0257] When the NNPA-EXP function is specified, for each element of the input tensor described by tensor descriptor 1, the corresponding element in the output tensor described by the output tensor descriptor is the exponent of that element.

[0258] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type=0), a response code, e.g., 0010 hex or 0011 hex, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0259] In one example, the shape, data layout, and data type of input tensor 1 and output tensor are identical; otherwise, a generic operand data exception will be recognized.

[0260] In one example, output tensor descriptor 2, input tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage area address field are ignored.

[0261] Function code 49: NNPA-RELU (Normalized Linear Unit)

[0262] When the NNPA-RELU function is specified, for each element of the input tensor described by tensor descriptor 1, if that element is less than or equal to zero, the corresponding element in the output tensor described by the output tensor descriptor is zero. Otherwise, the corresponding element in the output tensor is the minimum value of the elements in the input tensor and the clipping value specified in the function-specific parameter 1.

[0263] For example, function-specific parameter 1 defines the clipping value for the RELU operation. For instance, the clipping value is bits 16-31 of function-specific parameter 1. The clipping value is specified, for example, in the NNPA data type 1 format. A clipping value of zero indicates the use of the largest positive value, in other words, no clipping is performed. If a negative value is specified, a generic operand data exception is recognized.

[0264] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type=0), a response code, e.g., 0010 hex or 0011 hex, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0265] In one example, the shape, data layout, and data type of input tensor 1 and output tensor are identical; otherwise, a generic operand data exception will be recognized.

[0266] In one example, output tensor descriptor 2, input tensor descriptor 2, input tensor descriptor 3, and the function-specific storage area address field are ignored. In one example, function-specific parameters 2-5 include zero.

[0267] Function code 50: NNPA-TANH (tangent)

[0268] When the NNPA-TANH function is specified, for each element of the input tensor described by tensor descriptor 1, the value of the corresponding element in the output tensor described by the output tensor descriptor is the hyperbolic tangent of that element.

[0269] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0), or if the data type in any of the specified tensor descriptors does not specify the NNP data type 1 (e.g., data type = 0), the response code, e.g., 0010 hex, or 0011 hex, is set in general-purpose register 0 respectively, and the instruction completes with a condition code, e.g., 1.

[0270] In one example, the shapes, data layouts, and data types of input tensor 1 and the output tensor are the same; otherwise, a general operand data exception is recognized.

[0271] In one example, output tensor descriptor 2, input tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1 to 5, and the function-specific save area address field are ignored.

[0272] Function code 51: NNPA-SIGMOID

[0273] When the NNPA-SIGMOID function is specified, for each element of the input tensor described by tensor descriptor 1, the corresponding element in the output tensor described by the output tensor descriptor is the sigmoid of that element.

[0274] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0), or if the data type in any of the specified tensor descriptors does not specify the NNP data type 1 (e.g., data type = 0), the response code, e.g., 0010 hex, or 0011 hex, is set in general-purpose register 0 respectively, and the instruction completes with a condition code, e.g., 1.

[0275] In one example, the shape, data layout, and data type of input tensor 1 and output tensor are identical; otherwise, a generic operand data exception will be recognized.

[0276] In one example, output tensor descriptor 2, input tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage area address field are ignored.

[0277] Function code 52: NNPA-SOFTMAX

[0278] When the NNPA-SOFTMAX function is specified, for each vector in dimension 1 of input tensor 1, the corresponding vector in the output tensor is calculated as described below.

[0279] * The maximum value of the vector is calculated.

[0280] * The sum of the exponents of the differences between each element in dimension 1 of the vector and the maximum value calculated above is calculated. If both the elements in dimension 1 of the input vector and the maximum value calculated above are numerical, and the difference is not numerical, the result of the exponent for that element is forced to zero.

[0281] * For each element in the vector, an intermediate quotient is formed from the exponent of the difference between the element and the maximum value calculated above, divided by the sum calculated above. An arbitrarily selected activation function is applied to this intermediate quotient to form the corresponding element in the output vector.

[0282] This process is iterated over, for example, for all dimension4 index size x dimension3 index size x dimension2 index size vectors in dimension1.

[0283] In one example, parameter 1, which is specific to the NNPA-SOFTMAX function, controls the activation function. For example, the ACT field of function-specific parameter 1 (e.g., bits 28-31) specifies the activation function. Example activation functions include the following:

[0284] ACT activation function

[0285] 0 The activation function is not executed.

[0286] 1 LOG

[0287] 2-15 are booked.

[0288] If a reserved value is specified for the ACT field, for example, a response code of F001 hex is reported, and the calculation is completed with a condition code of, for example, 1.

[0289] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type=0), a response code, e.g., 0010 hex or 0011 hex, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0290] In one example, if the size of the input tensor's dimension 3 index is not equal to 1, a response code of, for example, F000 hex is stored, and the instruction is completed with a condition code of, for example, 1.

[0291] In one example, the shape, data layout, and data type of input tensor 1 and output tensor are identical; otherwise, a generic operand data exception will be recognized.

[0292] In one example, output tensor descriptor 2, input tensor descriptor 2, and input tensor descriptor 3 are ignored. In one example, function-specific parameters 2-5 include zero.

[0293] The 8KB function-specific storage area may be used by this function.

[0294] In one embodiment, when obtaining a vector in dimension 1, the elements may not be contiguous in memory, depending on the specified data layout format. The result may be more inaccurate if all elements of the dimension 1 vector of input tensor 1 contain the largest possible negative number that can be represented in the specified data type.

[0295] Function code 64: NNPA-BATCHNORM (Batch Normalization)

[0296] When the NNPA-BATCHNORM function is specified, for each vector in dimension 1 of the input tensor, the corresponding vector in dimension 1 of the output tensor is calculated by multiplying each element of the vector by the corresponding element in the dimension 1 vector that makes up the input tensor. Then, the full-precision product is added to the corresponding element in the dimension 1 vector that makes up the input tensor, and then rounded to the precision of the specified data type of the output tensor. This process is iterated over, for example, for all dimension 4 index size x dimension 3 index size x dimension 2 index size vectors in dimension 1.

[0297] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type=0), a response code, e.g., 0010 hex or 0011 hex, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0298] In one example, the following conditions are true; otherwise, a generic operand data exception is recognized.

[0299] * The shape and data layout of input tensor 1 and output tensor are identical.

[0300] * The input tensor and output tensor have the same data type.

[0301] * The dimension 1 index size of input tensors 1, 2, and 3 and the output tensor are identical.

[0302] * The index sizes of dimensions 2, 3, and 4 of input tensors 2 and 3 are 1.

[0303] In one example, the output tensor descriptor 2 and the function-specific storage area address field are ignored. In another example, function-specific parameters 2-5 include zero.

[0304] Function code 80: NNPA-MAXPOOL2D Function code 81: NNPA-AVGPOOL2D

[0305] When either the NNPA-MAXPOOL2D or NNPA-AVGPOOL2D function is specified, input tensor 1, described by the input tensor 1 descriptor, is reduced by the specified operation and the input window is combined. The input window is selected by moving a 2D sliding window across dimension indices 2 and 3. The combined window is the element in the output tensor. The sliding window dimension is described, for example, by function-specific parameters 4 and 5. The amount of sliding window moved across the input tensor when computing adjacent output tensor elements is called the stride. The sliding window stride is specified, for example, by function-specific parameters 2 and 3. When the NNPA-MAXPOOL2D operation is specified, the Max operation defined below is performed on the window. When the NNPA-AVGPOOL2D operation is specified, the AVG operation defined below is performed on the window. If the specified padding type is valid, all elements in the window are added to the set used to compute the resulting output elements. If the specified padding type is Same, depending on the window's position, only a subset of elements from the window may be added to the set used to calculate the resulting output elements.

[0306] In one example, the CollectElements operation adds an element to a set of elements and increments the number of elements in the set. The set becomes empty each time the window's starting position moves. It is unpredictable whether elements not needed to perform the operation will be accessed.

[0307] Max operation: In one example, the maximum value of a set of elements in a window is calculated by comparing all elements in the set with each other and returning the largest value.

[0308] AVG (Average Value) calculation: In one example, the average value of a set of elements in a window is calculated by dividing the sum of all elements in the set by the number of elements in the set.

[0309] In one example, the fields are assigned as follows:

[0310] * Parameter 1, specific to the pooling function, controls the padding type. For example, bits 29-31 of function-specific parameter 1 contain the PAD field, which specifies the padding type. Examples of types include, for example, the following:

[0311] PAD Padding type

[0312] 0 Valid

[0313] 1. Same

[0314] 2-7 Booked

[0315] If a reserved value is specified for the PAD field, for example, a response code of F000 hex is reported, and the calculation is completed with a condition code, for example, 1.

[0316] In one example, bit positions 0-28 for function-specific parameter 1 are reserved, including zero.

[0317] * Function-specific parameter 2 may include, for example, a 32-bit unsigned binary integer specifying the dimension 2 stride (D2S), which specifies the number of elements the sliding window moves across in dimension 2.

[0318] * Function-specific parameter 3 includes, for example, a 32-bit unsigned binary integer that specifies the dimension 3 stride (D3S), which specifies the number of elements the sliding window moves across in dimension 3.

[0319] * Function-specific parameter 4 includes, for example, a 32-bit unsigned binary integer that specifies the dimension 2 window size (D2WS), which specifies the number of elements in dimension 2 that the sliding window contains.

[0320] * Function-specific parameter 5 includes, for example, a 32-bit unsigned binary integer that specifies the dimension 3 window size (D3WS), which specifies the number of elements in dimension 3 that the sliding window contains.

[0321] In one example, the specified values ​​for function-specific parameters 2-5 are less than or equal to the maximum dimension index size, and the specified values ​​for function-specific parameters 4-5 are greater than zero; otherwise, a response code, e.g., 0012 hex, is reported, and the operation is completed with a condition code, e.g., 1.

[0322] If both the Dimension 2 stride and Dimension 3 stride are zero, and either the Dimension 2 window size or the Dimension 3 window size is greater than, for example, 1024, a response code, e.g., F001 hex, is stored. If both the Dimension 2 stride and Dimension 3 stride are greater than, for example, zero, and either the Dimension 2 window size or the Dimension 3 window size is greater than, for example, 64, a response code, e.g., F002 hex, is stored. If both the Dimension 2 stride and Dimension 3 stride are greater than, for example, zero, and either the Dimension 2 stride or the Dimension 3 stride is greater than, for example, 30, a response code, e.g., F003 hex, is stored. If both the Dimension 2 stride and Dimension 3 stride are greater than, for example, zero, and either the input tensor Dimension 2 index size or the input tensor Dimension 3 index size is greater than, for example, 1024, a response code, e.g., F004 hex, is stored. For all of the above conditions, the instruction completes with a condition code, e.g., 1.

[0323] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data type=0), a response code, e.g., 0010 hex or 0011 hex, is set in general register 0, and the instruction is completed with a condition code, e.g., 1.

[0324] In one example, if the following conditions are true, a generic operand data exception will be recognized.

[0325] * The input tensor and the output tensor have the same Dimension 4 index size and Dimension 1 index size.

[0326] * The data layout and data types of the input and output tensors are identical.

[0327] * If both Dimension 2 stride and Dimension 3 stride are zero, in one example the following additional condition is true:

[0328] * The input tensor dimension 2 index size is equal to the dimension 2 window size.

[0329] * The input tensor's 3-index size is equal to the 3-window size.

[0330] * The output tensor has a dimension 2 index size and a dimension 3 index size of 1.

[0331] * The specified padding is valid.

[0332] * If either the Dimension 2 stride or the Dimension 3 stride is non-zero, then in one example, both strides are non-zero.

[0333] * If both Dimension 2 stride and Dimension 3 stride are greater than zero, in one example the following additional condition is true:

[0334] * When specified padding is enabled, the dimension2 window size is less than or equal to the dimension2 index size of the input tensor.

[0335] * When specified padding is enabled, the dimension3 window size is less than or equal to the dimension3 index size of the input tensor.

[0336] * When the specified padding is Same, the following relationship is satisfied between the dimension 2 index size and the dimension 3 index size of the input and output tensors (Pooling Same Padding):

[0337]

number

[0338]

number

[0339] Here,

[0340] IxDyIS: The size of the index y of the dimension y of the input tensor x as defined in the tensor descriptor x.

[0341] OxDyIS: The size of the index y of the dimension y of the output tensor x defined in the tensor descriptor x.

[0342] D2S Dimension 2 Stride

[0343] D3S Dimension 3 Stride

[0344] * When the specified padding is valid, the following relationship is satisfied between the dimension 2 index size and the dimension 3 index size of the input and output tensors (Pooling Valid Padding):

[0345]

number

[0346]

number

[0347] Here, D2WS is the window size for dimension 2, and D3WS is the window size for dimension 3.

[0348] The output tensor descriptor 2, input tensor descriptors 2 and 3, and the function-specific storage area address field are ignored.

[0349] Function code 96: NNPA-LSTMACT (Long-Short-Term Memory Activation)

[0350] When the NNPA-LSTMACT function is specified, input tensor 1, which is separated into four subtensors for each dimension 4 index value and described by input tensor 1 descriptor, is input to the LSTMACT operation along with input tensor 2, which is separated into four subtensors for each dimension 4 index value and described by input tensor 2 descriptor, and input tensor 3, which is described by input tensor 3 descriptor. At the end of the LSTMACT operation, the results are written to output tensor 1, described by output tensor 1 descriptor, and output tensor 2, described by output tensor 2 descriptor.

[0351] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data type=0), the response code 0010 hex or 0011 hex is set in general register 0, respectively, and the instruction completes with a condition code, e.g., 1.

[0352] In one embodiment, a generic operand data exception is recognized if the following condition is true, otherwise.

[0353] * The size of the Dimension 4 index for input tensor 3 and output tensors 1 and 2 is, for example, equal to 1.

[0354] * The size of the 4-index for dimension 4 for input tensor 1 and input tensor 2 is, for example, equal to 4.

[0355] * For example, the dimension 3 index size for all input tensors and the two output tensors is equal to, for example, 1.

[0356] * For example, all input tensors and the two output tensors have identical data layouts and data types.

[0357] * For example, the dimension-1 index size of all input tensors and the two output tensors are identical.

[0358] * For example, all input tensors and the two output tensors have the same dimension-2 index size.

[0359] In one example, the function-specific storage area address field is ignored. In another example, function-specific parameters 1-5 include zero.

[0360] Function code 97: NNPA-GRUACT (Gated Regressive Unit Activation)

[0361] When the NNPA-GRUACT function is specified, input tensor 1, described by the input tensor 1 descriptor and separated into three subtensors for each dimension 4 index value, is input to the GRUACT operation along with input tensor 2, described by the input tensor 2 descriptor and separated into three subtensors for each dimension 4 index value, and input tensor 3, described by the input tensor 3 descriptor. At the end of the GRUACT operation, the output tensor described by the output tensor descriptor is stored.

[0362] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data type=0), a response code, e.g., 0010 hex or 0011 hex, is set in general register 0, and the instruction is completed with a condition code, e.g., 1.

[0363] In one embodiment, a generic operand data exception is recognized if the following condition is true, otherwise.

[0364] * The size of the 4-index of the output tensor and the input tensor 3 is, for example, equal to 1.

[0365] * The size of the Dimension 4 index for input tensor 1 and input tensor 2 is, for example, equal to 3.

[0366] * For example, the dimension 3 index size for all input and output tensors is equal to, for example, 1.

[0367] * For example, the dimension-1 index size is the same for all input and output tensors.

[0368] * For example, the dimension 2 index size is the same for all input and output tensors.

[0369] * For example, all input and output tensors have the same data layout and data type.

[0370] In one example, the output tensor descriptor 2 and the function-specific storage area address field are ignored. In another example, function-specific parameters 2-5 include zero.

[0371] Function code 112: NNPA-CONVOLUTION

[0372] When the NNPA-CONVOLUTION function is specified, for each output element in the output tensor described by the output tensor 1 descriptor, a 3-dimensional input 1 window consisting of dimension indices 3, 2, and 1 is selected from input tensor 1 described by the input tensor 1 descriptor. A 3-dimensional input 2 window of the same size consisting of dimension indices 4, 3, and 2 is selected from tensor 2 described by the input tensor 2 descriptor. The elements in the input 1 window are multiplied with the corresponding elements in the input 2 window, and all the products are added together to produce the first sum. This first sum is added to the corresponding elements in input tensor 3 to calculate the value of the intermediate sum. The elements of the output tensor are the results of the specified activation function performed on the intermediate sum. If no activation function is specified, the output elements are equal to the intermediate sum.

[0373] If the specified padding type is valid, all elements in the window are used to calculate the resulting initial sum. If the specified padding type is Same, when calculating the resulting initial sum, some elements of the input-1 window may be implied to be zero, depending on the window's position.

[0374] It is unpredictable whether elements not necessary for performing the operation will be accessed.

[0375] In one example, the function-specific parameter fields used by the convolution function are assigned as follows:

[0376] * Parameter 1, specific to the NNPA-CONVOLUTION function, controls the padding type and activation function. In one example, bits 29-31 of function-specific parameter 1 include a PAD field that specifies the padding type. Exemplary types are as follows:

[0377] PAD Padding type

[0378] 0 Valid

[0379] 1 Same

[0380] 2-7 Booked

[0381] If a reserved value is specified for the PAD field, for example, a response code of F000 hex is reported, and the calculation is completed with a condition code, for example, 1.

[0382] Furthermore, in one example, bits 24-27 of parameter 1, which is specific to the NNPA-CONVOLUTION function, contain an activation field that specifies the activation function. An example function is as follows:

[0383] ACT activation function

[0384] 0 The activation function is not executed.

[0385] 1 RELU

[0386] 2-15 are booked.

[0387] If the RELU activation function is specified, the resulting output element values ​​are determined as follows: if the value of the intermediate sum is less than or equal to zero, the corresponding element in the output tensor is zero; otherwise, the corresponding element in the output tensor is the value of the intermediate sum and the minimum clipping value specified by the function-specific parameter 4.

[0388] If a reserved value is specified for the ACT field, for example, a response code of F001 hex will be reported, and the calculation will be completed with a condition code, for example, 1.

[0389] * Function-specific parameter 2 includes, for example, a 32-bit unsigned binary integer that specifies the dimension 2 (D2S) stride, which specifies the number of elements the sliding window moves in dimension 2.

[0390] * Function-specific parameter 3 includes, for example, a 32-bit unsigned binary integer that specifies the dimension 3 stride (D3S), which specifies the number of elements the sliding window moves across in dimension 3.

[0391] The specified values ​​for function-specific parameters 2-3 are less than the maximum dimension index size; otherwise, a response code, e.g., 0012 hex, is reported, and the operation is completed with a condition code, e.g., 1.

[0392] * Function-specific parameter 4 defines the clipping value for an optional RELU operation. In one example, the clipping value is located in bits 16-31 of function-specific parameter 4.

[0393] In one example, if the ACT field is zero, this field is ignored. If the ACT field specifies RELU, the clipping value is specified in the NNP data type 1 format. A clipping value of zero indicates that the largest positive value is used, in other words, no clipping is performed. If a non-zero value is specified, a generic operand data exception is recognized.

[0394] In one example, if the specified data layout in any of the specified tensor descriptors other than input tensor 2 does not specify a 4D feature tensor (e.g., data layout=0), or if the specified data layout in input tensor 2 does not specify a 4D kernel tensor (e.g., data layout=1), then a response code, e.g., 0010 hex, is set in general register 0, and the instruction completes with a condition code, e.g., 1. In another example, if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data type=0), then a response code, e.g., 0011 hex, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0395] If both the Dimension 2 stride and Dimension 3 stride are zero, and the Dimension 3 index size or Dimension 4 index size of input tensor 2 is greater than, for example, 448, a response code, e.g., F002 hex, is stored. If both the Dimension 2 stride and Dimension 3 stride are greater than zero, and either the Dimension 3 index size or Dimension 4 index size of input tensor 2 is greater than, for example, 64, a response code, e.g., F003 hex, is stored, and the operation is completed with a condition code, e.g., 1. If either the Dimension 2 stride or Dimension 3 stride is greater than, for example, 13, a response code, e.g., F004 hex, is stored, and the operation is completed with a condition code, e.g., 1.

[0396] In one example, if the following conditions are true, a generic operand data exception will be recognized.

[0397] * The data layouts of input tensor 1, input tensor 3, and output tensor are identical.

[0398] * All input and output tensors have the same data type.

[0399] * The input 3-tensor has index sizes of 1 for dimensions 2, 3, and 4.

[0400] * The size of the 4-index dimension of the output tensor is equal to the size of the 4-index dimension of the input 1-tensor.

[0401] * The size of the dimension 1 index of the output tensor is equal to the size of the dimension 1 index of the input tensor 2 and the size of the dimension 1 index of the input tensor 3.

[0402] * The size of the 1st index of the input tensor is equal to the size of the 2nd index of the input tensor.

[0403] * If both Dimension 2 stride and Dimension 3 stride are zero, in one example the following additional condition is true:

[0404] * The size of the input tensor dimension 2 (index size) is equal to the size of the input tensor dimension 3 (index size).

[0405] * The input tensor's input 1 tensor dimension 3 index size is equal to the input 2 tensor's dimension 4 index size.

[0406] * The output tensor has a dimension 2 index size and a dimension 3 index size of 1.

[0407] * The specified padding is valid.

[0408] * If either the Dimension 2 stride or the Dimension 3 stride is non-zero, then both strides are non-zero.

[0409] * If both Dimension 2 stride and Dimension 3 stride are greater than zero, in one example the following additional condition is true:

[0410] * When the specified padding is valid, the dimension 2 index size of input tensor 1 is greater than or equal to the dimension 3 index size of input tensor 2.

[0411] * When the specified padding is valid, the size of the 3-index of the input tensor is greater than or equal to the size of the 4-index of the input tensor.

[0412] * When the specified padding is Same, in one example, the following relationship is satisfied between the dimension 2 index size and dimension 3 index size of the input 1 tensor and the output tensor (convolution identical padding).

[0413]

number

[0414]

number

[0415] Here,

[0416] O1D2IS Output Tensor Dimension 2 Index Size

[0417] O1D3IS Output Tensor Dimension 3 Index Size

[0418] I1D2IS Input 1 Tensor Dimension 2 Index Size

[0419] I1D3IS Input 1 Tensor Dimension 3 Index Size

[0420] D2S Dimension 2 Stride

[0421] D3S Dimension 3 Stride

[0422] * When the specified padding is valid, in one example, the following relationship is satisfied between the dimension 2 index size and dimension 3 index size of the input tensor 1, and the dimension 3 index size and dimension 4 index size of the input tensor 2 and the output tensor (convolution-valid padding).

[0423]

number

[0424]

number

[0425] Here,

[0426] O1D2IS Output Tensor Dimension 2 Index Size

[0427] O1D3IS Output Tensor Dimension 3 Index Size

[0428] I1D2IS Input 1 Tensor Dimension 2 Index Size

[0429] I1D3IS Input 1 Tensor Dimension 3 Index Size

[0430] I2D3IS Input 2 Tensor Dimension 3 Index Size

[0431] I2D4IS Input 2 Tensor Dimension 4 Index Size

[0432] D2S Dimension 2 Stride

[0433] D3S Dimension 3 Stride

[0434] In one example, the output tensor descriptor 2 and the function-specific storage and address fields are ignored. In another example, the function-specific parameter 5 includes zero.

[0435] Function code 113: NNPA-MATMUL-OP (matrix multiplication operation)

[0436] When the NNPA-MATMUL-OP function is specified, in one example, each element in the output tensor described by the output tensor descriptor is computed as described below.

[0437] * Using the dimension-1 vector acquisition operation described below, a dimension-1 vector is selected from input tensor 1 described by the input tensor 1 descriptor.

[0438] * Using the dimension 2 vector acquisition operation described below, a dimension 2 vector is selected from the input tensor 2 described by the input tensor 2 descriptor.

[0439] * The intermediate dot product of a dimension 1 vector and a dimension 2 vector is calculated using the dot product operation described below.

[0440] * An intermediate dot product and an operation are performed on the elements of input tensor 3, described by an input tensor 3 descriptor having the same dimension index 4 and dimension index 1 values ​​as the output tensor elements. The resulting elements are stored in the output tensor. The fused operation is determined by a function-specific parameter 1, which is described below.

[0441] Dimension 1 Vector Acquisition Operation: For a specified output element, a Dimension 1 vector is selected from the Input 1 tensor, where the input Dimension 4 index is the output Dimension 4 index, the input Dimension 3 index is the output Dimension 3 index, and the input Dimension 2 index is the output Dimension 2 index.

[0442] Dimension 2 Vector Acquisition Operation: For a specified output element, a dimension 2 vector is selected from input tensor 2, where the input dimension 4 index is the output dimension 4 index, the input dimension 3 index is the output dimension 3 index, and the input dimension 1 index is the output dimension 1 index.

[0443] Dot product operation: The intermediate dot product of two vectors of the same size and data type is calculated as the sum of the products of each element in input vector 1 and the corresponding elements in input vector 2.

[0444] Fusion operations: Function-specific parameter 1 controls the operations performed on the intermediate dot product and the corresponding elements from the input tensor 3. In one example, the NNPA-MATMUL-OP function-specific parameter 1 includes, for example, the operation field in bits 24-31. The operation field specifies the operation to be performed. An example operation is shown below.

[0445] Calculation Calculation type

[0446] Add 0

[0447] 1. Compare when the dot product is high.

[0448] 2. Compare when the dot product is not low.

[0449] 3. Compare when the dot product and elements are equal.

[0450] 4. Dot product and comparison when elements are not equal.

[0451] 5. Compare when the dot product is not high.

[0452] 6. Compare when the dot product is low.

[0453] In one example, for the addition operation, the three elements of the input tensor are added to an intermediate dot product. For the comparison operation, the intermediate dot product is compared to the three elements of the input tensor, and if the comparison is true, the result is set to a value of, for example, +1 in the data type specified for the output tensor; otherwise, it is set to a value of, for example, +0.

[0454] In one example, all other values ​​in the calculated field are reserved. If a reserved value is specified for a calculation field, for example, a response code of F000 hex is reported, and the calculation is completed with a condition code, for example, 1.

[0455] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data type=0), a response code, e.g., 0010 hex or 0011 hex, is set in general register 0, and the instruction is completed with a condition code, e.g., 1.

[0456] In one embodiment, a generic operand data exception is recognized if the following condition is true, otherwise.

[0457] * All input and output tensors have the same dimension 4 index size.

[0458] * The dimension 3 index size of all input and output tensors is equal to 1.

[0459] * The size of the index of dimension 2 of the input tensor 3 is equal to 1.

[0460] * The input tensor 1 and the output tensor have the same dimension 2 index size.

[0461] * The index size of dimension 1 of input tensor 1 and the index size of dimension 2 of input tensor 2 are the same.

[0462] * The dimension 1 index size of input tensor 2, input tensor 3, and output tensor are the same.

[0463] * All input and output tensors have identical data layouts and data types.

[0464] In one embodiment, the output tensor descriptor 2 and the function-specific storage area address field are ignored. In the example, the function-specific parameters 2-5 include zero.

[0465] Function code 114: NNPA-MATMUL-OP-BCAST23 (Matrix multiplication operation - broadcast 23)

[0466] When the NNPA-MATMUL-OP-BCAST23 function is specified, in one example, each element in the output tensor described by the output tensor descriptor is computed as described below.

[0467] * Using the dimension-1 vector acquisition operation described below, the dimension-1 vector is selected from the described input tensor 1 by the input tensor 1 descriptor.

[0468] * Using the dimension 2 vector acquisition operation described below, a dimension 2 vector is selected from the input tensor 2 described by the input tensor 2 descriptor.

[0469] * The dot product of a dimension 1 vector and a dimension 2 vector is calculated using the dot product operation described below.

[0470] * The elements of input tensor 3 described by input tensor 3 descriptors that have the same dimension index 1 value as the output tensor elements are added to the previously calculated dot product and stored in the output tensor.

[0471] Dimension 1 Vector Acquisition Operation: For a specified output element, a Dimension 1 vector is selected from the Input 1 tensor, where the input Dimension 4 index is the output Dimension 4 index, the input Dimension 3 index is the output Dimension 3 index, and the input Dimension 2 index is the output Dimension 2 index.

[0472] Dimension 2 Vector Acquisition Operation: For a specified output element, a dimension 2 vector is selected from input tensor 2, where the input dimension 4 index is 1, the input dimension 3 index is the output dimension 3 index, and the input dimension 1 index is the output dimension 1 index.

[0473] Dot product: The intermediate product of two vectors of the same size and data type is calculated as the sum of the products of each element in input vector 1 and the corresponding elements in input vector 2.

[0474] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data type=0), a response code, e.g., 0010 hex or 0011 hex, is set in general register 0, and the instruction is completed with a condition code, e.g., 1.

[0475] In one embodiment, a generic operand data exception is recognized if the following condition is true, otherwise.

[0476] * The input tensor 1 and the output tensor have the same dimension 4 index size.

[0477] * The size of the 4th index of input tensor 2 and input tensor 3 is equal to 1.

[0478] * The dimension 3 index size of all input and output tensors is equal to 1.

[0479] * The size of the index of dimension 2 of the input tensor 3 is equal to 1.

[0480] * The input tensor 1 and the output tensor have the same dimension 2 index size.

[0481] * The index size of dimension 1 of input tensor 1 and the index size of dimension 2 of input tensor 2 are the same.

[0482] * The dimension 1 index size of input tensor 2, input tensor 3, and output tensor are the same.

[0483] * All input and output tensors have identical data layouts and data types.

[0484] In one embodiment, the output tensor descriptor 2 and the function-specific storage area address field are ignored. In one example, the function-specific parameters 1 to 5 include zero.

[0485] In one embodiment of the neural network processing support instructions, if the output tensor overlaps with either the input tensor or the parameter block, the result is unpredictable.

[0486] A specification exception is recognized when an attempt is made to execute a neural network processing support instruction, and the parameter block is not specified, for example, on a double word boundary.

[0487] An attempt is made to execute a neural network processing support instruction, and if, for example, there is a tensor descriptor mismatch, a generic operand data exception is recognized.

[0488] The resulting condition codes for neural network processing support instructions include, for example, 0 - successful completion; 1 - response code is set; 2 -; 3 - a quantity of data determined by the CPU is processed.

[0489] In one embodiment, the execution priority for neural network processing support instructions includes, for example, the following:

[0490] 1-7. Exceptions having the same priority as the program interrupt condition priority in general cases.

[0491] 8.A Condition code 1 resulting from the specification of a function code that is not assigned or installed.

[0492] 8.B Specification exceptions caused by parameter blocks not specified on double word boundaries

[0493] 9. Access exceptions regarding access to parameter blocks

[0494] 10. Condition code 1 resulting from a specified format of parameter block not supported by the model.

[0495] 11.A Condition code 1 resulting from the specified tensor data layout not being supported.

[0496] 11.B Generic operand data exceptions caused by different data layouts between tensor descriptors

[0497] 12.A Condition code 1 arising from conditions other than those included in items 8.A, 10, and 11.A above and 12.B.1 below.

[0498] 12.B.1 Condition Code 1 due to invalid output tensor data type for NNPA-RELU and NNPA-CONVOLUTION

[0499] 12.B.2 General operand data exceptions for invalid values ​​for parameter 1 specific to the NNPA-RELU function and parameter 4 specific to the NNPA-CONVOLUTION function.

[0500] 13.A Access exception for accessing the output tensor

[0501] 13.B Access exceptions for accessing input tensors

[0502] 13. Access exceptions regarding access to function-specific storage areas.

[0503] 14 Condition code 0

[0504] As described herein, a single instruction (e.g., a neural network processing support instruction) is configured to execute multiple functions, including a query function and multiple non-query functions. Each non-query function can operate on a tensor, such as a 4D tensor. To facilitate processing using tensors, according to one or more aspects of the present invention, a tensor (e.g., a 4D tensor) is reformatted into multiple 2D tensors (also referred to as subtensors) having, for example, specific characteristics, thereby improving processing. As shown, subtensors have readily computable addresses, can be loaded / stored in a single operation, increase bandwidth, and improve system performance. This is, for example, a result of starting the subtensor on a memory boundary and having a fixed dimension (made possible by using padding).

[0505] In one example, tensor reformatting is performed based on a processor (e.g., general-purpose processor 104) obtaining a data request and / or a neural network processing support instruction specifying a non-query function. The specified 4D tensor is reformatted using, for example, tensor descriptor information provided in a parameter block (e.g., tensor descriptors 360, 365 in Figure 3G). Address information associated with the reformatted tensor is provided to a dedicated processor (e.g., neural network processor 105) for use in executing the function specified by the instruction.

[0506] One or more aspects of the present invention are closely related to computer technology, facilitating processing within computers and improving their performance. Subtensors and / or instructions can be used in many technical fields, including computer processing, artificial intelligence, recurrent neural networks, medical processing, engineering, automotive technology, and manufacturing. As described herein, by using subtensors of reformatted original tensors, certain optimizations are provided, including optimizations in the execution of complex calculations used in various technical fields, improving those fields by increasing bandwidth, providing efficiency, and / or reducing execution time.

[0507] Further details of one embodiment that facilitates processing in a computing environment relating to one or more aspects of the present invention are described with reference to Figures 7A and 7B.

[0508] Referring to Figure 7A, in one embodiment, a tensor of a first choice dimension is obtained 700. The tensor is reformatted to provide one or more subtensors of a second choice dimension 702. Reformatting includes, for example, determining several subtensors to be used to represent the tensor 704, generating several subtensors (the subtensors start at the boundaries of memory units) 706, and reorganizing the data of the tensor so that it fits within several subtensors 708.

[0509] By generating subtensors that start on the memory unit boundaries, processing is facilitated by reducing the number of address translation requests and increasing the data transfer rate. Furthermore, the memory address for the subtensor is easily computable, reducing the overhead of complex address calculations.

[0510] In the example, determining several subtensors involves using one or more ceil functions based on at least the number of element values ​​contained in the tensor and the size of the memory units.710

[0511] In one example, reconstructing data for at least one subtensor of several subtensors involves padding at least one subtensor of several subtensors in at least one dimension to complete at least one subtensor.712 Padding provides a fixed-dimension subtensor and provides efficiency in address calculation and data transfer.

[0512] In one example, reconstructing data for selected subtensors of several subtensors involves selecting data at the same position in multiple matrices representing tensor 716, placing the selected data in one row of the selected subtensor 718, and repeating the selection and placement for multiple positions in multiple matrices 720.

[0513] Furthermore, referring to Figure 7B, in one example, reconstructing the data involves padding one or more rows of the selected subtensor based on the fact that one or more rows have less data than can be accommodated by the size of the selected subtensor 730; and padding the selected subtensor with one or more additional padded rows based on the fact that the selected subtensor has fewer rows than can be accommodated by the size of the selected subtensor 732. Similarly, padding provides a fixed-dimension subtensor and provides efficiency in address calculation and data transfer.

[0514] For example, a memory unit is a memory page, and the boundary of a memory unit is a page boundary.734 Processing is facilitated by generating subtensors that start on page boundaries, thereby reducing the number of address translation requests and increasing the data transfer rate.

[0515] In one example, one or more subtensors are provided to a processor for use in tensor computation 740. Providing one or more subtensors includes, for example, providing the processor used to store the data of one or more subtensors in a single operation 742, address information and information related to a second selection dimension. This enables block storage of data in a single operation and improves system performance.

[0516] In one example, providing one or more subtensors includes providing address information and information related to the second choice dimension to a processor used to load the data of one or more subtensors in a single operation.744 This enables block loading of data in a single operation and improves system performance.

[0517] In one example, the first chosen dimension is 4 dimensions, and the second chosen dimension is 2 dimensions, where one or more 2-dimensional subtensors represent one 4-dimensional tensor.

[0518] Other variations and embodiments are also possible.

[0519] Aspects of the present invention can be used in many types of computing environments. Another example of a computing environment for incorporating and using one or more aspects of the present invention is described with reference to Figure 8A. As an example, the computing environment in Figure 8A is based on the z / Architecture® instruction set architecture provided by International Business Machines Corporation in Armonk, New York. However, the z / Architecture instruction set architecture is just one exemplary architecture. Similarly, the computing environment may be based on other architectures, including but not limited to the Intel® x86 architecture, other architectures from International Business Machines Corporation, and / or architectures from other companies. Intel is a trademark or registered trademark of Intel Corporation or its subsidiaries in the United States and other countries.

[0520] In one example, the computing environment 10 includes a central electronic processing unit (CEC) 11. The central electronic processing unit 11 includes, for example, one or more processors such as one or more general-purpose processors (i.e., a central processing unit (CPU) 13) and one or more application-specific processors (e.g., a neural network processor 31), and multiple components such as memory 12 (i.e., system memory, main memory, main storage, central storage, and storage) coupled to an input / output (I / O) subsystem 14.

[0521] For example, one or more application-specific processors may be separate from one or more general-purpose processors, and / or at least one dedicated processor may be incorporated within at least one general-purpose processor. Other variations are also applicable.

[0522] The I / O subsystem 14 may be part of the central electronic processing unit or separate from it. It directs the flow of information between the main storage 12 and the input / output control unit 15 and input / output (I / O) devices 16 connected to the central electronic processing unit.

[0523] Many types of I / O devices may be used. One particular type is a data storage device 17. The data storage device 17 may store one or more programs 18, one or more computer-readable program instructions 19, and / or data, etc. The computer-readable program instructions may be configured to perform functions of embodiments of the present invention.

[0524] The central electronic processing unit 11 may be included in and / or linked to removable / non-removable, volatile / non-volatile computer system storage media. For example, it may be included in and / or linked to a magnetic disk drive for reading from or writing to a non-removable non-volatile magnetic medium (commonly called a “hard drive”), a removable non-volatile magnetic disk (e.g., a “floppy disk”), and / or an optical disk drive for reading from or writing to a removable non-volatile optical disk such as a CD-ROM, DVD-ROM, or other optical medium. It should be understood that other hardware and / or software components may be used in conjunction with the central electronic processing unit 11. Examples include, but are not limited to, microcode or millicode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0525] Furthermore, the central electronic processing unit 11 may be able to operate with a number of other general-purpose or dedicated computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that may be suitable for use with the central electronic processing unit 11 include, but are not limited to, personal computer (PC) systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments, which may include any of the systems or devices described above and similarly.

[0526] The central electronic processing unit 11 provides logical partition and / or virtualization support in one or more embodiments. In one embodiment, as shown in Figure 8B, the memory 12 includes, for example, one or more logical partitions 20, a hypervisor 21 that manages the logical partitions, and processor firmware 22. One example of the hypervisor 21 is the Processor Resource / System Manager (PR / SMTM) provided by International Business Machines Corporation of Armonk, New York. PR / SM is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction.

[0527] Each logical partition 20 can function as a separate system. That is, each logical partition can be reset independently and run a guest operating system 23, such as the z / OS® operating system provided by International Business Machines Corporation in Armonk, New York, or other control code 24, such as Coupling Facility Control Code (CFCC), and perform calculations using different programs 25. An operating system or application program running on a logical partition appears to have access to the whole and complete system, but in reality, only a portion of it is available. While the z / OS operating system is provided as an example, other operating systems provided by International Business Machines Corporation and / or other companies may be used according to one or more aspects of the present invention.

[0528] Memory 12 is coupled, for example, to a CPU 13 (Figure 8A), which is a physical processor resource that can be allocated to a logical partition. For example, a logical partition 20 may contain one or more logical processors, each of which represents all or part of the physical processor resources 13 that can be dynamically allocated to the logical partition.

[0529] In yet another embodiment, the central electronic processing unit provides virtual machine support (with or without logical partition support). As shown in Figure 8C, the memory 12 of the central electronic processing unit 11 includes, for example, one or more virtual machines 26, a virtual machine manager such as a hypervisor 27 that manages the virtual machines, and processor firmware 28. One example of a hypervisor 27 is the z / VM® hypervisor provided by International Business Machines Corporation of Armonk, New York. The hypervisor is sometimes referred to as a host. z / VM is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction.

[0530] The virtual machine support of the central electronic processing unit provides the capability to compute a large number of virtual machines 26, each capable of computed using a different program 29 and running a guest operating system 30, such as the Linux® operating system. Each virtual machine 26 can function as a separate system; that is, each virtual machine can be reset independently, run a guest operating system, and operate with different programs. The operating system or application program running in the virtual machine appears to have access to the whole and complete system, but in reality, only a portion of it is available. While z / VM and Linux are provided as examples, other virtual machine managers and / or operating systems may be used according to one or more aspects of the present invention. The registered trademark Linux® is used under a sublicense from the Linux Foundation, a worldwide exclusive licensee of Linus Torvalds, the owner of the trademark.

[0531] Another embodiment of a computing environment for incorporating and using one or more aspects of the present invention is described with reference to Figure 9A. In this example, the computing environment 36 includes, for example, a native central processing unit (CPU) 37, memory 38, and one or more input / output devices and / or interfaces 39, which are connected to each other via, for example, one or more buses 40 and / or other types of connections. For example, the computing environment 36 may include a PowerPC® processor provided by International Business Machines Corporation in Armonk, New York; an HP Superdome using an Intel® Itanium® II processor provided by Hewlett Packard, California; and / or other machines based on architectures provided by International Business Machines Corporation, Hewlett Packard, Intel Corporation, Oracle and / or other companies. (PowerPC is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction). Itanium is a trademark or registered trademark of Intel Corporation or its subsidiaries in the United States and other countries.

[0532] The native central processing unit 37 includes one or more native registers 41, such as one or more general-purpose registers and / or one or more application-specific registers, which are used during processing within the environment. These registers contain information representing the state of the environment at any given point in time.

[0533] Furthermore, the native central processing unit 37 executes instructions and code stored in memory 38. In one particular example, the central processing unit executes emulator code 42 stored in memory 38. This code enables a computing environment configured in one architecture to emulate another architecture. For example, the emulator code 42 enables machines based on architectures other than the z / Architecture instruction set architecture, such as PowerPC processors, HP Superdome servers, or others, to emulate the z / Architecture instruction set architecture and execute software and instructions developed based on the z / Architecture instruction set architecture.

[0534] Further details relating to the emulator code 42 are described with reference to Figure 9B. The guest instructions 43 stored in memory 38 consist of software instructions (e.g., those associated with machine instructions) developed to run on architectures other than the native CPU 37. For example, the guest instructions 43 may be designed to run on a processor based on the z / Architecture instruction set architecture, but instead are emulated on the native CPU 37, which may be, for example, an Intel Itanium II processor. In one example, the emulator code 42 includes an instruction fetch routine 44 for retrieving one or more guest instructions 43 from memory 38 and optionally providing local buffering for the retrieved instructions. It also includes an instruction translation routine 45 for determining the type of the retrieved guest instruction and translating the guest instruction into one or more corresponding native instructions 46. This translation includes, for example, identifying the function performed by the guest instruction and selecting a native instruction to perform that function.

[0535] Furthermore, the emulator code 42 includes an emulation control routine 47 for executing native instructions. The emulation control routine 47 may cause the native CPU 37 to execute a native instruction routine that emulates one or more previously obtained guest instructions, and upon completion of such execution, return control to an instruction fetch routine for emulating the acquisition of the next guest instruction or group of guest instructions. The execution of the native instruction 46 may include loading data from memory 38 into registers; returning data from registers to memory and storing it; or performing some type of arithmetic or logical operation as determined by the translation routine.

[0536] Each routine is implemented, for example, in software stored in memory and executed by the native central processing unit 37. In other examples, one or more routines or operations are implemented in firmware, hardware, software, or a combination thereof. Registers of the emulated processor may be emulated using the registers 41 of the native CPU or by using their locations in memory 38. In embodiments, the guest instruction 43, native instruction 46, and emulator code 42 may reside in the same memory or be distributed across different memory devices.

[0537] Instructions that can be emulated include neural network-assisted processing instructions described herein, according to one or more aspects of the present invention. Furthermore, other instructions and / or one or more aspects of tensor processing can be emulated according to one or more aspects of the present invention.

[0538] The computing environments described above are merely examples of computing environments that may be used. Other environments may be used, including but not limited to, unpartitioned environments, partitioned environments, cloud environments, and / or emulated environments, and embodiments are not limited to any one environment. While various examples of computing environments are described herein, one or more embodiments of the present invention may be used with many types of environments. The computing environments provided herein are merely examples.

[0539] Each computing environment can be configured to include one or more aspects of the present invention.

[0540] One or more aspects may relate to cloud computing.

[0541] While this disclosure includes a detailed description of cloud computing, it should be understood that the implementation of the teachings described herein is not limited to cloud computing environments. Rather, embodiments of the present invention can be implemented in conjunction with any other type of computing environment currently known or to be developed in the future.

[0542] Cloud computing is a service delivery model that enables convenient on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with service providers. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0543] The features are as follows:

[0544] On-demand self-service: Cloud consumers can unilaterally provision computing power, such as server time and network storage, automatically as needed, without requiring human interaction with the service provider.

[0545] Broad network access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin-client or thick-client platforms (e.g., mobile phones, laptops, and PDAs®).

[0546] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with various physical and virtual resources dynamically allocated and reallocated according to demand. Consumers generally do not have control or knowledge of the exact location of the resources provided, but they may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center), thus exhibiting a kind of location independence.

[0547] Rapid Scalability: Capabilities are provisioned quickly and flexibly, sometimes automatically, allowing for instant scaling out or rapid release and instant scaling in. To consumers, the available capacity for provisioning often appears unlimited and can be purchased at any time in any quantity.

[0548] Measuring Services: Cloud systems automatically control and optimize resource usage by leveraging metric capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, thereby providing transparency to both service providers and consumers.

[0549] The service model is as follows:

[0550] Software as a Service (SaaS): The capability offered to consumers is the use of a provider's applications running on a cloud infrastructure. These applications are accessible from various client devices through thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application capabilities, with the conceivable exception of limited, user-specific application configuration settings.

[0551] Platform as a Service (PaaS): The capability offered to consumers is the ability to deploy applications they have created or acquired, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they do have control over the deployed applications and, in some cases, the configuration of the application hosting environment.

[0552] Infrastructure as a Service (IaaS): The ability provided to consumers is to provision processing, storage, networking, and other basic computing resources, allowing consumers to deploy and run any software, including operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but have control over the operating system, storage, deployed applications, and, in some cases, limited control over selected networking components (e.g., host firewalls).

[0553] The deployment model is as follows:

[0554] Private Cloud: The cloud infrastructure is operated solely for the organization. It may be managed by the organization or a third party and may reside on-premises or off-premises.

[0555] Community Cloud: Cloud infrastructure is shared by multiple organizations to support a specific community that shares common interests (e.g., mission, security requirements, policies, and compliance considerations). Community clouds may be managed by those organizations or third parties and may reside on-premises or off-premises.

[0556] Public cloud: Cloud infrastructure is made available to the general public or large industry groups and is owned by organizations that sell cloud services.

[0557] Hybrid Cloud: Cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain separate entities but are bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing across clouds).

[0558] Cloud computing environments are service-oriented, emphasizing statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing lies an infrastructure that includes a network of interconnected nodes.

[0559] Referring here to Figure 10, an exemplary cloud computing environment 50 is shown. As illustrated, the cloud computing environment 50 includes one or more cloud computing nodes 52 that can communicate with local computing devices used by cloud consumers, such as a personal digital assistant (PDA) or cellular phone 54A, a desktop computer 54B, a laptop computer 54C, and / or an automotive computer system 54N. The nodes 52 can communicate with each other. They may be physically or virtually grouped (not shown) in one or more networks, such as a private cloud, community cloud, public cloud, or hybrid cloud, or a combination thereof, as described above. This enables the cloud computing environment 50 to provide infrastructure, platform, and / or software as a service that does not require cloud consumers to maintain resources on their local computing devices for that purpose. The types of computing devices 54A-N shown in Figure 10 are intended to be illustrative only, and it is understood that the computing nodes 52 and the cloud computing environment 50 can communicate with any type of computerized device via any type of network and / or network addressable connection (e.g., using a web browser).

[0560] Referring now to Figure 11, a set of functional abstraction layers provided by the cloud computing environment 50 (Figure 10) is shown. It should be understood in advance that the components, layers, and functionalities shown in Figure 11 are for illustrative purposes only and that embodiments of the present invention are not limited thereto. As illustrated, the following layers and corresponding functionalities are provided:

[0561] The hardware and software layer 60 comprises hardware and software components. Examples of hardware components include a mainframe 61; a RISC (Reduced Instruction Set Computer) architecture-based server 62; a server 63; a blade server 64; a storage device 65; and network and networking components 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0562] The virtualization layer 70 provides an abstraction layer that may provide examples of virtual entities, such as virtual servers 71, virtual storage 72, virtual networks 73 including virtual private networks, virtual applications and operating systems 74, and virtual clients 75.

[0563] In one example, the management layer 80 may provide the following functions: Resource provisioning 81 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing 82 provides cost tracking as resources are used within the cloud computing environment and billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification of cloud consumers and tasks, as well as protection of data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides cloud computing resource allocation and management to ensure that required service levels are met. Service level agreement (SLA) planning and execution 85 provides advance preparation and procurement of cloud computing resources where future requirements are anticipated in accordance with the SLA.

[0564] The workload layer 90 provides examples of functions that can be utilized in a cloud computing environment. Examples of workloads and functions that can be provided from this layer include mapping and navigation 91, software development and lifecycle management 92, virtual classroom education delivery 93, data analysis processing 94, transaction processing 95, and tensor and / or neural network-assisted processing 96.

[0565] Aspects of the present invention may be systems, methods, and / or computer program products in any possible level of technical detail integration. A computer program product may include a computer-readable storage medium (or a set of mediums) having computer-readable program instructions that cause a processor to execute aspects of the present invention.

[0566] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium may, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any preferred combination thereof. A non-exhaustive list of further specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or grooved structures on which instructions are recorded, and any preferred combination thereof. Computer-readable storage media as used herein should not be interpreted as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0567] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives computer-readable program instructions from the network and transfers them for storage in a computer-readable storage medium within the respective computing / processing device.

[0568] The computer-readable program instructions that perform the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk® or C++, and procedural programming languages ​​such as the C programming language or similar programming languages. The computer-readable program instructions can run as a standalone software package, either entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be to an external computer (for example, via the Internet using an Internet Service Provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may be personalized by executing computer-readable program instructions using state information of computer-readable program instructions in order to perform aspects of the present invention.

[0569] Aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0570] These computer-readable program instructions may be provided to a computer or other programmable data processing device processor to create a machine, thereby creating means for instructions executed via the computer or other programmable data processing device processor to implement functions / operations specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can instruct a computer, programmable data processing device and / or other device to function in a particular manner, thereby having a product in which a computer-readable storage medium storing instructions therein has instructions that implement modes of functions / operations specified in one or more blocks of a flowchart and / or block diagram.

[0571] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing device, or other device to execute a series of operational steps on the computer, other programmable device, or other device, thereby generating a computer implementation process in which the instructions executed on the computer, other programmable device, or other device implement the functions / operations specified in one or more blocks of a flowchart and / or block diagram.

[0572] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions described in a block may be performed in an order different from the order shown in the drawings. For example, two consecutively shown blocks may actually be implemented as a single step, executed simultaneously, substantially simultaneously, partially or entirely, with overlapping timelines, or blocks may be executed in reverse order depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented by a special-purpose hardware-based system that performs a specified function or operation, or executes a combination of special-purpose hardware instructions and computer instructions.

[0573] In addition to the above, one or more aspects may be offered, provided, deployed, managed, and serviced by a service provider offering management of customer environments. For example, a service provider may create, maintain, and support computer code and / or computer infrastructure that runs one or more aspects for one or more customers. In return, the service provider may receive payments from customers under subscription and / or fee agreements, for example. In addition or alternatively, the service provider may receive payments from the sale of advertising content to one or more third parties.

[0574] In one embodiment, the application may be deployed to perform one or more embodiments. As one example, the deployment of the application includes providing a computer infrastructure capable of performing one or more embodiments.

[0575] In a further embodiment, a computing infrastructure may be deployed that includes integrating computer-readable code into a computing system, where the code combined with the computing system can perform one or more embodiments.

[0576] In yet another embodiment, a process for integrating a computing infrastructure may be provided, which includes integrating computer-readable code into a computer system. The computer system includes a computer-readable medium, where the computer medium includes one or more embodiments. The code combined with the computer system is capable of performing one or more embodiments.

[0577] Various embodiments are described above, but these are merely examples. For example, computing environments of other architectures may be used to embed and / or use one or more embodiments. Furthermore, different instructions or operations may be used. In addition, different types of registers and / or different registers may be used. Furthermore, other data formats, data layouts, and / or data sizes may be supported. In one or more embodiments, one or more general-purpose processors, one or more application-specific processors, or a combination of general-purpose and dedicated processors may be used. Many modifications are possible.

[0578] Various embodiments are described herein. Furthermore, many modifications can be made without departing from the spirit of the embodiments of the invention. It should be noted that, unless otherwise contradictory, each embodiment or feature and its variations described herein may be combined with any other embodiment or feature.

[0579] Furthermore, other types of computing environments may be beneficial and utilized. For example, a data processing system suitable for storing and / or executing program code may be available, comprising at least two processors directly or indirectly connected to memory elements via a system bus. The memory elements may include, for example, local memory used while the program code is actually executed, mass storage, and cache memory providing temporary storage for at least some of the program code to reduce the number of times the code must be retrieved from mass storage during execution.

[0580] Input / output or I / O devices (including, but not limited to, keyboards, displays, pointing devices, DASDs, tapes, CDs, DVDs, thumb drives, and other memory media) can be connected to the system directly or via an intermediary I / O controller. Network adapters can also be connected to the system via an intermediary private or public network to enable data processing systems to be coupled with other data processing systems or remote printers or storage devices. Modems, cable modems, and Ethernet® cards are just a few of the types of network adapters available.

[0581] The terms used herein are intended to describe only specific embodiments and are not intended to be limiting. Where used herein, unless the context otherwise explicitly indicates, the singular forms "a," "an," and "the" are intended to include the plural forms as well. It should be further understood that the terms "contains" and / or "include," when used herein, specify the presence of the described features, integers, stages, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, stages, operations, elements, components, and / or groups thereof.

[0582] Any means or steps plus functional elements in the following claims are intended to include corresponding structures, materials, movements, and, if there are equivalents thereto, structures, materials, or actions for performing a function in combination with other claimed elements, especially if claimed. The descriptions of one or more embodiments are presented for illustrative and explanatory purposes, but are not intended to be exhaustive or to limit the disclosed forms. Many modifications and variations will be apparent to those skilled in the art. Embodiments have been selected and described in order to best illustrate various aspects and practical applications, and to enable those skilled in the art to understand various embodiments with various modifications to suit a particular intended use. [Item 1] A computer program product for facilitating processing within a computing environment, wherein the computer program product is Procedure for obtaining the tensor of the first selected dimension; and A procedure for reformatting the aforementioned tensor to provide one or more subtensors of the second choice dimension. A method comprising a program instruction stored collectively on one or more computer-readable storage media, wherein the reformatting is To determine several subtensors used to represent the aforementioned tensor; To generate several subtensors, where each subtensor starts at the boundary of a memory unit; and Reconstructing the tensor data so that it fits within the aforementioned subtensors. including, Computer program products. [Item 2] The computer program product described in item 1, wherein determining the aforementioned subtensors involves using one or more ceil functions based on at least the number of element values ​​contained in the tensor and the size of the memory unit. [Item 3] The computer program product according to item 1, wherein the reconstruction of data for at least one subtensor of the several subtensors includes padding the at least one subtensor of the several subtensors in at least one dimension to complete the at least one subtensor. [Item 4] Reconstructing the data for a selected subtensor of the aforementioned several subtensors is: Selecting data at the same position in multiple matrices representing the aforementioned tensor; Placing the selected data in one row of the selected subtensor; and Repeating the selection and placement for multiple positions in the aforementioned multiple matrices. Computer program products, including those listed in item 1. [Item 5] Reconstructing the data as described above further, Padding one or more rows of the selected subtensor on the basis that the one or more rows have less data than can be accommodated by the size of the selected subtensor; and Padding the selected subtensor with one or more additional padded rows, based on the fact that the selected subtensor has fewer rows than can be accommodated by the size of the selected subtensor. Computer program products, including those listed in item 4. [Item 6] A computer program product according to any one of items 1 to 5, wherein the memory unit is a memory page, and the boundary of the memory unit is a page boundary. [Item 7] The computer program product according to any one of items 1 to 6, further comprising a step of providing the one or more subtensors to a processor for use in tensor computation. [Item 8] The computer program product described in item 7, wherein the procedure for providing the one or more subtensors includes providing address information and information relating to the second selection dimension to the processor used to store the data of the one or more subtensors in a single operation. [Item 9] The computer program product described in item 7, wherein the procedure for providing the one or more subtensors includes providing address information and information relating to the second selection dimension to the processor used to load the data of the one or more subtensors in a single operation. [Item 10] The computer program product described in any one of items 1 to 9, wherein the first selection dimension is 4-dimensional, and the second selection dimension is 2-dimensional, where one or more 2-dimensional subtensors represent one 4-dimensional tensor. [Item 11] A computer system for facilitating processing within a computing environment, wherein the computer system is memory; and At least one processor that communicates with the aforementioned memory The computer system is equipped with, Procedure for obtaining the tensor of the first selected dimension; and A procedure for reformatting the aforementioned tensor to provide one or more subtensors of the second choice dimension. It is configured to perform a method comprising, where the reformatting is To determine several subtensors used to represent the aforementioned tensor; To generate several subtensors, where each subtensor starts at the boundary of a memory unit; and Reconstructing the data of the tensor so that it fits into the aforementioned subtensors. A computer system, including a computer system. [Item 12] Reconstructing the data for a selected subtensor of the aforementioned several subtensors is: Selecting data at the same position in multiple matrices representing the aforementioned tensor; Placing the selected data in one row of the selected subtensor; and Repeating the selection and placement for multiple positions in the aforementioned multiple matrices. The computer systems described in item 11, including the computer systems described in item 11. [Item 13] Reconstructing the data further involves, Padding one or more rows of the selected subtensor on the basis that the one or more rows have less data than can be accommodated by the size of the selected subtensor; and Padding the selected subtensor with one or more additional padded rows, based on the fact that the selected subtensor has fewer rows than can be accommodated by the size of the selected subtensor. The computer systems described in item 12, including the computer systems described in item 12. [Item 14] The method further comprises a step of providing the one or more subtensors to a processor for use in tensor computation, wherein the step of providing the one or more subtensors includes providing the processor used to store the data of the one or more subtensors in a single operation, address information and information relating to the second selection dimension. [Item 15] The computer system according to item 11, further comprising a step of providing the one or more subtensors to a processor for use in tensor computation, wherein the step of providing the one or more subtensors includes providing the processor used to load the data of the one or more subtensors in a single operation, address information and information relating to the second selection dimension. [Item 16] A computer implementation method that facilitates processing within a computing environment, wherein the computer implementation method is The step of obtaining the tensor of the first selected dimension; and The step of reformatting the tensor to provide one or more subtensors of the second selected dimension. The reformatting is provided, To determine several subtensors used to represent the aforementioned tensor; To generate several subtensors, where the subtensors start on the boundary of a memory unit; and Reconstructing the tensor data so that it fits within the aforementioned subtensors. Computer implementation methods, including those mentioned above. [Item 17] Reconstructing the data for a selected subtensor of the aforementioned several subtensors is: Selecting data at the same position in multiple matrices representing the aforementioned tensor; Placing the selected data in one row of the selected subtensor; and Repeating the selection and placement for multiple positions in the aforementioned multiple matrices. The computer implementation methods described in item 16, including the methods described in item 16. [Item 18] Reconstructing the data as described above further, Padding one or more rows of the selected subtensor on the basis that the one or more rows have less data than can be accommodated by the size of the selected subtensor; and Padding the selected subtensor with one or more additional padded rows, based on the fact that the selected subtensor has fewer rows than can be accommodated by the size of the selected subtensor. The computer implementation methods described in item 17, including the methods described in item 17. [Item 19] The computer implementation method according to item 16, further comprising the step of providing the one or more subtensors to a processor for use in tensor computation, wherein the step of providing the one or more subtensors includes providing the processor used to store the data of the one or more subtensors in a single operation, address information and information relating to the second selection dimension. [Item 20] The computer implementation method according to item 16, further comprising the step of providing the one or more subtensors to a processor for use in tensor computation, wherein the step of providing the one or more subtensors includes providing the processor used to load the data of the one or more subtensors in a single operation, address information and information relating to the second selection dimension.

Claims

1. A computer program for facilitating processing within a computing environment, wherein the computer program, when executed by a computer, Procedure for obtaining the tensor of the first selected dimension; A procedure for reformatting the tensor using a processor to provide one or more subtensors of the second selected dimension; and A procedure for providing at least the address information of one or more subtensors to another processor. The computer is made to perform a method comprising the following, where the reformatting is: Determining the number of subtensors used to represent the aforementioned tensor; To generate the aforementioned number of subtensors, where each subtensor starts at the boundary of a memory unit; and Reconstructing the tensor data so that it fits within the determined number of subtensors. Includes, The one or more subtensors are used by another processor to execute a function specified by an instruction executed on the processor. The output subtensor generated based on the execution of the aforementioned function is stored in memory in a single storage operation. Computer program.

2. The computer program according to claim 1, wherein determining the number of subtensors includes using one or more ceil functions based on at least the number of element values ​​included in the tensor and the size of the memory unit.

3. The computer program according to claim 1, wherein the reconstruction of data for at least one subtensor of the determined number of subtensors includes padding the at least one subtensor of the determined number of subtensors in at least one dimension to complete the at least one subtensor.

4. Reconstructing the data for the selected subtensors of the determined number of subtensors is: Selecting data at the same position in multiple matrices representing the aforementioned tensor; Placing the selected data in one row of the selected subtensor; and Repeating the selection and placement for multiple positions in the aforementioned multiple matrices. The computer program according to claim 1, including the computer program described in claim 1.

5. Reconstructing the data as described above further, Padding one or more rows of the selected subtensor on the basis that the one or more rows have less data than can be accommodated by the size of the selected subtensor; and Padding the selected subtensor with one or more additional padded rows, based on the fact that the selected subtensor has fewer rows than can accommodate by the size of the selected subtensor. The computer program according to claim 4, including the computer program described in claim 4.

6. The computer program according to claim 1, wherein the memory unit is a memory page, and the boundary of the memory unit is a page boundary.

7. The computer program according to claim 1, wherein the function performs one or more tensor computations.

8. The computer program according to claim 1, wherein the procedure for providing at least the address information includes providing information relating to the second selection dimension to the other processor.

9. The computer program according to claim 1, wherein the step of providing at least the address information includes providing information relating to the second selection dimension to the other processor used to load the data of the one or more subtensors in a single operation.

10. The computer program according to claim 1, wherein the first selection dimension is four-dimensional, and the second selection dimension is two-dimensional, where one or more two-dimensional subtensors represent one four-dimensional tensor.

11. A computer system for facilitating processing within a computing environment, wherein the computer system is memory; and At least one processor that communicates with the memory The computer system is equipped with, Procedure for obtaining the tensor of the first selected dimension; A procedure for reformatting the tensor using a processor to provide one or more subtensors of the second selected dimension; and A procedure for providing at least the address information of one or more subtensors to another processor. It is configured to perform a method comprising, where the reformatting is Determining the number of subtensors used to represent the aforementioned tensor; To generate the aforementioned number of subtensors, where each subtensor begins at the boundary of a memory unit; and Reconstruct the tensor data so that it fits into the number of subtensors determined above. Includes, The one or more subtensors are used by another processor to execute a function specified by an instruction executed on the processor. A computer system in which the output subtensor generated based on the execution of the aforementioned function is stored in the memory in a single storage operation.

12. Reconstructing the data for the selected subtensors of the determined number of subtensors is: Selecting data at the same position in multiple matrices representing the aforementioned tensor; Placing the selected data in one row of the selected subtensor; and Repeating the selection and placement for multiple positions in the aforementioned multiple matrices. The computer system according to claim 11, including the above.

13. Reconstructing the data further involves, Padding one or more rows of the selected subtensor on the basis that the one or more rows have less data than can be accommodated by the size of the selected subtensor; and Padding the selected subtensor with one or more additional padded rows, based on the fact that the selected subtensor has fewer rows than can accommodate by the size of the selected subtensor. The computer system according to claim 12, including the above.

14. The computer system according to claim 11, wherein the function performs one or more tensor computations.

15. The computer system according to claim 11, wherein the step of providing at least the address information includes providing information relating to the second selection dimension to the other processor used to load the data of the one or more subtensors in a single operation.

16. A computer implementation method that facilitates processing within a computing environment, wherein the computer implementation method is The step of obtaining the tensor of the first selected dimension; The steps include: reformatting the tensor using a processor to provide one or more subtensors of the second selected dimension; and A procedure for providing at least the address information of one or more subtensors to another processor. The reformatting is provided, Determining the number of subtensors used to represent the aforementioned tensor; To generate the determined number of subtensors, where each subtensor starts on the boundary of a memory unit; and Reconstructing the tensor data so that it fits within the determined number of subtensors. Includes, The one or more subtensors are used by another processor to execute a function specified by an instruction executed on the processor. A computer implementation method in which the output subtensor generated based on the execution of the aforementioned function is stored in memory in a single storage operation.

17. Reconstructing the data for the selected subtensors of the determined number of subtensors is: Selecting data at the same position in multiple matrices representing the aforementioned tensor; Placing the selected data in one row of the selected subtensor; and Repeating the selection and placement for multiple positions in the aforementioned multiple matrices. The computer implementation method according to claim 16, including the method described in claim 16.

18. Reconstructing the data as described above further, Padding one or more rows of the selected subtensor on the basis that the one or more rows have less data than can be accommodated by the size of the selected subtensor; and Padding the selected subtensor with one or more additional padded rows, based on the fact that the selected subtensor has fewer rows than can accommodate by the size of the selected subtensor. The computer implementation method according to claim 17, including the method described in claim 17.

19. The computer implementation method according to claim 16, wherein the function performs one or more tensor computations.

20. The computer implementation method according to claim 16, wherein at least the step of providing the address information includes providing information relating to the second selection dimension to the other processor used to load the data of the one or more subtensors in a single operation.

Images