Recurrent neural network cell activation to perform multiple actions in a single launch.

Abstract

Instructions are executed to perform recurrent neural network cell activations. The executing includes executing a plurality of operations of the recurrent neural network cell activations to provide a result of the recurrent neural network cell activations. The plurality of operations are executed in a single invocation of the instructions. The recurrent neural network cell activations are, for example, long short-term memory cell activations or gated recurrent unit cell activations.

Description

Technical Field

[0001] One or more aspects generally relate to facilitating processing within a computing environment, and more particularly to improving such processing.

Background Art

[0002] To improve processing in computing environments that are data and / or compute intensive, coprocessors such as artificial intelligence accelerators (also referred to as neural network processors or neural network accelerators) are utilized. Such accelerators provide, for example, a large amount of computing power used when performing involved computations such as computations on matrices or tensors.

[0003] Tensor calculations are used, for example, in complex processing including deep learning, which is a subset of machine learning. Deep learning or machine learning, which is one aspect of artificial intelligence, is used in a variety of technologies including, but not limited to, engineering, manufacturing, medical technology, automotive technology, computer processing, and the like.

[0004] Tensors and tensor calculations enable the input of large amounts of data and / or detailed data into deep learning processing. However, accelerators are limited by the data bandwidth to the accelerators. Previously, efforts have been made to address this limitation by using data locality and data reuse in the accelerators. Advancements in the use of tensors and / or the processing that uses such tensors will improve technologies that use machine learning, including computer processing.

Summary of the Invention

[0005] The shortcomings of prior art are overcome and further advantages are provided through the provisioning of computer program products to facilitate processing within a computing environment. A computer program product includes one or more computer-readable storage media and program instructions collectively stored on one or more computer-readable storage media to perform a method. The method includes performing an instruction to perform recurrent neural network cell activation. The execution includes performing multiple operations of recurrent neural network cell activation to provide the result of recurrent neural network cell activation. The multiple operations are performed in a single invocation of the instruction.

[0006] Using a single instruction call to perform multiple operations reduces complexity, lowers system resource usage, and improves system performance.

[0007] In one example, multiple operations include one or more sigmoid functions and one or more tangent functions. In another example, multiple operations include tensor element-wise addition and tensor element-wise multiplication operations.

[0008] For example, multiple operations include one or more sigmoid functions, one or more tangent functions, one or more tensor element-wise addition operations, and one or more tensor element-wise multiplication operations.

[0009] In one example, one or more inputs to an instruction include one or more concatenated tensors. These concatenated tensors may be used directly by instructions executed on an accelerator performing cell activation in a recurrent neural network, for example. The concatenated tensors may be accessed in a single operation, saving processing time and increasing processing speed. Furthermore, fewer tensor pointers need to be managed, and there are reductions in copying or reorganizing tensor data between accelerator invocations, resulting in improved processing speed.

[0010] In one example, the result is an output tensor, which, as an example, is the input to another invocation of the instruction.

[0011] For example, recurrent neural network cell activation includes long-term memory cell activation or gated recurrent unit cell activation.

[0012] In one example, performing multiple operations to activate recurrent neural network cells is carried out by an accelerator, generating intermediate computation data. This intermediate computation data is stored in the accelerator, for example.

[0013] In one example, performing multiple operations involves performing multiple operations on spatially close input data.

[0014] Computer implementation methods and systems relating to one or more embodiments are also described and claimed herein. Furthermore, services relating to one or more embodiments are also described and may be claimed herein.

[0015] Additional features and advantages are realized through the techniques described herein. Other embodiments and aspects are described in detail herein and are considered to be part of the claimed embodiments. [Brief explanation of the drawing]

[0016] One or more embodiments are specifically pointed out and expressly claimed as examples of the claims at the end of this specification. The purposes, features, and advantages of the foregoing and one or more embodiments will become apparent from the following detailed description when read in conjunction with the accompanying drawings. [Figure 1A] This figure shows one example of a computing environment that incorporates one or more aspects of the present invention. [Figure 1B] This figure shows further details of the processor shown in Figure 1A, according to one or more embodiments of the present invention. [Figure 2A]A diagram showing an example of a result tensor according to one or more aspects of the present invention. [Figure 2B] A diagram showing an example of multiplying concatenation weights by input features to provide an intermediate result used according to one or more aspects of the present invention. [Figure 2C] A diagram showing an example of a bias added to the intermediate result of FIG. 2B to provide the result tensor of FIG. 2A according to one or more aspects of the present invention. [Figure 2D] A diagram showing an example of a concatenated output tensor according to one or more aspects of the present invention. [Figure 3A] A diagram showing an example of a 2D tensor according to one or more aspects of the present invention. <## [Figure 3B] A diagram showing an example of a process used to create a tensor of a selected dimension according to one or more aspects of the present invention. [Figure 3C] A diagram showing an example of a process used to create a tensor of a selected dimension according to one or more aspects of the present invention. [[ID=)19]] [Figure 4A] A diagram showing an example of long short-term memory cell activation according to one or more aspects of the present invention. [Figure 4B] A diagram showing an example of gated recurrent unit cell activation according to one or more aspects of the present invention. [Figure 5A] A diagram showing an example of long short-term memory cell activation using chaining according to one or more aspects of the present invention. [Figure 5B] A diagram showing an example of long short-term memory cell activation using chaining according to one or more aspects of the present invention. [Figure 6A] A diagram showing an example of the format of a neural network processing support instruction according to one or more aspects of the present invention. [Figure 6B] A diagram showing an example of a general-purpose register used by a neural network processing support instruction according to one or more aspects of the present invention. [Figure 6C]A diagram showing an example of function code supported by neural network processing support instructions according to one or more aspects of the present invention. [Figure 6D] A diagram showing an example of another general-purpose register used by neural network processing support instructions according to one or more aspects of the present invention. [Figure 6E] A diagram showing an example of a parameter block used by a query function of neural network processing support instructions according to one or more aspects of the present invention. [Figure 6F] A diagram showing an example of a parameter block used by one or more non-query functions of neural network processing support instructions according to one or more aspects of the present invention. [Figure 6G] A diagram showing an example of a tensor descriptor used by neural network processing support instructions according to one or more aspects of the present invention. [Figure 7] A diagram showing an example of the format of a neural network processing (NNP) data type 1 data type according to one or more aspects of the present invention. [Figure 8A] A diagram showing an example of an input data layout used by neural network processing support instructions according to one or more aspects of the present invention. [Figure 8B] A diagram showing an example of an input data layout used by neural network processing support instructions according to one or more aspects of the present invention. [Figure 8C] A diagram showing an example of an input data layout used by neural network processing support instructions according to one or more aspects of the present invention. [Figure 9] (A) of FIG. 9 is a diagram showing an exemplary output corresponding to the input data layout of FIG. 8A according to one or more aspects of the present invention. (B) of FIG. 9 is a diagram showing an exemplary output corresponding to the input data layout of FIG. 8B according to one or more aspects of the present invention. (C) of FIG. 9 is a diagram showing an exemplary output corresponding to the input data layout of FIG. 8C according to one or more aspects of the present invention. [Figure 10A] This figure shows one example of facilitating processing within a computing environment according to one or more aspects of the present invention. [Figure 10B] This figure shows one example of facilitating processing within a computing environment according to one or more aspects of the present invention. [Figure 11A] This figure shows another example of a computing environment that incorporates or uses one or more aspects of the present invention. [Figure 11B] This figure shows one example of further details of the memory shown in Figure 11A, according to one or more embodiments of the present invention. [Figure 11C] This figure shows another example of further details of the memory shown in Figure 11A, according to one or more embodiments of the present invention. [Figure 12A] This figure shows yet another example of a computing environment that incorporates one or more aspects of the present invention. [Figure 12B] This figure shows further details of the memory shown in Figure 12A, according to one or more embodiments of the present invention. [Figure 13] This figure shows one embodiment of a cloud computing environment according to one or more aspects of the present invention. [Figure 14] This figure shows one example of an abstraction model layer according to one or more aspects of the present invention. [Modes for carrying out the invention]

[0017] According to one or more aspects of the present invention, the ability is provided to create tensors of a selected data layout format for use in recurrent neural networks, such as recurrent neural networks on long short-term memory (LSTM) architectures and / or gated recurrent unit (GRU) architectures. As an example, the selected data layout format includes, for example, a concatenated input and / or output format used in long short-term memory cell activation and / or gated recurrent unit cell activation.

[0018] Long-term memory is typically an artificial recurrent neural network architecture that includes, for example, cells that store states and multiple gates that control the flow of information into and out of the cells. These gates include, for example, input gates, output gates, and forget gates. A gated recurrent unit is another recurrent neural network architecture. It is similar to the long-term memory architecture but may have fewer parameters and lacks an output gate. Each network uses timesteps, where, at each timestep, an operation is performed on an input that produces an output. The output of one timestep may be the input to the next timestep. For each timestep, multiple activations (e.g., sigmoid, tanh) and other operations (e.g., addition, multiplication) are applied to the hidden state (H), input, and cell (c) states. While each of these small steps (e.g., activations, operations) can be performed efficiently locally to the processor, calling an accelerator for each of these steps can be detrimental to the overall performance of the recurrent neural network and / or system, for example, due to accelerator startup time. Therefore, according to one or more aspects of the present invention, individual activations and operations (for example, for one time step) are combined and executed as part of a single invocation of an instruction. This significantly increases processing speed and efficiency, for example, since there is only one invocation; intermediate computation data can be stored in the accelerator instead of being written back to memory; more computations can be performed in parallel in fewer cycles per computation by using the accelerator's SIMD (Single Instruction Multiple Data) width and pipelining properties; and higher precision can be used for intermediate results, resulting in improved accuracy and greater stability for LSTM / GRU operations.

[0019] Furthermore, in one or more embodiments, a single instruction uses a selected data layout format, providing spatially close input and / or output data, reducing address translation requests, and improving processing speed. The selected data layout format provides efficiency, where, for example, operations such as cell activation in a recurrent neural network can be chained without requiring a general-purpose processor to inspect / reconstruct data for each time step of cell activation.

[0020] One example of a selected data layout format is a concatenated input format according to one or more aspects of the present invention. To provide such a format, in one example, weight tensors used by recurrent neural network cells are transformed into reformatted weight tensors of a selected dimension (e.g., 2D reformatted tensors), which are then concatenated linearly to form a larger concatenated tensor. This allows, for example, cell activation and other operations performed on the resulting concatenated tensor to be executed in a single instruction invocation executed on an accelerator. The resulting concatenated tensor is a selected input format used directly by instructions to an accelerator performing cell activation on a recurrent neural network.

[0021] A further example of a selected data layout format is a concatenated output format, such as a 2D output tensor, according to one or more aspects of the present invention. The format is selected, for example, so that the output tensor can be accessed as a memory-contiguous subtensor that can be fed, for example, to the next time step of the computation for each time step. The time steps maintain their adjacency state in memory to return a final result consisting of the time steps as a single memory adjacency tensor.

[0022] One or more aspects of the present invention include reformatting a tensor to provide a reformatted tensor (which may also be called a subtensor) of a selected dimension (e.g., a 2D tensor) representing the original tensor. This optimizes operations including, but not limited to, memory address calculation, load / storage operations, and / or prefetching. As an example, a tensor is reformatted such that the reformatted tensor starts on the boundary of a memory unit (e.g., a memory page) and the information of the original tensor is rearranged to fit within the reformatted tensor (known as a tile) of a selected dimension (e.g., 2D). The reformatted tensor has an readily computable address and may be block-loaded and / or stored (e.g., loaded / stored in one operation), providing efficiency when using the reformatted tensor.

[0023] One example of an instruction that uses a concatenated input / output data format provided according to one or more aspects of the present invention and / or combines multiple operations (e.g., activation and / or other operations) of recurrent neural network cell activation is a neural network processing support instruction, which is a single instruction (e.g., a single designed hardware machine instruction in a hardware / software interface) configured to perform multiple functions. Each of the functions is configured as part of a single instruction (e.g., a single designed instruction), reducing the use of system resources and complexity and improving system performance.

[0024] The instructions may be part of a general-purpose processor instruction set architecture (ISA), which is dispatched by a program on a processor such as a general-purpose processor. They may be executed by the general-purpose processor, and / or one or more functions of the instructions may be executed by a dedicated processor, such as a coprocessor or accelerator configured for a specific function, which is coupled to or part of the general-purpose processor. Other variations are possible.

[0025] One embodiment of a computing environment that incorporates and uses one or more aspects of the present invention is described with reference to Figure 1A. As an example, the computing environment is based on the z / Architecture® instruction set architecture provided by International Business Machines Corporation, Armonk, New York. One embodiment of the z / Architecture instruction set architecture is described in the publication “z / Architecture Principles of Operation,” IBM Publication No. SA22-7832-12, 13th edition, September 2019, which is incorporated herein by reference in its entirety. However, the z / Architecture instruction set architecture is merely an illustrative architecture; other architectures and / or other types of computing environments of International Business Machines Corporation and / or other entities may include and / or use one or more aspects of the present invention. z / Architecture and IBM are trademarks or registered trademarks of International Business Machines Corporation in at least one jurisdiction.

[0026] Referring to Figure 1A, the computing environment 100 includes, for example, a computer system 102, which is shown in the form of a general-purpose computing device. The computer system 102 may include, but is not limited to, one or more general-purpose processors or processing units 104 (e.g., a central processing unit (CPU)), at least one dedicated processor such as a neural network processor 105, memory 106 (for example, known as system memory, main memory, main storage, central storage, or storage), and one or more input / output (I / O) interfaces 108, all coupled to one or more buses and / or other connections. For example, processors 104, 105 and memory 106 are coupled to the I / O interface 108 via one or more buses 110, and processors 104, 105 are coupled to one or more buses 111.

[0027] Bus 111 is, for example, a memory or cache coherence bus, and Bus 110 represents one or more of several types of bus structures, including, for example, a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of the various bus architectures. Such architectures include, but are not limited to, Industry Standard Architecture (ISA), Microchannel Architecture (MCA), Enhanced ISA (EISA), Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI).

[0028] For example, one or more dedicated processors (e.g., neural network processors) may be separate from, but coupled to, and / or embedded within, one or more general-purpose processors. Many variations are possible.

[0029] Memory 106 may include a cache 112, such as a shared cache, which may be coupled, for example, to the local cache 114 of the processor 104 and / or to the neural network processor 105 via one or more buses 111. Furthermore, memory 106 may include one or more programs or applications 116 and at least one operating system 118. Exemplary operating systems include the z / OS® operating system provided by International Business Machines Corporation, located in Armonk, New York. z / OS is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction. Other operating systems provided by International Business Machines Corporation and / or other entities may be used. Memory 106 may also include one or more computer-readable program instructions 120, which may be configured to perform functions of embodiments of the present invention.

[0030] Furthermore, in one or more embodiments, the memory 106 includes processor firmware 122. The processor firmware includes, for example, the processor's microcode or millicode. It includes, for example, data structures used in the implementation of hardware-level instructions and / or higher-level machine code. In one embodiment, it includes, for example, proprietary code typically delivered as microcode or millicode, including trusted software, microcode or millicode specific to the underlying hardware, which controls the operating system's access to the system hardware.

[0031] The computer system 102 may communicate with one or more external devices 130 such as a user terminal, tape drive, pointing device, display, etc., and one or more data storage devices 134, etc., for example via an I / O interface 108. The data storage device 134 may store one or more programs 136, one or more computer-readable program instructions 138, and / or data, etc. The computer-readable program instructions may be configured to perform functions of embodiments of the present invention.

[0032] The computer system 102 may communicate with the network interface 132, for example, via the I / O interface 108, which enables the computer system 102 to communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet), and provides communication with other computing devices or systems.

[0033] Computer system 102 may include and / or be coupled with removable / non-removable volatile / non-volatile computer system storage media. For example, it may include and / or be coupled with non-removable non-volatile magnetic media (typically called “hard drives”), magnetic disk drives for reading and writing to removable non-volatile magnetic disks (e.g., “floppy disks”), and / or optical disk drives for reading and writing to removable non-volatile optical disks such as CD-ROMs, DVD-ROMs, or other optical media. It should be understood that other hardware and / or software components may be used in conjunction with computer system 102. Examples include, but are not limited to, microcode or millicode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0034] Computer system 102 may be able to operate with a number of other general-purpose or dedicated computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with computer system 102 include, but are not limited to, personal computer (PC) systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices.

[0035] In one example, a processor (e.g., processor 104 and / or processor 105) includes several functional components (or subsets thereof) used to execute instructions. As shown in Figure 1B, these functional components include, for example, an instruction fetch component 150 that fetches the instructions to be executed; an instruction decoding unit 152 that decodes the fetched instructions and retrieves the operands of the decoded instructions; one or more instruction execution components 154 that execute the decoded instructions; a memory access component 156 that accesses memory for instruction execution if necessary; and a writeback component 158 ​​that provides the results of the executed instructions. One or more of these components may access and / or use one or more registers 160 in instruction processing. Furthermore, one or more of the components may include, or have access to, at least some of, other components used in, according to one or more aspects of the present invention, when providing concatenated input and / or output data formats, when combining multiple operations of cell activation functions, when performing tensor processing (including, but not limited to, generating and / or using reformatted tensors), and / or when performing, for example, neural network processing support operations of neural network processing support instructions (or other processing that may use one or more aspects of the present invention). One or more other components may include, for example, one or more combination / concatenation components 170, tensor component 171, and / or neural network processing support component 172 (and / or one or more other components).

[0036] According to one or more aspects of the present invention, processing within a computing environment is facilitated by providing an improved data format for use by a processor such as a dedicated processor (e.g., a neural network processor 105). For example, a concatenated input data format layout is provided, where multiple tensors of selected dimensions, such as multiple 2D tensors, are concatenated to create a concatenated tensor. Similarly, in one example, a concatenated output data format is provided, where multiple output tensors are concatenated. Further details regarding the concatenated input / output data layout format are described with reference to Figures 2A to 2D. In the figures, t refers to the time step, Nmb refers to the batch size, s refers to the size, and l is the length of the input feature.

[0037] Referring to Figure 2A, an example of a connected tensor input (also referred to herein as a result tensor) 200 is shown. In this example, multiple 2D tensors 202 having size s are concatenated (e.g., linearly) to create a larger connected tensor 200 having size 4s. In one example, the connected tensor 200 is multiplied by multiple (e.g., four) connected weight tensors (e.g., W) multiplied by the feature input X(Xi). f , W i ,

number

[0038] In one example, each weight tensor in Figure 2B is a reformatted 2D tensor provided to facilitate tensor processing. The weight tensors are independently converted to 2D reformatted tensors, which are then concatenated to provide a larger tensor. The resulting tensor is, according to one aspect of the present invention, an input format directly used by instructions (e.g., neural network processing support instructions) on an accelerator (e.g., processor 105) that performs cell activation of a recurrent neural network. This makes it possible to perform matrix multiplication of cell activation over multiple time steps in a single instruction executed on the accelerator. Each reformatted 2D tensor, according to one aspect of the present invention, starts on the boundary of a memory unit (e.g., a memory page boundary), and the information of the original tensor is rearranged in the reformatted tensor. The dimension of the tensor in the dimension of the reformatted tensor is rounded up to the next full tile in that dimension (e.g., padding is provided to create a fixed-size tensor, e.g., a 2D tensor). For example, row padding 216 and / or page padding 218 are provided to create a fixed-size tensor, as described herein. This allows access to each tensor on a memory unit boundary (e.g., a page boundary) and facilitates the calculation of the address of any 2D tensor. By providing placement on memory unit boundaries, address translation requests are reduced and data transfer rates are increased. Furthermore, in one example, each 2D tensor may be loaded via an operation such as direct memory access (DMA), which accesses one memory unit (e.g., a page) in accelerator memory at a time. This significantly increases bandwidth.

[0039] Similarly, in one example, the bias tensor 220 is a concatenated bias tensor containing multiple bias tensors 222. Each bias tensor is of a selected fixed size, and thus row padding 224 and / or page padding 226 are provided as described herein.

[0040] In addition to concatenated input tensors, according to one or more aspects of the present invention, a concatenated output tensor is provided, an example of which is shown in Figure 2D. As shown in Figure 2D, the concatenated output tensor 250 includes, for each input, a hidden state (h) tensor 260 concatenated to an internal cell state (c) tensor 270. In one example, each tensor 260, 270 is a reformatted tensor of a selected dimension (e.g., 2D) and a selected size. To provide a tensor of the selected size, row padding 280 and / or page padding 282 are provided as described herein. The concatenated output tensor is, for example, a concatenated 2D reformatted output tensor. The concatenated output tensor is accessible as a memory-continuous subtensor that can be fed to the next time step of the computation, while, as an example, all time steps maintain adjacent states in memory to return a final result consisting of all time steps as a single memory adjacent tensor. Similar to the input tensor, the dimension of the reformatted tensor in its dimension is rounded up to the next complete tile in that dimension (for example, padding is provided to create a fixed-size tensor, e.g., a 2D tensor).

[0041] Further details regarding the 2D tensor are described with reference to Figure 3A according to one or more aspects of the present invention. As shown, the 2D tensor 300 starts on a memory boundary and uses multiple memory units such as multiple 4K pages (e.g., pages 0-11 numbered in the 2D tensor). Each page contains a pre-selected number of rows (e.g., 32)302, and each row contains a pre-selected number of elements, e.g., 64 elements. If a row has fewer data than the pre-selected number of elements, it is padded with a pre-specified value such as zeros or spaces (304). Furthermore, as shown in Figure 3A, if there is not enough data to provide the pre-selected number of rows, additional padding 306 (e.g., unpredictable data, existing data, arbitrary values, etc.) is provided to add additional padded rows.

[0042] The designed data format for 2D tensors provides easily computable address and memory-unit adjacent tensor units, thereby reducing the overhead of multiple complex address calculations. This supports hardware-supported block load / storage operations and prefetch engines, resulting in a significant increase in effective data bandwidth for accelerators (e.g., neural network processor 105) (e.g., 2x~5x).

[0043] Further details regarding the creation of 2D tensors are described with reference to Figures 3B-3C according to one aspect of the present invention. In one example, the process creates a tensor (e.g., 2D, 3D, 4D, and / or other tensors) based on a 4D feature data layout described herein. As an example, this process is performed by a processor such as a general-purpose processor 104. This process can, but is not limited to, generate 2D, 3D, or 4D tensors, for example.

[0044] Referring to Figure 3B, in one example, e2_limit is set to equal to ceil(E2 / 32)*32 (352), indicating that the 2D tensor being created has, for example, 32 rows, where E2 refers to the 2D index size. Furthermore, e1_limit is set to equal to ceil(E1 / 64)*64 (354), indicating that the 2D tensor being created has, for example, 64 elements per row, where E1 refers to the 1D index size. The index e4x is initialized to zero (356).

[0045] Following initialization, it is determined whether e4x is less than E4 (358), where E4 refers to the 4-dimensional index size. If e4x is not less than E4, the process terminates (360); otherwise, the process continues to initialize index e3x to zero (362). It is determined whether e3x is less than E3 (364), where E3 refers to the 3-dimensional index size. If e3x is not less than E3, the process repeats, where e4x is incremented by, for example, 1 (366), and the process continues to 358. On the other hand, if e3x is less than E3, index e2x is initialized to zero (368). It is determined whether e2x is less than e2_limit (370). If e2x is not less than e2_limit, the process repeats, where e3x is incremented by, for example, 1 (372), and the process continues to 364. If e2x is less than e2_limit, the index e1x is initialized to zero (374).

[0046] Referring to Figure 3C, a determination is made as to whether e1x is less than e1_limit (376). If e1x is not less than e1_limit, the process is repeated, where e2x is incremented by, for example, 1 (e2x = e2x + 1) (378), and the process continues to 370 (Figure 3B). If e1x is less than e1_limit, arr_pos (for example, the position in a certain row) is,

number

number

[0047] A check is made to determine whether e2x is greater than or equal to E2 (384). If e2x is not less than E2, a further check is made to determine whether e1x is greater than or equal to e1 (386). If e1x is less than E1, the value is set to equal input_array[e4x][e3x][e2x][e1x] (388); if e1x is greater than or equal to E1, the value is E1pad (390) (the row is padded). Furthermore, if e2x is greater than or equal to E2 (384), the value is E2pad (392) (an additional row to be padded is added to the 2D tensor). Following the setting of the value, OutputTensor[arr_pos] = value. Furthermore, the index e1x is incremented by, for example, 1 (e1x = e1x + 1) (394), and the process continues in 376.

[0048] As a further example, tensors may be created based on the 4D_kernel layout described herein. The processes shown in Figures 3B-3C are used to create 2D, 3D, 4D and / or other tensors, except that 382 is...

number

[0049] The created tensor (e.g., a reformatted tensor created from reformatting the original tensor) may be used by one or more instructions. For example, address information (e.g., the start of a 4D or 2D tensor), tensor dimensions, etc., are transferred from the general-purpose processor to a dedicated processor (e.g., neural network 105) for use when loading / storing the data in the correct format (e.g., in the correct location on a page of memory) and for using the data (e.g., in tensor computation). In other embodiments, the general-purpose processor uses the created reformatted tensor. Other modifications are possible.

[0050] According to one or more embodiments, multiple reformatted tensors are concatenated to provide a concatenated input and / or output tensor. In one example, one or more concatenated input tensors are input to a recurrent neural network cell activation, such as a long short-term memory cell activation or a gated recurrent unit cell activation, which generates one or more concatenated output tensors. Further details regarding the exemplary cell activations are described with reference to Figures 4A and 4B.

[0051] As an example, referring to Figure 4A, a first input tensor 400a (e.g., input tensor 1) and a second input tensor 400b (e.g., input tensor 2) are input to the long short-term memory cell activation 401. For example, the first input tensor 400a and the second input tensor 400b are concatenated tensors (e.g., result tensors), each containing, for example, the concatenation of four individual tensors 400a1 to 400a4 and 400b1 to 400b4, and each of these individual tensors is input to the addition operation of the long short-term memory cell activation 401. For example, input tensors 400a1 and 400b1 are input to addition operation 402a; input tensors 400a2 and 400b2 are input to addition operation 402b; input tensors 400a3 and 400b3 are input to addition operation 402c; and input tensors 400a4 and 400b4 are input to addition operation 402d. Each addition operation is equivalent to, for example, an NNPA-ADD operation, an example of which is described herein. The output of addition operation 402a is input to sigmoid activation 404a; the output of addition operation 402b is input to sigmoid activation 404b; the output of addition operation 402c is input to tangent activation 406; and the output of addition operation 402d is input to sigmoid activation 404c. The sigmoid activations 404a, 404b, and 404c and the tangent activation 406 are equivalent to, for example, the NNPA-SIGMOID function and the NNPA-TANH function, respectively, examples of which are described herein. The outputs of the sigmoid activation 404b and the tangent activation 406 are input to the multiplication operator 408, which is equivalent to, for example, the NNPA-MUL function, an example of which is described herein.

[0052] The outputs of the sigmoid activation 404a and the multiplication operation 408 are input to the join operation 410, along with a third input tensor 400c (e.g., input tensor 3). In this example, the input tensor 400c is not a concatenated tensor, but the output from the previous time step. For example, the input tensor 400c is the cell state portion of a concatenated output tensor. The join operation 410 is, for example, a fused sum-of-products (FMA) operation, which is equivalent to, for example, the NNPA-BATCHNORM function, an example of which is described herein. (In other examples, individual operations may be used instead of a join operation.) In operation 410, the output from the sigmoid activation 404a and the input tensor 400c are multiplied to provide an intermediate result. The intermediate result is added to the output of the multiplication operation 408 to provide another intermediate result. The other intermediate result (e.g., the result of the join operation 410) is input to the tangent activation 412, which is equivalent to, for example, the NNPA-TANH function, an example of which is described herein. The output of the tangent function 412 and the output of the sigmoid function 404c are input to the multiplication operation 414, which is equivalent to, for example, the NNPA-MUL function, an example of which is described herein. The output of NNPA-MUL 414 is the output tensor 420a (e.g., output tensor 1). Furthermore, in one example, the output of the join operation 410 is the output tensor 420b (e.g., output tensor 2). As an example, output tensors 420a and 420b are concatenated output tensors, such as those described with reference to Figure 2D.

[0053] Referring to Figure 4B, an example of a gated recurrent unit cell activation is illustrated. In one example, a first input tensor 450a (e.g., input tensor 1) and a second input tensor 450b (e.g., input tensor 2) are input to a gated recurrent unit cell activation 451. For example, the first input tensor 450a and the second input tensor 450b are concatenated tensors (e.g., result tensors), each containing, for example, the concatenation of three individual tensors 450a1 to 450a3 and 450b1 to 450b3, each of which is input to an operation of the gated recurrent unit cell activation 451. For example, input tensors 450a1 and 450b1 are input to an addition operation 452a; input tensors 450a2 and 450b2 are input to an addition operation 452b. Each addition operation is equivalent to, for example, the NNPA-ADD operation, an example of which is described herein. The output of addition operation 452a is input to sigmoid activation 454a; the output of addition operation 452b is input to sigmoid activation 454b. Sigmoid activations 454a and 454b are equivalent to, for example, the NNPA-SIGMOID function, an example of which is described herein. The outputs of sigmoid activations 454a and 454b are input to multiplication operations 456a and 456b, respectively, which are equivalent to, for example, the NNPA-MUL function, an example of which is described herein. Another input to multiplication operation 456a is the input tensor 450c. In this example, the input tensor 450c is not a concatenated tensor, but the output from the previous time step. For example, the input tensor 450c is the cell state portion of a concatenated output tensor. Furthermore, another input to the multiplication operation 456b is the input tensor 450b3.

[0054] In one example, the output of the sigmoid function 454a is also input to the subtraction operation 458, along with the numerical value 1. One example of a subtraction operation is the NNPA-SUB function, an example of which is described herein.

[0055] The output of the multiplication operation 456b and the input tensor 450a3 are input to the addition operation 460, which is equivalent to, for example, the NNPA-ADD function, an example of which is described herein. The output of the addition operation 460 is input to the tangent activation 462, which is equivalent to, for example, the NNPA-TANH function, an example of which is described herein. The outputs of the subtraction operation 458 and the tangent activation 462 are input to the multiplication operation 464, which is equivalent to, for example, the NNPA-MUL function, an example of which is described herein. The outputs of the multiplication operation 464 and the multiplication operation 456a are input to the addition operation 466, which is equivalent to, for example, the NNPA-ADD function, an example of which is described herein. The output of the addition operation 466 is the output tensor 468. For example, the output tensor 468 is a concatenated output tensor, such as the one described with reference to Figure 2D.

[0056] As described above, multiple activations (e.g., sigmoid, tangent) and other operations (e.g., addition, subtraction, and / or multiplication) are combined and executed as part of a single cell activation, which is executed based on the invocation of a single instruction (e.g., a neural network processing support instruction) (e.g., on an accelerator such as a neural network processor 105). The single instruction is implemented to combine the individual activations and other operations. This provides higher accuracy, for example, by combining multiplication and addition operations together, without losing precision for intermediate results. Furthermore, higher numerical accuracy can be achieved by storing intermediate calculations in the accelerator with higher precision. In addition, according to one or more aspects of the present invention, the activations and other operations of a cell activation are separate from the matrix multiplication used to create a concatenated input tensor, reducing the complexity of a single operation and allowing the reuse of basic blocks for other recurrent neural networks. In other words, a recurrent neural network (e.g., on a long-term short-term memory architecture or a gated recurrent unit architecture) relies on several matrix multiplications between input features (e.g., X in Figure 2B) and different weight tensors (e.g., the unconnected, unreformatted weight tensors in Figure 2B), followed by several activation functions (e.g., sigmoid, tangent in Figures 4A-4B) on the resulting intermediates. Typically, the matrix multiplications and activation functions are performed separately on independent tensor buffers, resulting in several separate instructions to compute the recurrent neural network timesteps, which may require copying / reorganizing data between these individual instructions, significantly degrading performance. For example, the advantages of an on-chip accelerator (e.g., neural network processor 105) are significantly reduced when data manipulation on a general-purpose processor is required between accelerator operations. This is due to lower bandwidth, required serialization, and setup time to start the accelerator.Accordingly, according to one or more aspects of the present invention, a data layout format (e.g., a reformatted concatenated tensor) is provided that is directly used by instructions on an accelerator that performs cell activation of a recurrent neural network.

[0057] Furthermore, according to one or more embodiments, a data layout format is selected in which a concatenated output tensor is generated based on computing cell activations at time steps that allow accelerator operations to be chained without the general-purpose processor needing to inspect / reconstruct the data. In addition, the instructions provide spatially close input and output sources to reduce address translation. By placing data adjacent to each other in memory, fewer address translations are required. This contributes to an overall increase in processing speed and higher precision within the accelerator (e.g., neural network processor 105).

[0058] One example of the overall chaining operation is illustrated with reference to Figures 5A and 5B. In Figures 5A and 5B, Nmb is the batch size, t is the time step, s is the size, and l is the feature length. In this example, the cell activation for using chaining is a long short-term memory cell activation 500, an example of which is illustrated herein with reference to Figure 4A. However, in other examples, it may be other cell activations, including, but not limited to, gated recurrent unit cell activations (an example of which is illustrated herein with reference to Figure 4B) and / or other cell activations.

[0059] Referring to Figure 5A, the output of the cell activation 500 includes a history (h) tensor 502 and a cell state (c) tensor 504, which are used to provide a concatenated output tensor 510. The concatenated output tensor is then input to the next time step of the cell activation 500 (i.e., chaining). For example, the history tensor 510a of the concatenated tensor 510 is input to a matrix multiplication operation 520, and the cell state tensor 510b of the concatenated tensor 510 is input to a join operation 530 (e.g., a fused sum-of-products operation such as NNPA-BATCHNORM). In other examples, individual operations may be used instead of a join operation.

[0060] In one example, a matrix multiplication operation 520 multiplies the history tensor 510a and the concatenated weighted matrix 540 to provide an intermediate result, which is added to a concatenated bias tensor 550 (Figure 5B) to provide a concatenated tensor (e.g., input tensor 2), which is input to the cell activation 500. Furthermore, in one example, another concatenated tensor (e.g., input tensor 1) is also input to the cell activation 500. Input tensor 1 is created by concatenating a plurality of weight tensors 560 to provide a concatenated weight tensor 562, as described herein and further described with reference to Figure 5B. The concatenated weight tensor 562 is multiplied by a feature input 566 using, for example, a matrix multiplication broadcast operation 564 (e.g., NNPA-MATMUL-OP-BCAST23) to provide an intermediate result, which is added to a concatenated bias tensor 570 using, for example, a matrix multiplication broadcast operation 564 to provide the resulting input tensor 1. The concatenated bias tensor 570 is constructed from a plurality of bias tensors 572, as described herein.

[0061] The concatenated weight tensor 562, the concatenated bias tensor 570, and / or the concatenated output tensor 510 (Figure 5A) are, according to one or more aspects of the present invention, for example, reformatted tensors. A reformatted tensor, as described herein, starts on a memory boundary (e.g., a page boundary) and includes padding to complete a tensor of a selected size. For example, if a tensor is to contain a selected number of rows (e.g., 32 rows) and the reformatted tensor has fewer rows, padded rows are added until the tensor contains the selected number of rows. In addition and / or alternatively, in one example, each row is to contain a selected number of elements (e.g., 64 elements), and if a row has fewer elements than it can contain, padding is added to the row until it contains the selected number of elements.

[0062] A layer of connected tensors (e.g., individual tensors of a connected tensor) is selected as input to cell activation. For example, referring to Figure 5A, individual input tensors of input tensor 1 are selected to be input to a particular operation (525). Other examples are possible.

[0063] According to one or more aspects of the present invention, a single designed instruction is provided that enables the creation and / or use of reformatted tensors and / or concatenated tensors and / or supports a data layout format that combines activation and operations in cell activation performed by a single invocation of the instruction. One example of such an instruction is a neural network processing support instruction. In one example, the instruction is initiated on a general-purpose processor (e.g., processor 104), and the function specified by the instruction is executed on either the general-purpose processor and / or a dedicated processor (e.g., neural network processor 105), depending on the function. For example, in one example, the query function of the neural network processing support instruction is executed on the general-purpose processor, and the non-query function is executed on the dedicated processor. However, other modifications are possible. If the function is to be executed on the dedicated processor (e.g., it is a non-query function, or in another example, one or more selected functions), information such as memory address information regarding the tensor data to be used in the neural network computation is provided to the dedicated processor by the general-purpose processor for use when executing the function, for example. The dedicated processor retrieves the information and executes the function. After the function has finished executing, processing is returned to the general-purpose processor, thereby completing the instruction. In other examples, the instruction is started, executed, and completed on one or more general-purpose processors or one or more dedicated processors. Other modifications are possible.

[0064] In one example, referring to Figure 6A, the neural network processing support instruction 600 has an RRE format indicating a register operation with registers and extended arithmetic codes (opcodes). As shown in Figure 6A, in one example, the neural network processing support instruction 600 includes an arithmetic code (opcode) field 602 (e.g., bits 0-15) indicating a neural network processing support operation. In one example, bits 16-31 of the instruction are reserved and will store zero. In the herein description of the function of the instruction and / or the instruction, specific locations, specific fields and / or specific sizes of fields are indicated (e.g., specific bytes and / or bits). However, other locations, fields and / or sizes may be provided. Furthermore, specific values ​​of bits, for example, setting to 1 or zero, may be specified, but this is merely an example. Bits may be set to different values, such as the opposite value, or to other values, where they are set in other examples. Many variations are possible.

[0065] In one example, an instruction uses several general-purpose registers implicitly specified by the instruction. For example, the neural network processing support instruction 600 uses the implicit registers general-purpose register 0 and general-purpose register 1, and these examples are illustrated with reference to Figures 6B and 6D, respectively.

[0066] Referring to Figure 6B, in one example, general-purpose register 0 contains a function code field and a status field that may be updated upon instruction completion. For example, general-purpose register 0 contains a response code field 610 (e.g., bits 0-15), an exception flag field 612 (e.g., bits 24-31), and a function code field 614 (e.g., bits 56-63). Furthermore, in one example, bits 16-23 and 32-55 of general-purpose register 0 are reserved and will store zeros. One or more fields are used by specific functions executed by the instruction. In one example, not all fields are used by all functions. Each of the fields is described below.

[0067] Response Code (RC) 610: This field (e.g., bit positions 0-15) stores the response code. If the execution of a neural network processing support instruction completes with a condition code of, for example, 1, the response code is stored. If an invalid input condition is encountered, a non-zero value is stored in the response code field, indicating the cause of the invalid input condition recognized during execution, and a selected condition code, for example, 1, is set. The codes stored in the response code field are defined, in one example, as follows:

[0068] Response code meaning

[0069] The parameter block format, such as that specified by the parameter block version number (0001), is not supported by the model.

[0070] 0002 The specified function is either not defined or not installed on the machine.

[0071] 0010 The specified tensor data layout format is not supported.

[0072] 0011 The specified tensor data type is not supported.

[0073] 0012 The specified single tensor dimension is greater than the maximum dimension index size.

[0074] 0013 The size of the specified tensor is greater than the maximum tensor size.

[0075] 0014 The specified tensor address is not aligned to a 4K byte boundary.

[0076] 0015 Function-specific storage area addresses are not aligned on 4K byte boundaries.

[0077] F000-FFFF are function-specific response codes. These response codes are defined for specific functions.

[0078] Exception Flags (EF) 612: This field (e.g., bit positions 24-31) contains exception flags. If an exception condition is detected during instruction execution, the corresponding exception flag control (e.g., bit) will be set to, for example, 1; otherwise, the control remains immutable. The exception flag field will be initialized to zero prior to the first invocation of the instruction. Reserved flags remain immutable during instruction execution. The flags stored in the exception flag field are defined in one example as follows:

[0079] EF (bit) meaning

[0080] 0 Range violation. This flag is set if a non-digital value is detected in the input tensor or stored in the output tensor. This flag is only valid, for example, if the instruction completes with a condition code, e.g., 0.

[0081] 1-7 are booked.

[0082] Function Code (FC) 614: This field (e.g., bit positions 56-63) contains the function code. An example of an assigned function code for neural network processing support instructions is shown in Figure 6C. All other function codes are immutable. If an unassigned or uninstalled function code is specified, the response code (e.g., 0002hex) and the selected condition code (e.g., 1) are set. This field is not modified during execution.

[0083] As shown, in addition to general-purpose register 0, neural network processing support instructions also use general-purpose register 1, an example of which is shown in Figure 6D. For example, bits 40-63 in 24-bit addressing mode, bits 33-63 in 31-bit addressing mode, or bits 0-63 in 64-bit addressing mode contain the address of parameter block 620. The contents of general-purpose register 1 specify, for example, the logical address of the leftmost byte of the parameter block in storage. The parameter block must be specified on a double-word boundary; otherwise, a specification exception is recognized. For all functions, the contents of general-purpose register 1 are not modified.

[0084] In access register mode, access register 1 specifies, for example, the address space that stores the parameter block, input tensor, output tensor, and function-specific storage area.

[0085] In one example, the parameter block may have a different format depending on the function specified by the instruction to be executed. For example, a query function might have a parameter block of one format, while other functions in an instruction might have parameter blocks of a different format. In another example, all functions might use the same parameter block format. Other variations are also possible.

[0086] An example of a parameter block used by query functions, such as NNPA query-available function (QAF) operators, is illustrated with reference to Figure 6E. As shown, in one example, the NNPA query-available function parameter block 630 includes, for example:

[0087] Installed Function Vector 632: This field of the parameter block (e.g., bytes 0-31) contains the installed function vector. In one example, bits 0-255 of the installed function vector correspond to function codes 0-255 of the neural network processing support instructions, respectively. If a bit is, for example, 1, the corresponding function is installed; otherwise, the function is not installed.

[0088] Installed Parameter Block Format Vector 634: This field of the parameter block (e.g., bytes 32-47) contains the installed parameter block format vector. In one example, bits 0-127 of the installed parameter block format vector correspond to parameter block formats 0-127 for non-query functions of neural network processing support instructions. If a bit is, for example, 1, the corresponding parameter block format is installed; otherwise, the parameter block format is not installed.

[0089] Installed Data Types 636: This field in the parameter block (e.g., bytes 48-49) contains the installed data types vector. In one example, bits 0-15 of the installed data types vector correspond to the installed data types. If a bit is, for example, 1, the corresponding data type is installed; otherwise, the data type is not installed. The example data types include (additional, fewer, and / or other data types are possible):

[0090] Bit Data Type

[0091] 0 NNP Data Type 1

[0092] 1-15 are reserved.

[0093] Installed Data Layout Format 638: This field in the parameter block (e.g., bytes 52-55) contains the installed data layout format vector. In one example, bits 0-31 of the installed data layout format vector correspond to the installed data layout formats. If a bit is, for example, 1, the corresponding data layout format is installed; otherwise, the data layout format is not installed. The example data layout format includes (additional, fewer, and / or other data types are possible):

[0094] Bit Data Layout Format

[0095] 0 4D Feature Tensor

[0096] 1 4D kernel tensor

[0097] 2-31 Booked

[0098] Maximum Dimension Index Size 640: This field in the parameter block (e.g., bytes 60-63) contains a 32-bit unsigned binary integer that specifies, for example, the maximum number of elements in the specified dimension index size for any given tensor. In another example, Maximum Dimension Index Size specifies the maximum number of bytes in the specified dimension index size for any given tensor. Other examples are possible.

[0099] Maximum Tensor Size 642: This field in the parameter block (e.g., bytes 64-71) contains a 32-bit unsigned binary integer specifying the maximum number of bytes in any given tensor, including any padding bytes required by the tensor format. In another example, Maximum Tensor Size specifies the maximum total number of elements in any given tensor, including any padding required by the tensor format. Other examples are possible.

[0100] Installed NNP Data Type 1 Conversion Vector 344: This field in the parameter block (e.g., bytes 72-73) contains the installed NNP Data Type 1 conversion vector. In one example, bits 0-15 of the installed NNP Data Type 1 conversion vector correspond to the installed data type conversions to the NNP Data Type 1 format. If a bit is 1, the corresponding conversion is installed; otherwise, the conversion is not installed. Additional, fewer, and / or other conversions may be specified.

[0101] Bit Data Type

[0102] 0 reserved

[0103] 1 BFP Tiny Format

[0104] 2 BFP Short Format

[0105] 3-15 are booked.

[0106] One example of a parameter block for a query function is illustrated with reference to Figure 6E, but other formats of parameter blocks for query functions, including NNPA query-available function operations, may be used. The format may depend, in one example, on the type of query function to be executed. Furthermore, the parameter block and / or each field of the parameter block may contain additional, lesser, and / or other information.

[0107] In addition to parameter blocks for query functions, there is also a parameter block format for non-query functions, such as the non-query function of a neural network processing support instruction, as one example. One example of a parameter block used by a non-query function, such as the non-query function of a neural network processing support instruction, is illustrated with reference to Figure 6F.

[0108] As shown, in one example, parameter block 650 used by a non-query function of a neural network processing support instruction includes, for example, the following:

[0109] Parameter Block Version Number 652: This field of the parameter block (e.g., bytes 0-1) specifies the version and size of the parameter block. In one example, bits 0-8 of the parameter block version number are reserved and store zero, while bits 9-15 store an unsigned binary integer specifying the format of the parameter block. The query function provides a mechanism to indicate available parameter block formats. If the specified parameter block size or format is not supported by the model, a response code, e.g., 0001hex, is stored in general register 0, and the instruction is completed by setting a condition code, e.g., condition code 1. The parameter block version number is specified by the program and is not modified during instruction execution.

[0110] Model version number 654: This field in the parameter block (e.g., byte 2) is an unsigned binary integer that identifies the model that executed the instruction (e.g., a particular non-query function). If the continuation flag (described below) is 1, the model version number may be input to the operation for the purpose of interpreting the contents of the continuation state buffer field (described below) of the parameter block in order to resume the operation.

[0111] Continuation Flag 656: This field in the parameter block (e.g., bit 63), e.g., 1, indicates that the operation is partially completed, and the contents of the continuation state buffer may be used to resume the operation. The program should initialize the continuation flag to zero and not change it if the instruction is to be re-executed for the purpose of resuming the operation; otherwise, the result is unpredictable.

[0112] If the continuation flag is set at the start of the operation and the contents of the parameter block have changed since the initial launch, the result is unpredictable.

[0113] Function-Specific Storage Area Address 658: This field in the parameter block (e.g., bytes 56-63) contains the logical address of the function-specific storage area. In one example, the function-specific storage area address would be aligned on a 4K byte boundary; otherwise, for example, a response code of 0015hex would be set in general-purpose register 0, and the instruction would complete with a condition code of 1, for example. The address takes on the current addressing mode. The size of the function-specific storage area depends on the function code.

[0114] If the entire function-specific storage area overlaps with a Program Event Log (PER) storage area designation, PER storage change events will be recognized for the function-specific storage area, if applicable. If only a portion of the function-specific storage area overlaps with a PER storage area designation, it is model-dependent which of the following occurs:

[0115] * PER storage change events are recognized for the entire function-specific storage area, where applicable.

[0116] * PER storage change events are recognized for a portion of the function-specific storage area being remembered, where applicable.

[0117] If the entire parameter block overlaps with the PER storage area specification, the PER storage change event will be recognized for the parameter block, if applicable. If only a portion of the parameter block overlaps with the PER storage area specification, it is model-dependent which of the following occurs:

[0118] * PER storage change events are recognized for the entire parameter block, if applicable.

[0119] * PER storage change events are recognized for a portion of the stored parameter block, where applicable.

[0120] The PER zero-address detection event is recognized for parameter blocks, where applicable. Zero-address detection does not apply to tensor addresses or function-specific storage area addresses, in one example.

[0121] Output tensor descriptor (e.g., 1-2) 660 / Input tensor descriptor (e.g., 1-3) 665: One example of a tensor descriptor is illustrated with reference to Figure 6G. In one example, tensor descriptors 660, 665 include:

[0122] Data layout format 682: This field of the tensor descriptor (e.g., byte 0) specifies the data layout format. Valid data layout formats include, for example, the following (additional, fewer, and / or other data layout formats are possible):

[0123] Format description: Alignment (bytes)

[0124] 0 4D Feature Tensor 4096

[0125] 1 4D kernel tensor 4096

[0126] 2-255 reserved

[0127] If an unsupported or reserved data layout format is specified, for example, a response code of 0010hex is stored in general-purpose register 0, and the instruction is completed by setting a condition code, for example, 1.

[0128] Data type 684: This field (e.g., byte 1) specifies the data type of the tensor. Examples of supported data types are described below (additional, fewer, and / or other data types are possible):

[0129] Value, Data Type, Data Size (bits)

[0130] 0 NNP Data Type 1 16

[0131] 1-255 Reserved

[0132] If an unsupported or reserved data type is specified, for example, a response code of 0011hex is stored in general-purpose register 0, and the instruction is completed by setting a condition code, for example, 1.

[0133] Dimensions 1-4 Index Size 686: Collectively, dimension index sizes 1-4 (e.g., E4, E3, E2, E1) specify the shape of the 4D tensor. Each dimension index size is greater than zero and less than or equal to the largest dimension index size (640, Figure 6E); otherwise, a response code, e.g., 0012hex, is stored in general-purpose register 0, and the instruction is completed by setting a condition code, e.g., 1. The total tensor size is less than or equal to the largest tensor size (642, Figure 6E); otherwise, a response code, e.g., 0013hex, is stored in general-purpose register 0, and the instruction is completed by setting a condition code, e.g., 1.

[0134] In one example, to determine the number of bytes in a 4D feature tensor having elements of NNPA data type 1 (i.e., total tensor size), the following is used: Dimension index 4 * Dimension index 3 * ceil(Dimension index 2 / 32) * 32 * ceil(Dimension index 1 / 64) * 64 * 2.

[0135] Tensor address 688: This field of the tensor descriptor (e.g., bytes 24-31) contains the logical address of the leftmost byte of the tensor. The address takes the current addressing mode.

[0136] If the address is not aligned on the boundary of the associated data layout format, for example, the response code 0014hex is stored in general-purpose register 0, and the instruction is completed by setting a condition code, for example, 1.

[0137] In access register mode, access register 1 specifies the address space in storage where all active input and output tensors are stored.

[0138] Returning to Figure 6F, the parameter block 650 further includes, in one example, function eigenparameters 1-5 (670), which may be used by the eigenfunction as described herein.

[0139] Furthermore, the parameter block 650, in one example, includes a continuation state buffer field 675, which contains data (or the location of data) that will be used if the operation of this instruction is to be resumed.

[0140] As input to the operation, reserved fields in a parameter block should store zero. Once the operation is complete, the reserved fields may be stored as zero or remain unchanged.

[0141] One example of a parameter block for a non-query function is illustrated with reference to Figure 6F, but other formats of parameter blocks for non-query functions, including non-query functions of neural network processing support instructions, may be used. The format may depend, in one example, on the type of function to be executed. Furthermore, one example of a tensor descriptor is illustrated with reference to Figure 6G, but other formats may be used. In addition, different formats may be used for input and output tensors. Other variations are possible.

[0142] Further details regarding the various functions supported by one embodiment of the neural network processing support instructions are described below. Additional, fewer, and / or other functions may be supported.

[0143] Function code 0: NNPA-QAF (Query-enabled function)

[0144] The Neural Network Processing Assisted (NNPA) query function provides a mechanism to indicate selected information, such as the availability of installed functions, installed parameter block formats, installed data types, installed data layout formats, maximum dimension index size, and maximum tensor size. This information is retrieved and placed in selected locations, such as parameter blocks (e.g., parameter block 630). Once the operation is complete, reserved fields in the parameter block may be stored as zero or remain immutable.

[0145] In the execution of one embodiment of the query function, a processor such as a general-purpose processor 104 obtains information about a specific model of a selected processor, such as a specific model of a neural network processor such as a neural network processor 105. A specific model of a processor or machine has specific capabilities. Another model of a processor or machine may have additional, less, and / or different capabilities, and / or be of a different generation (e.g., current or future generation) with additional, less, and / or different capabilities. The information obtained is placed in a parameter block (e.g., parameter block 630), or in other structures accessible to and / or used with one or more applications that may use this information in further processing. In one example, the parameter block and / or the information in the parameter block is maintained in memory. In other embodiments, the parameter block and / or the information may be maintained in one or more hardware registers. In another example, the query function may be a preferred operation performed by the operating system, which makes the application programming interface available to make this information available to applications or non-preferential programs. In yet another example, the query function is executed by a dedicated processor such as a neural network processor 105. Other forms of modification are possible.

[0146] The information is obtained, for example, by the firmware of the processor executing the query function. The firmware has knowledge of the attributes of a particular model of a particular processor (e.g., a neural network processor). This information may be stored, for example, in control blocks, registers and / or memory, and / or otherwise accessible to the processor executing the query function.

[0147] The information retrieved includes, for example, model-dependent details about at least one or more data attributes of a particular processor, including, for example, one or more installed or supported data types, one or more installed or supported data layout formats, and / or one or more installed or supported data sizes for selected models of a particular processor. This information is model-dependent in that other models (e.g., previous and / or future models) may not support the same data attributes, such as the same data types, data sizes, and / or data layout formats. When the execution of a query function (e.g., the NNPA-QAF function) is complete, condition code 0 is set, for example. Condition codes 1, 2, and 3 are not applicable to the query function, for example. Further information regarding the retrieved information is described below.

[0148] As shown, in one example, the information retrieved includes, for example, model-dependent information about one or more data attributes of a particular model of a neural network processor. One example of a data attribute is the installed data type of the neural network processor. For example, a particular model of a neural network processor (or other processor) may support one or more data types, such as, for example, the NNP Data Type 1 data type (also referred to as the Neural Network Processing Data Type 1 data type) and / or other data types. The NNP Data Type 1 data type is a 16-bit floating-point format that offers several advantages for deep learning training and inference computation, including, for example: preserving the accuracy of deep learning networks; eliminating abnormal formats that simplify rounding modes and corner case handling; automatically rounding to nearest neighbor values ​​for arithmetic operations; and the special entities of infinity and Not-Number (NaN) being combined into a single value (NINF), which is accepted and handled by arithmetic operations. NINF provides better defaults for exponential overflow and invalid operations (such as division by zero). This allows many programs to continue execution without hiding such errors or using special exception handlers. Other model-dependent data types are also supported.

[0149] One example of the format for an NNP data type 1 data type is shown in Figure 7. As shown, in one example, an NNP data type 1 data may be represented in format 700, which includes, for example, a sign 702 (e.g., bit 0), an exponent +31 704 (e.g., bits 1-6), and a fractional part 706 (e.g., bits 7-15).

[0150] The exemplary characteristics of the NNP data type 1 format are shown below:

[0151] Characteristics NNP Data Type 1

[0152] Format length (bits): 16 bits

[0153] Biased exponential length (bits) 6 bits

[0154] 9 bits

[0155] Precision (p): 10 bits

[0156] Maximum left unit display exponent (exponent) Emax 32

[0157] Minimum left unit display exponent (exponent) Emin -31

[0158] Left-hand unit display (LUV) bias 31

[0159]

number

[0160]

number

[0161] Dmin ---

[0162]

number

[0163] Further details regarding the NNP data type 1 are described below:

[0164] Biased exponent: The bias used to enable the exponent to be represented as an unsigned number is shown above. The biased exponent is similar to the characteristics of the binary floating-point format, except that biased exponents of all zeros and all ones are not given any special meaning, as described below with reference to the NNP data type 1 data type class.

[0165] Mantissa: The binary fractional point of an NNP data type 1 number is assumed to be to the left of the leftmost fractional bit. To the left of the binary fractional point is an implied unit bit, which is assumed to be 1 for normal numbers and zero for zero. The fractional part with the implied unit bit appended to the left is the mantissa of the number.

[0166] The value of a normalized NNP data type 1 is a mantissa multiplied by the base 2 raised to the power of the unbiased exponential part.

[0167] Non-zero values: Non-zero values ​​are shown below:

[0168] Number of classes of values

[0169] Normal number ±2 e-31 ×(1.f)

[0170] Here, e is the biased exponent in decimal form, and f is the fractional part in binary form.

[0171] In one embodiment, there are three classes of NNP data type 1 data, including numeric and associated non-numeric entities. Each data item includes a sign, exponent, and mantissa. The exponent is biased such that all biased exponents are non-negative unsigned numbers and the smallest biased exponent is zero. The mantissa includes an explicit fractional part and an implicit unit bit to the left of the binary decimal point. The sign bit is zero for positive numbers and one for negative numbers.

[0172] All allowed non-zero finite numbers have a unique NNP data type 1 representation. There are no denormalized numbers; these numbers can have multiple representations for the same value, and there are no denormalized arithmetic operations. The three classes include, for example:

[0173] Data class Sign Biased exponent Unit bit * Fractional part

[0174] Zero ± 0 0 0

[0175] Normal numbers ± 0 1 Non-zero

[0176] Normal numbers ± Non-zero, not all 1s 1 Arbitrary

[0177] Normal numbers ± all 1 - not all 1

[0178] NINF ± All 1 - All 1

[0179] Here, :- indicates not applicable, * indicates that a unit bit is implied, and NINF is not number or infinite.

[0180] Further details about each class are provided below:

[0181] Zero: Zero has a zero-biased exponent and a zero fractional part. The implied unit bit is zero.

[0182] Normalized numbers: Normalized numbers can have biased exponents of any value. If the biased exponent is 0, the fractional part will be non-zero. If all biased exponents are 1, the fractional part will not be all 1. Other biased exponent values ​​may have any fractional part values. The implied unit bit is 1 for all normalized numbers.

[0183] NINF: NINF is represented by a biased exponent part of all ones and a fractional part of all ones. NINF represents a value that is not within the range of representable values ​​in NNP data type 1 (i.e., a 16-bit floating-point number designed for deep learning with 6 exponent bits and 9 fractional bits). Typically, NINF is simply propagated during computation so that it remains visible in the end.

[0184] In one example, the NNP data type 1 data type is supported, but other special or non-standard data types, as well as one or more standard data types, including, but not limited to, IEEE 754 short-precision binary floating-point 16-bit, IEEE half-precision floating-point, 8-bit floating-point, 4-bit integer format and / or 8-bit integer format, may also be supported. These data formats have different qualities for neural network processing. For example, smaller data types (e.g., fewer bits) may be processed faster and use less cache / memory, while larger data types provide higher accuracy of results in neural networks. Each data type to be supported may have one or more allocated bits in the query parameter block (e.g., in the installed data type field 636 of parameter block 630). For example, special or non-standard data types supported by a particular processor may be indicated in the installed data type field, while standard data types may not. In other embodiments, one or more standard data types may also be indicated. Other modifications are possible.

[0185] In one particular example, bit 0 of the installed data type field 636 is reserved for the NNP data type 1 data type, and if it is set to, for example, 1, it indicates that the processor supports NNP data type 1. In one example, the bit vector of installed data types is configured to represent up to 16 data types, where bits are assigned to each data type. However, bit vectors in other embodiments may support more or fewer data types. Furthermore, the vector may be configured, where one or more bits are assigned to one data type. Many examples are possible, and / or additional, fewer, and / or other data types may be supported and / or shown in the vector.

[0186] In one example, the query function retrieves an indication of a data type installed on the model-dependent processor and places the indication in the parameter block by, for example, setting one or more bits in the installed data type field 636 of the parameter block 630. Furthermore, in another example, the query function retrieves an indication of an installed data layout format (another data attribute) and places the information in the parameter block by, for example, setting one or more bits in the installed data layout format field 638. Exemplary data layout formats include, for example, 4D feature tensor layouts and 4D kernel tensor layouts. These data layout formats arrange data in storage for tensors in a manner that enhances the processing efficiency of the execution of the neural network processing support instructions. For example, to operate efficiently, the neural network processing support instructions use input tensors provided in a particular data layout format. While exemplary layouts are provided, additional, fewer, and / or other layouts may be provided for the functions and / or other functions described herein.

[0187] The use or availability of a layout for a particular processor model is provided by a vector of the installed data layout format (e.g., field 638 of parameter block 630). The vector is a bit vector of the installed data layout format that allows the CPU to communicate to the application which layouts are supported. For example, bit 0 is reserved for the 4D feature tensor layout, and if it is set to, for example, 1, it indicates that the processor supports the 4D feature tensor layout; bit 1 is reserved for the 4D kernel tensor layout, and if it is set to, for example, 1, it indicates that the processor supports the 4D kernel tensor layout. In one example, the bit vector of the installed data layout format is configured to represent up to 16 data layouts, where bits are assigned to each data layout. However, in other embodiments, the bit vector may support more or fewer data layouts. Furthermore, the vector may be configured such that one or more bits are assigned to a single data layout. Many examples are possible. Further details regarding the 4D feature tensor layout and the 4D kernel tensor layout are described below. Here again, other layouts may be used here or in the future to optimize performance.

[0188] In one example, neural network processing support instructions operate using 4D tensors, i.e., tensors having four dimensions. These 4D tensors are obtained, for example, row-major from the general-purpose input tensors described herein, i.e., when enumerating tensor elements in ascending order of memory addresses, the internal dimension called E1 is first incremented through E1 index size values ​​starting from 0 up to E1 index size 1, then the index of the E2 dimension is incremented, and so on, repeating the increment through the E1 dimension. The index of the external dimension called E4 dimension is incremented last.

[0189] Tensors with a lower number of dimensions (e.g., 3D or 1D tensors) will be represented as 4D tensors where one or more dimensions of the original tensor with dimensions exceeding the original tensor dimension are set to 1.

[0190] This specification describes the conversion of a row-first general-purpose 4D tensor having dimensions E4, E3, E2, and E1 to a 4D feature tensor layout (also referred to herein as NNPA data layout format 0 4D feature tensor).

[0191] The resulting tensor can be represented, for example, as a 4D tensor with 64 elements, or as a 5D tensor with the following dimensions:

[0192]

number

number

[0193] The elements [e4][e3][e2][e1] of the general-purpose tensor may be mapped to the following elements of the resulting 5D tensor:

[0194]

number

number

number

number

number

[0195] The resulting tensor can be larger than the general-purpose tensor. Elements of the resulting tensor that do not have corresponding elements in the general-purpose tensor are called padded elements.

[0196] Consider the elements [fe4][fe1][fe3][fe2][fe0] of a 64-element vector NNPA data layout format 0 4D feature tensor, or their equal representation as a 5D tensor. These elements are either padded elements or corresponding elements in a generic 4D tensor, and their dimensions E4, E3, E2, and E1 can be determined using the following formula:

[0197] If fe2 ≥ E2, then this is an E2 (or page) padded element.

[0198] else if fe1*64+fe0≧E1 then this is an E1 (or row) pad element.

[0199] The corresponding elements in the generic 4D tensor are as follows:

[0200] [fe4][fe3][fe2][fe1*64+fe0]

[0201] For convolutional neural network-based artificial intelligence models, the four-dimensional meaning of a feature tensor can generally be mapped as follows:

[0202] E4: N-Mini Batch Size

[0203] E3: H-3D tensor / image height

[0204] E2:W-3D tensor / image width

[0205] E1: Channel or class of a C-3D tensor

[0206] For machine learning or recurrent neural network-based artificial intelligence models, the four-dimensional meaning of a 4D feature tensor can generally be mapped as follows:

[0207] • E4: T - Number of time steps or models

[0208] • E3: Reserved, generally set to 1

[0209] ·E2:N mb - Mini badge size

[0210] ·E1:L-Features

[0211] The NNPA Data Layout Format 0 provides two-dimensional data locality, for example, by using 4kbyte blocks of data (pages) and 4kbyte block data alignment for the external dimensions of the generated tensor.

[0212] Pad element bytes are ignored for the input tensor and unpredictable for the output tensor. PER storage changes on pad bytes are unpredictable.

[0213] An example of an input data layout for a 4D feature tensor layout having dimensions E1, E2, E3, and E4 is shown in Figures 8A-8C, and an example of an output for a 4D feature tensor layout is shown in Figures 9(A)-9(C). Referring to Figure 8A, a 3D tensor 800 is shown, which has dimensions E1, E2, and E3. In one example, each 3D tensor contains multiple 2D tensors 802. Therefore, in the example shown, multiple 2D tensors (e.g., three 2D tensors) create a 3D tensor, and multiple 3D tensors (e.g., three 3D tensors) create a 4D tensor. The numbers in each 2D tensor 802 describe the memory offset where each of its elements resides in memory. The input is used to lay out the data of the original tensor in memory (for example, the original 4D tensor in Figures 8A to 8C), as shown in Figures 9(A) to 9(C), which correspond to Figures 8A to 8C.

[0214] In Figure 9(A), as an example, a unit of memory 900 (e.g., a memory page) contains a pre-selected number (e.g., 32) rows 902, each identified by, for example, e2_page_idx; each row has a pre-selected number (e.g., 64) elements 904, each identified by, for example, e1_page_idx. If a row does not contain the pre-selected number of elements, it is padded (906) and referred to as row padding or E1 padding; if a memory unit does not have the pre-selected number of rows, it is padded (908) and referred to as page padding or E2 padding. For example, row padding is, for example, zero or other values, and page padding is, for example, an existing value, zero, or other values.

[0215] In one example, the output elements of a row are provided in memory (e.g., on a page) based on the element's position in the E1 direction of its corresponding input. For example, referring to Figure 8A, the element positions 0, 1, and 2 of the three matrices shown (e.g., the element positions at the same location in each matrix) are shown as row 0 on page 0 in Figure 9(A), etc. In this example, the 4D tensor is small, and all the elements of each 2D tensor representing the 4D tensor fit on a single page. However, this is just one example. A 2D tensor may contain one or more pages. As shown in Figure 3A, the 2D tensor in that example contains 12 pages. However, this is just one example. Again, a 2D tensor may contain one or more pages. If a 2D tensor is created based on a reformatting of a 4D tensor, the number of pages in the 2D tensor is based on the size of the 4D tensor. In one example, one or more ceil functions are used to determine the number of rows in a 2D tensor and the number of elements in each row, which will indicate the number of pages that will be used. Other variations are possible.

[0216] The reformatted 2D tensor (e.g., concatenated) is stored in memory based on a 4D feature tensor layout, as described herein, according to one or more aspects of the present invention. The 2D tensor input to cell activation is, for example, a 4D tensor with E3 and E4 set to 1.

[0217] In addition to 4D feature tensor layouts, one example is that a neural network processor may support 4D kernel tensors, which rearrange the elements of a 4D tensor to reduce the number of memory accesses and data acquisition steps when performing artificial intelligence operations (e.g., neural network processing assistance) such as convolution. As an example, a row-major general-purpose 4D tensor having dimensions E4, E3, E2, E1 is converted to an NNPA Data Layout Format 1 4D kernel tensor (4D kernel tensor) as described herein:

[0218] The resulting tensor can be represented, for example, as a 4D tensor with 64 elements, or as a 5D tensor with the following dimensions:

[0219]

number

number

[0220] The elements [e4][e3][e2][e1] of the general-purpose tensor may be mapped to the following elements of the resulting 5D tensor:

[0221]

number

number

number

number

number

[0222] The resulting tensor can be larger than the general-purpose tensor. Elements of the resulting tensor that do not have corresponding elements in the general-purpose tensor are called padded elements.

[0223] Consider the NNPA data layout format 1 for 64-element vectors. We examine the elements [fe1][fe4][fe3][fe2][fe0] of a 4D feature tensor, or their equal representation as a 5D tensor. These elements are either padded elements or their corresponding elements in a generic 4D tensor, and their dimensions E4, E3, E2, and E1 can be determined using the following formula:

[0224] If fe2 ≥ E2, then this is an E2 (or page) padded element.

[0225] else if fe1*64+fe0≧E1 then this is the padded element of E1 (or row).

[0226] The corresponding elements in the generic 4D tensor are as follows:

[0227] [fe4][fe3][fe2][fe1*64+fe0]

[0228] For convolutional neural network-based artificial intelligence models, the four-dimensional meaning of a kernel tensor can generally be mapped as follows:

[0229] E4: H-3D tensor / image height

[0230] E3:W-3D tensor / image width

[0231] E2: Number of channels in a C-3D tensor

[0232] E1: Number of K kernels

[0233] The NNPA Data Layout Format 1 provides two-dimensional kernel parallelism within a 4k-byte block data alignment for, for example, 4k-byte blocks of data (pages) and for the outer dimensions of generated tensors for efficient processing.

[0234] Pad bytes are ignored for input tensors. PER storage changes on pad bytes are unpredictable.

[0235] Here again, the example data layout formats include 4D feature tensor layouts and 4D kernel tensor layouts, but other data layout formats may be supported by the processor (e.g., neural network processor 105). Indications of supported data layouts are obtained and placed in the query parameter block by setting one or more bits, for example, in field 638.

[0236] The query parameter block also includes other data attribute information according to one or more aspects of the present invention, which includes, for example, supported size information for the data. Processors such as neural network processors typically have limitations based on internal buffer size, processing units, data bus structure, firmware limitations, etc., which can limit the maximum size of the tensor dimension and / or the overall size of the tensor. Therefore, the query function provides fields for communicating these limitations to the application. For example, based on executing the query function, the processor obtains various data sizes such as the maximum dimension index size (e.g., 65,536 elements) and the maximum tensor size (e.g., 8 GB), and includes this information in fields 640 and 642 of the parameter block (e.g., parameter block 630), respectively. Additional, lesser, and / or other size information may also be supported by the processor (e.g., neural network processor 105) and therefore obtained and placed in the parameter block, for example, fields 640, 642, and / or other fields. In other embodiments, the limit may be smaller or larger, and / or the size may be in bytes instead of elements, or in other units such as elements instead of bytes. Furthermore, other embodiments allow for different maximum sizes for each dimension, rather than the same maximum for all dimensions. Many modifications are possible.

[0237] According to one or more aspects of the present invention, a query function is provided to determine model - dependent information regarding a particular processor. (The processor may also support standard data attributes such as standard data types, standard data layouts, etc., which are implied and not necessarily presented by the query function; however, in other embodiments, the query function may indicate all or various selected subsets of data attributes, etc.) Although exemplary information is provided, other information may be provided in other embodiments. The information obtained, which can vary for different models of the processor and / or different processors, is used to perform artificial intelligence and / or other processing. The artificial intelligence and / or other processing may utilize, for example, one or more non - query functions of neural network processing support instructions. The particular non - query functions utilized in the processing are executed by executing the neural network processing support instructions one or more times and specifying non - query - specific functions.

[0238] Further details of exemplary non - query functions supported by neural network processing support instructions are described below (additional, fewer, and / or other functions may be supported in other embodiments):

[0239] Function code 16: NNPA - ADD (Addition)

[0240] When the NNPA - ADD function is specified, each element of input tensor 1 described by tensor descriptor 1 is added to the corresponding element of input tensor 2 described by tensor descriptor 2, and the resulting sum is placed in the corresponding element of the output tensor described by the output tensor descriptor.

[0241] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any of the specified tensor descriptors does not specify the NNP data type 1 (e.g., data type = 0), response codes, e.g., 0010hex or 0011hex respectively, are set in general-purpose register 0 and the instruction completes with a condition code, e.g., 1.

[0242] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and the output tensor are to be the same; otherwise, a global operand data exception is recognized.

[0243] In one example, output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1 - 5, and the function-specific save area address field are ignored.

[0244] Function code 17: NNPA - SUB (subtraction)

[0245] When the NNPA - SUB function is specified, each element of input tensor 2 described by tensor descriptor 2 is subtracted from the corresponding element of input tensor 1 described by tensor descriptor 1, and the resulting difference is placed in the corresponding element of the output tensor.

[0246] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any of the specified tensor descriptors does not specify the NNP data type 1 (e.g., data type = 0), response codes, e.g., 0010hex or 0011hex respectively, are set in general-purpose register 0 and the instruction completes with a condition code, e.g., 1.

[0247] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and output tensor will be the same; otherwise, a whole operand data exception will be recognized.

[0248] The output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage area address fields are ignored in one example.

[0249] Function code 18: NNPA-MUL (multiplication)

[0250] When the NNPA-MUL function is specified, the product of each element of input tensor 1 (multiplier) described by tensor descriptor 1 and the corresponding element of input tensor 2 (multiplicand) described by tensor descriptor 2 is placed in the corresponding element of the output tensor.

[0251] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010hex or 0011hex, respectively, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0252] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and output tensor will be the same; otherwise, a whole operand data exception will be recognized.

[0253] The output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage area address fields are ignored in one example.

[0254] Function code 19: NNPA-DIV (division)

[0255] When the NNPA-DIV function is specified, each element of input tensor 1, described by tensor descriptor 1 (dividend), is divided by the corresponding element of input tensor 2 (divisor), described by tensor descriptor 2, and the quotient is placed in the corresponding element of the output tensor.

[0256] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010hex or 0011hex, respectively, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0257] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and output tensor will be the same; otherwise, a whole operand data exception will be recognized.

[0258] The output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage area address fields are ignored in one example.

[0259] Function code 20: NNPA-MIN (minimum)

[0260] When the NNPA-MIN function is specified, each element of input tensor 1 described by tensor descriptor 1 is compared to the corresponding element of input tensor 2 described by tensor descriptor 2. The smaller of the two values ​​is placed in the corresponding element of the output tensor descriptor. If both values ​​are equal, the value is placed in the corresponding element of the output tensor.

[0261] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or the data type in any of the specified tensor descriptors does not specify the NNP data type 1 (e.g., data type = 0), response codes, e.g., 0010hex or 0011hex respectively, are set in general-purpose register 0 and the instruction completes with a condition code, e.g., 1.

[0262] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and the output tensor will be the same; otherwise, a global operand data exception is recognized.

[0263] In one example, output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1 - 5, and the function-specific save area address field are ignored.

[0264] Function code 21: NNPA - MAX (maximum)

[0265] When the NNPA - MAX function is specified, each element of input tensor 1 described by tensor descriptor 1 is compared with the corresponding element of input tensor 2 described by tensor descriptor 2. The larger of the two values is placed in the corresponding element of the output tensor descriptor. If both values are the same, the value is placed in the corresponding element of the output tensor.

[0266] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or the data type in any of the specified tensor descriptors does not specify the NNP data type 1 (e.g., data type = 0), response codes, e.g., 0010hex or 0011hex respectively, are set in general-purpose register 0 and the instruction completes with a condition code, e.g., 1.

[0267] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and output tensor will be the same; otherwise, a whole operand data exception will be recognized.

[0268] The output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage area address fields are ignored in one example.

[0269] Function code 32: NNPA-LOG(natural logarithm)

[0270] When the NNPA-LOG function is specified, for each element of the input tensor described by tensor descriptor 1, if the element is greater than zero, the corresponding element in the output tensor described by the output tensor descriptor is the natural logarithm of that element. Otherwise, the corresponding element in the output tensor is not numerically representable and stores the value associated with negative infinity in the target data type.

[0271] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010hex or 0011hex, respectively, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0272] In one example, the shape, data layout, and data type of input tensor 1 and output tensor will be the same; otherwise, a whole operand data exception will be recognized.

[0273] The output tensor descriptor 2, input tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage area address fields are ignored in one example.

[0274] Function code 33: NNPA-EXP (exponential function)

[0275] When the NNPA-EXP function is specified, for each element of the input tensor described by tensor descriptor 1, the corresponding element in the output tensor described by the output tensor descriptor is the exponential function of that element.

[0276] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010hex or 0011hex, respectively, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0277] In one example, the shape, data layout, and data type of input tensor 1 and output tensor will be the same; otherwise, a whole operand data exception will be recognized.

[0278] The output tensor descriptor 2, input tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage area address fields are ignored in one example.

[0279] Function code 49: NNPA-RELU (Normalized Linear Unit)

[0280] When the NNPA-RELU function is specified, for each element of the input tensor described by tensor descriptor 1, if that element is less than or equal to zero, the corresponding element in the output tensor described by the output tensor descriptor is zero. Otherwise, the corresponding element in the output tensor is the smallest element in the input tensor, which is the clipping value specified in function-specific parameter 1.

[0281] For example, function-specific parameter 1 defines the clipping value for the ReLU operation. For instance, the clipping value is located within bits 16-31 of function-specific parameter 1. The clipping value is specified, for example, in the NNPA data type 1 format. A clipping value of zero indicates the use of the largest positive value; in other words, no clipping is performed. If a negative value is specified, a global operand data exception is recognized.

[0282] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010hex or 0011hex, respectively, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0283] In one example, the shape, data layout, and data type of input tensor 1 and output tensor will be the same; otherwise, a whole operand data exception will be recognized.

[0284] The output tensor descriptor 2, input tensor descriptor 2, input tensor descriptor 3, and the function-specific storage area address field are ignored in one example. Function-specific parameters 2-5 will store zero in one example.

[0285] Function code 50: NNPA-TANH (tangent)

[0286] When the NNPA-TANH function is specified, for each element of the input tensor described by tensor descriptor 1, the corresponding element value in the output tensor described by the output tensor descriptor is the hyperbolic tangent of that element.

[0287] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010hex or 0011hex, respectively, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0288] In one example, the shape, data layout, and data type of input tensor 1 and output tensor will be the same; otherwise, a whole operand data exception will be recognized.

[0289] The output tensor descriptor 2, input tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage area address fields are ignored in one example.

[0290] Function code 51: NNPA-SIGMOID

[0291] When the NNPA-SIGMOID function is specified, for each element of the input tensor described by tensor descriptor 1, the corresponding element in the output tensor described by the output tensor descriptor is the sigmoid function of that element.

[0292] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010hex or 0011hex, respectively, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0293] In one example, the shape, data layout, and data type of input tensor 1 and output tensor will be the same; otherwise, a whole operand data exception will be recognized.

[0294] The output tensor descriptor 2, input tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage area address fields are ignored in one example.

[0295] Function code 52: NNPA-SOFTMAX

[0296] When the NNPA-SOFTMAX function is specified, for each vector in dimension 1 of input tensor 1, the corresponding vector in the output tensor is calculated as described below.

[0297] * The maximum value of the vector is calculated.

[0298] * The sum of the exponential functions of the differences between each element in dimension 1 of the vector and the maximum value calculated above is calculated. If both the element in dimension 1 of the input vector and the maximum value calculated above are numerical, and the difference is not numerical, the result of the exponential function for that element is forced to zero.

[0299] * For each element in the vector, the intermediate quotient is formed by dividing the exponential function of the difference between the element and the maximum value calculated above by the sum calculated above. An arbitrarily chosen activation function is applied to this intermediate quotient to form the corresponding element in the output vector.

[0300] This process is repeated, for example, for all dimensional 4 index size × dimensional 3 index size × dimensional 2 index size vectors in dimensional 1.

[0301] In one example, the NNPA-SOFTMAX function-specific parameter 1 controls the activation function. For example, the ACT field of function-specific parameter 1 (e.g., bits 28-31) specifies the activation function. Example activation functions include:

[0302] ACT activation function

[0303] 0 The activation function is not executed.

[0304] 1 LOG

[0305] 2-15 are booked.

[0306] If a reserved value is specified for the ACT field, for example, a response code of F001hex will be reported, and the operation will be completed with a condition code, for example, 1.

[0307] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010hex or 0011hex, respectively, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0308] In one example, if the 3-dimensional index size of the input tensor is not equal to 1, for example, a response code of F000hex is stored, and the instruction completes with a condition code, for example, 1.

[0309] In one example, the shape, data layout, and data type of input tensor 1 and output tensor will be the same; otherwise, a whole operand data exception will be recognized.

[0310] Output tensor descriptor 2, input tensor descriptor 2, and input tensor descriptor 3 are ignored in one example. Function-specific parameters 2-5 will store zero in one example.

[0311] An 8KB function-specific storage area may be used by this function.

[0312] In one embodiment, when obtaining a vector in dimension 1, the elements may not be contiguous in memory, depending on the specified data layout format. If all elements of the dimension 1 vector of input tensor 1 store negative numbers of the largest size representable in the specified data type, the result may become less accurate.

[0313] Function code 64: NNPA-BATCHNORM (Batch Normalization)

[0314] When the NNPA-BATCHNORM function is specified, for each vector in dimension 1 of the input tensor, the corresponding vector in dimension 1 of the output tensor is calculated by multiplying each element of the vector by the corresponding element in the dimension 1 vector that makes up the input tensor. The full-precision product is then added to the corresponding element in the dimension 1 vector that makes up the input tensor, and then rounded to the precision of the specified data type of the output tensor. This process is repeated, for example, for all dimension 4 index size × dimension 3 index size × dimension 2 index size vectors in dimension 1.

[0315] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010hex or 0011hex, respectively, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0316] In one example, the following condition would be true; otherwise, a global operand data exception would be recognized:

[0317] * The shape and data layout of input tensor 1 and output tensor will be the same.

[0318] * The data types of the input tensor and the output tensor will be the same.

[0319] * The 1-dimensional index sizes of input tensors 1, 2, and 3, and the output tensor, will be the same.

[0320] * The 2, 3, and 4 index sizes of the input tensors 2 and 3 will be the same.

[0321] The output tensor descriptor 2 and the function-specific storage area address field are ignored in one example. Function-specific parameters 2-5 will store zero in one example.

[0322] Function code 80: NNPA-MAXPOOL2D Function code 81: NNPA-AVGPOOL2D

[0323] If either the NNPA-MAXPOOL2D or NNPA-AVGPOOL2D function is specified, input tensor 1, described by the input tensor 1 descriptor, is reduced by the specified operation to summarize the input window. The input window is selected by moving a 2D sliding window across dimension indices 2 and 3. The window summary is the elements in the output tensor. The sliding window dimension is described, for example, by function-specific parameters 4 and 5. The amount the sliding window moves across input tensor 1 when computing adjacent output tensor elements is called the stride. The sliding window stride is specified, for example, by function-specific parameters 2 and 3. If the NNPA-MAXPOOL2D operation is specified, the maximum value operation, as defined below, is performed on the window. If the NNPA-AVGPOOL2D operation is specified, the AVG operation, as defined below, is performed on the window. If the specified padding type is valid, all elements in the window are added to the set used to compute the resulting output elements. If the specified padding types are the same, depending on the window's location, only a subset of elements from the window may be added to the set used to calculate the resulting output elements.

[0324] In one example, the CollectElements operation adds an element to a set of elements, incrementing the number of elements in the set. The set is emptied each time the window's starting position moves. It is unpredictable whether elements not needed for the operation to be performed will be accessed.

[0325] Maximum value calculation: In one example, the maximum value of a set of elements in a window is calculated by comparing all elements in the set with each other and returning the maximum value.

[0326] Avg (average) calculation: In one example, the average of a set of elements in a window is calculated by dividing the sum of all elements in the set by the number of elements in the set.

[0327] In one example, the fields are assigned as follows:

[0328] * The pooling function-specific parameter 1 controls the padding type. For example, bits 29-31 of function-specific parameter 1 contain the PAD field, which specifies the padding type. Examples of types include, for example:

[0329] PAD Padding type

[0330] 0 Valid

[0331] 1 Same

[0332] 2-7 Booked

[0333] If a reserved value is specified for the PAD field, for example, a response code of F000hex will be reported, and the operation will complete with a condition code, for example, 1.

[0334] In one example, bit positions 0-28 for function-specific parameter 1 are reserved and will store zero.

[0335] * Function-specific parameter 2 stores a 32-bit unsigned binary integer that specifies, for example, the dimensional 2 stride (D2S), which specifies the number of elements the sliding window moves in dimension 2.

[0336] * Function-specific parameter 3 stores a 32-bit unsigned binary integer that specifies, for example, the 3-dimensional stride (D3S), which specifies the number of elements the sliding window moves in 3 dimensions.

[0337] * Function-specific parameter 4 stores a 32-bit unsigned binary integer that specifies, for example, the dimension 2 window size (D2WS), which specifies the number of elements in dimension 2 that the sliding window will store.

[0338] * Function-specific parameter 5 stores a 32-bit unsigned binary integer that specifies, for example, the dimension 3 window size (D3WS), which specifies the number of elements in dimension 3 that the sliding window will store.

[0339] In one example, the specified values ​​for function-specific parameters 2-5 will be less than or equal to the maximum dimension index size, and the specified values ​​for function-specific parameters 4-5 will be greater than zero; otherwise, a response code, e.g., 0012hex, will be reported, and the operation will complete with a condition code, e.g., 1.

[0340] If both the Dimension 2 stride and Dimension 3 stride are zero, and either the Dimension 2 window size or the Dimension 3 window size is greater than, for example, 1024, a response code, e.g., F001hex, is stored. If both the Dimension 2 stride and Dimension 3 stride are greater than, for example, zero, and either the Dimension 2 window size or the Dimension 3 window size is greater than, for example, 64, a response code, e.g., F002hex, is stored. If both the Dimension 2 stride and Dimension 3 stride are greater than, for example, zero, and either the Dimension 2 window stride or the Dimension 3 stride is greater than, for example, 30, a response code, e.g., F003hex, is stored. If both the Dimension 2 stride and Dimension 3 stride are greater than, for example, zero, and either the input tensor Dimension 2 index size or the input tensor Dimension 3 index size is greater than, for example, 1024, a response code, e.g., F004hex, is stored. For all of the above conditions, the instruction completes with a condition code, e.g., 1.

[0341] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010hex or 0011hex, respectively, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0342] In one example, the following condition would be true; otherwise, a global operand data exception would be recognized:

[0343] * The 4-dimensional index size and 1-dimensional index size of the input and output tensors will be the same.

[0344] * The data layout and data type of the input tensor and the output tensor will be the same.

[0345] * If both the 2-dimensional stride and the 3-dimensional stride are zero, the following additional condition will be true in one example:

[0346] * The input tensor's 2-dimensional index size will be equal to its 2-dimensional window size.

[0347] * The input tensor's 3-dimensional index size is equal to the 3-dimensional window size.

[0348] * The 2-dimensional and 3-dimensional index sizes of the output tensor will be 1.

[0349] * The specified padding will be considered valid.

[0350] * If either the 2-dimensional stride or the 3-dimensional stride is non-zero, then both strides will be non-zero in one example.

[0351] * If both the 2-dimensional stride and the 3-dimensional stride are greater than zero, the following additional condition will be true in one example:

[0352] * If the specified padding is enabled, the 2D window size will be less than or equal to the 2D index size of the input tensor.

[0353] * If the specified padding is enabled, the 3D window size will be less than or equal to the 3D index size of the input tensor.

[0354] * If the specified padding is the same, the following relationship will be satisfied between the 2-dimensional index sizes and 3-dimensional index sizes of the input and output tensors (pooling of the same padding):

[0355]

number

[0356]

number

[0357] Here:

[0358] IxDyIS: The dimension y of the input tensor x defined in tensor descriptor x is the index size.

[0359] OxDyIS: The dimension y of the output tensor x defined in tensor descriptor x is the index size.

[0360] D2S Dimension 2 Stride.

[0361] D3S Dimensional 3 Stride.

[0362] * If the specified padding is enabled, the following relationship will be satisfied between the 2-dimensional index size and the 3-dimensional index size of the input and output tensors (pooling of enabled padding):

[0363]

number

[0364]

number

[0365] Here, D2WS is the 2-dimensional window size, and D3WS is the 3-dimensional window size.

[0366] The output tensor descriptor 2, input tensor descriptors 2 and 3, and the function-specific storage area address field are ignored.

[0367] Function code 96: NNPA-LSTMACT (Long-term short-term memory activation)

[0368] When the NNPA-LSTMACT function is specified, the inputs to the LSTMACT operation are input tensor 1 (e.g., reformatted, concatenated input tensor) (e.g., reformatted, concatenated input tensor) (e.g., reformatted, concatenated input tensor) (e.g., reformatted, concatenated input tensor) (e.g., reformatted, concatenated input tensor) (e.g., reformatted, concatenated output tensor) (e.g., reformatted, concatenated output tensor)) (e.g., reformatted, concatenated output tensor) (e.g., reformatted, concatenated output tensor) (e.g., reformatted, concatenated output tensor) (e.g., reformatted, concatenated output tensor) (e.g., reformatted, concatenated output tensor

[0369] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type = 0), response codes 0010hex and 0011hex are set in general register 0, respectively, and the instruction completes with a condition code, e.g., 1.

[0370] In one embodiment, the following condition will be true; otherwise, a global operand data exception will be recognized:

[0371] * The 4-dimensional index size of input tensor 3 and output tensors 1 and 2 will be equal to, for example, 1.

[0372] * The 4-dimensional index size for input tensor 1 and input tensor 2 will be equal to, for example, 4.

[0373] * For example, the 3-dimensional index size for all input tensors and the two output tensors will be equal to, for example, 1.

[0374] * For example, the data layout and data type of all input tensors and the two output tensors will be the same.

[0375] * For example, the dimensional 1 index size of all input tensors and the two output tensors will be the same.

[0376] * For example, the dimensional 2 index size of all input tensors and the two output tensors will be the same.

[0377] The function-specific storage area address field is ignored in one example. Function-specific parameters 1-5 will store zero in one example.

[0378] Further details regarding one embodiment of long- and short-term cell activation are described herein with reference, for example, to Figures 4A and 5A-5B.

[0379] Function code 97: NNPA-GRUACT (Activation of gated recurrent unit)

[0380] When the NNPA-GRUACT function is specified, the inputs to the GRUACT operation are input tensor 2 (e.g., reformatted, concatenated input tensor), described by the input tensor 2 descriptor and divided into three subtensors for each dimension's 4 index values; input tensor 1 (e.g., reformatted, concatenated input tensor), described by the input tensor 1 descriptor and divided into three subtensors for each dimension's 4 index values; and input tensor 3, described by the input tensor 3 descriptor. At the end of the GRUACT operation, the output tensor (e.g., reformatted, concatenated output tensor), described by the output tensor descriptor, is stored.

[0381] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010hex or 0011hex, respectively, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0382] In one embodiment, the following condition will be true; otherwise, a global operand data exception will be recognized:

[0383] * The 4-dimensional index size for the output tensor and input tensor 3 will be equal to, for example, 1.

[0384] * The 4-dimensional index size for input tensor 1 and input tensor 2 will be equal to, for example, 3.

[0385] * For example, the 3-dimensional index size for all input and output tensors will be equal to, for example, 1.

[0386] * For example, the dimensional 1 index size of all input and output tensors will be the same.

[0387] * For example, the dimensional 2 index size of all input and output tensors will be the same.

[0388] * For example, the data layout and data type of all input and output tensors will be the same.

[0389] The output tensor descriptor 2 and the function-specific storage area address field are ignored in one example. Function-specific parameters 2-5 will store zero in one example.

[0390] Further details regarding one embodiment of gated recurrent unit cell activation are described herein with reference, for example, to Figure 4B.

[0391] Function code 112: NNPA-CONVOLUTION

[0392] When the NNPA-CONVOLUTION function is specified, for each output element in the output tensor described by the output tensor 1 descriptor, a 3D input 1 window consisting of dimensional indices 3, 2, and 1 is selected from input tensor 1 described by the input tensor 1 descriptor. A 3D input 2 window of the same size consisting of dimensional indices 4, 3, and 2 is selected from tensor 2 described by the input tensor 2 descriptor. Elements in the input 1 window are multiplied by the corresponding elements in the input 2 window, and all products are added together to create an initial sum. The initial sum is added to the corresponding elements of input tensor 3 to calculate the intermediate sum. The elements of the output tensor are the results of a specified activation function applied to the intermediate sum. If no activation function is specified, the output elements are equal to the intermediate sum.

[0393] If the specified padding type is valid, all elements in the window are used to calculate the resulting initial sum. If the specified padding type is the same, when calculating the resulting initial sum, some elements of the input 1 window may be implicitly zero, depending on the window's location.

[0394] It is unpredictable whether elements not required to perform the operation will be accessed.

[0395] In one example, the fields of function-specific parameters used by the convolution function are assigned as follows:

[0396] * The NNPA-CONVOLUTION function-specific parameter 1 controls the padding type and activation function. In one example, bits 29-31 of function-specific parameter 1 contain the PAD field, which specifies the padding type. The example types are as follows:

[0397] PAD Padding type

[0398] 0 Valid

[0399] 1 Same

[0400] 2-7 Booked

[0401] If a reserved value is specified for the PAD field, for example, a response code of F000hex will be reported, and the operation will complete with a condition code, for example, 1.

[0402] Furthermore, in one example, bits 24-27 of the NNPA-CONVOLUTION function-specific parameter 1 contain the activation field that specifies the activation function. The example function is as follows:

[0403] ACT activation function

[0404] 0 The activation function is not executed.

[0405] 1 RELU

[0406] 2-15 are booked.

[0407] If the RELU activation function is specified, the resulting output element values ​​are determined as follows: if the intermediate sum is less than or equal to zero, the corresponding element in the output tensor is zero; otherwise, the corresponding element in the output tensor is the minimum of the intermediate sum and the clipping value specified in function-specific parameter 4.

[0408] If a reserved value is specified for the ACT field, for example, a response code of F001hex will be reported, and the operation will be completed with a condition code, for example, 1.

[0409] * Function-specific parameter 2 stores a 32-bit unsigned binary integer that specifies, for example, the dimensional 2 stride (D2S), which specifies the number of elements the sliding window moves in dimension 2.

[0410] * Function-specific parameter 3 stores a 32-bit unsigned binary integer that specifies, for example, the 3-dimensional stride (D3S), which specifies the number of elements the sliding window moves in 3 dimensions.

[0411] The specified values ​​for function-specific parameters 2-3 must be less than the maximum dimension index size; otherwise, a response code such as 0012hex is reported, and the operation completes with a condition code, such as 1.

[0412] * Function-specific parameter 4 defines the clipping value for an optional RELU operation. In one example, the clipping value is located in bits 16-31 of function-specific parameter 4.

[0413] In one example, if the ACT field is zero, this field is ignored. If the ACT field specifies RELU, the clipping value is specified in the NNP data type 1 format. A clipping value of zero indicates that the largest positive value is used; in other words, no clipping is performed. If a non-zero value is specified, a global operand data exception is recognized.

[0414] In one example, if the specified data layout in any of the specified tensor descriptors other than input tensor 2 does not specify a 4D feature tensor (e.g., data layout=0) or if the specified data layout in input tensor 2 does not specify a 4D kernel tensor (e.g., data layout=1), a response code, e.g., 0010hex, is set in general-purpose register 0, and the instruction completes with a condition code, e.g., 1. In another example, if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type=0), a response code, e.g., 0011hex, is set in general-purpose register 0, and the instruction completes with a condition code, e.g., 1.

[0415] If both the Dimension 2 stride and the Dimension 3 stride are zero, and the Dimension 3 index size or Dimension 4 index size of input tensor 2 is greater than, for example, 448, a response code, e.g., F002hex, is stored. If both the Dimension 2 stride and the Dimension 3 stride are greater than zero, and either the Dimension 3 index size or Dimension 4 index size of input tensor 2 is greater than, for example, 64, a response code, e.g., F003hex, is stored, and the operation completes with a condition code, e.g., 1. If either the Dimension 2 stride or the Dimension 3 stride is greater than, for example, 13, a response code, e.g., F004hex, is stored, and the operation completes with a condition code, e.g., 1.

[0416] In one example, the following condition would be true; otherwise, a global operand data exception would be recognized:

[0417] * The data layouts of input tensor 1, input tensor 3, and output tensor will be the same.

[0418] * All input and output tensors will have the same data type.

[0419] * The index sizes for dimensions 2, 3, and 4 of the input 3-tensor will be 1.

[0420] * The 4-dimensional index size of the output tensor will be equal to the 4-dimensional index size of the input 1 tensor.

[0421] * The dimensional 1 index size of the output tensor will be equal to the dimensional 1 index size of the input 2 tensor and the dimensional 1 index size of the input 3 tensor.

[0422] * The dimensional 1 index size of the input tensor 1 is equal to the dimensional 2 index size of the input tensor 2.

[0423] * If both the 2-dimensional stride and the 3-dimensional stride are zero, the following additional condition will be true in one example:

[0424] * The index size of a 1-dimensional tensor input is equal to the index size of a 3-dimensional tensor input tensor.

[0425] * The input tensor with a dimension of 3 (indices) will have an index size equal to the input tensor with a dimension of 4 (indices).

[0426] * The 2-dimensional and 3-dimensional index sizes of the output tensor will be 1.

[0427] * The specified padding will be considered valid.

[0428] * If either the 2-dimensional stride or the 3-dimensional stride is non-zero, then both strides will be non-zero.

[0429] * If both the 2-dimensional stride and the 3-dimensional stride are greater than zero, the following additional condition will be true in one example:

[0430] * If the specified padding is enabled, the dimensional 2 index size of input tensor 1 will be greater than or equal to the dimensional 3 index size of input tensor 2.

[0431] * If the specified padding is enabled, the dimensional 3 index size of the input 1 tensor will be greater than or equal to the dimensional 4 index size of the input 2 tensor.

[0432] * If the specified padding is the same, the following relationship between the Dimension 2 index size and Dimension 3 index size of the input 1 tensor and output tensor will be satisfied in one example (convolution with the same padding):

[0433]

number

[0434]

number

[0435] Here:

[0436] O1D2IS: The 2-dimensional index size of the output tensor.

[0437] O1D3IS: The 3-index size of the output tensor.

[0438] I1D2IS: Dimension 2 index size of an input tensor.

[0439] I1D3IS: Dimension 3 index size of an input tensor.

[0440] D2S Dimension 2 Stride.

[0441] D3S Dimensional 3 Stride.

[0442] * If the specified padding is effective, the following relationship between the dimensional 2 index size and dimensional 3 index size of the input tensor 1, and the dimensional 3 index size and dimensional 4 index size of the input tensor 2 and the output tensor will be satisfied in one example (convolution with effective padding):

[0443]

number

[0444]

number

[0445] Here:

[0446] O1D2IS: The 2-dimensional index size of the output tensor.

[0447] O1D3IS: The 3-index size of the output tensor.

[0448] I1D2IS: Dimension 2 index size of an input tensor.

[0449] I1D3IS: Dimension 3 index size of an input tensor.

[0450] I2D3IS: Dimension 3 index size of a 2-dimensional tensor input.

[0451] I2D4IS: Dimension 4 index size of a 2-tensor input.

[0452] D2S Dimension 2 Stride.

[0453] D3S Dimensional 3 Stride.

[0454] The output tensor descriptor 2 and the function-specific storage area address field are ignored in one example. The function-specific parameter 5 will store zero in one example.

[0455] Function code 113: NNPA-MATMUL-OP (matrix multiplication operation)

[0456] When the NNPA-MATMUL-OP function is specified, each element in the output tensor described by the output tensor descriptor is computed as described below in one example:

[0457] * Using the dimension 1 vector acquisition operation described below, a dimension 1 vector is selected from input tensor 1 described by the input tensor 1 descriptor.

[0458] * Using the dimensional 2 vector acquisition operation described below, a dimensional 2 vector is selected from the input tensor 2 described by the input tensor 2 descriptor.

[0459] * The intermediate dot product of a 1-dimensional vector and a 2-dimensional vector is calculated using the dot product operation described below.

[0460] * The operation is performed on the elements of input tensor 3 described by the intermediate dot product and input tensor 3 descriptor, along with the same-dimensional index 4 and dimensional index 1 values ​​as output tensor elements. The resulting elements are stored in the output tensor. The fusion operation is determined by function-specific parameter 1, which is described below.

[0461] Dimension 1 vector acquisition operation: For a specified output element, a dimensional 1 vector is selected from the input 1 tensor, where an input dimensional 4 index becomes an output dimensional 4 index, an input dimensional 3 index becomes an output dimensional 3 index, and an input dimensional 2 index becomes an output dimensional 2 index.

[0462] Dimension 2 vector acquisition operation: For a specified output element, a dimensional 2 vector is selected from the input 2 tensor, where an input dimension 4 index becomes an output dimension 4 index, an input dimension 3 index becomes an output dimension 3 index, and an input dimension 1 index becomes an output dimension 1 index.

[0463] Dot product operation: The intermediate dot product of two vectors of the same size and data type is calculated as the sum of the products of each element in input vector 1 and the corresponding elements in input vector 2.

[0464] Fusion Operation: Function-specific parameter 1 controls the operation performed on the intermediate dot product and the corresponding elements from the input tensor 3. In one example, the NNPA-MATMUL-OP function-specific parameter 1 includes, for example, an operation field in bits 24-31. The operation field specifies the operation to be performed. An example operation is shown below:

[0465] Calculation Calculation type

[0466] Add 0

[0467] 1. Compare whether the dot product is high or not.

[0468] 2. Compare whether the dot product is low or not.

[0469] 3. Compare whether the dot product and elements are equal.

[0470] 4. Compare the dot product and whether the elements are unequal.

[0471] 5. Compare whether the dot product is high or not.

[0472] 6. Compare whether the dot product is low or not.

[0473] In one example, for the addition operation type, the three elements of the input tensor are added together by the intermediate dot product. For the comparison operation type, the intermediate dot product is compared with the three elements of the input tensor, and for the output tensor of the specified data type, if the comparison is true, the result is set to a value of, for example, +1; otherwise, it is set to a value of, for example, +0.

[0474] In one example, all other values ​​in the OPERATION field are reserved. If a reserved value is specified for the OPERATION field, for example, a response code of F000hex will be reported, and the operation will be completed with a condition code, for example, 1.

[0475] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010hex or 0011hex, respectively, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0476] In one embodiment, the following condition will be true; otherwise, a global operand data exception will be recognized:

[0477] * The 4-dimensional index size of all input and output tensors will be the same.

[0478] * The 3-dimensional index size of all input and output tensors will be equal to 1.

[0479] * The 2-dimensional index size of the input tensor 3 will be equal to 1.

[0480] * The Dimension 2 index size of the input tensor 1 and the output tensor will be the same.

[0481] * The dimensional 1 index size of input tensor 1 and the dimensional 2 index size of input tensor 2 will be the same.

[0482] * The dimensional 1 index sizes of input tensor 2, input tensor 3, and output tensor will be the same.

[0483] * The data layout and data type of all input and output tensors will be the same.

[0484] In one embodiment, the output tensor descriptor 2 and the function-specific storage area address field are ignored. Function-specific parameters 2-5 will, in one example, store zero.

[0485] Function code 114: NNPA-MATMUL-OP-BCAST23 (Matrix multiplication operation - broadcast 23)

[0486] When the NNPA-MATMUL-OP-BCAST23 function is specified, each element in the output tensor described by the output tensor descriptor is computed as described below in one example:

[0487] * Using the dimension 1 vector acquisition operation described below, a dimension 1 vector is selected from input tensor 1 described by the input tensor 1 descriptor.

[0488] * Using the dimensional 2 vector acquisition operation described below, a dimensional 2 vector is selected from the input tensor 2 described by the input tensor 2 descriptor.

[0489] * The dot product of a 1-dimensional vector and a 2-dimensional vector is calculated using the dot product operation described below.

[0490] * The elements of the input tensor 3 described by the input tensor 3 descriptor are added to the previously calculated dot product along with the same-dimensional index 1 values ​​as output tensor elements and stored in the output tensor.

[0491] Dimension 1 vector acquisition operation: For a specified output element, a dimensional 1 vector is selected from the input 1 tensor, where an input dimensional 4 index becomes an output dimensional 4 index, an input dimensional 3 index becomes an output dimensional 3 index, and an input dimensional 2 index becomes an output dimensional 2 index.

[0492] Dimension 2 vector acquisition operation: For a specified output element, a dimensional 2 vector is selected from the input 2 tensor, where the input dimension 4 index is 1, the input dimension 3 index is the output dimension 3 index, and the input dimension 1 index is the output dimension 1 index.

[0493] Dot product operation: The intermediate dot product of two vectors of the same size and data type is calculated as the sum of the products of each element in input vector 1 and the corresponding elements in input vector 2.

[0494] In one example, if the specified data layout in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0) or if the data type in any specified tensor descriptor does not specify an NNP data type 1 (e.g., data type = 0), a response code, e.g., 0010hex or 0011hex, respectively, is set in general register 0, and the instruction completes with a condition code, e.g., 1.

[0495] In one embodiment, the following condition will be true; otherwise, a global operand data exception will be recognized:

[0496] * The input tensor 1 and the output tensor will have the same 4-dimensional index size.

[0497] * The 4-dimensional index size of input tensor 2 and input tensor 3 will be equal to 1.

[0498] * The 3-dimensional index size of all input and output tensors will be equal to 1.

[0499] * The 2-dimensional index size of the input tensor 3 will be equal to 1.

[0500] * The Dimension 2 index size of the input tensor 1 and the output tensor will be the same.

[0501] * The dimensional 1 index size of input tensor 1 and the dimensional 2 index size of input tensor 2 will be the same.

[0502] * The dimensional 1 index sizes of input tensor 2, input tensor 3, and output tensor will be the same.

[0503] * The data layout and data type of all input and output tensors will be the same.

[0504] In one embodiment, the output tensor descriptor 2 and the function-specific storage area address field are ignored. Function-specific parameters 1-5 will store zero in one example.

[0505] In one embodiment of the neural network processing support instructions, the result is unpredictable if the output tensor overlaps with the input tensor or parameter block.

[0506] For example, if an attempt is made to execute a neural network processing support instruction and the parameter block is not specified, for example, on a double word boundary, a specification exception will be recognized.

[0507] An attempt is made to execute neural network processing support instructions, and if, for example, a tensor descriptor mismatch exists, a global operand data exception is recognized.

[0508] The resulting condition codes for neural network processing support instructions include, for example, the following: 0 - normal completion; 1 - response code is set; 2 -; 3 - amount of data to be processed determined by the CPU.

[0509] In one embodiment, the execution priority for neural network processing support instructions includes, for example, the following:

[0510] 1.-7 Exceptions that have the same priority as the program interrupt conditions in general cases.

[0511] 8.A Condition code 1 resulting from a specified function code that is not assigned or installed.

[0512] 8.B Specification exception caused by a parameter block not specified on a double word boundary.

[0513] 9. Access exception for accessing parameter blocks.

[0514] 10. Condition code 1 resulting from a specified format of a parameter block not supported by the model.

[0515] 11.A Condition code 1, resulting from the specified tensor data layout not being supported.

[0516] 11.B Global operand data exception caused by different data layouts between tensor descriptors.

[0517] 12.A Condition code 1 resulting from conditions other than those included in items 8.A, 10, and 11.A above and 12.B.1 below.

[0518] 12.B.1 Condition code 1 resulting from an invalid output tensor data type for NNPA-RELU (Normalized Linear Unit) and NNPA-CONVOLUTION (other available functions not described herein).

[0519] 12.B.2 Overall operand data exception due to invalid values ​​for NNPA-RELU function-specific parameter 1 and NNPA-CONVOLUTION function-specific parameter 4.

[0520] 13.A Access exception for accessing the output tensor.

[0521] 13.B Access exceptions for accessing input tensors.

[0522] 13. Access exceptions for accessing function-specific storage areas.

[0523] 14. Condition code 0.

[0524] As described herein, a single instruction (e.g., a neural network processing support instruction) is configured to perform multiple functions, including a query function and a plurality of non-query functions. Each non-query function operates on a tensor, such as a 4D tensor (or a tensor of other sizes). To facilitate processing using tensors, according to one or more aspects of the present invention, the tensor is reformatted into a plurality of, for example, 2D tensors having specific characteristics to improve processing. For example, the reformatted tensor has an easily computable address, may be loaded / stored in a single operation, and increases bandwidth and improves system performance. This is, for example, a result of starting a tensor on a memory boundary and having a fixed dimension (made possible using padding).

[0525] In one example, tensor reformatting is performed by a processor (e.g., general-purpose processor 104) that obtains a neural network processing support instruction specifying a non-query function. The specified tensor is reformatted using, for example, tensor descriptor information provided in a parameter block (e.g., tensor descriptors 660, 665 in Figure 6G). Address information for the reformatted tensor is provided to a dedicated processor (e.g., neural network processor 105) for use when performing the function specified by the instruction.

[0526] In one example, an instruction (e.g., a neural network processing support instruction) implements recurrent neural network cell activation (e.g., long short-term memory cell activation, gated recurrent unit cell activation, and / or other cell activation), where the input and / or output data use a concatenated data layout in tensor memory to prevent data reformatting between operations. For example, for concatenation of input data, weight tensors are independently 2D transformed and concatenated within a time step preceding the multiplication operation. A single invocation of the instruction computes all multiplications of the input features across the time step at once for an intermediate result. The intermediate result is provided in a memory address contiguous tensor for computing the activation.

[0527] For concatenating the output data, the result tensor contains the concatenation of the 2D reformatted results of the time steps. Each time step result tensor contains a contiguous tensor of memory addresses of the completed results of the recurrent neural network computation. The time step result tensor can be used directly in the computation of the next time step without any data manipulation or copy operations.

[0528] In one or more further embodiments, individual activations and operations are combined into a single instruction executed at once in the accelerator. The recurrent neural network, in one example, relies on a long short-term memory network or a gated recurrent unit network. For each time step (per operation), multiple activations (e.g., sigmoid, tanh) and other operations (e.g., addition, subtraction, and / or multiplication) are applied to hidden states (e.g., previously learned), input and state, and cell state. Calling an accelerator (e.g., neural network processor 105) for each of these stages is detrimental to the overall performance of the processor and / or system, at least due to the accelerator startup time. According to one embodiment of the present invention, significant speedup is achieved based on individual activations and operations being combined into a single instruction executed at once in the accelerator. According to one embodiment of the present invention, individual activations and coupling functions are combined to implement a single instruction. Therefore, only one execution exists; intermediate computation data is stored in the accelerator rather than written back to memory; the SIMD width and pipelining properties of the accelerator allow for more computations to be performed in parallel in fewer cycles per computer; and higher precision is used for intermediate results, resulting in improved accuracy and greater stability for long short-term storage and / or gated recurrent unit operations. For example, the combination of multiplication and addition operations provides higher accuracy without loss of precision for intermediate results. Furthermore, higher numerical accuracy can be achieved by storing intermediate computations in the accelerator with higher precision.

[0529] Furthermore, according to one or more aspects of the present invention, the matrix multiplication operation used to provide the concatenated result tensor input to the cell activation is separate from the cell activation, reducing the complexity of a single operation and allowing the reuse of basic blocks for other recurrent neural networks. The designed instructions provide spatially close input and output data sources to reduce address translation.

[0530] In one or more embodiments, the activation of an input in an internal format is computed, the computed values ​​are combined, and one or more outputs in an input numerical format are generated. For example, the internal format is, for example, a model-dependent format for a neural network processor. In one example, the internal format used may have a different numerical precision than the input / output numerical format in order to increase accuracy or reduce computation time and power.

[0531] Furthermore, according to one or more embodiments, multiple activations are encapsulated in a single instruction. The instruction provides modularity without breaking down the activations into very small chunks. In addition, the instruction uses a concatenated input and output format for the activations, providing savings in processing time and improvements in processing speed.

[0532] One or more aspects of the present invention are closely tied to computer technology, facilitating processing within computers and improving their performance. Reformatted concatenated tensors and / or instructions for defining and / or using such tensors may be used in many technical fields such as computer processing, artificial intelligence, recurrent neural networks, medical processing, engineering, automotive technology, and manufacturing. As described herein, the use of reformatted concatenated tensors provides certain optimizations, including optimizations when performing complex calculations used in various technical fields, improving those fields by increasing bandwidth, providing efficiency, and / or reducing execution time.

[0533] Further details of one embodiment that facilitates processing in a computing environment, such as one or more aspects of the present invention, are described with reference to Figures 10A and 10B.

[0534] Referring to Figure 10A, an instruction to perform recurrent neural network cell activation is executed (1000). Execution includes, for example, performing multiple operations of recurrent neural network cell activation to provide the result of recurrent neural network cell activation (1002). As an example, multiple operations are performed in a single invocation of the instruction (1004).

[0535] Using a single instruction call to perform multiple operations reduces complexity, lowers system resource usage, and improves system performance.

[0536] In one example, multiple operations include one or more sigmoid functions and one or more tangent functions (1006). In another example, multiple operations include tensor element-wise addition and tensor element-wise multiplication operations (1008).

[0537] For example, multiple operations include one or more sigmoid functions, one or more tangent functions, one or more tensor element-wise addition operations, and one or more tensor element-wise multiplication operations (1010).

[0538] In one example, one or more inputs to an instruction include one or more connected tensors (1012). A connected tensor may be used directly by an instruction executed on an accelerator performing cell activation in a recurrent neural network, for example. The connected tensor may be accessed in a single operation, saving processing time and increasing processing speed. Furthermore, fewer tensor pointers need to be managed, and there are reductions in copying or reorganizing tensor data between accelerator invocations, resulting in improved processing speed.

[0539] In one example, referring to Figure 10B, the result is an output tensor (1014), and the output tensor is, as an example, the input to another invocation of the instruction (1016).

[0540] For example, recurrent neural network cell activation includes long-term memory cell activation (1020), or recurrent neural network cell activation includes gated recurrent unit cell activation (1022).

[0541] In one example, the multiple operations of recurrent neural network cell activation are performed by an accelerator, generating intermediate computation data (1024). The intermediate computation data is stored in the accelerator, for example (1026).

[0542] In one example, performing multiple operations includes performing multiple operations on spatially close input data (1028).

[0543] Other modifications and embodiments are possible.

[0544] Aspects of the present invention may be used in many types of computing environments. Another example of a computing environment incorporating and using one or more aspects of the present invention is described with reference to Figure 11A. As an example, the computing environment in Figure 11A is based on the z / Architecture® instruction set architecture provided by International Business Machines Corporation, located in Armonk, New York. However, the z / Architecture instruction set architecture is merely one illustrative architecture. Again, the computing environment may be based on other architectures, including but not limited to the Intel® x86 architecture, other architectures of International Business Machines Corporation, and / or architectures of other companies. Intel is a trademark or registered trademark of Intel Corporation or its subsidiaries in the United States and other countries.

[0545] In one example, the computing environment 10 includes a central electronic processing unit (CEC) 11. The central electronic processing unit 11 includes one or more processors, such as one or more general-purpose processors (known as central processing units (CPUs) 13) and one or more dedicated processors (e.g., neural network processors 31), and multiple components, such as memory 12 (known as system memory, main memory, main storage, central storage, or storage) coupled to an input / output (I / O) subsystem 14.

[0546] For example, one or more dedicated processors may be separate from one or more general-purpose processors, and / or at least one dedicated processor may be embedded within at least one general-purpose processor. Other variations are also possible.

[0547] The I / O subsystem 14 may be part of the central electronic processing unit or separate from it. It directs the flow of information between the main storage 12 and the input / output control unit 15 and input / output (I / O) devices 16 coupled to the central electronic processing unit.

[0548] Many types of I / O devices may be used. One particular type is a data storage device 17. The data storage device 17 can store one or more programs 18, one or more computer-readable program instructions 19, and / or data, etc. The computer-readable program instructions may be configured to perform functions of embodiments of the present invention.

[0549] The central electronic processing unit 11 includes and / or may be coupled with removable / non-removable volatile / non-volatile computer system storage media. For example, it may include and / or be coupled with non-removable non-volatile magnetic media (typically called “hard drives”), magnetic disk drives for reading from or writing to removable non-volatile magnetic disks (e.g., “floppy disks”), and / or optical disk drives for reading from or writing to removable non-volatile optical disks such as CD-ROMs, DVD-ROMs, or other optical media. It should be understood that other hardware and / or software components may be used in conjunction with the central electronic processing unit 11. Examples include, but are not limited to, microcode or millicode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0550] Furthermore, the central electronic processing unit 11 may be able to operate with a number of other general-purpose or dedicated computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with the central electronic processing unit 11 include, but are not limited to, personal computer (PC) systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems or devices.

[0551] The central electronic processing unit 11 provides logical partition and / or virtualization support in one or more embodiments. In one embodiment, as shown in Figure 11B, the memory 12 includes, for example, one or more logical partitions 20, a hypervisor 21 that manages the logical partitions, and processor firmware 22. One example of the hypervisor 21 is the Processor Resource / System Manager (PR / SM®) provided by International Business Machines Corporation, located in Armonk, New York. PR / SM is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction.

[0552] Each logical partition 20 can function as a separate system. That is, each logical partition can be independently reset and run a guest operating system 23, such as the z / OS® operating system provided by International Business Machines Corporation in Armonk, New York, or other control code 24, such as Coupling Facility Control Code (CFCC), and operate with different programs 25. An operating system or application program running on a logical partition appears to have access to the whole and complete system, but in reality, only a portion of it is available. While the z / OS operating system is provided as an example, other operating systems provided by International Business Machines Corporation and / or other companies may be used according to one or more aspects of the present invention.

[0553] The memory 12 is, for example, coupled to the CPU 13 (Figure 11A), and these are physical processor resources that can be allocated to logical partitions. For example, a logical partition 20 may contain one or more logical processors, each of which represents all or a certain percentage of the physical processor resources 13 that can be dynamically allocated to the logical partition.

[0554] In further embodiments, the central electronic processing unit provides virtual machine support (with or without logical partition support). As shown in Figure 11C, the memory 12 of the central electronic processing unit 11 includes, for example, one or more virtual machines 26, a virtual machine manager such as a hypervisor 27 that manages the virtual machines, and processor firmware 28. One example of a hypervisor 27 is the z / VM® hypervisor provided by International Business Machines Corporation, located in Armonk, New York. A hypervisor is sometimes referred to as a host. z / VM is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction.

[0555] The virtual machine support of the central electronic processing unit provides the ability to operate a number of virtual machines 26, each capable of running a different program 29 and executing a guest operating system 30 such as the Linux® operating system. Each virtual machine 26 can function as a separate system; that is, each virtual machine can be reset independently, execute a guest operating system, and run with a different program. The operating system or application program running in the virtual machine appears to have access to the whole and complete system, but in reality, only a portion of it is available. While z / VM and Linux are provided as examples, other virtual machine managers and / or operating systems may be used in accordance with one or more aspects of the present invention. The registered trademark Linux® is used under a sublicense from the Linux Foundation, which is the exclusive licensee of Linus Torvalds, the owner of the trademark worldwide.

[0556] Another embodiment of a computing environment using one or more aspects of the present invention is described with reference to Figure 12A. In this example, the computing environment 36 includes, for example, native central processing units (CPUs) 37, memory 38, and one or more input / output devices and / or interfaces 39, coupled to one or more buses 40 and / or other connections. For example, the computing environment 36 may include a PowerPC® processor provided by International Business Machines Corporation, located in Armonk, New York; an HP Superdome using an Intel® Itanium® II processor provided by Hewlett Packard Co., located in Palo Alto, California; and / or other machines based on architectures provided by International Business Machines Corporation, Hewlett Packard, Intel Corporation, Oracle and / or other companies. PowerPC is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction. Itanium is a trademark or registered trademark of Intel Corporation or its subsidiaries in the United States and other countries.

[0557] The native central processing unit 37 includes one or more native registers 41, such as one or more general-purpose registers and / or one or more dedicated registers, which are used during processing within the environment. These registers contain information representing the state of the environment at any given point in time.

[0558] Furthermore, the native central processing unit 37 executes instructions and code stored in memory 38. In one particular example, the central processing unit executes emulator code 42 stored in memory 38. This code enables a computing environment configured in one architecture to emulate another architecture. For example, the emulator code 42 enables machines based on architectures other than the z / Architecture instruction set architecture, such as PowerPC processors, HP Superdome servers, or others, to emulate the z / Architecture instruction set architecture and execute software and instructions developed based on the z / Architecture instruction set architecture.

[0559] Further details regarding the emulator code 42 are described with reference to Figure 12B. The guest instructions 43 stored in memory 38 consist of software instructions (e.g., correlated with machine instructions) developed to run on architectures other than the native CPU 37. For example, the guest instructions 43 may be designed to run on a processor based on the z / Architecture instruction set architecture, but instead are emulated on the native CPU 37, which may be, for example, an Intel Itanium II processor. In one example, the emulator code 42 includes an instruction fetch routine 44 for retrieving one or more guest instructions 43 from memory 38 and, optionally, providing local buffering for the retrieved instructions. It also includes an instruction translation routine 45 for determining the type of the retrieved guest instruction and translating the guest instruction into one or more corresponding native instructions 46. This translation includes, for example, identifying the function that the guest instruction will perform and selecting the native instruction that will perform that function.

[0560] Furthermore, the emulator code 42 includes an emulation control routine 47 for executing native instructions. The emulation control routine 47 may cause the native CPU 37 to execute a native instruction routine that emulates one or more previously obtained guest instructions, and at the end of such execution, return control to an instruction fetch routine for emulating the acquisition of the next guest instruction or group of guest instructions. The execution of the native instruction 46 may include loading data from memory 38 into a register; returning data from the register to memory and storing it; or performing some type of arithmetic or logical operation as determined by the translation routine.

[0561] Each routine is implemented, for example, in software stored in memory and executed by the native central processing unit 37. In other examples, one or more routines or operations are implemented in firmware, hardware, software, or any combination thereof. The emulated processor registers may be emulated by using the native CPU registers 41 or by using locations in memory 38. In embodiments, the guest instruction 43, native instruction 46, and emulator code 42 may reside in the same memory or be distributed across different memory devices.

[0562] Instructions that can be emulated include, according to one or more aspects of the present invention, the neural network-assisted processing instructions described herein. Furthermore, one or more aspects of other instructions and / or tensor processing (including, but not limited to, tensor definition, generation, reformatting, and / or concatenation) may be emulated according to one or more aspects of the present invention.

[0563] The computing environments described above are merely examples of computing environments that may be used. Other environments may be used, but are not limited to unpartitioned environments, partitioned environments, cloud environments, and / or emulated environments; embodiments are not limited to any one environment. While various examples of computing environments are described herein, one or more aspects of the present invention may be used with many types of environments. The computing environments provided herein are merely examples.

[0564] Each computing environment can be configured to include one or more aspects of the present invention.

[0565] One or more aspects may relate to cloud computing.

[0566] While this disclosure includes a detailed description of cloud computing, it should be understood that the implementation of the teachings described herein is not limited to cloud computing environments. Rather, embodiments of the present invention can be implemented in combination with any other type of computing environment that is currently known or may be developed in the future.

[0567] Cloud computing is a service delivery model that enables convenient on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and deployed with minimal administrative effort or interaction with service providers. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0568] The characteristics are as follows:

[0569] On-demand self-service: Cloud consumers can unilaterally provision computing power, such as server time and network storage, automatically as needed, without requiring human interaction with service providers.

[0570] Broad network access: This capability is available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs®).

[0571] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically allocated and reallocated according to demand. While consumers generally have no control or knowledge of the exact location of the resources provided, there is location independence in that they may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0572] Rapid resilience: This capability allows for rapid and elastic provisioning, sometimes automatically, enabling quick scaling out and rapid release and rapid scaling in. To consumers, the capacity available for provisioning often appears unlimited and can be purchased in any quantity at any time.

[0573] Services measured: Cloud systems automatically control and optimize resource usage by leveraging metric capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, thereby providing transparency to both service providers and consumers.

[0574] The service model is as follows:

[0575] Software as a Service (SaaS): The ability provided to consumers is the use of a provider's applications running on cloud infrastructure. These applications are accessible from various client devices through thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0576] Platform as a Service (PaaS): The ability provided to consumers is to deploy applications they have created or acquired, written using programming languages ​​and tools supported by the provider, on cloud infrastructure. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they do control the deployed applications and, in some cases, the configuration of the application hosting environment.

[0577] Infrastructure as a Service (IaaS): The ability provided to consumers is to provision processing, storage, networking, and other basic computing resources, where consumers can deploy and run any software, including operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but they do control the operating system, storage, and deployed applications, and in some cases have limited control over selected networking components (e.g., host firewalls).

[0578] The deployment model is as follows:

[0579] Private Cloud: This cloud infrastructure operates solely for a specific organization. It may be managed by that organization or a third party, and may reside on-premises or off-premises.

[0580] Community Cloud: This cloud infrastructure is shared by several organizations and supports a specific community that shares common interests (e.g., mission, security requirements, policies, and compliance considerations). It may be managed by those organizations or a third party and may reside on-premises or off-premises.

[0581] Public Cloud: This cloud infrastructure is made available to the general public or large industry groups and is owned by an organization that sells cloud services.

[0582] Hybrid Cloud: This cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain separate entities but are joined together by standard or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0583] Cloud computing environments are service-oriented, focusing on statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing lies an infrastructure that includes a network of interconnected nodes.

[0584] Referring here to Figure 13, an exemplary cloud computing environment 50 is shown. As shown, the cloud computing environment 50 comprises one or more cloud computing nodes 52 that can communicate with local computing devices used by cloud consumers, such as personal digital assistants (PDAs) or mobile phones 54A, desktop computers 54B, laptop computers 54C, and / or automotive computer systems 54N. The nodes 52 may communicate with each other. They may be physically or virtually grouped within one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or a combination thereof, as described above in this specification (not shown). This enables the cloud computing environment 50 to provide infrastructure, platforms, and / or software as services that do not require cloud consumers to maintain resources on their local computing devices for that purpose. The types of computing devices 54A-N shown in Figure 13 are intended for illustrative purposes only, and it should be understood that the computing nodes 52 and the cloud computing environment 50 can communicate with any type of computerized device via any type of network and / or network addressable connection (e.g., using a web browser).

[0585] Referring now to Figure 14, a set of functional abstraction layers provided by the cloud computing environment 50 (Figure 13) is shown. It should be understood in advance that the components, layers, and functionalities shown in Figure 14 are for illustrative purposes only and that embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functionalities are provided:

[0586] The hardware and software layer 60 comprises hardware and software components. Examples of hardware components include: a mainframe 61; a RISC (Reduced Instruction Set Computer) architecture-based server 62; a server 63; a blade server 64; a storage device 65; and network and networking components 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0587] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities may be provided: virtual servers 71; virtual storage 72; virtual networks 73 including virtual private networks; virtual applications and operating systems 74; and virtual clients 75.

[0588] In one example, the management layer 80 may provide the functions described below. Resource provisioning 81 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Measurement and pricing 82 provides cost tracking as resources are used within the cloud computing environment and billing or invoices for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides consumers and system administrators with access to the cloud computing environment. Service level management 84 provides cloud computing resource allocation and management to ensure that required service levels are met. Service level agreement (SLA) planning and execution 85 provides pre-arrangements and procurement of cloud computing resources where future requirements are expected in accordance with the SLA.

[0589] The workload layer 90 provides examples of functions that can be utilized in a cloud computing environment. Examples of workloads and functions that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analysis processing 94; transaction processing 95; and tensor and / or neural network-assisted processing 96.

[0590] Aspects of the present invention may be systems, methods, and / or computer program products at any possible level of technical detail of integration. A computer program product may include a computer-readable storage medium (or multiple mediums) having computer-readable program instructions for causing a processor to execute aspects of the present invention.

[0591] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of those described above. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital multipurpose disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or grooved raised structures recording instructions, and any suitable combination of those described above. When used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0592] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives computer-readable program instructions from the network and transfers such instructions for storage in a computer-readable storage medium within the respective computing / processing device.

[0593] The computer-readable program instructions that perform the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages, the one or more programming languages ​​including object-oriented programming languages ​​such as Smalltalk®, C++, etc., and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially executed on the user's computer as a standalone software package, partially executed on the user's computer and partially executed on a remote computer, or fully executed on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be to an external computer (for example, via the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) can be personalized by executing computer-readable program instructions using state information of computer-readable program instructions in order to perform an aspect of the present invention.

[0594] Aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0595] These computer-readable program instructions can be provided to a computer or other programmable data processing device processor to generate a machine, thereby creating means for implementing functions / operations specified in one or more blocks of a flowchart and / or block diagram, through which instructions executed via the computer or other programmable data processing device processor. Furthermore, these computer-readable program instructions can be stored in a computer-readable storage medium, which can instruct a computer, programmable data processing device, and / or other device to function in a specific manner, thereby resulting in a computer-readable storage medium containing instructions that include instructions for implementing functions / operations specified in one or more blocks of a flowchart and / or block diagram.

[0596] Furthermore, computer-readable program instructions can be loaded into a computer, other programmable data processing device, or other device to execute a series of operational steps on the computer, other programmable device, or other device, thereby generating a computer implementation process in which the instructions executed on the computer, other programmable device, or other device implement the functions / operations specified in one or more blocks of a flowchart and / or block diagram.

[0597] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions containing one or more executable instructions that implement a specified logical function. In some alternative implementations, the functions described in the blocks may be performed in an order different from the order shown in the figures. For example, two blocks shown consecutively may actually be achieved as a single step, or they may be executed simultaneously, substantially simultaneously, partially or entirely in overlapping time, or the blocks may be executed in reverse order depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart diagram, and combinations of blocks in the block diagram and / or flowchart diagram, may be implemented by a dedicated hardware-based system that performs a specified function or operation, or a combination of dedicated hardware and computer instructions.

[0598] In addition to the above, one or more aspects of the above may be offered, provided, deployed, managed, and serviced by a service provider offering to manage the customer environment. For example, a service provider may create, maintain, and support computer code and / or computer infrastructure that executes one or more aspects of the above for one or more customers. In return, the service provider may receive payments from customers, for example, under subscription and / or fee agreements. In addition or alternatively, the service provider may receive payments from the sale of advertising content to one or more third parties.

[0599] In one embodiment, the application may be deployed to perform one or more embodiments. As one example, the deployment of the application includes providing a computer infrastructure capable of performing one or more embodiments.

[0600] In a further embodiment, a computing infrastructure may be deployed that includes integrating computer-readable code into a computing system, where the code combined with the computing system can perform one or more embodiments.

[0601] In a further embodiment, a process for integrating a computing infrastructure is provided, which includes integrating computer-readable code into a computer system. The computer system includes a computer-readable medium, where the computer medium includes one or more embodiments. The code combined with the computer system is capable of performing one or more embodiments.

[0602] Various embodiments have been described above, but these are merely examples. For example, computing environments of other architectures may be used to incorporate and / or use in one or more aspects. Furthermore, different instructions or operations may be used. In addition, different types of registers and / or different registers may be used. Furthermore, other data formats, data layouts and / or data sizes may be supported. In one or more embodiments, one or more general-purpose processors, one or more dedicated processors, or a combination of general-purpose and dedicated processors may be used. Many variations are possible.

[0603] Various embodiments are described herein. Furthermore, many modifications are possible without departing from the spirit of the embodiments of the present invention. It should be noted that, unless otherwise contradictory, each embodiment or feature and its variations described herein may be combined with any other embodiment or feature.

[0604] Furthermore, other types of computing environments may be beneficial and utilized. For example, a data processing system suitable for storing and / or executing program code is available, which includes at least two processors directly or indirectly coupled to memory elements via a system bus. The memory elements include, for example, local memory used while the program code is actually executed, mass storage, and cache memory providing temporary storage for at least some of the program code to reduce the number of times the code must be retrieved from mass storage during execution.

[0605] Input / output or I / O devices (including, but not limited to, keyboards, displays, pointing devices, DASDs, tapes, CDs, DVDs, thumb drives, and other memory media) can be connected to the system either directly or via an intermediary I / O controller. Network adapters can also be connected to the system to enable the data processing system to connect to other data processing systems or remote printers or storage devices through a private or public network. Modems, cable modems, and Ethernet® cards are just some of the types of network adapters available.

[0606] The terminology used herein is intended solely to describe and not to limit specific embodiments. Where used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context explicitly indicates otherwise. Where used herein, the terms "comprises" and / or "comprising" specify the presence of the described features, integers, stages, actions, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, stages, actions, elements, components, and / or groups thereof.

[0607] All means or step-plus-function elements in the following claims are intended to include, in combination with other claimed elements, any corresponding structures, materials, actions, and equivalents for performing a function, if any. The descriptions of one or more embodiments are presented for illustrative and explanatory purposes, but are not intended to be exhaustive or limitful to the disclosed forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments have been selected and described in order to best illustrate various aspects and practical applications, and to enable others skilled in the art to understand various embodiments with various modifications to suit specific uses to be contemplated. 。 [Item 1] A computer program product for facilitating processing within a computing environment, wherein the computer program product is: One or more computer-readable storage media, and program instructions collectively stored on the one or more computer-readable storage media for executing the method. The method comprises: The stage where instructions are executed to perform recurrent neural network cell activation. The steps to be performed are: A step of performing multiple operations for recurrent neural network cell activation and providing the results of the recurrent neural network cell activation. A computer program product that includes, wherein the plurality of operations are performed in a single invocation of the instruction. [Item 2] The aforementioned operations include one or more sigmoid functions and one or more tangent functions, as described in the computer program product described in the preceding item. [Item 3] The aforementioned operations include tensor element-level addition and tensor element-level multiplication operations, and are computer program products as described in any one of the above items. [Item 4] The computer program product described in any one of the above items, wherein the aforementioned operations include one or more sigmoid functions, one or more tangent functions, one or more tensor element unit addition operations, and one or more tensor element unit multiplication operations. [Item 5] One or more inputs to the instruction are computer program products as described in any one of the preceding items, including one or more connected tensors. [Item 6] The result is an output tensor, the output tensor is the input to another invocation of the instruction, as described in any one of the preceding items. [Item 7] The computer program product described in any one of the above items, which includes the activation of long-term memory cells, as described above. [Item 8] The aforementioned recurrent neural network cell activation is a computer program product as described in any one of the preceding items, including gated recurrent unit cell activation. [Item 9] The computer program product according to any one of the preceding items, wherein the step of performing the plurality of operations for recurrent neural network cell activation is performed by an accelerator to generate intermediate computation data, and the method further comprises the step of storing the intermediate computation data in the accelerator. [Item 10] The computer program product described in any one of the above items, wherein the step of performing the aforementioned operations includes the step of performing the aforementioned operations on spatially close input data. [Item 11] A computer system for facilitating processing within a computing environment, wherein the computer system is: memory; and At least one processor that communicates with the aforementioned memory The computer system is configured to perform a method, and the method is: The stage where instructions are executed to perform recurrent neural network cell activation. The steps to be performed are: A step of performing multiple operations for recurrent neural network cell activation and providing the results of the recurrent neural network cell activation. A computer system that includes such operations, wherein the multiple operations are performed in a single invocation of the instruction. [Item 12] The computer system described in the preceding item, wherein the aforementioned operations include one or more sigmoid functions, one or more tangent functions, one or more tensor element-level addition operations, and one or more tensor element-level multiplication operations. [Item 13] One or more inputs to the instruction are computer systems, as described in either of the two preceding items, which include one or more connected tensors. [Item 14] The computer system described in any one of the three preceding items, wherein the recurrent neural network cell activation includes long-term memory cell activation or gated recurrent unit cell activation. [Item 15] The computer system according to any one of the four preceding items, wherein the step of performing the plurality of operations of recurrent neural network cell activation is performed by an accelerator to generate intermediate computation data, and the method further comprises the step of storing the intermediate computation data in the accelerator. [Item 16] A computer implementation method for facilitating processing within a computing environment, wherein the computer implementation method is: The stage where instructions are executed to perform recurrent neural network cell activation. The steps to be performed are: A step of performing multiple operations for recurrent neural network cell activation and providing the results of the recurrent neural network cell activation. A computer implementation method having such that the plurality of operations are performed in a single invocation of the instruction. [Item 17] The computer implementation method described in the preceding item, wherein the plurality of operations include one or more sigmoid functions, one or more tangent functions, one or more tensor element unit addition operations, and one or more tensor element unit multiplication operations. [Item 18] One or more inputs to the instruction are computer implementation methods described in any one of the two preceding items, comprising one or more connected tensors. [Item 19] The computer implementation method described in any one of the three preceding items, wherein the recurrent neural network cell activation includes long-term memory cell activation or gated recurrent unit cell activation. [Item 20] The computer implementation method according to any one of the four preceding items, further comprising the steps of performing the plurality of operations for recurrent neural network cell activation by an accelerator, generating intermediate computation data, and storing the intermediate computation data in the accelerator.

Claims

1. A computer program for facilitating processing within a computing environment, which for a computer: Procedure for executing the command to perform recurrent neural network cell activation The procedure for executing the above is: A procedure for performing multiple operations of the recurrent neural network cell activation and providing the results of the recurrent neural network cell activation. The plurality of operations are performed in a single invocation of the instruction, One or more inputs to the instruction include one or more connected tensors, which are made up of multiple tensors linked together. The first and second tensors connected in the aforementioned connected tensor are input to different operations. Computer program.

2. The computer program according to claim 1, wherein the plurality of operations include one or more sigmoid functions and one or more tangent functions.

3. The computer program according to claim 1, wherein the plurality of operations include tensor element-level addition and tensor element-level multiplication operations.

4. The computer program according to claim 1, wherein the plurality of operations include one or more sigmoid functions, one or more tangent functions, one or more tensor element unit addition operations, and one or more tensor element unit multiplication operations.

5. The computer program according to claim 1, wherein the result is an output tensor, the output tensor is an input to another invocation of the instruction.

6. The computer program according to claim 1, wherein the recurrent neural network cell activation includes long-term memory cell activation.

7. The computer program according to claim 1, wherein the recurrent neural network cell activation includes gated recurrent unit cell activation.

8. The computer program according to claim 1, wherein the procedure for performing the plurality of operations for recurrent neural network cell activation is performed by an accelerator, generates intermediate calculation data, and causes the computer to further perform a procedure for storing the intermediate calculation data in the accelerator.

9. The computer program according to claim 1, wherein the procedure for performing the plurality of operations includes a procedure for performing the plurality of operations on spatially close input data.

10. A computer system for facilitating processing within a computing environment, wherein the computer system is: memory; and At least one processor that communicates with the memory The computer system is configured to perform a method, and the method is: The stage in which instructions are executed to perform recurrent neural network cell activation. The steps to be performed are: A step of performing multiple operations for recurrent neural network cell activation and providing the results of the recurrent neural network cell activation. The plurality of operations are performed in a single invocation of the instruction, One or more inputs to the instruction include one or more connected tensors, which are made up of multiple tensors linked together. The first and second tensors connected in the aforementioned connected tensor are input to different operations. Computer system.

11. The computer system according to claim 10, wherein the plurality of operations include one or more sigmoid functions, one or more tangent functions, one or more tensor element unit addition operations, and one or more tensor element unit multiplication operations.

12. The computer system according to claim 10, wherein the recurrent neural network cell activation includes long-term memory cell activation or gated recurrent unit cell activation.

13. The computer system according to claim 10, wherein the step of performing the plurality of operations for recurrent neural network cell activation is performed by an accelerator to generate intermediate computation data, and the method further comprises the step of storing the intermediate computation data in the accelerator.

14. A computer implementation method for facilitating processing within a computing environment, wherein the computer implementation method is: The stage in which instructions are executed to perform recurrent neural network cell activation. The steps included are: A step of performing multiple operations for recurrent neural network cell activation and providing the results of the recurrent neural network cell activation. The plurality of operations are performed in a single invocation of the instruction, One or more inputs to the instruction include one or more connected tensors, which are made up of multiple tensors linked together. The first and second tensors connected in the aforementioned connected tensor are input to different operations. Computer implementation method.

15. The computer implementation method according to claim 14, wherein the plurality of operations include one or more sigmoid functions, one or more tangent functions, one or more tensor element unit addition operations, and one or more tensor element unit multiplication operations.

16. The computer implementation method according to claim 14, wherein the recurrent neural network cell activation includes long-term memory cell activation or gated recurrent unit cell activation.

17. The computer implementation method according to claim 14, further comprising the steps of performing the plurality of operations for recurrent neural network cell activation by an accelerator, generating intermediate computation data, and storing the intermediate computation data in the accelerator.

Images