Program event recording storage change processing for neural network accelerator instructions

Abstract

Instruction processing of the instruction is performed. The instruction is configured to perform a plurality of functions, where one of the plurality of functions is to be performed in a plurality of processing stages. The processing stages are defined to store up to a selected amount of data. The selected amount of data is based on the function being performed. At least one of the plurality of functions has a value of the selected amount of data that is different from at least one other function. Based on processing the selected processing stage of the selected function, a determination is made as to whether a store to the designated area has occurred. Based on a determination that a store to the designated area has occurred, an interrupt is presented and based on a determination that a store to the designated area has not occurred, instruction processing continues.

Description

[Technical Field]

[0001] One or more aspects relate generally to facilitating processing within a computing environment, and more particularly to improving such processing. [Background technology]

[0002] Coprocessors, such as artificial intelligence accelerators (also called neural network processors or neural network accelerators), are used to improve processing in data-intensive and / or computationally intensive computing environments. Such accelerators provide large amounts of computing power that are used in performing complex calculations, such as calculations on matrices or tensors.

[0003] As an example, tensor calculus is used in complex processing, including deep learning, which is a subset of machine learning. Deep learning or machine learning, an aspect of artificial intelligence, is used in a variety of technologies, including, but not limited to, design, manufacturing, medical technology, automotive technology, computer processing, etc.

[0004] When certain events occur during computational processes, it is useful to obtain information related to those events and / or processes. Therefore, progress should be made in obtaining such information. Summary of the Invention

[0005] Shortcomings of the prior art are overcome and additional advantages are provided by the provision of a computer program product for facilitating processing within a computing environment. The computer program product includes one or more computer-readable storage media and program instructions collectively stored on the one or more computer-readable storage media for performing a method. The method includes executing instruction processing of the instructions. The instructions are configured to perform a plurality of functions, one of the plurality of functions being performed in a plurality of processing stages. One of the plurality of processing stages is defined to store up to a selected amount of data. The selected amount of data is based on one of the plurality of functions being performed. At least one of the plurality of functions has a value of the selected amount of data that is different from at least one other function of the plurality of functions. A determination is made based on processing a selected one of the plurality of processing stages of the selected function as to whether a store to a designated area has occurred. Based on a determination that a store to a designated area has occurred, an interrupt is presented, and based on a determination that a store to a designated area has not occurred, execution of the instruction processing continues.

[0006] The use of a single designed machine instruction configured to perform a variety of functions improves performance within a computing environment by reducing complexity, reducing resource usage, or increasing processing speed, or a combination thereof.

[0007] By checking whether a store to the specified area occurred after a processing step, before terminating the instruction, a store to the specified area only occurs within a single step. Because a selected amount of data is stored during each step (e.g., when processing multiple contiguous 4K pages), and the selected amount may be larger than 4K, a reduced number of checks may be required to test whether a store to the specified area occurred.

[0008] In one example, continuing to execute the instruction processing includes proceeding to process the next processing stage of the selected function. In another example, continuing to execute the instruction processing includes terminating the instruction.

[0009] By way of example, the specified region may include a region of memory defined by a starting address and an ending address, the starting address and ending address being provided, for example, by one or more control registers.

[0010] In one embodiment, determining whether a store to the designated region has occurred includes checking whether the memory address to which the data is being written is within the designated region, and based on the memory address being within the designated region, this determination indicates that a store to the designated region has occurred.

[0011] In one embodiment, determining whether a storage in the designated area has occurred includes checking whether at least a portion of a parameter block used by the selected function overlaps with the designated area, and based on at least a portion of the parameter block overlapping with the designated area, this determination indicates that a storage in the designated area exists.

[0012] Checking for parameter block duplication may cause an interrupt after each step and provides easy information about memory accesses.

[0013] In one embodiment, determining whether storage to the designated area has occurred includes checking whether at least a portion of a function-specific storage area used by the selected function overlaps with the designated area, and based on at least a portion of the function-specific storage area overlapping with the designated area, this determination indicates that storage to the designated area exists.

[0014] Function-specific save area overlap checks, if used, may cause an interrupt after each step and provide easy information about memory accesses.

[0015] In one or more embodiments, a particular prediction may be made. For example, a particular processing stage of a selected function may be predicted to cause an interrupt, the prediction being based on a memory access pattern of the selected function. As a further example, a prediction that an interrupt will occur may be made based on a determination that at least a portion of a parameter block used by the selected function overlaps with a designated region. In another example, a prediction that an interrupt will occur may be made based on a determination that at least a portion of a function-specific storage area used by the selected function overlaps with a designated region.

[0016] At the very least, the complexity of a verification program that can predict a storage alteration interrupt is reduced, improving verification quality. For example, based on the memory access pattern corresponding to a function code, the verification program can accurately predict after which stage an instruction will interrupt and present a program event record interrupt. Because an interrupt is presented after each stage, the verification programming can predict the interrupt if the specified area for the program event record is located within a parameter block and / or a function-specific save area.

[0017] Computer-implemented methods and systems relating to one or more aspects are also described and claimed herein. Additionally, services relating to one or more aspects may also be described and claimed herein.

[0018] Additional features and advantages are realized by the techniques described herein. Other embodiments and aspects are described in detail herein and are considered a part of the claimed aspects.

[0019] One or more aspects are particularly pointed out and distinctly claimed as examples in the claims at the end of this specification. The foregoing, as well as objects, features, and advantages of one or more aspects, will become apparent from the following detailed description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0020] [Figure 1A] FIG. 1 illustrates an example computing environment for incorporating and using one or more aspects of the present invention. [Figure 1B] 1B illustrates further details of the processor of FIG. 1A in accordance with one or more aspects of the present invention. [Figure 2A] FIG. 1 illustrates an example of processing related to program event recording associated with Neural Network Processing Assist instructions in accordance with one or more aspects of the present invention. [Figure 2B] FIG. 10 illustrates an example of a storage change event that occurs during the processing stage of a neural network processing assistance instruction function in accordance with one or more aspects of the present invention. [Figure 3A] FIG. 10 illustrates one example of a format for neural network processing assistance instructions in accordance with one or more aspects of the present invention. [Figure 3B] FIG. 2 illustrates an example of a general-purpose register used by neural network processing assistance instructions in accordance with one or more aspects of the present invention. [Figure 3C] FIG. 10 illustrates example function codes supported by neural network processing assistance instructions in accordance with one or more aspects of the present invention. [Figure 3D] FIG. 10 illustrates one example of another general-purpose register used by neural network processing assistance instructions in accordance with one or more aspects of the present invention. [Figure 3E]FIG. 10 illustrates an example of a parameter block used by a query function of a neural network processing assistance instruction in accordance with one or more aspects of the present invention. [Figure 3F] FIG. 10 illustrates one example of a parameter block used by one or more non-query functions of a neural network processing assistance instruction in accordance with one or more aspects of the present invention. [Figure 3G] FIG. 10 illustrates one example of a tensor descriptor used by neural network processing assistance instructions in accordance with one or more aspects of the present invention. [Figure 4] FIG. 1 illustrates an example of a Neural Network Processing (NNP) Data Type 1 data type format in accordance with one or more aspects of the present invention. [Figure 5] 1A, 1B, and 1C are diagrams illustrating example input data layouts used by neural network processing assistance instructions in accordance with one or more aspects of the present invention. [Figure 6] 5(A), (B), and (C) are diagrams illustrating exemplary outputs corresponding to the input data layouts of FIGS. 5(A)-5(C), in accordance with one or more embodiments of the present invention. [Figure 7A] FIG. 1 illustrates an example that facilitates processing within a computing environment in accordance with one or more aspects of the present invention. [Figure 7B] FIG. 1 illustrates an example that facilitates processing within a computing environment in accordance with one or more aspects of the present invention. [Figure 8A] FIG. 1 illustrates another example of a computing environment for incorporating and using one or more aspects of the present invention. [Figure 8B] 8B illustrates an example of further details of the memory of FIG. 8A in accordance with one or more aspects of the present invention. [Figure 8C] 8B illustrates another example of further details of the memory of FIG. 8A in accordance with one or more aspects of the present invention. [Figure 9A]FIG. 1 illustrates yet another example of a computing environment for incorporating and using one or more aspects of the present invention. [Figure 9B] FIG. 9B illustrates further details of the memory of FIG. 9A in accordance with one or more aspects of the present invention. [Figure 10] FIG. 1 illustrates one embodiment of a cloud computing environment in accordance with one or more aspects of the present invention. [Figure 11] FIG. 2 illustrates an example of an abstract model layer in accordance with one or more aspects of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0021] In accordance with one or more aspects of the present invention, capabilities are provided for facilitating processing within a computing environment. By way of example, capabilities are provided for facilitating obtaining information related to events that may occur during stages of instruction processing. This information is useful in determining whether a program executing within the computing environment is performing as expected, or determining whether optimizations can be performed to reduce errors and / or complexity, improve system performance, or both. Furthermore, in one example, this information allows a verification program to predict that an event (e.g., a storage modification causing an interrupt) will occur, thereby reducing the complexity of the verification program and improving verification quality, reducing errors, improving system performance, or a combination thereof. This capability includes program event recording capabilities, e.g., used to debug programs, enhanced for certain types of processing, including processing associated with instructions executed by specialized processors such as neural network processors (also known as neural network accelerators).

[0022] One example of an instruction covered by enhanced program event recording is a neural network processing support instruction, which is a single instruction (e.g., a single designed hardware machine instruction at a hardware / software interface) configured to perform multiple functions, each of which is configured as part of a single instruction (e.g., a single designed instruction), reducing system resource usage and complexity and improving system performance.

[0023] The instructions may be part of an instruction set architecture (ISA) of a general-purpose processor, dispatched by a program on a processor such as a general-purpose processor. The instructions may be executed by the general-purpose processor, or one or more functions of the instructions may be executed by a special-purpose processor, such as a co-processor configured for a specific function, coupled to or part of the general-purpose processor, or both. Other variations are possible.

[0024] One embodiment of a computing environment for incorporating and using one or more aspects of the present invention is described with reference to FIG. 1A. As an example, the computing environment is based on the z / Architecture® instruction set architecture offered by International Business Machines Corporation (Armonk, New York). One embodiment of the z / Architecture instruction set architecture is described in the publication “z / Architecture Principles of Operation,” IBM Publication No. SA22-7832-12, Thirteenth Edition, September 2019, which is incorporated herein by reference in its entirety. However, the z / Architecture instruction set architecture is only one exemplary architecture, and other architectures or types of computing environments of International Business Machines Corporation or other entities or both may include and / or use one or more aspects of the present invention. z / Architecture and IBM are trademarks or registered trademarks of International Business Machines Corporation in at least one jurisdiction.

[0025] 1A, computing environment 100 includes computer system 102, shown, for example, in the form of a general-purpose computing device. Computer system 102 may include, but is not limited to, one or more general-purpose processors or processing units 104 (e.g., central processing units (CPUs)), at least one special-purpose processor, such as neural network processor 105, coupled together via one or more buses and / or other connections, memory 106 (e.g., also known as system memory, main memory, primary storage, central storage, or storage), and one or more input / output (I / O) interfaces 108. For example, processors 104, 105 and memory 106 are coupled to I / O interface 108 via one or more buses 110, and processors 104, 105 are coupled together via one or more buses 111.

[0026] Bus 111 may be, for example, a memory or cache coherence bus, and bus 110 may represent any one or more of several types of bus structures, including, for example, a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures, including, by way of example only, Industry Standard Architecture (ISA), Micro Channel Architecture (MCA), Enhanced ISA (EISA), Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnects (PCI).

[0027] As an example, one or more special-purpose processors (e.g., neural network processors) may be separate from but coupled to one or more general-purpose processors, or may be incorporated within one or more general-purpose processors, or both. Many variations are possible.

[0028] For example, memory 106 may include a cache 112, such as a shared cache, that may be coupled to a local cache 114 of processor 104 and / or neural network processor 105, e.g., via one or more buses 111. Additionally, memory 106 may include one or more programs or applications 116 and at least one operating system 118. An exemplary operating system includes the z / OS® operating system offered by International Business Machines Corporation (Armonk, New York). z / OS is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction. Other operating systems offered by International Business Machines Corporation and / or other entities may also be used. Memory 106 may include one or more computer-readable program instructions 120 that may be configured to perform functions of embodiments of aspects of the present invention.

[0029] Additionally, in one or more embodiments, memory 106 includes processor firmware 122. Processor firmware includes, for example, processor microcode or millicode. Processor firmware includes, for example, hardware-level instructions and / or data structures used in implementing higher-level machine code. In one embodiment, processor firmware includes, for example, microcode or millicode containing trusted software, proprietary code that is typically provided as microcode or millicode specific to the underlying hardware and that controls operating system access to the system's hardware.

[0030] The computer system 102 may communicate with one or more external devices 130, such as a user terminal, a tape drive, a pointing device, a display, and one or more data storage devices 134, for example, via the I / O interface 108. The data storage devices 134 may store one or more programs 136, one or more computer-readable program instructions 138, or data, or a combination thereof. The computer-readable program instructions may be configured to perform the functions of embodiments of aspects of the present invention.

[0031] The computer system 102 may communicate, for example, via the I / O interface 108, with a network interface 132 that enables the computer system 102 to communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), or a public network (e.g., the Internet), or a combination thereof, to facilitate communication with other computing devices or systems.

[0032] Computer system 102 may include and / or be coupled to removable / non-removable, volatile / non-volatile computer system storage media. For example, computer system 102 may include and / or be coupled to non-removable, non-volatile magnetic media (commonly referred to as a "hard drive"), a magnetic disk drive for reading from and writing to removable, non-volatile magnetic disks (e.g., "floppy disks"), and / or an optical disk drive for reading from and writing to removable, non-volatile optical disks, such as CD-ROMs, DVD-ROMs, or other optical media. It should be understood that other hardware and / or software components may be used with computer system 102, including, but not limited to, microcode or millicode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0033] Computer system 102 may operate in numerous other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, or configurations, or combinations thereof, that may be suitable for use with computer system 102 include, but are not limited to, personal computer (PC) systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, microcomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of these systems or devices.

[0034] In one example, a processor (e.g., processor 104 and / or processor 105) includes multiple functional components (or a subset thereof) used to execute instructions. As shown in FIG. 1B , these functional components include, for example, an instruction fetch component 150 for fetching instructions to be executed, an instruction decode unit 152 for decoding the fetched instructions and obtaining operands for the decoded instructions, one or more instruction execution components 154 for executing the decoded instructions, a memory access component 156 for accessing memory to execute the instructions, if necessary, and a writeback component 158 ​​for providing results of the executed instructions. One or more of the components may access and / or use one or more registers 160 in instruction processing. Additionally, one or more of the components may include at least a portion of or have access to one or more other components used in performing, for example, program event recording and / or neural network processing assistance processing of neural network processing assistance instructions (or other processing that may employ one or more aspects of the present invention) as described herein in accordance with one or more aspects of the present invention. The one or more other components may include, for example, program event recording component 170 and / or neural network processing assistance component 172 (or one or more other components or combinations thereof).

[0035] In accordance with one or more aspects of the present invention, program event recording is improved such that an interrupt may be generated due to a detected event after a selected amount of data has been written to a selected location (e.g., memory) as defined by the specific function of the instruction being executed, prior to completion of the instruction. By way of example, detection of one or more selected events (e.g., storage modification of a designated area) may occur after processing one of multiple processing stages of the function being executed. During the processing stage, up to a selected amount of data may be stored in memory, the selected amount of data being stored being based on the function of the instruction being executed. Different functions may have the same or different selected amounts of data that may be stored during a particular stage. If the storage of the selected amount of data includes storing it in the designated area, the selected event (e.g., storage modification of the designated area) is detected and an interrupt is generated.

[0036] Further details regarding program event recording processing for selected instructions, such as neural network processing-assisted instructions, are described with reference to Figures 2A-2B. Referring first to Figure 2A, an instruction, such as a neural network processing-assisted instruction, is initiated by a general-purpose processor, such as general-purpose processor 104 (200). The general-purpose processor performs instruction processing of the instruction, including, for example, determining one function (e.g., a non-query function) of multiple functions that the instruction is configured to perform.

[0037] According to aspects of the invention, functions are performed in processing stages, and up to a selected amount of data may be stored in a selected location (e.g., memory) during each processing stage. The selected amount of data processed within a stage depends on the function. Exemplary functions and, based on, for example, function codes of neural network processing assistance instructions, exemplary maximum amounts of data that may be stored in a target memory during a processing stage are provided below (other embodiments may provide additional, fewer, or other functions, or combinations thereof, function codes, or maximum amounts, etc.).

[0038] NNPA-ADD (Function Code 16): Target 65536 bytes per stage (16 contiguous 4K pages)

[0039] NNPA-SUB (Function Code 17): 65536 bytes of target per stage (16 contiguous 4K pages)

[0040] NNPA-MUL (Function Code 18): 65536 bytes of target per stage (16 contiguous 4K pages)

[0041] NNPA-DIV (Function Code 19): 65536 bytes of target per stage (16 contiguous 4K pages)

[0042] NNPA-MIN (Function Code 20): 65536 bytes (16 contiguous 4K pages) of target per stage

[0043] NNPA-MAX (Function Code 21): 65536 bytes (16 contiguous 4K pages) of target per stage

[0044] NNPA-LOG (Function Code 32): 65536 bytes (16 contiguous 4K pages) of target per stage

[0045] NNPA-EXP (Function Code 33): 65536 bytes of target per stage (16 contiguous 4K pages)

[0046] NNPA-REUL (Function Code 49): 65536 bytes of target per stage (16 contiguous 4K pages)

[0047] NNPA-TANH (Function Code 50): 65536 bytes of target per stage (16 contiguous 4K pages)

[0048] NNPA-SIGMOID (Function Code 51): Target 65536 bytes per stage (16 contiguous 4K pages)

[0049] NNPA-SOFTMAX (function code 52): 65536 bytes of target per stage (16 non-contiguous 4K pages), plus two contiguous 4K pages called "function-specific save areas"

[0050] NNPA-BATCHNORM (Function Code 64): Target 65536 bytes per stage (16 non-contiguous 4K pages)

[0051] NNPA-MAXPOOL2D (Function Code 80): Target 131072 bytes per stage (32 non-contiguous 4K pages)

[0052] NNPA-AVGPOOL2D (Function Code 81): Target 131072 bytes per stage (32 non-contiguous 4K pages)

[0053] NNPA-LSTM (function code 96): Target 16384 bytes per stage (4 non-contiguous 4K pages)

[0054] NNPA-GRU (Function Code 97): Target 8192 bytes per stage (two non-contiguous 4K pages)

[0055] NNPA-CONVOLUTION (Function Code 112): Target 131072 bytes per step (32 non-contiguous 4K pages)

[0056] NNPA-MATMUL-OP (Function Code 113): 131072 bytes of target per step (32 non-contiguous 4K pages)

[0057] NNPA-MATMUL-OP-BCAST23 (Function code 114): Target 131072 bytes per stage (32 non-contiguous 4K pages)

[0058] As shown, a function of an instruction (e.g., a neural network processing assistance instruction) is executed during a processing stage. Accordingly, based on initiating the instruction, processing of a processing stage of a function begins (202). In one embodiment, the function selected for execution (referred to herein as a selected function) is executed by a dedicated processor (e.g., neural network processor 105). Thus, as part of an initial processing stage, the general-purpose processor provides the dedicated processor with an indication of the selected function to be executed. Further, in one example, processing of a processing stage includes the general-purpose processor (e.g., firmware 122 of general-purpose processor 104) sending a memory address to the dedicated processor for use in executing the selected function during the processing stage. The firmware, in one example, also provides an indication of the end of the processing stage to the dedicated processor. In one example, as part of sending the memory address to the dedicated processor, the general-purpose processor (e.g., firmware 122) performs one or more memory-related checks, including checks related to program event records. For example, before providing the address during a particular processing stage, the firmware determines whether the memory address being presented during the processing stage falls within a designated area of ​​the program event record. If one or more addresses fall within the specified range, the firmware indicates to the dedicated processor that processing should stop after this stage of processing.

[0059] The dedicated processor (e.g., neural network processor 105) receives the address as well as information about the end of a stage and, for example, whether a program event recording event exists, performs the selected function on the input data (e.g., input tensor data), and provides output data (e.g., output tensor data). When a processing stage is complete, the dedicated processor passes the counter that was incremented at the end of the current stage to the firmware and stops processing until it receives an additional memory address for the next stage.

[0060] The general-purpose processor (e.g., firmware 122) receives an indication that processing of the last stage is complete and makes a determination as to whether all data has been processed (204). If all data has been processed, the instruction is completed (206). However, if not all data has been processed, a check is made (e.g., by firmware) as to whether there was a store to the designated area during the last stage's write to memory (208). As an example, in the case of program event recording, the designated area is defined via control registers, such as control register 10, which provides the start address of the designated area, and control register 11, which provides the end address of the designated area. In other examples, other control registers may be used to provide the start and end addresses, or other mechanisms may be used, or both. If data is written to one or more memory addresses that overlap the designated area, it is determined that there is a store to the designated area, and, assuming program event recording storage modification is enabled, that a program event recording storage modification event has been detected. In one example, program event recording storage changes are enabled when a storage change event control of a selected control register (e.g., a selected bit (e.g., bit 34) of a selected control register (e.g., control register 9)) is set (e.g., to 1), a storage change space control (e.g., a selected bit (e.g., bit 42) of a selected control register (e.g., control register 9)) is set (e.g., to 0), or a selected bit (e.g., bit 56) of an address space control element is set (e.g., to 1).

[0061] Additionally, in one example, if at least a portion of a parameter block used by a function, or at least a portion of a function-specific storage area, if used by the function, overlaps with a designated area, in one embodiment, a store to the designated area (e.g., a storage change event) is determined to have occurred, regardless of whether such a store occurred. In another example, a store to the designated area is determined to have occurred only if there is a store to the portion that overlaps with the designated area. In a further example, if the entire function's storage area, or the entire parameter block, if used by the function, overlaps with a designated area, a store to the designated area is determined to have occurred, regardless of whether the parameter block or the function-specific storage area was updated. Other variations are possible.

[0062] If it is determined that there was no store to the designated area, processing continues to the next processing stage 202, where the general-purpose processor provides an additional memory address to the dedicated processor for further processing of the function in the next stage. However, if it is determined that a store to the designated area occurred during the last stage, an interrupt is presented (210) and the instruction is terminated (206), e.g., by firmware. This interrupt is presented without waiting until all data is processed. Furthermore, in accordance with aspects of the present invention, amounts of data greater than 4K may be stored during a processing stage prior to the interrupt. For example, the amount of resulting output data produced during a processing stage depends on the function code, and different function codes may have different amounts of data. Furthermore, data may be stored in multiple 4K pages, which may or may not be contiguous in the logical address space, during a processing stage; parameter blocks may be updated during execution of a function; the parameter blocks may be updated multiple times; function-specific storage areas may be updated based on the function being executed; and function-specific storage areas may be updated multiple times. Other possibilities exist.

[0063] In one embodiment, when an instruction completes, a determination may be made as to whether the last stage of processing caused a store to a designated area. This information may be used, for example, for debugging purposes.

[0064] One example depiction of a program event record storage change event is shown in Figure 2B. As shown in the figure, in one example, phase 0 of a selected function of a neural network processing assistance instruction is processed without a store to a designated area, but during the processing of phase 1, a write to the designated area is performed (250). Thus, at the end of the phase, a program event record storage change event occurs (252), causing an interruption of the instruction (e.g., the instruction ends with a condition code of, say, 3). Phase 2 is not executed until the instruction is resumed.

[0065] As described herein, in one example, the instructions being processed are neural network processing assistance instructions. In one example, the instructions are initiated on a general-purpose processor (e.g., processor 104), and the function specified by the instructions is executed on either the general-purpose processor or the specialized processor (e.g., neural network processor 105), or both, depending on the function. For example, in one example, query functions of the neural network processing assistance instructions are executed on the general-purpose processor, and non-query functions are executed on the specialized processor. However, other variations are possible. When a function is executed on a specialized processor (e.g., the function is a non-query function, or in another example, one or more selected functions), information for use in executing the function, such as memory address information associated with tensor data used in neural network calculations, is provided to the specialized processor by the general-purpose processor. The specialized processor obtains the information and executes the function. After execution of the function is completed, processing returns to the general-purpose processor, and the general-purpose processor completes the instruction. In another example, the instructions are initiated, executed, and completed on one or more general-purpose processors or one or more specialized processors. Other variations are possible.

[0066] In one example, referring to FIG. 3A , a neural network processing assistance instruction 300 has an RRE format that indicates a register and register operation along with an extended operation code (opcode). As shown in FIG. 3A , in one example, the neural network processing assistance instruction 300 includes an operation code (opcode) field 302 (e.g., bits 0-15) that indicates a neural network processing assistance operation. In one example, bits 16-31 of the instruction are reserved and should contain 0. In the descriptions herein of the instructions and / or the instruction's function, specific locations of fields, specific fields, or specific sizes of fields, or combinations thereof (e.g., specific bytes and / or bits), are indicated. However, other locations, fields, or sizes, or combinations thereof, may be provided. Furthermore, while setting a bit to a particular value (e.g., 1 or 0) may be specified, this is merely an example. When set, the bit may be set to a different value, such as the opposite value or another value, in other examples. Many variations are possible.

[0067] In one example, an instruction uses multiple general purpose registers implicitly specified by the instruction, such as neural network processing assistance instruction 300, which uses implicit registers general purpose register 0 and general purpose register 1, examples of which are described with reference to Figures 3B and 3D, respectively.

[0068] Referring to FIG. 3B, in one example, general register 0 includes a function code field and a status field that may be updated upon completion of the instruction. By way of example, general register 0 includes a response code field 310 (e.g., bits 0-15), an exception flags field 312 (e.g., bits 24-31), and a function code field 314 (e.g., bits 56-63). Additionally, in one example, bits 16-23 and 32-55 of general register 0 are reserved and should contain zeros. One or more fields are used by a particular function performed by the instruction. In one example, not all fields are used by all functions. Each of the fields is described below.

[0069] Response Code (FC) 310: This field (e.g., bit positions 0-15) contains a response code. The response code is stored when execution of a neural network processing support instruction is completed with a condition code of, for example, 1. If an invalid input condition occurs, a non-zero value indicating the cause of the invalid input condition recognized during execution is stored in the response code field, and a selected condition code (e.g., 1) is set. The code stored in the response code field is defined as follows in one example:

[0070] Response Code meaning

[0071] 0001 The parameter block format specified by the parameter block version number is not supported by the model.

[0072] 0002 The specified feature is not defined or is not installed on the machine.

[0073] 0010 The specified tensor data layout format is not supported.

[0074] 0011 The specified tensor data type is not supported.

[0075] 0012 The dimensions of the specified single tensor are greater than the maximum dimension index size.

[0076] 0013 The specified tensor size is larger than the maximum tensor size.

[0077] 0014 The specified tensor address is not aligned to a 4K byte boundary.

[0078] 0015 The address of the function-specific save area is not aligned to a 4K byte boundary.

[0079] F000-FFFF Function-specific response codes. These response codes are defined for specific functions.

[0080] Exception Flags (EF) 312: This field (e.g., bit positions 24-31) contains exception flags. If an exception condition is detected during the execution of an instruction, the corresponding exception flag control (e.g., bit) is set, e.g., to 1; otherwise, this control remains unchanged. Before the first invocation of an instruction, the exception flags field is initialized to 0. During the execution of the instruction, the reserved flags do not change. In one example, the flags stored in the exception flags field are defined as follows:

[0081] EF (bit) meaning

[0082] 0 Range violation. This flag is set if a non-numeric value is found in an input tensor or stored in an output tensor. This flag is only valid if the instruction completes with, for example, a condition code of 0.

[0083] 1-7 spare.

[0084] Function Code (FC) 314: This field (e.g., bit positions 56-63) contains the function code. Examples of function codes assigned to neural network processing support instructions are shown in Figure 3C. All other function codes are unassigned. If an unassigned or uninstalled function code is specified, a response code of, for example, hexadecimal 0002 and a selected condition code (e.g., 1) are set. This field is not changed during execution.

[0085] In addition to general register 0, neural network processing support instructions also use general register 1, an example of which is shown in Figure 3D. For example, bits 40 through 63 in 24-bit addressing mode, bits 33 through 63 in 31-bit addressing mode, or bits 0 through 63 in 64-bit addressing mode contain the address 320 of the parameter block. The contents of general register 1 specify the logical address of the leftmost byte of the parameter block in storage, for example. The parameter block must be specified on a doubleword boundary; otherwise, a specification exception is recognized. For all functions, the contents of general register 1 are unchanged.

[0086] In access register mode, for example, access register 1 specifies the address space containing the parameter block, input tensors, output tensors, and function-specific storage areas.

[0087] In one example, the parameter blocks may have different formats depending on the function specified by the instruction being executed. For example, a query function may include a parameter block of one format, while other functions of the instruction may include a parameter block of another format. In another example, all functions use the same parameter block format. Other variations are possible.

[0088] One example of a parameter block used by a query function such as an NNPA-Query Available Functions (QAF) operation is described with reference to Figure 3E. As shown in the figure, in one example, the NNPA-Query Available Functions parameter block 330 includes, for example:

[0089] Installed Feature Vector 332: This field (e.g., bytes 0-31) of the parameter block contains the installed feature vector. In one example, bits 0-255 of the installed feature vector correspond to function codes 0-255 of the neural network processing assistance instructions, respectively. If a bit is, for example, 1, the corresponding feature is installed; otherwise, the feature is not installed.

[0090] Installed Parameter Block Type Vector 334: This field (e.g., bytes 32-47) of the parameter block contains the installed parameter block type vector. In one example, bits 0-127 of the installed parameter block type vector correspond to parameter block types 0-127 of the non-query function of the neural network processing assistance command. If a bit is, for example, 1, the corresponding parameter block type is installed; otherwise, the parameter block type is not installed.

[0091] Installed Data Types 336: This field of the parameter block (e.g., bytes 48-49) contains an installed data type vector. In one example, bits 0-15 of the installed data type vector correspond to installed data types. If a bit is, for example, 1, the corresponding data type is installed; otherwise, the data type is not installed. Examples of data types include the following (additional, fewer, or other data types, or combinations thereof, are possible):

[0092] bit Data Types

[0093] 0 NNP data type 1

[0094] 1~15 Reserve

[0095] Installed Data Layout Formats 338: This field (e.g., bytes 52-55) of the parameter block contains an installed data layout format vector. In one example, bits 0-31 of the installed data layout format vector correspond to installed data layout formats. If a bit is, for example, 1, the corresponding data layout format is installed; otherwise, the data layout format is not installed. Examples of data layout formats include the following (additional, fewer, or other data layout formats, or combinations thereof, are possible):

[0096] bit Data Layout Format

[0097] 0 4D feature tensor

[0098] 1. 4D Kernel Tensor

[0099] 2~31 Reserve

[0100] Maximum Dimension Index Size 340: This field of the parameter block (e.g., bytes 60-63) contains a 32-bit unsigned binary integer that specifies, for example, the maximum number of elements in the specified dimension index size for any specified tensor. In another example, the maximum dimension index size specifies the maximum number of bytes in the specified dimension index size for any specified tensor. Other examples are possible.

[0101] Maximum Tensor Size 342: This field of the parameter block (e.g., bytes 64 through 71) contains a 32-bit unsigned binary integer that specifies the maximum number of bytes in any specified tensor, including any pad bytes required by the tensor format, for example. In another example, the maximum tensor size specifies the maximum number of total elements in any specified tensor, including any padding required by the tensor format. Other examples are possible.

[0102] Installed NNP Data Type 1 Conversion Vector 344: This field of the parameter block (e.g., bytes 72-73) contains the installed NNP Data Type 1 Conversion Vector. In one example, bits 0-15 of the installed NNP Data Type 1 Conversion Vector correspond to installed data type conversions to and from the NNP Data Type 1 format. If a bit is 1, the corresponding conversion is installed; otherwise, the conversion is not installed. Additional conversions, fewer conversions, or other conversions, or a combination thereof, may be specified.

[0103] bit Data Types

[0104] 0 Reserve

[0105] 1 BFP small quantity format

[0106] 2 BFP short form

[0107] 3~15 Reserve

[0108] Although one example of a query function parameter block is described with reference to Figure 3E, other formats of the query function parameter block may be used, including query operations for NNPA-enabled functions. The format may depend, in one example, on the type of query function being performed. Furthermore, the parameter block and / or individual fields of the parameter block may contain additional information, less information, other information, or combinations thereof.

[0109] In addition to the parameter block for query functions, in one example, there is a parameter block format for non-query functions, such as the non-query function of a neural network processing assistance instruction. One example of a parameter block used by a non-query function, such as the non-query function of a neural network processing assistance instruction, is described with reference to Figure 3F.

[0110] As shown in the figure, in one example, a parameter block 350 used by, for example, a non-query function of a neural network processing assistance instruction may include, for example:

[0111] Parameter Block Version Number 352: This field of the parameter block (e.g., bytes 0-1) specifies the version and size of the parameter block. In one example, bits 0-8 of the parameter block version number are reserved and contain 0, and bits 9-15 of the parameter block version number contain an unsigned binary integer that specifies the parameter block format. The query function provides a mechanism to indicate available parameter block formats. If the specified parameter block size or format is not supported by the model, a response code, e.g., hex 0001, is stored in general register 0 and the instruction is completed by setting a condition code (e.g., condition code 1). The parameter block version number is specified by the program and does not change during instruction execution.

[0112] Model Version Number 354: This field (e.g., byte 2) of the parameter block is an unsigned binary integer that identifies the model that executed the command (e.g., a particular non-query function). If the Continue Flag (described below) is 1, the model version number may be input to the operation for the purpose of interpreting the contents of the Continue Status Buffer field (described below) of the parameter block and resuming the operation.

[0113] Continuation flag 356: This field (e.g., bit 63) of the parameter block, when set to 1, indicates that the operation is partially complete and the contents of the continuation state buffer may be used to resume the operation. Programs should initialize the continuation flag to 0 and not change it if instructions are re-executed with the intent of resuming the operation; otherwise, results will be unpredictable.

[0114] If the continue flag is set at the start of the operation and the contents of the parameter block have changed since the initial call, the results will be unpredictable.

[0115] Function-specific save area address 358: This field of the parameter block (e.g., bytes 56-63) contains the logical address of the function-specific save area. In one example, the function-specific save area address should be aligned on a 4K byte boundary; if not, a response code of, for example, hex 0015 is set in general register 0, and the instruction completes with a condition code of, for example, 1. This address is affected by the current addressing mode. The size of the function-specific save area depends on the function code.

[0116] If the entire function-specific save area overlaps with a program event recording (PER) storage area specification, a PER storage change event is recognized, if applicable, for the function-specific save area. If only a portion of the function-specific save area overlaps with a PER storage area specification, which of the following occurs is model-dependent:

[0117] * PER storage alteration events are recognized, where applicable, for the entire function-specific save area.

[0118] * PER storage change events are recognized, where applicable, for some of the saved areas specific to the stored function.

[0119] If an entire parameter block overlaps with a PER storage region specification, a PER storage change event is recognized, if applicable, for the parameter block. If only part of a parameter block overlaps with a PER storage region specification, which of the following occurs is model-dependent:

[0120] * PER storage change events are recognized, if applicable, for the entire parameter block.

[0121] * For parts of the stored parameter block, PER storage change events are recognized, if applicable.

[0122] PER zero address detection events are recognized when applicable for parameter blocks. Zero address detection does not apply to tensor addresses or function-specific storage addresses, for example.

[0123] Output tensor descriptor (e.g., 1-2) 360 / Input tensor descriptor (e.g., 1-3) 365: One example of tensor descriptors is described with reference to Figure 3G. In one example, tensor descriptors 360, 365 include:

[0124] Data Layout Format 382: This field (e.g., byte 0) of the tensor descriptor specifies the data layout format. Valid data layout formats include, for example, the following (additional, fewer, or other data layout formats, or combinations thereof, are possible):

[0125] format explanation Alignment (byte)

[0126] 0 4D feature tensor 4096

[0127] 1 4D kernel tensor 4096

[0128] 2~255 Reserve --

[0129] If an unsupported or reserved data layout format is specified, a response code, e.g., hex 0010, is stored in general register 0 and the instruction is completed by, e.g., setting condition code 1.

[0130] Data Type 384: This field (e.g., byte 1) specifies the data type of the tensor. Examples of supported data types are described below (additional, fewer, or other data types, or combinations thereof, are possible).

[0131] value Data Types Data size (bits)

[0132] 0 NNP data type 1 16

[0133] 1~255 Reserved --

[0134] If an unsupported or reserved data type is specified, a response code, for example hex 0011, is stored in general register 0 and the instruction is completed, for example, by setting a condition code of 1.

[0135] Dimension 1-4 Index Size 386: Collectively, dimension index sizes 1-4 specify the shape of a 4D tensor. Each dimension index size should be greater than 0 and less than or equal to the maximum dimension index size (340 in Figure 3E); if not, a response code, e.g., hex 0012, is stored in general register 0 and the instruction completes, e.g., by setting condition code 1. The total tensor size should be less than or equal to the maximum tensor size (342 in Figure 3E); if not, a response code, e.g., hex 0013, is stored in general register 0 and the instruction completes, e.g., by setting condition code 1.

[0136] In one example, to determine the number of bytes (i.e., total tensor size) in a 4D feature tensor containing elements of NNP data type 1, the following is used: dimension index 4 * dimension index 3 * ceil(dimension index 2 / 32) * 32 * ceil(dimension index 1 / 64) * 64 * 2.

[0137] Tensor Address 388: This field of the tensor descriptor (e.g., bytes 24-31) contains the logical address of the leftmost byte of the tensor. This address is affected by the current addressing mode.

[0138] If this address is not aligned to the boundary of the relevant data layout format, a response code, e.g. hex 0014, is stored in general register 0 and the instruction is completed, e.g. by setting condition code 1.

[0139] In access register mode, access register 1 specifies the address space containing all active input and output tensors in storage.

[0140] Returning to FIG. 3F, parameter block 350, in one example, further includes function-specific parameters 1-5 (370) that may be used by particular functions as described herein.

[0141] Additionally, parameter block 350, in one example, includes a continuation state buffer field 375, which contains data (or the location of data) to be used if operation of this instruction is resumed.

[0142] As input to an operation, the reserved field of the parameter block should contain 0. When the operation completes, the reserved field may be filled with 0 or may remain unchanged.

[0143] Although one example of a non-query function parameter block is described with reference to FIG. 3F, other formats of the non-query function parameter block may be used, including non-query functions of neural network processing assistance instructions. The format may depend, in one example, on the type of function being performed. Furthermore, although one example of a tensor descriptor is described with reference to FIG. 3G, other formats may be used. Furthermore, different formats for the input tensor and output tensor may be used. Other variations are possible.

[0144] Below, further details regarding the various functions supported by one embodiment of the neural network processing assistance instructions are provided.

[0145] Function code 0: NNPA-QAF (Query available functions)

[0146] The Neural Network Processing Assist (NNPA) query function provides a mechanism for indicating selected information, such as installed features, installed parameter block formats, installed data types, installed data layout formats, maximum dimension index size, and maximum tensor size availability. This information is retrieved and placed in a selected location, such as a parameter block (e.g., parameter block 330). When an operation terminates, the reserved fields of the parameter block may be filled with zeros or may remain unchanged.

[0147] In one embodiment of the query function, a processor, such as general-purpose processor 104, obtains information related to a particular model of a selected processor, such as a particular model of a neural network processor, such as neural network processor 105. A particular model of a processor or machine has particular capabilities. Another model of a processor or machine may have additional, less, or different capabilities, or a combination thereof, or may be a model of a different generation (e.g., a current or future generation) having additional, less, or different capabilities, or a combination thereof. The obtained information is placed in a parameter block (e.g., parameter block 330) or other structure that is accessible by and / or for use by one or more applications that may use this information for further processing. In one example, the parameter block and / or the parameter block information are maintained in memory. In other embodiments, the parameter block and / or the information may be maintained in one or more hardware registers. In another embodiment, the query function may be a privileged operation performed by an operating system, which makes available an application programming interface to make this information available to applications or non-privileged programs. In a further example, the query function is performed by a dedicated processor, such as neural network processor 105. Other variations are possible.

[0148] This information is obtained, for example, by firmware of the processor executing the query function. The firmware has knowledge of the attributes of the particular model of the particular processor (e.g., neural network processor). This information may be stored, for example, in a control block, register, or memory, or a combination thereof, and / or may be otherwise accessible by the processor executing the query function.

[0149] For example, the retrieved information may include model-dependent detailed information regarding at least one or more data attributes of a particular processor, including, for example, one or more installed or supported data types, one or more installed or supported data layout formats, or one or more installed or supported data sizes, or any combination thereof, of a selected model of a particular processor. Illustratively, condition code 0 is set upon completion of execution of an inquiry function (e.g., an NNPA-QAF function). In one example, condition codes 1, 2, and 3 are not applicable to an inquiry function. Details related to the retrieved information are described below.

[0150] As shown, in one example, the retrieved information includes model-dependent information regarding one or more data attributes, e.g., of a particular model of a neural network processor. One example of a data attribute is the installed data type of the neural network processor. For example, a particular model of a neural network processor (or other processor) may support one or more data types, such as, for example, the NNP Data Type 1 data type (also referred to as the Neural Network Processing Data Type 1 data type) and / or other data types. The NNP Data Type 1 data type is a 16-bit floating-point format that provides multiple benefits to deep learning training and inference calculations, including, for example, maintaining precision in deep learning networks, eliminating denormalized forms and simplifying rounding modes and corner case handling, automatic rounding to nearest for arithmetic operations, and combining the special entities for infinity and not-a-number (NaN) into a single value (NINF) that is accepted and processed by arithmetic operations. NINF provides better defaults for exponent overflow and invalid operations (e.g., division by zero). This allows many programs to continue executing without hiding such errors and without using special exception handlers. Other model-dependent data types are also possible.

[0151] One example of a format for the NNP Data Type 1 data type is shown in Figure 4. As shown, in one example, NNP Data Type 1 data may be represented in format 400, which may include, for example, a sign 402 (e.g., bit 0), an exponent +31 404 (e.g., bits 1-6), and a fraction 406 (e.g., bits 7-15).

[0152] Exemplary properties of the NNP Data Type 1 format are shown below.

[0153] characteristics NNP data type 1

[0154] Format length (bits) 16 bits

[0155] Biased exponent length (bits) 6 bits

[0156] Fraction length (bits) 9 bits

[0157] Precision (p) 10 bits

[0158] Maximum left unit exponent (Emax) 32

[0159] Minimum left unit display exponent (Emin) -31

[0160] Left-units-view (LUV) bias 31

[0161] Nmax (1-2 -9 )×2 33 ≒8.6×10 9

[0162] Nmin (1+2 -9 )×2 -31 ≒4.6×10 -10

[0163] Dmin -- where ≒ indicates that the value is approximate, Nmax is the largest representable finite number (in magnitude), and Nmin is the smallest representable number (in magnitude).

[0164] Further details relating to the NNP Data Type 1 data type are described below.

[0165] Biased Exponent: Shown above is the bias used to allow the exponent to be represented as an unsigned number. Biased exponents are similar in properties to those of binary floating-point formats, except that no special meaning is attached to all-zero and all-one biased exponents, as explained below with reference to the NNP Data Type 1 data type class.

[0166] Mantissa: The binary point of a number with NNP data type 1 is considered to be to the left of the leftmost fraction bit. To the left of the binary point there is an implied unit bit that is considered to be 1 for normalized numbers and 0 for 0. The fraction with the implied unit bit added to the left becomes the mantissa of the number.

[0167] A value of the regular NNP data type 1 is the product of the mantissa and a base 2 number raised to an unbiased exponent.

[0168] Non-zero numeric values: Non-zero numeric values ​​are shown below.

[0169] Number classes value

[0170] Normal number ±2 e-31 ×(1.f) where e is the biased exponent in decimal and f is the fraction in binary.

[0171] In one embodiment, there are three classes of NNP Data Type 1 data, including numeric entities and associated non-numeric entities. Each data item includes a sign, an exponent, and a mantissa. The exponent is biased so that all biased exponents are non-negative unsigned numbers, and the smallest biased exponent is 0. The mantissa includes an explicit fraction and an implicit unit bit to the left of the binary point. The sign bit is 0 for positive and 1 for negative.

[0172] Every finite number other than 0 that is allowed has a unique NNP data type 1 representation. Numeric values ​​may allow multiple representations of the same value; there are no denormalized numbers, and there are no arithmetic operations on denormalized numbers. The three classes include, for example:

[0173] Class of Data sign Biased Exponential Unit bit* Decimal Part

[0174] 0 ± 0 0 0

[0175] Normal number ± 0 1 not 0

[0176] Normal number ± 1 Not 0, not all 1s Any

[0177] normal number ± all ones - not all ones

[0178] NINF ± all 1 - all 1 where - indicates not applicable, * indicates the unit bit is implicit, and NINF is not a number or infinity.

[0179] Further details regarding each of the classes are provided below.

[0180] 0:0 has a biased exponent of 0 and a fraction of 0. The implicit unit bit is 0.

[0181] Normal Numbers: Normal numbers may contain a biased exponent of any value. If the biased exponent is 0, the fraction will be non-zero. If the biased exponent is all ones, the fraction will not be all ones. Other biased exponent values ​​may have any fraction value. The implicit unit bit for all normal numbers is 1.

[0182] NINF: NINF is represented by a biased exponent of all ones and a fraction of all ones. NINF represents values ​​that are not within the range of values ​​that can be represented by the NNP data type 1 (i.e., a 16-bit floating point designed for deep learning, with 6 bits for the exponent and 9 bits for the fraction). Typically, NINF is simply propagated through a computation so that it remains visible at the end.

[0183] In one example, the NNP data type 1 data type is supported, but other specialized or non-standard data types may be supported in addition to one or more standard data types, including, but not limited to, IEEE 754 short precision, binary floating point 16-bit, IEEE half precision floating point, 8-bit floating point, 4-bit integer format, or 8-bit integer format, to name a few. These data formats have different qualities with respect to neural network processing. For example, smaller data types (e.g., fewer bits) can be processed faster and can use less cache / memory, while larger data types result in higher accuracy of results in neural networks. Supported data types may have one or more allocated bits in the query parameter block (e.g., in the installed data types field 336 of the parameter block 330). For example, specialized or non-standard data types supported by a particular processor are indicated in the installed data types field, while standard data types are not. In other embodiments, one or more standard data types are also indicated. Other variations are possible.

[0184] In one particular example, bit 0 of installed data types field 336 is reserved for the NNP Data Type 1 data type, and when set to 1, for example, indicates that the processor supports NNP Data Type 1. By way of example, a bit vector of installed data types may be configured to represent up to 16 data types, with a bit in the bit vector assigned to each data type. However, in other embodiments, the bit vector may support more or fewer data types. Additionally, vectors may be configured in which one or more bits within are assigned to data types. Many examples are possible, and / or additional, fewer, or other data types, or combinations thereof, may be supported and / or indicated within the vector.

[0185] In one example, the query function obtains an indication of the data types installed in the model-dependent processor and places this indication in the parameter block, e.g., by setting one or more bits in the installed data type field 336 of the parameter block 330. Additionally, in one example, the query function obtains an indication of the installed data layout format (another data attribute) and places this information in the parameter block, e.g., by setting one or more bits in the installed data layout format field 338. Examples of data layout formats include, e.g., a 4D feature tensor layout and a 4D kernel tensor layout. In one example, a 4D feature tensor layout is used by the functions described herein, and in one example, a convolution function uses a 4D kernel tensor layout. These data layout formats arrange tensor data for storage in a manner that improves processing efficiency in the execution of the neural network processing assistance instruction's function. For example, to operate efficiently, the neural network processing assistance instruction uses input tensors provided in a particular data layout format. Although example layouts are provided, additional, fewer, or other layouts, or combinations thereof, may be provided for the functions described herein or other functions or both.

[0186] The use or availability of layouts for a particular processor model is realized by an installed data layout format vector (e.g., field 338 of parameter block 330). This vector is, for example, an installed data layout format bit vector that allows the CPU to communicate which layouts are supported to an application. For example, bit 0 is reserved for the 4D feature tensor layout and, when set to 1, indicates that the processor supports the 4D feature tensor layout. Bit 1 is reserved for the 4D kernel tensor layout and, when set to 1, indicates that the processor supports the 4D kernel tensor layout. In one example, the installed data layout format bit vector is configured to represent up to 16 data layouts, with a bit in the bit vector assigned to each data layout. However, in other embodiments, the bit vector may support more or fewer data layouts. Furthermore, a vector may be configured in which one or more bits are assigned to a data layout. Many examples are possible. Further details regarding 4D feature tensor layouts and 4D kernel tensor layouts are provided below. Again, other layouts may be used now or in the future to optimize performance.

[0187] In one example, neural network processing assistance instructions operate using 4D tensors (i.e., tensors with four dimensions). These 4D tensors are obtained from the general input tensors described herein, e.g., row-major, i.e., when enumerating tensor elements in increasing memory address order, the inner dimension, called E1, is first incremented by the E1 index size value, starting from 0 to E1 index size 1, then the index of the E2 dimension is incremented, and so on through the E1 dimension. The index of the outer dimension, called E4 dimension, is incremented last.

[0188] A tensor with a fewer number of dimensions (e.g., a 3D tensor or a 1D tensor) is represented as a 4D tensor with one or more dimensions of the 4D tensor beyond the dimensions of the original tensor set to 1.

[0189] The conversion of a row-major general 4D tensor with dimensions E4, E3, E2, E1 to a 4D feature tensor layout (also referred to herein as an NNPA data layout format 0 4D feature tensor) is described herein.

[0190] For example, the resulting tensor can be represented as a 4D tensor, e.g., a 64 element vector, or a 5D tensor with the following dimensions:

[0191] E4,

number

number

number

[0192] The elements [e4][e3][e2][e1] of the general tensor may be mapped to the next elements of the resulting 5D tensor.

[0193]

number

number

number

number

number

[0194] The resulting tensor may be larger than the general tensor. Elements of the resulting tensor that have no corresponding elements in the general tensor are called pad elements.

[0195] Consider an element [fe4][fe1][fe3][fe2][fe0] of a 4D feature tensor in NNPA data layout format 0, which is a 64-element vector, or an equivalent representation as a 5D tensor. This element is a pad element, or the corresponding element in a general 4D tensor with dimensions E4, E3, E2, E1 can be determined using the following formula:

[0196] If fe2 ≥ E2, this element is a pad element of E2 (or page)

[0197] Otherwise, if fe1*64+fe0≧E1, then this element is a pad element for E1 (or row).

[0198] Otherwise, the corresponding elements in a general 4D tensor are:

[0199] [fe4][fe3][fe2][fe1*64+fe0]

[0200] For artificial intelligence models based on convolutional neural networks, the meaning of the four dimensions of a feature tensor can usually be mapped to the following:

[0201] E4:N - minibatch size

[0202] E3: H-3D tensor / image height

[0203] E2:W - 3D tensor / image width

[0204] E1:C - Channel or class of a 3D tensor

[0205] For artificial intelligence models based on machine learning or recurrent neural networks, the meaning of the four dimensions of a 4D feature tensor may usually be mapped to:

[0206] E4:T - number of time steps or models

[0207] E3: Reserved, normally set to 1

[0208] E2:N mb -Minibatch size

[0209] E1:L-Features

[0210] NNPA data layout format 0, for example, provides 2-dimensional data locality with 4k-byte blocks of data (pages) in addition to 4k-byte block data alignment in the outer dimensions of the generated tensors.

[0211] Pad element bytes are ignored for input tensors and unpredictable for output tensors. PER storage changes to pad bytes are unpredictable.

[0212] An example of an input data layout for a 4D feature tensor layout having dimensions E1, E2, E3, and E4 is shown in FIGS. 5(A)-5(C), and an example output of the 4D feature tensor layout is shown in FIGS. 6(A)-6(C). Referring to FIG. 5(A), a 3D tensor 500 is shown having dimensions E1, E2, and E3. In one example, each 3D tensor contains multiple 2D tensors 502. The numbers in each 2D tensor 502 represent the memory offsets where each of the elements reside in memory. These inputs are used to layout the data of the original tensor (e.g., the original 4D tensor of FIGS. 5(A)-5(C)) in memory as shown in FIGS. 6(A)-6(C), which correspond to FIGS. 5(A)-5(C).

[0213] In FIG. 6A, for example, a unit of memory 600 (e.g., a memory page) includes a preselected number (e.g., 32) of rows 602, each identified by, for example, e2_page_idx, and each row includes a preselected number (e.g., 64) of elements 604, each identified by, for example, e1_page_idx. If a row does not include the preselected number of elements, the row is padded (606), referred to as row padding or E1 padding. If the unit of memory does not include the preselected number of rows, the unit of memory is padded (608), referred to as page padding or E2 padding. For example, the row padding may be, for example, 0 or another value, and the page padding may be, for example, the existing value, 0, or another value.

[0214] In one example, output elements for a row are provided in memory (e.g., within a page) based on the element position in the E1 direction of the corresponding input. For example, referring to FIG. 5(A), element positions 0, 1, and 2 (e.g., element positions at the same position within each matrix) of the three illustrated matrices are shown in row 0 of page 0 of FIG. 6(A), etc. In this example, the 4D tensor is small, and all of the elements of each 2D tensor representing the 4D tensor fit on one page. However, this is just one example. A 2D tensor may include one or more pages. If a 2D tensor is created based on reformatting a 4D tensor, the number of pages for the 2D tensor is based on the size of the 4D tensor. In one example, one or more ceil functions are used to determine the number of rows in the 2D tensor and the number of elements in each row, which indicates the number of pages to be used. Other variations are possible.

[0215] In addition to 4D feature tensor layouts, in one example, a neural network processor may support 4D kernel tensors, which rearrange the elements of a 4D tensor to reduce the number of memory accesses and data collection steps when performing certain artificial intelligence (e.g., neural network processing aids) operations such as convolution. By way of example, as described herein, a typical 4D tensor in row-major order with dimensions E4, E3, E2, and E1 is converted to a 4D kernel tensor (4D kernel tensor) in NNPA data layout format 1.

[0216] The resulting tensor can be represented as a 4D tensor, for example a 64 element vector, or as a 5D tensor with the following dimensions:

[0217]

number

number

number

[0218] The elements [e4][e3][e2][e1] of the general tensor may be mapped to the next elements of the resulting 5D tensor.

[0219]

number

number

number

number

number

[0220] The resulting tensor may be larger than the general tensor. Elements of the resulting tensor that have no corresponding elements in the general tensor are called pad elements.

[0221] Consider an element [fe1][fe4][fe3][fe2][fe0] of a 4D feature tensor in NNPA data layout format 1, which is a 64-element vector, or an equivalent representation as a 5D tensor. This element is a pad element, or the corresponding element in a general 4D tensor with dimensions E4, E3, E2, E1 can be determined using the following formula:

[0222] If fe2 ≥ E2, this element is a pad element of E2 (or page)

[0223] Otherwise, if fe1*64+fe0≧E1, then this element is a pad element for E1 (or row).

[0224] Otherwise, the corresponding elements in a general 4D tensor are:

[0225] [fe4][fe3][fe2][fe1*64+fe0]

[0226] For artificial intelligence models based on convolutional neural networks, the meaning of the four dimensions of the kernel tensor can usually be mapped to the following:

[0227] E4: H-3D tensor / image height

[0228] E3:W - 3D tensor / image width

[0229] E2: Number of channels in the C-3D tensor

[0230] E1: K - Number of kernels

[0231] NNPA Data Layout Format 1, for example, provides for parallelism of 2D kernels within 4k-byte blocks of data (pages), in addition to alignment of 4k-byte blocks of data in the outer dimensions of generated tensors for efficient processing.

[0232] Pad bytes are ignored for input tensors. PER storage changes to pad bytes are unpredictable.

[0233] Again, exemplary data layout formats include a 4D feature tensor layout and a 4D kernel tensor layout, although other data layout formats may be supported by a processor (e.g., neural network processor 105). An indication of the supported data layouts is obtained and placed in the query parameter block, for example, by setting one or more bits in field 338.

[0234] The query parameter block also includes other data attribute information, including, for example, information about the supported sizes of data, in accordance with one or more aspects of the present invention. Processors, such as neural network processors, typically have limitations based on internal buffer sizes, processing unit, data bus structure, firmware limitations, etc., that may limit the maximum size of a tensor dimension and / or the overall size of a tensor. Therefore, the query function provides fields for communicating these limitations to an application. For example, the processor may obtain various data sizes, such as the maximum dimension index size (e.g., 65,536 elements) and the maximum tensor size (e.g., 8 GB), based on executing the query function and include this information in fields 340 and 342, respectively, of the parameter block (e.g., parameter block 330). Additional, fewer, or other size information, or combinations thereof, may be supported by a processor (e.g., neural network processor 105) and thus may be obtained and placed in the parameter block (e.g., fields 340, 342, or other fields). In other embodiments, these limits may be smaller or larger, or the sizes may be in other units, such as bytes instead of elements, elements instead of bytes, etc. Additionally, other embodiments allow different maximum sizes for each dimension, rather than the same maximum for all dimensions. Many variations are possible.

[0235] In accordance with one or more aspects of the present invention, a query function is provided that conveys detailed information related to a particular model of a selected processor (e.g., neural network processor 105). The detailed information may include, for example, information dependent on the model associated with the particular processor (a processor may support standard data attributes, such as standard data types, standard data layouts, etc., which are implicit and not necessarily represented by the query function, although in other embodiments, the query function may indicate all or various selected subsets of the data attributes). While exemplary information is provided, other information may be provided in other embodiments. The obtained information, which may vary for different models of a single processor and / or different processors, is used to perform artificial intelligence and / or other processing. The artificial intelligence and / or other processing may use, for example, one or more non-query functions of neural network processing assistance instructions. The specific non-query functions used in processing are implemented by executing the neural network processing assistance instructions one or more times and specifying the specific non-query functions.

[0236] Further details of exemplary non-query functions supported by the neural network processing assistance instructions are provided below (other embodiments may support additional, fewer, or other functions, or combinations thereof).

[0237] Function code 16: NNPA-ADD (addition)

[0238] If the NNPA-ADD function is specified, each element of input tensor 1, described by tensor descriptor 1, is added to the corresponding element of input tensor 2, described by tensor descriptor 2, and the resulting sum is placed in the corresponding element of the output tensor, described by output tensor descriptor.

[0239] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0240] In one example, the shape, data layout, and data type of input tensor1, input tensor2, and output tensor should be the same; if not, a general operand-data exception is recognized.

[0241] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage address fields are ignored.

[0242] Function code 17: NNPA-SUB (Subtraction)

[0243] When the NNPA-SUB function is specified, each element of input tensor 2, described by tensor descriptor 2, is subtracted from the corresponding element of input tensor 1, described by tensor descriptor 1, and the resulting difference is placed in the corresponding element of the output tensor.

[0244] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0245] In one example, the shape, data layout, and data type of input tensor1, input tensor2, and output tensor should be the same; if not, a general operand-data exception is recognized.

[0246] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage address fields are ignored.

[0247] Function code 18: NNPA-MUL (multiplication)

[0248] If the NNPA-MUL function is specified, the product of each element (multiplier) of input tensor 1 described by tensor descriptor 1 and the corresponding element (multiplicand) of input tensor 2 described by tensor descriptor 2 is placed into the corresponding element of the output tensor.

[0249] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0250] In one example, the shape, data layout, and data type of input tensor1, input tensor2, and output tensor should be the same; if not, a general operand-data exception is recognized.

[0251] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage address fields are ignored.

[0252] Function code 19: NNPA-DIV (Division)

[0253] If the NNPA-DIV function is specified, each element (dividend) of input tensor 1 described by tensor descriptor 1 is divided by the corresponding element (divisor) of input tensor 2 described by tensor descriptor 2, and the quotient is placed in the corresponding element of the output tensor.

[0254] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0255] In one example, the shape, data layout, and data type of input tensor1, input tensor2, and output tensor should be the same; if not, a general operand-data exception is recognized.

[0256] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage address fields are ignored.

[0257] Function code 20: NNPA-MIN (minimum value)

[0258] When the NNPA-MIN function is specified, each element of input tensor 1, described by tensor descriptor 1, is compared to the corresponding element of input tensor 2, described by tensor descriptor 2. The smaller of these two values ​​is placed in the corresponding element of the output tensor descriptor. If both values ​​are equal, they are placed in the corresponding element of the output tensor.

[0259] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0260] In one example, the shape, data layout, and data type of input tensor1, input tensor2, and output tensor should be the same; if not, a general operand-data exception is recognized.

[0261] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage address fields are ignored.

[0262] Function code 21: NNPA-MAX (maximum value)

[0263] When the NNPA-MAX function is specified, each element of input tensor 1, described by tensor descriptor 1, is compared to the corresponding element of input tensor 2, described by tensor descriptor 2. The greater of these two values ​​is placed in the corresponding element of the output tensor descriptor. If both values ​​are the same, they are placed in the corresponding element of the output tensor.

[0264] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0265] In one example, the shape, data layout, and data type of input tensor1, input tensor2, and output tensor should be the same; if not, a general operand-data exception is recognized.

[0266] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific storage address fields are ignored.

[0267] Function code 32: NNPA-LOG (natural logarithm)

[0268] If the NNPA-LOG function is specified, then for each element of the input tensor described by tensor descriptor 1, if that element is greater than 0, the corresponding element in the output tensor described by the output tensor descriptor is the natural logarithm of that element. Otherwise, the corresponding element in the output tensor cannot be represented numerically and the value associated with negative infinity in the target data type is stored.

[0269] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0270] In one example, the shape, data layout, and data type of the input tensor and the output tensor should be the same; if not, a general operand-data exception is recognized.

[0271] In one example, the Output Tensor Descriptor 2, Input Tensor Descriptor 2, Input Tensor Descriptor 3, Function-Specific Parameters 1-5, and Function-Specific Storage Address fields are ignored.

[0272] Function code 33: NNPA-EXP (exponential function)

[0273] If the NNPA-EXP function is specified, then for each element of the input tensor described by tensor descriptor 1, the corresponding element in the output tensor described by output tensor descriptor 1 is the exponential function of that element.

[0274] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0275] In one example, the shape, data layout, and data type of the input tensor and the output tensor should be the same; if not, a general operand-data exception is recognized.

[0276] In one example, the Output Tensor Descriptor 2, Input Tensor Descriptor 2, Input Tensor Descriptor 3, Function-Specific Parameters 1-5, and Function-Specific Storage Address fields are ignored.

[0277] Function code 49: NNPA-RELU (rectified linear unit)

[0278] If an NNPA-RELU function is specified, then for each element of the input tensor described by tensor descriptor 1, if the element is less than or equal to 0, the corresponding element in the output tensor described by output tensor descriptor will be 0. Otherwise, the corresponding element in the output tensor will be the minimum of the element in the input tensor and the clipping value specified in function-specific parameter 1.

[0279] For example, Function Specific Parameter 1 defines the clipping value for a RELU operation. For example, the clipping value is in bits 16-31 of Function Specific Parameter 1. The clipping value is specified, for example, in NNPA Data Type 1 format. A clipping value of 0 indicates that the maximum positive value is to be used; in other words, no clipping is performed. If a negative value is specified, a general operand data exception is recognized.

[0280] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0281] In one example, the shape, data layout, and data type of the input tensor and the output tensor should be the same; if not, a general operand-data exception is recognized.

[0282] In one example, Output Tensor Descriptor 2, Input Tensor Descriptor 2, Input Tensor Descriptor 3, and the function-specific storage address fields are ignored. In one example, function-specific parameters 2 through 5 should contain 0.

[0283] Function code 50:NNPA-TANH

[0284] If the NNPA-TANH function is specified, then for each element of the input tensor described by tensor descriptor 1, the corresponding element value in the output tensor described by output tensor descriptor 1 is the hyperbolic tangent of that element.

[0285] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0286] In one example, the shape, data layout, and data type of the input tensor and the output tensor should be the same; if not, a general operand-data exception is recognized.

[0287] In one example, the Output Tensor Descriptor 2, Input Tensor Descriptor 2, Input Tensor Descriptor 3, Function-Specific Parameters 1-5, and Function-Specific Storage Address fields are ignored.

[0288] Function code 51: NNPA-SIGMOID

[0289] If the NNPA-SIGMOID function is specified, then for each element of the input tensor described by tensor descriptor 1, the corresponding element in the output tensor described by output tensor descriptor 1 will be the sigmoid of that element.

[0290] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0291] In one example, the shape, data layout, and data type of the input tensor and the output tensor should be the same; if not, a general operand-data exception is recognized.

[0292] In one example, the Output Tensor Descriptor 2, Input Tensor Descriptor 2, Input Tensor Descriptor 3, Function-Specific Parameters 1-5, and Function-Specific Storage Address fields are ignored.

[0293] Function code 52:NNPA-SOFTMAX

[0294] If the NNPA-SOFTMAX function is specified, then for each vector in dimension 1 of input tensor 1, the corresponding vector in the output tensor is calculated as described below.

[0295] *The maximum value of the vector is calculated.

[0296] *The sum of the exponentials of the difference between each element of the vector in dimension 1 and the maximum computed above is computed. If both an element of the input vector in dimension 1 and the maximum computed above are numeric and the difference is non-numeric, the result of the exponential for that element is forced to 0.

[0297] *For each element in the vector, an intermediate quotient is formed by dividing the exponential of the difference between the element and the maximum value calculated above by the sum calculated above. An optional activation function is applied to this intermediate quotient to form the corresponding element in the output vector.

[0298] For example, this process is repeated for all dimension 4 index size × dimension 3 index size × dimension 2 index size vectors in dimension 1.

[0299] In one example, NNPA-SOFTMAX feature-specific parameter 1 controls the activation function. Illustratively, the ACT field (e.g., bits 28-31) of feature-specific parameter 1 specifies the activation function. Exemplary activation functions include:

[0300] ACT Activation Function

[0301] 0 Activation function not executed

[0302] 1 LOG

[0303] 2~15 spare

[0304] If a reserved value in the ACT field is specified, a response code of, for example, hex F001 is reported and the operation is completed with, for example, a condition code of 1.

[0305] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0306] In one example, if the dimension 3 index size of the input tensor is not equal to 1, a response code of, say, hex F000 is stored and the instruction completes with, say, a condition code of 1.

[0307] In one example, the shape, data layout, and data type of the input tensor and the output tensor should be the same; if not, a general operand-data exception is recognized.

[0308] In one example, output tensor descriptor 2, input tensor descriptor 2, and input tensor descriptor 3 are ignored. In one example, function-specific parameters 2 through 5 should contain 0.

[0309] This function may use 8K bytes of function-specific storage.

[0310] In one embodiment, when retrieving a vector in dimension 1, the elements may not be contiguous in memory, depending on the specified data layout format. If all elements of the input tensor1's dimension 1 vector contain negative numbers of the maximum magnitude that can be represented by the specified data type, the results may be less accurate.

[0311] Function code 64: NNPA-BATCHNORM (Batch normalization)

[0312] When the NNPA-BATCHNORM function is specified, for each vector in dimension 1 of the input 1 tensor, the corresponding vector in dimension 1 of the output tensor is calculated by multiplying each element in the vector by the corresponding element in the dimension 1 vectors that make up the input 2 tensor. The full-precision product is then added to the corresponding element in the dimension 1 vectors that make up the input 3 tensor, and then rounded to the precision of the specified data type of the output tensor. For example, this process is repeated for all dimension 4 index size × dimension 3 index size × dimension 2 index size vectors in dimension 1.

[0313] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0314] In one example, the following condition should be true; if not, a general operand-data exception is recognized:

[0315] *The shape and data layout of the input tensor1 and output tensor should be the same.

[0316] *The data types of the input and output tensors should be the same.

[0317] *The index sizes of input tensors 1, 2, 3 and dimension 1 of the output tensor should be the same.

[0318] *The index size of input tensors 2 and 3 should be 1, dimensions 2, 3, and 4.

[0319] In one example, the output tensor descriptor 2 and function-specific storage address fields are ignored. In one example, function-specific parameters 2 through 5 should contain 0.

[0320] Function code 80:NNPA-MAXPOOL2D Function code 81: NNPA-AVGPOOL2D

[0321] If either the NNPA-MAXPOOL2D or NNPA-AVGPOOL2D function is specified, the input tensor 1 described by the input tensor 1 descriptor is reduced by the specified operation to summarize a window of the input. The window of the input is selected by moving a 2D sliding window over dimension indices 2 and 3. The window summary becomes the elements of the output tensor. The dimension of the sliding window is described, for example, by function-specific parameter 4 and function-specific parameter 5. The amount by which the sliding window moves over the input 1 tensor when computing adjacent output tensor elements is called the stride. The stride of the sliding window is specified, for example, by function-specific parameter 2 and function-specific parameter 3. If the NNPA-MAXPOOL2D operation is specified, the Max operation defined below is performed on the window. If the NNPA-AVGPOOL2D operation is specified, the AVG operation defined below is performed on the window. If the specified padding type is "enabled", all elements in the window are added to the set used to compute the resulting output elements. If the specified padding type is "same", then depending on the window's position, only a subset of elements from the window may be added to the set used to compute the resulting output elements.

[0322] In one example, the CollectElements operation adds elements to a collection of elements and increments the number of elements in the collection. Each time the start position of the window moves, the collection is emptied. It is unpredictable whether elements that are not needed to perform the operation are accessed.

[0323] Max Operation: In one example, the maximum value of a set of elements within a window is calculated by comparing all elements in the set with each other and returning the largest value.

[0324] Avg operation: In one example, the average value of a set of elements within a window is calculated as the sum of all elements in the set divided by the number of elements in the set.

[0325] In one example, the fields are assigned as follows:

[0326] *Pool Function Specific Parameter 1 controls the type of padding. For example, bits 29-31 of Function Specific Parameter 1 contain a Pad field that specifies the type of padding. Example types include, for example, the following:

[0327] pad Padding types

[0328] 0 Enabled

[0329] 1 Same

[0330] 2~7 spare

[0331] If a reserved value for the pad field is specified, a response code of, say, hex F000 is reported and the operation completes with, say, a condition code of 1.

[0332] In one example, bit positions 0-28 of function-specific parameter 1 are reserved and should contain zeros.

[0333] *Function-specific parameter 2, for example, contains a 32-bit unsigned binary integer specifying the dimension-2-stride (D2S), which specifies the number of elements in dimension 2 by which the sliding window moves.

[0334] *Function-specific parameter 3, for example, contains a 32-bit unsigned binary integer that specifies the dimension-3-stride (D3S), which specifies the number of elements in dimension 3 that the sliding window moves by.

[0335] *Function-specific parameter 4, for example, contains a 32-bit unsigned binary integer that specifies the dimension-2-window-size (D2WS), which specifies the number of elements in dimension 2 that the sliding window contains.

[0336] *Function-specific parameter 5, for example, contains a 32-bit unsigned binary integer that specifies the dimension-3-window-size (D3WS), which specifies the number of elements in dimension 3 that the sliding window contains.

[0337] In one example, the values ​​specified in function-specific parameters 2-5 should be less than or equal to the maximum dimension index size, and the values ​​specified in function-specific parameters 4-5 should be greater than 0; otherwise, a response code (e.g., hex 0012) is reported and the operation completes, for example, with a condition code of 1.

[0338] If the dimension 2 stride and dimension 3 stride are both 0 and either the dimension 2 window size or the dimension 3 window size is greater than, for example, 1024, a response code (e.g., hex F001) is stored. If the dimension 2 stride and dimension 3 stride are both greater than, for example, 0, and either the dimension 2 window size or the dimension 3 window size is greater than, for example, 64, a response code (e.g., hex F002) is stored. If the dimension 2 stride and dimension 3 stride are both greater than, for example, 0, and either the dimension 2 stride or the dimension 3 stride is greater than, for example, 30, a response code (e.g., hex F003) is stored. If the dimension 2 stride and dimension 3 stride are both greater than, for example, 0, and either the input tensor's dimension 2 index size or the input tensor's dimension 3 index size is greater than, for example, 1024, a response code (e.g., hex F004) is stored. For all of the above conditions, the instruction completes, for example, with a condition code of 1.

[0339] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0340] In one example, the following condition should be true; if not, a general operand-data exception is recognized:

[0341] *The dimension 4 index size and dimension 1 index size of the input tensor and output tensor should be the same.

[0342] *The data layout and data types of the input and output tensors should be the same.

[0343] *If the dimension 2 stride and dimension 3 stride are both 0, then in one example the following additional condition should be true:

[0344] *The dimension 2 index size of the input tensor should be equal to the dimension 2 window size.

[0345] *The dimension 3 index size of the input tensor should be equal to the dimension 3 window size.

[0346] * The dimension 2 index size and dimension 3 index size of the output tensor should be 1.

[0347] * The specified padding should be valid.

[0348] *If either dimension 2 stride or dimension 3 stride is non-zero, then in one instance both strides should be non-zero.

[0349] *If dimension 2 stride and dimension 3 stride are both greater than 0, then in one example the following additional condition should be true:

[0350] *If the specified padding is "enabled", the dimension 2 window size should be less than or equal to the dimension 2 index size of the input tensor.

[0351] *If the specified padding is 'enabled', the dimension 3 window size should be less than or equal to the dimension 3 index size of the input tensor.

[0352] *If the specified padding is "same", the following relationship between the dimension 2 index size and dimension 3 index size of the input and output tensors should be satisfied (same padding for Pooling):

[0353]

number

[0354]

number

[0355] where:

[0356] IxDyIS The dimension y index size of the input tensor x defined by the tensor descriptor x.

[0357] OxDyIS The dimension y index size of the output tensor x defined by the tensor descriptor x.

[0358] D2S Dimension 2 Stride.

[0359] D3S Dimension 3 Stride.

[0360] *If the specified padding is "valid", the following relationship between the dimension 2 index size and dimension 3 index size of the input and output tensors should be satisfied (same padding for Pooling):

[0361]

number

[0362]

number

[0363] where D2WS is the dimension 2 window size and D3WS is the dimension 3 window size.

[0364] Output tensor descriptor 2, input tensor descriptors 2 and 3, and function-specific storage address fields are ignored.

[0365] Function code 96: NNPA-LSTMACT (Long Short-Term Memory Activation)

[0366] When the NNPA-LSTMACT function is specified, input tensor 1, described by the input tensor 1 descriptor, split into four subtensors per index value of dimension 4, along with input tensor 2, described by the input tensor 2 descriptor, and input tensor 3, described by the input tensor 3 descriptor, also split into four subtensors per index value of dimension 4, are input to the LSTMACT operation. At the end of the LSTMACT operation, the results are written to output tensor 1, described by the output tensor 1 descriptor, and output tensor 2, described by the output tensor 2 descriptor.

[0367] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data type = 0), a response code (hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, for example, with a condition code of 1.

[0368] In one embodiment, the following conditions should be true; if they are not, a general operand-data exception is recognized:

[0369] *The dimension 4 index size of input tensor 3, and output tensors 1 and 2 should be equal to 1, for example.

[0370] *The dimension 4 index size of input tensor1 and input tensor2 should be equal to 4 for example.

[0371] *For example, all input tensors and the two output tensors have dimension 3. The index size should be equal to, say, 1.

[0372] *For example, the data layout and data types of all input tensors and the two output tensors should be the same.

[0373] *For example, the index size of dimension 1 of all input tensors and the two output tensors should be the same.

[0374] *For example, the dimension 2 index size of all input tensors and the two output tensors should be the same.

[0375] In one example, the function-specific storage address field is ignored. In one example, function-specific parameters 1 through 5 should contain 0.

[0376] Function code 97: NNPA-GRUACT (Gated Recurrent Unit Enable)

[0377] When the NNPA-GRUACT function is specified, input tensor 1, described by the input tensor 1 descriptor, split into three subtensors per index value of dimension 4, along with input tensor 2, described by the input tensor 2 descriptor, and input tensor 3, described by the input tensor 3 descriptor, split into three subtensors per index value of dimension 4, are input to the GRUACT operation. At the end of the GRUACT operation, the output tensor, described by the output tensor descriptor, is stored.

[0378] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0379] In one embodiment, the following conditions should be true; if they are not, a general operand-data exception is recognized:

[0380] *The dimension 4 index size of the output tensor and input tensor 3 should be equal to 1 for example.

[0381] *The dimension 4 index size of input tensor1 and input tensor2 should be equal to 3 for example.

[0382] *For example, all input tensors and output tensors dimension 3 should have index size equal to e.g. 1.

[0383] *For example, the index size of dimension 1 of all input tensors and output tensors should be the same.

[0384] *For example, the dimension 2 index size of all input tensors and output tensors should be the same.

[0385] *For example, the data layout and data types of all input and output tensors should be the same.

[0386] In one example, the output tensor descriptor 2 and function-specific storage address fields are ignored. In one example, function-specific parameters 2 through 5 should contain 0.

[0387] Function code 112:NNPA-CONVOLUTION

[0388] When the NNPA-CONVOLUTION function is specified, for each output element in the output tensor described by the Output Tensor 1 descriptor, a 3-dimensional Input 1 window consisting of dimension indices 3, 2, and 1 is selected from Input Tensor 1 described by the Input Tensor 1 descriptor. An equally sized 3-dimensional Input 2 window consisting of dimension indices 4, 3, and 2 is selected from Tensor 2 described by the Input Tensor 2 descriptor. Elements in the Input 1 window are multiplied by the corresponding elements in the Input 2 window, and all products are added together to create an initial sum. This initial sum is added to the corresponding element of Input Tensor 3 to calculate an intermediate sum value. The elements of the output tensor are the result of the specified activation function applied to the intermediate sums. If no activation function is specified, the output elements are equal to the intermediate sums.

[0389] If the specified padding type is "valid", all elements in the window are used to compute the resulting initial sum. If the specified padding type is "same", depending on the window position, some elements in the input 1 window may have implicit zeros when computing the resulting initial sum.

[0390] It is unpredictable whether elements that are not required to perform an operation are accessed.

[0391] In one example, the function-specific parameter fields used by the convolution function are assigned as follows:

[0392] *NNPA-CONVOLUTION function-specific parameter 1 controls the padding type and activation function. In one example, bits 29-31 of function-specific parameter 1 contain a pad field that specifies the padding type. Example types are:

[0393] pad Padding types

[0394] 0 Enabled

[0395] 1 Same

[0396] 2~7 spare

[0397] If a reserved value for the pad field is specified, a response code of, say, hex F000 is reported and the operation completes with, say, a condition code of 1.

[0398] Additionally, in one example, bits 24-27 of parameter 1 specific to the NNPA-CONVOLUTION function include an activation field that specifies an activation function. An example function is:

[0399] ACT Activation Function

[0400] 0 Activation function not executed

[0401] 1 RELU

[0402] 2~15 spare

[0403] When a RELU activation function is specified, the resulting output element value is determined as follows: if the intermediate sum value is less than or equal to 0, the corresponding element in the output tensor is set to 0; otherwise, the corresponding element in the output tensor is set to the minimum of the intermediate sum value and the clipping value specified in function-specific parameter 4.

[0404] If a reserved value in the ACT field is specified, a response code of, for example, hex F001 is reported and the operation is completed with, for example, a condition code of 1.

[0405] *Function-specific parameter 2, for example, contains a 32-bit unsigned binary integer that specifies the dimension 2 stride (D2S), which specifies the number of elements in dimension 2 that the sliding window moves by.

[0406] *Function-specific parameter 3, for example, contains a 32-bit unsigned binary integer that specifies the dimension 3 stride (D3S), which specifies the number of elements in dimension 3 that the sliding window moves by.

[0407] The values ​​specified in function-specific parameters 2-3 should be less than the maximum dimension index size; if not, a response code (e.g., hex 0012) is reported and the operation completes with a condition code of 1.

[0408] *Function-specific parameter 4 defines the clipping value for the optional RELU operation. In one example, the clipping value is in bits 16-31 of function-specific parameter 4.

[0409] In one example, if the ACT field is 0, this field is ignored. If the ACT field specifies RELU, a clipping value is specified in the NNP data type 1 format. A clipping value of 0 indicates that the most positive value is to be used; in other words, no clipping is performed. If a value other than 0 is specified, a general operand data exception is recognized.

[0410] In one example, if the data layout specified in any of the specified tensor descriptors except input tensor 2 does not specify a 4D feature tensor (e.g., data layout = 0), or if the data layout specified in input tensor 2 does not specify a 4D kernel tensor (e.g., data layout = 1), a response code (e.g., hex 0010) is set in general register 0 and the instruction completes, e.g., with a condition code of 1. In one example, if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data type = 0), a response code (e.g., hex 0011) is set in general register 0 and the instruction completes, e.g., with a condition code of 1.

[0411] If the dimension 2 stride and dimension 3 stride are both 0 and either the dimension 3 index size or the dimension 4 index size of input tensor 2 is greater than, for example, 448, a response code (e.g., hex F002) is stored. If the dimension 2 stride and dimension 3 stride are both greater than 0 and either the dimension 3 index size or the dimension 4 index size of input tensor 2 is greater than, for example, 64, a response code (e.g., hex F003) is stored and the operation completes, for example, with a condition code of 1. If either the dimension 2 stride or dimension 3 stride is greater than, for example, 13, a response code (e.g., hex F004) is stored and the operation completes, for example, with a condition code of 1.

[0412] In one example, the following condition should be true; if not, a general operand-data exception is recognized:

[0413] *The data layout of input tensor1, input tensor3, and output tensor should be the same.

[0414] *The data types of all input and output tensors should be the same.

[0415] *The index sizes of dimension 2, dimension 3, and dimension 4 of the input 3 tensor should be 1.

[0416] *The dimension 4 index size of the output tensor should be equal to the dimension 4 index size of the input 1 tensor.

[0417] *The dimension 1 index size of the output tensor should be equal to the dimension 1 index size of the input 2 tensor and the dimension 1 index size of the input 3 tensor.

[0418] *The dimension 1 index size of the input 1 tensor should be equal to the dimension 2 index size of the input 2 tensor.

[0419] *If the dimension 2 stride and dimension 3 stride are both 0, then in one example the following additional condition should be true:

[0420] *The dimension 2 index size of the input 1 tensor should be equal to the dimension 3 index size of the input 2 tensor.

[0421] *The dimension 3 index size of the input 1 tensor of the input tensor should be equal to the dimension 4 index size of the input 2 tensor.

[0422] * The dimension 2 index size and dimension 3 index size of the output tensor should be 1.

[0423] * The specified padding should be "valid".

[0424] *If either dimension 2 stride or dimension 3 stride is non-zero, then both strides should be non-zero.

[0425] *If dimension 2 stride and dimension 3 stride are both greater than 0, then in one example the following additional condition should be true:

[0426] *If the specified padding is "valid", the dimension 2 index size of input tensor 1 should be greater than or equal to the dimension 3 index size of input tensor 2.

[0427] *If the specified padding is "valid", the dimension 3 index size of the input 1 tensor should be greater than or equal to the dimension 4 index size of the input 2 tensor.

[0428] *If the specified padding is "same", then in one example the following relationship between dimension 2 index size and dimension 3 index size of input 1 tensor and output tensor should be satisfied (same padding for convolutions):

[0429]

number

[0430]

number

[0431] where:

[0432] O1D2IS The dimension 2 index size of the output tensor.

[0433] O1D3IS The dimension 3 index size of the output tensor.

[0434] I1D2IS Dimension 2 index size of input 1 tensor.

[0435] I1D3IS Dimension 3 index size of input 1 tensor.

[0436] D2S Dimension 2 Stride.

[0437] D3S Dimension 3 Stride.

[0438] *If the specified padding is "valid", then in one example the following relationship should be satisfied between the dimension 2 index size and dimension 3 index size of the input 1 tensor, and between the dimension 3 index size and dimension 4 index size of the input 2 tensor and output tensor (valid padding for convolutions):

[0439]

number

[0440]

number

[0441] where:

[0442] O1D2IS The dimension 2 index size of the output tensor.

[0443] O1D3IS The dimension 3 index size of the output tensor.

[0444] I1D2IS Dimension 2 index size of input 1 tensor.

[0445] I1D3IS Dimension 3 index size of input 1 tensor.

[0446] I2D3IS Dimension 3 index size of input 2 tensor.

[0447] I2D4IS Dimension 4 index size of input 2 tensor.

[0448] D2S Dimension 2 Stride.

[0449] D3S Dimension 3 Stride.

[0450] In one example, the output tensor descriptor 2 and function-specific storage address fields are ignored. In one example, function-specific parameter 5 should contain 0.

[0451] Function code 113: NNPA-MATMUL-OP (matrix multiplication operation)

[0452] If the NNPA-MATMUL-OP function is specified, in one example, each element in the output tensor described by the output tensor descriptor is computed as described below.

[0453] *Using the Get Dimension 1 Vector operation described below, a dimension 1 vector is selected from input tensor 1 described by the input tensor 1 descriptor.

[0454] *Using the Get Dimension 2 Vector operation described below, a dimension 2 vector is selected from the input tensor 2 described by the input tensor 2 descriptor.

[0455] *The intermediate dot product of a dimension 1 vector and a dimension 2 vector is calculated using the dot product operation described below.

[0456] An operation is performed on the intermediate dot products and elements of input tensor 3 described by the input tensor 3 descriptors that have the same values ​​of dimension index 4 and dimension index 1 as the elements of the output tensor. The resulting elements are stored in the output tensor. The fused operation is determined by function-specific parameter 1 and is described below.

[0457] Get dimension 1 vector behavior: For a given output element, a vector of dimension 1 is selected from the input 1 tensor, with input dimension 4 indices becoming output dimension 4 indices, input dimension 3 indices becoming output dimension 3 indices, and input dimension 2 indices becoming output dimension 2 indices.

[0458] Get dimension 2 vector operation: For a given output element, a dimension 2 vector is selected from the input 2 tensor, with input dimension 4 indices becoming output dimension 4 indices, input dimension 3 indices becoming output dimension 3 indices, and input dimension 1 indices becoming output dimension 1 indices.

[0459] Dot Product Operation: The intermediate dot product of two vectors of the same size and data type is calculated as the sum of the products of each element in Input Vector 1 and the corresponding element in Input Vector 2.

[0460] Fused Operation: Function-specific parameters 1 control the operation performed on the intermediate dot products and the corresponding elements from input tensor 3. In one example, NNPA-MATMUL-OP function-specific parameters 1 includes an operation field, e.g., in bits 24-31. The operation field specifies the operation to be performed. Exemplary operations are shown below:

[0461] operation Type of action

[0462] Add 0

[0463] 1. Compare if dot product is higher

[0464] 2. Compare if the dot product is low

[0465] 3 Dot product and element equality comparison

[0466] 4 Dot product and comparing elements for inequality

[0467] 5. Compare high dot products

[0468] 6. Compare if dot product is lower

[0469] In one example, for an addition operation type, the elements of input tensor 3 are added to the intermediate dot product. For a comparison operation type, the intermediate dot product is compared to the elements of input tensor 3, and if the comparison is true, the result is set to a value, for example +1, in the data type specified for the output tensor, otherwise it is set to a value, for example +0.

[0470] In one example, all other values ​​of the operation field are reserved. If a reserved value of the operation field is specified, a response code of, for example, hex F000 is reported and the operation completes with, for example, a condition code of 1.

[0471] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0472] In one embodiment, the following conditions should be true; if they are not, a general operand-data exception is recognized:

[0473] *All input tensors and output tensors should have the same dimension 4 index size.

[0474] *All input and output tensors dimension 3 index size should be equal to 1.

[0475] *The index size of dimension 2 of input tensor 3 should be equal to 1.

[0476] *The index size of input tensor 1 and dimension 2 of the output tensor should be the same.

[0477] *The dimension 1 index size of input tensor 1 and the dimension 2 index size of input tensor 2 should be the same.

[0478] *The index sizes of input tensor 2, input tensor 3, and dimension 1 of the output tensor should be the same.

[0479] *The data layout and data types of all input and output tensors should be the same.

[0480] In one embodiment, the Output Tensor Descriptor 2 and Function-Specific Save Area Address fields are ignored. In one example, Function-Specific Parameters 2 through 5 should contain 0.

[0481] Function code 114: NNPA-MATMUL-OP-BCAST23 (Matrix multiplication operation - broadcast 23)

[0482] If the NNPA-MATMUL-OP-BCAST23 function is specified, in one example, each element in the output tensor described by the output tensor descriptor is computed as described below.

[0483] *Using the Get Dimension 1 Vector operation described below, a dimension 1 vector is selected from input tensor 1 described by the input tensor 1 descriptor.

[0484] *Using the Get Dimension 2 Vector operation described below, a dimension 2 vector is selected from the input tensor 2 described by the input tensor 2 descriptor.

[0485] *The dot product of a dimension 1 vector and a dimension 2 vector is calculated using the dot product operation described below.

[0486] *Elements of input tensor 3 described by input tensor 3 descriptors that have the same value of dimension index 1 as elements of the output tensor are added to the previously computed dot product and stored in the output tensor.

[0487] Get dimension 1 vector behavior: For a given output element, a vector of dimension 1 is selected from the input 1 tensor, with input dimension 4 indices becoming output dimension 4 indices, input dimension 3 indices becoming output dimension 3 indices, and input dimension 2 indices becoming output dimension 2 indices.

[0488] Get dimension 2 vector operation: For a given output element, a dimension 2 vector is selected from the input 2 tensor, with the input dimension 4 index becoming 1, the input dimension 3 index becoming the output dimension 3 index, and the input dimension 1 index becoming the output dimension 1 index.

[0489] Dot product operation: The intermediate product of two vectors of the same size and data type is calculated as the sum of the products of each element in input vector 1 and the corresponding element in input vector 2.

[0490] In one example, if the data layout specified in any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data_layout = 0), or if the data type in any of the specified tensor descriptors does not specify an NNP data type 1 (e.g., data_type = 0), a response code (e.g., hex 0010 or hex 0011) is set in general register 0, respectively, and the instruction completes, e.g., with a condition code of 1.

[0491] In one embodiment, the following conditions should be true; if they are not, a general operand-data exception is recognized:

[0492] *The index size of input tensor 1 and dimension 4 of the output tensor should be the same.

[0493] *The index size of dimension 4 of input tensor 2 and input tensor 3 should be equal to 1.

[0494] *For example, all input and output tensors of dimension 3 should have an index size equal to 1.

[0495] *The index size of dimension 2 of input tensor 3 should be equal to 1.

[0496] *The index size of input tensor 1 and dimension 2 of the output tensor should be the same.

[0497] *The dimension 1 index size of input tensor 1 and the dimension 2 index size of input tensor 2 should be the same.

[0498] *The index sizes of input tensor 2, input tensor 3, and dimension 1 of the output tensor should be the same.

[0499] *The data layout and data types of all input and output tensors should be the same.

[0500] In one embodiment, the Output Tensor Descriptor 2 and Function-Specific Save Area Address fields are ignored. In one example, Function-Specific Parameters 1 through 5 should contain 0.

[0501] For neural network processing aided instructions, in one embodiment, results become unpredictable if the output tensor overlaps with any of the input tensors or parameter blocks.

[0502] For example, if an attempt is made to execute a neural network processing instruction and the parameter block is not specified on a double-word boundary, for example, a specification exception is recognized.

[0503] When an attempt is made to execute a neural network processing instruction and, for example, there is an inconsistency in the tensor descriptors, a general operand-data exception is recognized.

[0504] Condition codes resulting from neural network processing assistance instructions include, for example, 0 - successful completion, 1 - response code is set, 2 - the amount of processed data determined by the CPU, and 3 - the amount of processed data determined by the CPU.

[0505] In one embodiment, the priority of execution of neural network processing assistance instructions includes, for example:

[0506] 1.~7. Exceptions with the same priority as the program interrupt condition for the general case.

[0507] 8.A Condition code 1 caused by specifying a feature code that is not assigned or installed.

[0508] 8.B Specification exception caused by parameter block not being specified on a double-word boundary.

[0509] 9. Access exception regarding access to parameter block.

[0510] 10. Condition code 1 caused by the specified format of the parameter block not being supported by the model.

[0511] 11.A Condition code 1 caused by the specified tensor data layout being unsupported.

[0512] 11.B General operand data exception due to different data layouts between tensor descriptors.

[0513] 12.A Condition Code 1 resulting from conditions other than those contained in Items 8.A, 10, and 11.A above and 12.B.1 below.

[0514] 12.B.1 Condition code 1 due to invalid output tensor data type for NNPA-RELU and NNPA-CONVOLUTION.

[0515] 12.B.2 General Operand Data Exception for invalid values ​​of parameter 1 specific to the NNPA-RELU function and parameter 4 specific to the NNPA-CONVOLUTION function.

[0516] 13.A Access exception on accessing output tensor.

[0517] 13.B Access exception on accessing input tensors.

[0518] 13.C Access Exception for Access to Function-Specific Storage Areas.

[0519] 14. Condition code 0.

[0520] As described herein, a single instruction (e.g., a neural network processing support instruction) is configured to perform multiple functions, including a query function and multiple non-query functions. Each non-query function may operate on a large amount of data, and thus the function is performed in multiple processing stages. During each processing stage, up to a selected amount of data may be stored to a selected location (e.g., memory). The selected amount of data depends on the function being performed. If a determination is made during a particular stage that a store to a designated area has occurred (e.g., an actual store to a designated area occurs; a parameter block and / or function-specific storage area, if defined to be used by the function, completely overlaps the designated area; a parameter block and / or function-specific storage area, if defined to be used by the function, partially overlaps the designated area and there is a store in the overlapping portion; a parameter block and / or function-specific storage area, if defined to be used by the function, partially overlaps the designated area regardless of whether there is a store in the overlapping portion; or a combination thereof), an interrupt is generated and the instruction is completed, e.g., with condition code 3. By checking whether a store to the designated area occurred after a processing step before terminating the instruction, stores to the designated area occur only within a single step. Because the selected amount of data is stored during each step (e.g., when processing multiple adjacent 4K pages), and the selected amount may be larger than 4K, a reduced number of checks may be required to test whether a store to the designated area occurred. For example, a test after each 4K page is not required. As an example, a neural network processing support instruction may store more than 4K additional bytes to each operand's location that intersects with a specified program event recording storage area before the event is indicated by an interrupt.

[0521] One or more aspects of the present invention are closely related to computer technology and facilitate processing within a computer and improve its performance. The use of a single, designed machine instruction configured to perform various functions improves performance within a computing environment by reducing complexity, reducing resource usage, and increasing processing speed. Furthermore, improving program event recording functionality facilitates program debugging, reduces program complexity, programming errors, or both, and improves system performance. For example, program event recording is improved by allowing instructions to store more than 4K additional bytes to each operand's location that intersects with a designated program event recording storage area before an event is indicated by an interrupt, by storing to the designated area only within a single processing stage before the interrupt, or by reducing the number of program event recording checks because more than 4K pages are stored before the check, or a combination thereof. Furthermore, in one or more embodiments, the complexity of a verification program that may predict a storage modification interrupt is reduced, improving verification quality, for example. For example, based on the memory access patterns corresponding to a function code, the verification program can accurately predict after which stage an instruction will present a program event recording interrupt. Because an interrupt is presented after each stage, the verification program can predict the interrupt if the program event recording's designated area is located within a parameter block and / or a function-specific storage area. The program can utilize the program event recording storage changes to, for example, identify unintentional memory updates. The acquired information, program event recording functions, and / or instructions can be used in many technical fields, such as computer processing, medical informatics, design, automotive engineering, and manufacturing. Achieving optimizations in acquiring information and / or improving program event recording can improve these technical fields by, for example, reducing errors and / or execution time.

[0522] Further details of one embodiment that facilitates processing within a computing environment as it relates to one or more aspects of the present invention are described with reference to Figures 7A and 7B.

[0523] Referring to FIG. 7A, in one embodiment, instruction processing of an instruction is performed (700). The instruction is configured to perform a plurality of functions (702). One of the plurality of functions is to be performed in a plurality of processing stages (704), with one of the plurality of processing stages defined to store up to a selected amount of data (706). The selected amount of data is based on one of the plurality of functions being executed (708). At least one of the plurality of functions has a value for the selected amount of data that is different from at least one other of the plurality of functions (710). Based on processing a selected one of the plurality of processing stages of the selected function, a determination is made as to whether a store to a designated area has occurred (712). Based on a determination that a store to a designated area has occurred, an interrupt is presented (714), and based on a determination that a store to a designated area has not occurred, execution of instruction processing continues (716).

[0524] The use of a single designed machine instruction configured to perform a variety of functions improves performance within a computing environment by reducing complexity, reducing resource usage, or increasing processing speed, or a combination thereof.

[0525] By checking whether a store to the specified area occurred after a processing step, before terminating the instruction, a store to the specified area only occurs within a single step. Because a selected amount of data is stored during each step (e.g., when processing multiple contiguous 4K pages), and the selected amount may be larger than 4K, a reduced number of checks may be required to test whether a store to the specified area occurred.

[0526] In one example, continuing to execute the instruction processing includes proceeding to process the next processing stage of the selected function (720). In another example, continuing to execute the instruction processing includes terminating the instruction (722).

[0527] By way of example, the specified region may include a region of memory defined by a starting address and an ending address 724. The starting address and ending address may be provided by, for example, one or more control registers 726.

[0528] In one embodiment, referring to FIG. 7B, determining whether a store to the designated region has occurred includes checking whether the memory address to which the data is being written is within the designated region (730), and based on the memory address being within the designated region, this determination indicates that a store to the designated region exists (732).

[0529] In one embodiment, determining whether storage in the designated area has occurred includes checking whether at least a portion of a parameter block used by the selected function overlaps with the designated area (734), and based on at least a portion of the parameter block overlapping with the designated area, this determination indicates that storage in the designated area exists (736).

[0530] Checking for parameter block duplication may cause an interrupt after each step and provides easy information about memory accesses.

[0531] In one embodiment, determining whether storage to the designated area has occurred includes checking whether at least a portion of a function-specific storage area used by the function overlaps with the designated area (738), and based on at least a portion of the function-specific storage area overlapping with the designated area, the determination indicates that storage to the designated area exists (740).

[0532] Function-specific save area overlap checks, if used, may cause an interrupt after each step and provide easy information about memory accesses.

[0533] In one or more aspects, a particular prediction may be made. For example, a particular processing stage of a selected function may be predicted to cause an interrupt (750), where the prediction is based on a memory access pattern of the selected function being executed (752). As a further example, the interrupt may be predicted to occur based on a determination that at least a portion of a parameter block used by the selected function overlaps with a designated region (754), or the interrupt may be predicted to occur based on a determination that at least a portion of a function-specific storage area used by the selected function overlaps with a designated region (756), or both.

[0534] At the very least, the complexity of a verification program that can predict storage modification interrupts is reduced, improving verification quality. For example, based on memory access patterns corresponding to function codes, the verification program can accurately predict after which stage an instruction will interrupt and present a program event recording interrupt. Because an interrupt is presented after each stage, the verification programming can predict the interrupt if the program event recording designated area is located within a parameter block and / or a function-specific save area.

[0535] Other variations and embodiments are possible.

[0536] Aspects of the present invention may be used with many types of computing environments. Another example of a computing environment for incorporating and using one or more aspects of the present invention is described with reference to FIG. 8A. By way of example, the computing environment of FIG. 8A is based on the z / Architecture® instruction set architecture offered by International Business Machines Corporation (Armonk, New York). However, the z / Architecture instruction set architecture is only one exemplary architecture. Additionally, computing environments may be based on other architectures, including, but not limited to, the Intel® x86 architecture, other architectures from International Business Machines Corporation, or architectures from other companies, or combinations thereof. Intel is a trademark or registered trademark of Intel Corporation or its subsidiaries in the United States and other countries.

[0537] In one example, computing environment 10 includes a central electronics complex (CEC) 11. Central electronics complex 11 includes multiple components, such as memory 12 (also known as system memory, main memory, primary storage, central storage, or storage), coupled to one or more processors, such as one or more general-purpose processors (also known as central processing units (CPUs) 13) and one or more special-purpose processors (e.g., neural network processor 31), and input / output (I / O) subsystem 14.

[0538] By way of example, the one or more special purpose processors may be separate from the one or more general purpose processors, or at least one special purpose processor may be embedded within at least one general purpose processor, or both. Other variations are possible.

[0539] I / O subsystem 14 may be part of or separate from the central electronics complex. I / O subsystem 14 directs the flow of information between main memory 12 and input / output control unit 15 and input / output (I / O) devices 16 coupled to the central electronics complex.

[0540] Many types of I / O devices may be used. One particular type is a data storage device 17. The data storage device 17 may store one or more programs 18, one or more computer-readable program instructions 19, or data, or a combination thereof. The computer-readable program instructions may be configured to perform the functions of embodiments of aspects of the present invention.

[0541] Central electronics complex 11 may include and / or be coupled to removable / non-removable, volatile / non-volatile computer system storage media. For example, central electronics complex 11 may include and / or be coupled to non-removable, non-volatile magnetic media (commonly referred to as "hard drives"), magnetic disk drives for reading from and writing to removable, non-volatile magnetic disks (e.g., "floppy disks"), and / or optical disk drives for reading from or writing to removable, non-volatile optical disks such as CD-ROMs, DVD-ROMs, or other optical media. It should be understood that other hardware and / or software components may be used in conjunction with central electronics complex 11, including, but not limited to, microcode or millicode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0542] Additionally, central electronics complex 11 can operate in numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, or configurations, or combinations thereof, that may be suitable for use with central electronics complex 11 include, but are not limited to, personal computer (PC) systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, microcomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of these systems or devices.

[0543] Central electronics complex 11, in one or more embodiments, provides support for logical partitioning and / or virtualization. In one embodiment, as shown in FIG. 8B, memory 12 includes, for example, one or more logical partitions 20, a hypervisor 21 that manages the logical partitions, and processor firmware 22. One example of hypervisor 21 is the Processor Resource / System Manager (PR / SM™) offered by International Business Machines Corporation (Armonk, New York). PR / SM is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction.

[0544] Each logical partition 20 can function as a separate system. That is, each logical partition can be reset independently, run a guest operating system 23, such as the z / OS® operating system provided by International Business Machines Corporation (Armonk, New York), or other control code 24, such as coupling facility control code (CFCC), and operate with different programs 25. An operating system or application program running within a logical partition appears to have access to the entire system, but in fact only a portion of it is available. While the z / OS operating system is provided as an example, other operating systems provided by International Business Machines Corporation, other companies, or both, may be used in accordance with one or more aspects of the present invention.

[0545] Memory 12 is coupled to physical processor resources, such as CPU 13 (FIG. 8A), that can be assigned to logical partitions. For example, logical partition 20 may include one or more logical processors, each of which represents all or a portion of physical processor resources 13 that can be dynamically assigned to a logical partition.

[0546] In yet another embodiment, the central electronics complex provides support for virtual machines (which may or may not support logical partitioning). As shown in FIG. 8C, memory 12 of central electronics complex 11 includes, for example, one or more virtual machines 26, a virtual machine manager (e.g., hypervisor 27) that manages the virtual machines, and processor firmware 28. One example of hypervisor 27 is the z / VM® hypervisor offered by International Business Machines Corporation (Armonk, New York). A hypervisor may be referred to as a host. z / VM is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction.

[0547] The central electronics complex's virtual machine support provides the ability to operate multiple virtual machines 26, each capable of running a different program 29 and a guest operating system 30, such as the Linux® guest operating system. Each virtual machine 26 can function as a separate system; that is, each virtual machine can be reset independently, run a guest operating system, and operate with different programs. An operating system or application program running within a virtual machine appears to have access to the entire system, but in fact only a portion of it is available. While z / VM and Linux are provided as examples, other virtual machine managers and / or operating systems may be used in accordance with one or more aspects of the present invention. The registered trademark Linux® is used pursuant to a sublicense from the Linux Foundation (an exclusive licensee of Linus Torvalds, owner of the trademark worldwide).

[0548] Another embodiment of a computing environment for incorporating and using one or more aspects of the present invention is described with reference to FIG. 9A. In this example, computing environment 36 includes a native central processing unit (CPU) 37, memory 38, and one or more input / output devices and / or interfaces 39, coupled together, for example, via one or more buses 40 or other connections. By way of example, computing environment 36 may include a PowerPC® processor offered by International Business Machines Corporation (Armonk, New York), an HP Superdome with an Intel® Itanium® II processor offered by Hewlett-Packard Company (Palo Alto, California), or other machines based on architectures offered by International Business Machines Corporation, Hewlett-Packard Company, Intel Corporation, Oracle Corporation, or other companies, or combinations thereof. PowerPC is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction. Itanium is a trademark or registered trademark of Intel Corporation or its subsidiaries in the United States and other countries.

[0549] The native central processing unit 37 includes one or more native registers 41, such as one or more general purpose registers and / or one or more special purpose registers used during processing within the environment. These registers contain information that represents the state of the environment at any particular point in time.

[0550] Additionally, native central processing unit 37 executes instructions and code stored in memory 38. In one particular example, central processing unit executes emulator code 42 stored in memory 38. This code enables a computing environment configured with one architecture to emulate another architecture. For example, emulator code 42 enables machines based on architectures other than the z / Architecture instruction set architecture (e.g., PowerPC processors, HP Superdome servers, etc.) to emulate the z / Architecture instruction set architecture and execute software and instructions developed based on the z / Architecture instruction set architecture.

[0551] Further details related to emulator code 42 are described with reference to FIG. 9B . Guest instructions 43 stored in memory 38 include software instructions (e.g., correlated to machine instructions) developed to execute on an architecture other than that of native CPU 37. For example, guest instructions 43 may be designed to execute on a processor based on the z / Architecture instruction set architecture, but are instead emulated on native CPU 37, which may be, for example, an Intel Itanium II processor. In one example, emulator code 42 includes an instruction fetch routine 44 for retrieving one or more guest instructions 43 from memory 38 and, optionally, providing local buffering of the retrieved instructions. Emulator code 42 also includes an instruction conversion routine 45 for determining the type of the retrieved guest instruction and for converting the guest instruction into one or more corresponding native instructions 46. This conversion includes, for example, identifying a function performed by the guest instruction and selecting a native instruction to perform this function.

[0552] Additionally, emulator code 42 includes an emulation control routine 47 for causing execution of native instructions. Emulation control routine 47 may cause native CPU 37 to execute a routine of native instructions that emulates one or more already-fetched guest instructions, and upon completion of such execution, return control to an instruction fetch routine to emulate the fetch of the next guest instruction or group of guest instructions. Execution of native instructions 46 may include reading data from memory 38 into a register, storing data from a register back to memory, or performing some type of arithmetic or logical operation as determined by a translation routine.

[0553] For example, each routine is implemented in software that is stored in memory and executed by native central processing unit 37. In other examples, one or more of the routines or operations are implemented in firmware, hardware, software, or some combination thereof. The emulated processor's registers may be emulated using the native CPU's registers 41 or by using locations in memory 38. In an embodiment, guest instructions 43, native instructions 46, and emulator code 42 may reside in the same memory or may be distributed among different memory devices.

[0554] Instructions that may be emulated include the neural network processing assistance instructions described herein in accordance with one or more aspects of the present invention. Additionally, other instructions and / or one or more aspects of program event recording functionality may be emulated in accordance with one or more aspects of the present invention.

[0555] The aforementioned computing environments are only examples of computing environments that can be used. Other environments may be used, including, but not limited to, non-partitioned, partitioned, cloud, or emulated environments, or combinations thereof, and embodiments are not limited to any one environment. Although various examples of computing environments are described herein, one or more aspects of the present invention may be used with many types of environments. The computing environments provided herein are only examples.

[0556] Each computing environment can be configured to include one or more aspects of the present invention.

[0557] One or more aspects may relate to cloud computing.

[0558] Although this disclosure includes detailed descriptions of cloud computing, it should be understood that implementation of the subject matter recited herein is not limited to cloud computing environments. Embodiments of the invention may be implemented in conjunction with any other type of computing environment now known or later developed.

[0559] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computational resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) and for rapidly provisioning and releasing these resources with minimal administrative effort or interaction with a service provider. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0560] The features are as follows:

[0561] On-demand self-service: Cloud customers can unilaterally and automatically provision computing power, such as server time and network storage, as needed, without the need for human interaction with the service provider.

[0562] Broad network access: Capabilities are available over the network and can be accessed using standard mechanisms, facilitating use by heterogeneous thin-client or thick-client platforms (e.g., mobile phones, laptops, and PDAs).

[0563] Resource Pool: The provider's computing resources are pooled and offered to multiple consumers using a multi-tenant model, with various physical and virtual resources dynamically allocated and reallocated according to demand. There is a sense of location independence, where consumers typically have no control or knowledge regarding the exact location of the resources offered, although at a higher level of abstraction they may be able to specify a location (e.g., country, state, or data center).

[0564] Rapid Elasticity: Capacity is quickly and elastically provisioned, sometimes automatically, and can be quickly scaled out and quickly released to quickly scale in. Capacity available for provisioning often appears to consumers as unlimited, available for purchase in any quantity at any time.

[0565] Metered Services: Cloud systems leverage metering capabilities to automatically control and optimize resource usage at an abstraction level appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the services used.

[0566] The service model is as follows:

[0567] SaaS (Software as a Service): The consumer is provided with the ability to use the provider's applications running on a cloud infrastructure. Those applications can be accessed from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or individual application features, except for the possibility of setting limited user-specific application configuration settings.

[0568] PaaS (Platform as a Service): The ability offered to a consumer is to deploy applications they create or acquire, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the configuration of the application hosting environment.

[0569] Infrastructure as a Service (IaaS): The capability provided to a consumer is the provisioning of processing, storage, network, and other basic computing resources, upon which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but does have control over the operating system, storage, deployed applications, and in some cases, limited control over selected network components (e.g., host firewalls).

[0570] The deployment model is as follows:

[0571] Private Cloud: This cloud infrastructure is operated solely for the organization, can be managed by the organization or a third party, and can reside on-premise or off-premise.

[0572] Community Cloud: This cloud infrastructure is shared by multiple organizations to support a specific community with shared interests (e.g., mission, security requirements, policy, and compliance considerations). It can be managed by these organizations or a third party and can reside on-premises or off-premises.

[0573] Public Cloud: This cloud infrastructure is available for use by the general public or large industry organizations and is owned by an organization that sells cloud services.

[0574] Hybrid cloud: This cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain distinct but are joined together by standardized or proprietary technologies that allow for data and application portability (e.g., cloud bursting to balance load between clouds).

[0575] A cloud computing environment is a service-oriented environment that emphasizes statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that contains a network of interconnected nodes.

[0576] Referring now to FIG. 10 , an exemplary cloud computing environment 50 is illustrated. As illustrated, the cloud computing environment 50 includes one or more cloud computing nodes 52 with which local computing devices used by cloud consumers (e.g., a personal digital assistant (PDA) or mobile phone 54A, a desktop computer 54B, a laptop computer 54C, and / or an automobile computer system 54N) can communicate. The nodes 52 may communicate with each other. The nodes 52 may be physically or virtually grouped in one or more networks (not shown), such as a private cloud, community cloud, public cloud, or hybrid cloud, or combinations thereof, as previously described herein. This enables the cloud computing environment 50 to provide an infrastructure, platform, and / or SaaS that does not require cloud consumers to maintain resources on their local computing devices. The types of computing devices 54A-N shown in FIG. 10 are intended to be illustrative only, and it is understood that computing node 52 and cloud computing environment 50 can communicate with any type of computer-controlled device via any type of network and / or network-addressable connection (e.g., a connection using a web browser).

[0577] Referring now to Figure 11, a set of functional abstraction layers provided by cloud computing environment 50 (Figure 10) is shown. It should be understood in advance that the components, layers, and functions shown in Figure 11 are intended to be illustrative only, and that embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0578] Hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframe 61, RISC (Reduced Instruction Set Computer) architecture-based server 62, server 63, blade server 64, storage device 65, and network and network components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0579] The virtualization layer 70 comprises an abstraction layer capable of providing virtual entities such as virtual servers 71 , virtual storage 72 , virtual networks including virtual private networks 73 , virtual applications and operating systems 74 , and virtual clients 75 .

[0580] By way of example, the management layer 80 may provide the following functions: Resource provisioning 81 dynamically procures computing and other resources used to execute tasks within the cloud computing environment; Metering and pricing 82 tracks costs as resources are utilized within the cloud computing environment and sends bills or invoices for the utilization of those resources; by way of example, those resources may include application software licenses; Security verifies the identity of cloud users and tasks and protects data and other resources; User portal 83 provides users and system administrators with access to the cloud computing environment; Service level management 84 allocates and manages cloud computing resources to meet required service levels; and Service Level Agreement (SLA) planning and execution 85 proactively prepares and procures cloud computing resources in accordance with SLAs in anticipation of future demand.

[0581] The Workload Layer 90 illustrates examples of functionality available in a cloud computing environment. Examples of workloads and functionality that may be provided from this layer include mapping and navigation 91, software development and lifecycle management 92, virtual classroom instruction delivery 93, data analytics processing 94, transaction processing 95, and program event recording and / or neural network processing-assisted processing 96.

[0582] Aspects of the invention may be systems, methods, and / or computer program products, at any possible level of technical detail of integration. The computer program product may include one or more computer-readable storage media containing computer-readable program instructions for causing a processor to perform aspects of the invention.

[0583] A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device, such as, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge-in-groove structures on which instructions are recorded, and any suitable combination thereof. As used herein, computer-readable storage media should not be construed as being ephemeral signals, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through fiber optic cable), or electrical signals transmitted over wires.

[0584] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or storage device over a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). This network may include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage on a computer-readable storage medium within each computing / processing device.

[0585] Computer-readable program instructions for carrying out the operations of the present invention may be source or object code written in any combination of one or more programming languages, including assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or object-oriented programming languages ​​such as Smalltalk®, C++, and procedural programming languages ​​such as the “C” programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, to carry out aspects of the present invention, electronic circuitry including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions to customize the electronic circuitry by utilizing state information of the computer-readable program instructions.

[0586] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0587] These computer-readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to create a machine, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for performing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may be stored on a computer-readable storage medium and capable of directing a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer-readable storage medium on which the instructions are stored comprises an article of manufacture containing instructions for performing aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0588] Computer-readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device such that the instructions, which execute on the computer, other programmable apparatus, or other device, perform the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams, thereby causing a series of operable steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process.

[0589] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, comprising one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions shown in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may actually be realized as a single step, executed concurrently, executed substantially concurrently in a partially or fully overlapping manner in time, or executed in the reverse order, depending on the functionality involved. It is also noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks included in the block diagrams and / or flowchart diagrams, may be implemented by a special-purpose hardware-based system that performs the specified function or operation or executes a combination of special-purpose hardware and computer instructions.

[0590] In addition to the above, one or more aspects may be provided, offered, deployed, managed, serviced, etc. by a service provider that offers management of a customer's environment. For example, a service provider may create, maintain, support, etc., computer code and / or computer infrastructure that implements one or more aspects for one or more customers. In return, the service provider may receive payments from the customer, for example, pursuant to a subscription and / or fee agreement. Additionally or alternatively, the service provider may receive payments from the sale of advertising content to one or more third parties.

[0591] In one aspect, an application may be deployed to perform one or more embodiments. As one example, deploying an application includes providing a computer infrastructure that functions to perform one or more embodiments.

[0592] As a further aspect, a computing infrastructure may be deployed that includes integrating computer readable code into a computing system, where the code in combination with the computing system is capable of executing one or more embodiments.

[0593] In yet another aspect, a process for integrating a computing infrastructure may be provided, comprising integrating computer-readable code into a computer system, the computer system comprising a computer-readable medium, the computer medium including one or more embodiments, the code in combination with the computer system capable of executing one or more embodiments.

[0594] While various embodiments have been described above, they are merely examples. For example, computing environments of other architectures may be used to incorporate one or more aspects and / or to use one or more aspects. Furthermore, various instructions or operations may be used. Furthermore, different types of registers and / or different registers may be used. Furthermore, other data formats, data layouts, and / or data sizes may be supported. In one or more embodiments, one or more general-purpose processors, one or more special-purpose processors, or a combination of general-purpose and special-purpose processors may be used. Many variations are possible.

[0595] Various embodiments are described herein. Furthermore, many variations are possible without departing from the spirit of the embodiments of the present invention. It should be noted that each embodiment or feature described herein and variations thereof can be combined with any other embodiment or feature unless specifically inconsistent.

[0596] Additionally, other types of computing environments may benefit and be used. By way of example, a data processing system may be used that includes at least two processors coupled directly or indirectly via a system bus to memory elements suitable for storing and / or executing program code. These memory elements may include, for example, local memory used during the actual execution of the program code, bulk storage, and cache memory for temporarily storing at least some program code to reduce the number of times the code must be retrieved from bulk storage during execution.

[0597] Input / output or I / O devices (including but not limited to keyboards, displays, pointing devices, DASDs, tapes, CDs, DVDs, thumb drives, and other storage media) may be coupled to the system either directly or through intervening I / O controllers. Network adapters may also be coupled to the system to enable the data processing system to be coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modems, and Ethernet cards are just a few of the types of network adapters available.

[0598] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used herein, indicate the presence of stated features, integers, steps, operations, elements, or components, or combinations thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups thereof, or combinations thereof.

[0599] Corresponding structures, materials, acts, and equivalents of all means or steps and functional elements within the scope of the claims below are intended to include any structure, material, or act for performing a function in combination with other claimed elements, if any, when specifically claimed. The description of one or more embodiments has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the disclosed form. Many modifications and variations will be apparent to those skilled in the art. The embodiments have been chosen and described to best explain various aspects and practical applications, and to enable others skilled in the art to appreciate various embodiments with various modifications as may be suited to the particular use contemplated.

Claims

1. 1. A computer program for facilitating processing within a computing environment, comprising: On the computer, executing instruction processing of instructions, the instructions configured to perform a plurality of functions, one of the plurality of functions to be performed in a plurality of processing stages, one of the plurality of processing stages defined to store up to a selected amount of data, the selected amount of data being based on the one of the plurality of functions being performed, and at least one of the plurality of functions having a value of the selected amount of data that is different from at least one other of the plurality of functions; determining whether a storage to a designated area occurred based on processing a selected one of the plurality of processing steps of a selected function; presenting an interrupt based on a determination that the storing in the designated area has occurred; and and continuing execution of instruction processing based on a determination that the store to the designated area did not occur.

2. 2. The computer program product of claim 1, wherein said continuing execution of instruction processing comprises proceeding to processing a next processing stage of said selected function.

3. 3. The computer program product of claim 1, wherein the continuing execution of instruction processing comprises terminating the instruction.

4. 3. The computer program product of claim 1, wherein the specified region comprises a region of memory defined by a starting address and an ending address, the starting address and the ending address being provided by one or more control registers.

5. 3. The computer program product of claim 1, wherein determining whether the store to the designated area has occurred includes checking whether a memory address to which data is being written is within the designated area, and wherein the determination indicates that a store to the designated area has occurred based on the memory address being within the designated area.

6. 3. The computer program product of claim 1, wherein determining whether the storage in the designated area has occurred includes checking whether at least a portion of a parameter block used by the selected function overlaps with the designated area, and wherein the determination indicates that storage in the designated area exists based on the at least a portion of the parameter block overlapping with the designated area.

7. 3. The computer program product of claim 1, wherein determining whether the storage to the designated area has occurred includes checking whether at least a portion of a function-specific storage area used by the selected function overlaps with the designated area, and wherein the determination indicates that storage to the designated area exists based on the at least a portion of the function-specific storage area overlapping with the designated area.

8. 3. The computer program product of claim 1, wherein the method further comprises predicting that a particular processing stage of the plurality of processing stages of the selected function will cause an interrupt, the prediction being based on a memory access pattern of the selected function.

9. 3. The computer program product of claim 1, wherein the method further comprises predicting that an interrupt will occur based on determining that at least a portion of a parameter block used by the selected function overlaps with the designated region.

10. 3. The computer program product of claim 1, wherein the method further comprises predicting that an interrupt will occur based on determining that at least a portion of a function-specific save area used by the function overlaps with the designated area.

11. 1. A computer system for facilitating processing within a computing environment, said computer system comprising: Memory and and at least one processor in communication with the memory, the computer system being configured to perform a method, the method comprising: executing instruction processing of instructions, the instructions configured to perform a plurality of functions, one of the plurality of functions to be performed in a plurality of processing stages, one of the plurality of processing stages defined to store up to a selected amount of data, the selected amount of data being based on the one of the plurality of functions being performed, and at least one of the plurality of functions having a value of the selected amount of data that is different from at least one other of the plurality of functions; determining whether a storage to a designated area occurred based on processing a selected one of the plurality of processing steps of a selected function; presenting an interrupt based on a determination that the storing in the designated area has occurred; and and continuing execution of instruction processing based on a determination that the store to the designated area did not occur.

12. 12. The computer system of claim 11, wherein determining whether the store to the designated region has occurred includes checking whether a memory address to which data is being written is within the designated region, and wherein based on the memory address being within the designated region, the determination indicates that a store to the designated region exists.

13. 13. The computer system of claim 11 or 12, wherein determining whether the storage to the designated area has occurred includes checking whether at least a portion of a parameter block used by the selected function overlaps with the designated area, and wherein the determination indicates that storage to the designated area exists based on the at least a portion of the parameter block overlapping with the designated area.

14. 13. The computer system of claim 11 or 12, wherein determining whether the storage to the designated area has occurred includes checking whether at least a portion of a function-specific storage area used by the selected function overlaps with the designated area, and wherein the determination indicates that storage to the designated area exists based on the at least a portion of the function-specific storage area overlapping with the designated area.

15. 13. The computer system of claim 11, wherein the method further comprises predicting that a particular processing stage of the plurality of processing stages of the selected function will cause an interrupt, the prediction being based on a memory access pattern of the selected function.

16. 1. A computer-implemented method for facilitating processing in a computing environment, the computer-implemented method comprising: executing instruction processing of instructions, the instructions configured to perform a plurality of functions, one of the plurality of functions to be performed in a plurality of processing stages, one of the plurality of processing stages defined to store up to a selected amount of data, the selected amount of data being based on the one of the plurality of functions being performed, and at least one of the plurality of functions having a value of the selected amount of data that is different from at least one other of the plurality of functions; determining whether a storage to a designated area occurred based on processing a selected one of the plurality of processing steps of a selected function; presenting an interrupt based on a determination that the storing in the designated area has occurred; and and continuing execution of instruction processing based on a determination that the store to the designated area did not occur.

17. 17. The computer-implemented method of claim 16, wherein determining whether the store to the designated region has occurred comprises checking whether a memory address to which data is being written is within the designated region, and wherein the determination indicates that a store to the designated region exists based on the memory address being within the designated region.

18. 18. The computer-implemented method of claim 16 or 17, wherein determining whether the storage in the designated area has occurred includes checking whether at least a portion of a parameter block used by the selected function overlaps with the designated area, and wherein the determination indicates that storage in the designated area exists based on the at least a portion of the parameter block overlapping with the designated area.

19. 18. The computer-implemented method of claim 16 or 17, wherein determining whether the storage to the designated area has occurred includes checking whether at least a portion of a function-specific storage area used by the selected function overlaps with the designated area, and wherein the determination indicates that storage to the designated area exists based on the at least a portion of the function-specific storage area overlapping with the designated area.

20. 18. The computer-implemented method of claim 16 or 17, further comprising predicting that a particular processing stage of the plurality of processing stages of the selected function will cause an interrupt, the prediction being based on a memory access pattern of the selected function.

Images