Summary of exceptions when an invalid value is detected during instruction execution

Abstract

An exception summary is provided for invalid values ​​detected during execution of the instructions. An indication is obtained that the value determined to be invalid was included in an input to or output from one of the one or more calculations. The value is determined to be invalid due to one of the plurality of exceptions. A summary indicator is set based on obtaining the indication that the value was determined to be invalid. The summary indicator collectively represents the plurality of exceptions.

Description

Technical Field

[0001] One or more aspects generally relate to facilitating processing within a computing environment, and more specifically to improving such processing.

Background Art

[0002] To improve processing in a computing environment that is data-intensive, or compute-intensive, or both, co-processors such as artificial intelligence accelerators (also called neural network processors or neural network accelerators) are utilized. Such accelerators provide a large amount of computing power used when performing related computations, such as matrix or tensor computations.

[0003] Tensor computations are used, for example, in complex processing including deep learning, which is a subset of machine learning. Deep learning or machine learning, which is one aspect of artificial intelligence, is used in various technologies (not particularly limited, such as engineering, manufacturing, medical technology, automotive technology, computer processing, etc.).

[0004] When performing tensor computations, a large amount of time and computing power are used. Therefore, improvements regarding such performance are sought to improve execution performance and system performance.

Summary of the Invention

[0005] The shortcomings of prior art are overcome and further advantages are obtained through the provision of computer program products that facilitate processing within a computing environment. The computer program product includes one or more computer-readable storage media and program instructions for executing a method, which are stored together on the one or more computer-readable storage media. The method includes obtaining an indication that a value determined to be invalid was included in the input data to one of one or more calculations, or in the output data obtained as a result of one or more calculations. The value is determined to be invalid by one of several exceptions. A summary indicator is set based on the acquisition of the indication that the value has been determined to be invalid. The summary indicator represents the several exceptions collectively.

[0006] Providing an indication that invalid values ​​have been detected in the input or output data makes debugging artificial intelligence models easier, for example. Higher computational performance is achieved by avoiding the need to specify a particular data element with an invalid value or distinguishing which of multiple exceptions was detected.

[0007] In one example, the summary indicator represents multiple exceptions without distinguishing between them. In another example, the summary indicator is set regardless of which of the multiple exceptions was used to determine if the value was invalid.

[0008] Using a single indicator to show invalid values, regardless of the type of invalid value (e.g., non-numeric, unrepresentable, out-of-range), simplifies coding and handling, reduces complexity, and improves system performance.

[0009] In one example, a summary indicator is a range violation indicator of an exception flag placed at a location specified by an instruction issued to perform one or more calculations.

[0010] For example, several exceptions include non-numeric values, unrepresentable numbers, and out-of-range values.

[0011] In one example, obtaining a notification that a value has been determined to be invalid is based on the execution of one or more instructions that perform calculations. This facilitates the processing of instructions and the reporting of invalid values.

[0012] For example, an instruction is configured to execute multiple functions that perform calculations, and these multiple functions use a summary indicator. Having a single indicator used by multiple functions reduces complexity and the effort required for coding and verification.

[0013] In one example, an instruction is a neural network instruction that performs computations on an input tensor to provide an output tensor used for artificial intelligence processing. For example, an instruction is configured to execute multiple functions that perform computations, and these functions use a summary indicator.

[0014] For example, a summary indicator is defined for a particular command, and another command uses a different summary indicator.

[0015] In one example, the value of a condition code is determined based on the execution of a specific command, and the effectiveness of the summary indicator is based on the determination that the value of the condition code is the selected value.

[0016] Computer implementation methods and systems relating to one or more embodiments are also described and claimed herein. Furthermore, services relating to one or more embodiments may also be described and claimed herein.

[0017] Further features and advantages are realized by the technology described herein. Other embodiments and aspects are described in detail herein and are considered to be part of the claimed embodiments. [Brief explanation of the drawing]

[0018] One or more embodiments are specifically cited as examples and expressly claimed in the claims at the end of this specification. The purposes, features, and advantages of the foregoing, as well as those of one or more embodiments, will become apparent from the following detailed description, made in conjunction with the accompanying drawings. [Figure 1A] This figure shows an example of a computing environment that incorporates one or more aspects of the present invention. [Figure 1B] This figure shows further details of the processor shown in Figure 1A, according to one or more embodiments of the present invention. [Figure 2A] This figure shows an example of processing related to the execution of a neural network processing assist instruction according to one or more aspects of the present invention. [Figure 2B] This figure shows an example of exception handling related to the execution of instructions, such as neural network processing assist instructions, according to one or more aspects of the present invention. [Figure 3A] This figure shows an example of a format for a neural network processing assist instruction according to one or more aspects of the present invention. [Figure 3B] This figure shows an example of a general-purpose register used by a neural network processing assist instruction according to one or more aspects of the present invention. [Figure 3C] This figure shows an example of function code supported by neural network processing assist instructions according to one or more aspects of the present invention. [Figure 3D] This figure shows an example of another general-purpose register used by a neural network processing assist instruction according to one or more aspects of the present invention. [Figure 3E] This figure shows an example of a parameter block used by a query function of a neural network processing assist instruction according to one or more aspects of the present invention. [Figure 3F] This figure shows an example of a parameter block used by one or more non-query functions of a neural network processing assist instruction according to one or more aspects of the present invention. [Figure 3G]A diagram showing an example of a tensor descriptor used by neural network processing assist instructions according to one or more aspects of the present invention. [Figure 4] A diagram showing an example of the format of a data type of neural network processing (NNP)-data type 1 according to one or more aspects of the present invention. [Figure 5] (A) to (C) are diagrams showing an example of an input data layout used by neural network processing assist instructions according to one or more aspects of the present invention. [Figure 6] (A) to (C) are diagrams showing output examples corresponding to the input data layouts of (A) to (C) in FIG. 5 according to one or more aspects of the present invention. [Figure 7A] A diagram showing an example of facilitating processing in a computing environment according to one or more aspects of the present invention. [Figure 7B] A diagram showing an example of facilitating processing in a computing environment according to one or more aspects of the present invention. [Figure 8A] A diagram showing another example of a computing environment incorporating and using one or more aspects of the present invention. [Figure 8B] A diagram showing an example of further details of the memory in FIG. 8A according to one or more aspects of the present invention. [Figure 8C] A diagram showing another example of further details of the memory in FIG. 8A according to one or more aspects of the present invention. [Figure 9A] A diagram showing yet another example of a computing environment incorporating and using one or more aspects of the present invention. [Figure 9B] A diagram showing further details of the memory in FIG. 9A according to one or more aspects of the present invention. [Figure 10] A diagram showing an embodiment of a cloud computing environment according to one or more aspects of the present invention. [Figure 11] A diagram showing an example of an abstraction model layer according to one or more aspects of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019] According to one or more aspects of the present invention, the ability to facilitate processing within a computing environment is provided. For example, a summary indicator is provided to indicate that an invalid value has been detected in input data to a computation or in output data obtained as a result of a computation. This value may be invalidated by one of several exceptions (e.g., a non-numeric value, a non-representable value, an out-of-range value, etc.), and one summary indicator represents multiple exceptions without distinguishing between them. In one example, the summary indicator is used by a selected instruction, while other instructions use different indicators. Furthermore, based on the fact that a selected instruction is configured to perform multiple functions that perform computations, the summary indicator is used by those functions. Thus, multiple functions use the same summary indicator.

[0020] In one example, the selected instruction is a neural network instruction, such as a Neural Network Processing Assist instruction, configured to execute multiple functions, including, for example, a query function and multiple non-query functions. Non-query functions include, for example, functions related to tensor computation. A Neural Network Processing Assist instruction is, for example, a single instruction that is part of an instruction set architecture (ISA) (e.g., a single architecturally defined hardware machine instruction in a hardware / software interface) and is processed on a general-purpose processor (e.g., at least partially decoded and / or executed). For example, the instruction is dispatched by a program on a general-purpose processor, and the program decodes and starts the instruction. The function specified by the instruction is executed by the general-purpose processor, or by a dedicated processor, such as a coprocessor configured for that specific function, which is coupled to or part of the general-purpose processor, or both. The instruction then completes on the general-purpose processor.

[0021] According to one or more aspects of the present invention, invalid values ​​(e.g., non-numeric, unrepresentable, or out-of-range values, or combinations thereof) are detected and summarized during the execution of a neural network processing assist instruction (or another instruction). This does not degrade the instruction or the system or its performance. For example, it is reported that an invalid value, such as a non-numeric, unrepresentable, or out-of-range value, is an input to a computation or was generated by the computation. This facilitates debugging of, for example, artificial intelligence models. In some examples, the invalid value is not precisely identified. However, in some examples, an indication can be provided or implied that which computation may have generated an invalid value.

[0022] An embodiment of a computing environment incorporating and using one or more aspects of the present invention will be described with reference to Figure 1A. As an example, the computing environment is based on the z / Architecture® instruction set architecture (manufactured by International Business Machines Corporation, Armonk, New York). One embodiment of the z / Architecture instruction set architecture is described in the publication entitled "z / Architecture Principles of Operation" (IBM publication number SA22-7832-12, 13th edition, September 2019), which is incorporated herein by reference in its entirety. However, the z / Architecture instruction set architecture is merely an example, and other architectures or other types of computing environments, or both, of International Business Machines Corporation and / or other entities may include, use, or both of, one or more aspects of the present invention. Note that z / Architecture and IBM are trademarks or registered trademarks of International Business Machines Corporation in at least one jurisdiction.

[0023] Referring to Figure 1A, the computing environment 100 includes, for example, a computer system 102. The computer system 102 is shown, for example, as a general-purpose computing device. The computer system 102 may include, but is not limited to, one or more general-purpose processors or processing units 104 (e.g., central processing units (CPUs)), at least one dedicated processor such as a neural network processor 105, memory 106 (also called, for example, system memory, main memory, main storage, central storage, or storage), and one or more input / output (I / O) interfaces 108, all coupled to each other via one or more buses or other connections, or both. For example, the processors 104, 105 and memory 106 are coupled to the I / O interface 108 via one or more buses 110, and the processors 104, 105 are coupled to each other via one or more buses 111.

[0024] Bus 111 is, for example, a memory or cache coherence bus. Bus 110 represents one or more of several types of bus structures, including, for example, a memory bus or memory controller, peripheral buses, accelerated graphics ports, and processor or local buses using any of the various bus architectures. Such architectures include, but are not limited to, the Industry Standard Architecture (ISA), Micro Channel Architecture (MCA), Enhanced ISA (EISA), Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI).

[0025] For example, one or more dedicated processors (e.g., neural network processors) may be separate from and coupled to one or more general-purpose processors, or incorporated into them, or both. Many variations are possible.

[0026] For example, memory 106 may include a cache 112, such as a shared cache, which can be connected to, for example, the local cache 114 of the processor 104 or a neural network processor 105 or both, via one or more buses 111. Furthermore, memory 106 may include one or more programs or applications 116 and at least one operating system 118. An example of an operating system is the z / OS® operating system (manufactured by International Business Machines Corporation, Armonk, New York). Note that z / OS is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction. Other operating systems provided by International Business Machines Corporation, other entities, or both may be used. Memory 106 may also include one or more computer-readable program instructions 120, which can be configured to perform functions of embodiments of the present invention.

[0027] Furthermore, in one or more embodiments, the memory 106 includes processor firmware 122. The processor firmware includes, for example, the processor's microcode or millicode. The processor firmware includes, for example, hardware-level instructions or data structures, or both, used in the implementation of higher-level machine code. In one embodiment, the processor firmware includes, for example, microcode or millicode containing trusted software, or proprietary code that controls the operating system's access to system hardware, which is typically provided as microcode or millicode specific to the underlying hardware.

[0028] The computer system 102 may communicate with one or more external devices 130, such as a user terminal, tape drive, pointing device, display, and one or more data storage devices 134, for example, via an I / O interface 108. The data storage device 134 may store one or more programs 136, one or more computer-readable program instructions 138, or data or a combination thereof. The computer-readable program instructions may be configured to perform functions of embodiments of the present invention.

[0029] The computer system 102 may also communicate with the network interface 132, for example, via the I / O interface 108. This allows the computer system 102 to communicate with one or more networks, such as a local area network (LAN), a general-purpose wide area network (WAN), or a public network (e.g., the Internet), or a combination thereof, and to communicate with other computing devices or systems.

[0030] The computer system 102 may include, or be combined with, removable / non-removable volatile / non-volatile computer system-readable media. For example, the computer system 102 may include, or be combined with, non-removable non-volatile magnetic media (commonly called "hard drives"), magnetic disk drives for reading and writing to removable non-volatile magnetic disks (e.g., "floppy disks"), or optical disk drives for reading and writing to removable non-volatile optical disks (such as CD-ROMs, DVD-ROMs, or other optical media), or a combination thereof. Other hardware components or software components, or both, may be used with the computer system 102. Examples, but not limited to, include microcode or millicode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0031] Computer system 102 may operate with many other general-purpose or dedicated computing system environments or configurations. Examples of well-known computing systems, environments, or configurations or combinations suitable for use with computer system 102 include, but are not limited to, personal computer (PC) systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of these systems or devices.

[0032] As an example, a processor (e.g., processor 104 or processor 105 or both) includes several functional components (or subsets thereof) used to execute instructions. As shown in Figure 1B, these functional components include, for example, an instruction fetch component 150 that fetches instructions to be executed, an instruction decode unit 152 that decodes the fetched instructions and obtains the operands of the decoded instructions, one or more instruction execution components 154 that execute the decoded instructions, a memory access component 156 that accesses memory for instruction execution as needed, and a write back component 158 ​​that provides the results of the executed instructions. One or more of the components may access, use, or both access one or more registers 160 in instruction processing. Furthermore, one or more of the components may include, or access, at least some of, one or more other components used to perform exception checking / reporting or neural network processing assist processing, or both, of neural network processing assist instructions (or other processing on which one or more aspects of the present invention can be used), for example, as described herein, according to one or more aspects of the present invention. These one or more other components include, for example, the exception checking / reporting component 170 and the neural network processing assist component 172 (or one or more other components or a combination thereof).

[0033] According to one or more aspects of the present invention, exception checking and reporting are performed during the execution of instructions, such as neural network processing assist instructions, but are not particularly limited. In one example, a neural network processing assist instruction is started on a general-purpose processor (e.g., processor 104), and the function specified by the instruction is executed on the general-purpose processor, a dedicated processor (e.g., neural network processor 105), or both, depending on the function. The instruction then completes on the general-purpose processor. In another example, the instruction is started, executed, and completed on one or more general-purpose processors or one or more dedicated processors. Other variations are also possible.

[0034] Further details regarding the execution of neural network processing assist instructions will be explained with reference to Figure 2A. Referring to Figure 2A, in one example, in step 200, the neural network processing assist instruction is acquired and decoded by a processor such as a general-purpose processor (e.g., processor 104). The decoded instruction is issued in step 210, for example, on the general-purpose processor. In step 220, the function to be executed is determined. In one example, this determination is made by checking the function code field of the instruction. An example of a function code field will be described later. Then, in step 230, the function is executed.

[0035] In one embodiment of function execution, step 232 determines whether the function is executed on a dedicated processor (e.g., neural network processor 105). For example, in one example, query functions of neural network processing assist instructions are executed on the general-purpose processor, and non-query functions are executed on the dedicated processor. However, other variations are possible. If the function is not executed on the dedicated processor (e.g., the function is a query function, or in another example, one or more selected functions), in one example, the function is executed on the general-purpose processor in step 234. On the other hand, if the function is executed on the dedicated processor (e.g., the function is a non-query function, or in another example, one or more selected functions), in step 236, information for use when executing the function is provided from the general-purpose processor to the dedicated processor, such as memory address information related to tensor data used in neural network computations. The dedicated processor receives this information and executes the function in step 238. Once the function has finished executing, in step 240, processing returns to the general-purpose processor, which completes the instruction in step 250 (in other examples, the instruction may start, execute, and complete on one or more general-purpose processors or one or more dedicated processors; other variations are possible).

[0036] According to one or more aspects of the present invention, during the execution of each non-query function (or a subset thereof), one or more checks are performed for invalid values, such as non-numeric, unrepresentable, or out-of-range values ​​or combinations thereof, as described with reference to Figure 2B, and based on the detection of such invalid values, a display of such invalid values ​​is reported. As an example, the process in Figure 2B is performed by a dedicated processor such as a neural network processor 105. However, in other examples, a general-purpose processor or other processor may perform this process.

[0037] Referring to Figure 2B, in one example, step 260 reads the input data (e.g., input tensor data) for the function executed by the instruction. Step 262 checks the input data for one or more invalid values, such as one or more non-numeric values, one or more unrepresentable values, one or more out-of-range values, one or more other exceptional values, or a combination thereof. Additional exceptions, fewer exceptions, other exceptions, or a combination thereof may also be checked. Examples of unrepresentable values ​​include infinity, division by zero, not-a-number, and a combination of infinity and a not-a-numeric value (NINF). If invalid values ​​are detected in step 264, step 266 provides a display of the invalid values.

[0038] In one example, the indication of an invalid value is sent back to the general-purpose processor that initiated the instruction, which sets a summary indicator to, for example, 1, indicating an invalid value. The summary indicator does not distinguish what type of exception caused the detection of an invalid value. The setting of the summary indicator is independent of the type of exception (e.g., non-numeric, one of the unrepresentable values, out-of-range value, etc.). However, in one embodiment, since the summary indicator is set and processing is returned to the general-purpose processor based on the detection of an invalid value, the calculation that resulted in the detection of an invalid value is known or easily identifiable.

[0039] As a concrete example, a summary indicator is a range indicator (e.g., a bit) within an exception flag, placed in a selected location accessible by the instruction that initiated the function performing the calculation in which the invalid value was detected. For example, the selected location could be a general-purpose register (e.g., general-purpose register 0) or a memory location used by the instruction. Various locations are possible. Examples of exception flags, particularly range indicators, are shown below.

[0040] Returning to the query in step 264, if no invalid values ​​are found in the input data, one or more calculations based on the function specified by the instruction are performed and output data is generated. In one example, step 270 checks whether there are one or more invalid values ​​in the output data. In step 272, if invalid values ​​are detected, step 266 provides a display of the invalid values. In one example, this display is provided to a general-purpose processor, which sets a summary indicator as described herein. However, in another example, this summary indicator may be a different indicator, which may be the same as or different from the indicator used for invalid input values.

[0041] Return to the query in step 272, and if no invalid values ​​are found, this exception handling is complete.

[0042] As described herein, when performing calculations, it is detected whether the values ​​of input or output data are invalid, such as non-numeric, unrepresentable numeric, or out-of-range values ​​or combinations thereof. Based on the detection of invalid values, a summary indicator (e.g., a range exception flag) is set. In one example, no indication is provided as to which data element provided the invalid value. This allows for higher performance in arithmetic operations.

[0043] An example of an instruction that uses a summary indicator is a neural network processing assist instruction. In one example, the summary indicator is used exclusively by the neural network processing assist instruction and its functions / operations. Other instructions may use other indicators. Further details regarding the neural network processing assist instruction and the functions supported by it are described herein with reference to Figures 3A-3G. In the descriptions of instructions, functions of instructions, or both herein, specific locations, fields, or specific sizes or combinations of fields are indicated (e.g., specific bytes or bits or both). However, other locations, fields, or sizes or combinations may be provided. Furthermore, it may be specified that a bit be set to a specific value, such as 1 or 0, but this is only an example. In other examples, where a bit is set, it may be set to a different value, such as the opposite value or another value. Many variations are possible.

[0044] In one example, referring to Figure 3A, the neural network processing assist instruction 300 has an RRE format that indicates registers and register operations by extended operation codes (opcodes). As shown in Figure 3A, in one example, the neural network processing assist instruction 300 includes an operation code (opcode) field 302 (e.g., bits 0-15) that indicates a neural network processing assist operation. In one example, bits 16-31 of the instruction are reserved and include 0.

[0045] In one example, an instruction uses several general-purpose registers implicitly specified by the instruction. For example, the neural network processing assist instruction 300 uses the implicit registers general-purpose register 0 and general-purpose register 1. These examples are illustrated with reference to Figures 3B and 3D, respectively.

[0046] Referring to Figure 3B, in one example, general-purpose register 0 includes a function code field and a status field that may be updated upon instruction completion. In one example, general-purpose register 0 includes a response code field 310 (e.g., bits 0-15), an exception flag field 312 (e.g., bits 24-31), and a function code field 314 (e.g., bits 56-63). Furthermore, in one example, bits 16-23 and 32-55 of general-purpose register 0 are reserved and contain 0. One or more fields are used by specific functions executed by the instruction. In one example, not all fields are used by all functions. Each field is described below.

[0047] Response Code (RC) 310: This field (e.g., bit positions 0-15) contains the response code. When the execution of a neural network processing assist instruction is completed with a condition code of, for example, "1", the response code is stored. If an invalid input condition is encountered, a non-zero value indicating the cause of the invalid input condition recognized during execution is stored in the response code field, and the selected condition code, for example, "1", is set. The codes stored in the response code field are defined as follows, for example:

[0048] Response code meaning

[0049] 0001 The parameter block format specified by the parameter block version number is not supported by the model.

[0050] 0002 The specified function is either not defined or not installed on the machine.

[0051] 0010 The specified tensor data layout format is not supported.

[0052] 0011 The specified tensor data type is not supported.

[0053] 0012 The specified single tensor dimension is greater than the maximum dimension index size.

[0054] 0013 The size of the specified tensor is greater than the maximum tensor size.

[0055] 0014 The address of the specified tensor is not aligned to a 4K-byte boundary.

[0056] 0015 The function-specific save area address is not aligned to a 4K byte boundary.

[0057] F000-FFFF are function-specific response codes. These responses are defined for a particular function.

[0058] Exception Flags (EF) 312: This field (e.g., bit positions 24-31) contains exception flags. If an exception condition is detected during instruction execution, the corresponding exception flag control (e.g., bit) is set to 1, for example; otherwise, the control remains unchanged. The exception flags field is initialized to 0 before the first call of the instruction. The spare flags are not changed during instruction execution. The flags stored in the exception flags field are defined as follows, for example:

[0059] EF (bit) meaning

[0060] 0 Range violation. This flag is set when an invalid value is detected in the input tensor or stored in the output tensor. This flag is valid, for example, only when an instruction completes with a condition code of "0". In one example, this flag is the summary indicator described herein.

[0061] 1-7 Spare

[0062] Function Code (FC) 314: This field (e.g., bit positions 56-63) contains the function code. An example of a function code assigned to a neural network processing assist instruction is shown in Figure 3C. All other function codes are unassigned. If an unassigned or uninstalled function code is specified, a response code of, for example, hexadecimal "0002" and a selection condition code of, for example, "1" are set. This field is not changed during execution.

[0063] As shown here, the neural network processing assist instruction uses general-purpose register 1 in addition to general-purpose register 0. An example of general-purpose register 1 is shown in Figure 3D. For example, bits 40-63 in 24-bit addressing mode, bits 33-63 in 31-bit addressing mode, or bits 0-63 in 64-bit addressing mode contain the address of parameter block 320. The contents of general-purpose register 1 specify, for example, the logical address of the leftmost byte of the parameter block in storage. The parameter block is specified by a doubleword boundary; otherwise, a specification exception is recognized. For all functions, the contents of general-purpose register 1 are not changed.

[0064] In access register mode, access register 1 specifies an address space that includes, for example, a parameter block, an input tensor, an output tensor, and a function-specific save area.

[0065] In one example, the parameter block may have different formats depending on the function specified by the instruction being executed. For instance, a query function might have a parameter block of one format, while other functions within an instruction might have a parameter block of a different format. In another example, all functions might use the same parameter block format. Other variations are also possible.

[0066] For example, a parameter block, or the information within a parameter block, or both, can be stored in memory, hardware registers, or a combination of memory and / or registers, or a combination thereof. Other examples are also possible.

[0067] An example of a parameter block used by query functions such as the NNPA-QAF (Query Available Functions) operation will be explained with reference to Figure 3E. As shown in the figure, in one example, the NNPA-QAF parameter block 330 includes, for example, the following:

[0068] Installed Functions Vector 332: This field in the parameter block (e.g., bytes 0-31) contains a vector of installed functions. In one example, bits 0-255 of the installed functions vector correspond to function codes 0-255 of the neural network processing assist instructions, respectively. If a bit is, for example, 1, the corresponding function is installed; otherwise, the function is not installed.

[0069] Installed parameter block formats vector 334: This field of the parameter block (e.g., bytes 32-47) contains a vector of installed parameter block formats. In one example, bits 0-127 of the installed parameter block formats vector correspond to parameter block formats 0-127 of the non-query functions of the neural network processing assist instruction. If a bit is, for example, 1, the corresponding parameter block format is installed; otherwise, the parameter block format is not installed.

[0070] Installed Data Types 336: This field in the parameter block (e.g., bytes 48-49) contains a vector of installed data types. In one example, bits 0-15 of the installed data types vector correspond to the data types to be installed. If a bit is, for example, 1, the corresponding data type is installed; otherwise, the data type is not installed. Examples of data types include the following (additional data types, fewer data types, other data types, or combinations thereof are also possible):

[0071] Bit Data Type

[0072] 0 NNP-Data Type 1

[0073] 1-15 spare

[0074] Installed Data Layout Format 338: This field in the parameter block (e.g., bytes 52-55) contains a vector of installed data layout formats. In one example, bits 0-31 of the installed data layout format vector correspond to the data layout formats to be installed. If a bit is, for example, 1, the corresponding data layout format will be installed; otherwise, the data layout format will not be installed. Examples of data layout formats include the following (additional data layout formats, fewer data layout formats, other data layout formats, or combinations thereof are also possible):

[0075] Bit Data Layout Format

[0076] 0 4-dimensional feature tensor

[0077] 1. 4-dimensional kernel tensor (4D-kernel tensor)

[0078] 2-31 Reserve

[0079] Maximum Dimension Index Size 340: This field in the parameter block (e.g., bytes 60-63) contains a 32-bit unsigned binary integer specifying, for example, the maximum number of elements within a specified dimensional index size for any given tensor. In another example, the maximum dimensional index size specifies the maximum number of bytes within a specified dimensional index size for any given tensor. Other examples are possible.

[0080] Maximum Tensor Size 342: This field in the parameter block (e.g., bytes 64-71) contains a 32-bit unsigned binary integer specifying the maximum number of bytes for any given tensor, including, for example, pad bytes required by the tensor format. In another example, the maximum tensor size specifies the maximum total number of elements for any given tensor, including any padding required by the tensor format. Other examples are possible.

[0081] Installed NNP-DataType1 Conversion Vector 344: This field in the parameter block (e.g., bytes 72-73) contains the vector of installed NNP-DataType1 conversions. In one example, bits 0-15 of the installed NNP-DataType1 conversion vector correspond to the conversion of the installed data type to / from the NNP-DataType1 format. If a bit is 1, the corresponding conversion is installed; otherwise, the conversion is not installed. Additional conversions, fewer conversions, other conversions, or a combination thereof may be specified.

[0082] Bit Data Type

[0083] 0 Reserve

[0084] 1. BFP Tiny Format

[0085] 2 BFP Short Format

[0086] 3-15 spare

[0087] An example of a query function parameter block is illustrated with Figure 3E, but other formats of query function parameter blocks, including NNPA-QAF operations, may be used. The format may depend, in one example, on the type of query function being executed. Furthermore, the parameter block, or each field of the parameter block, or both, may contain additional information, less information, other information, or a combination thereof.

[0088] In addition to parameter blocks for query functions, there are parameter block formats for non-query functions, such as the non-query functions of neural network processing assist instructions, for example. An example of a parameter block used by non-query functions, such as the non-query functions of neural network processing assist instructions, is explained with reference to Figure 3F.

[0089] As illustrated, in one example, the parameter block 350 adopted by, for example, the non-query function of a neural network processing assist instruction includes, for example, the following:

[0090] Parameter Block Version Number 352: This field of the parameter block (e.g., bytes 0-1) specifies the version and size of the parameter block. In one example, bits 0-8 of the parameter block version number are spare and can contain 0, while bits 9-15 of the parameter block version number contain an unsigned binary integer that specifies the format of the parameter block. The query function provides a mechanism to indicate available parameter block formats. If the specified parameter block size or format is not supported by the model, a response code, for example, hexadecimal "0001", is stored in general register 0, and the instruction is completed by setting a condition code (e.g., condition code "1"). The parameter block version number is specified by the program and is not changed during instruction execution.

[0091] Model version number 354: This field in the parameter block (e.g., byte 2) is an unsigned binary integer that identifies the model that executed the instruction (e.g., a specific non-query function). If the continuation flag (described below) is 1, the model version number can be input to the operation for the purpose of resuming the operation by interpreting the contents of the continuation state buffer field (described below) of the parameter block.

[0092] Continuation Flag 356: This field in the parameter block (e.g., bit 63) indicates, for example, that the operation is partially complete and can be resumed using the contents of the continuation state buffer, when it is set to 1. The program initializes the continuation flag to 0 and does not change it when the instruction is re-executed for the purpose of resuming the operation. Otherwise, the result is unpredictable.

[0093] If the continuation flag is set at the beginning of the operation, and the contents of the parameter block have changed since the first call, the result will be unpredictable.

[0094] Function-specific save area address 358: This field in the parameter block (e.g., bytes 56-63) contains the logical address of the function-specific save area. In one example, the function-specific save area address is aligned to a 4K byte boundary. Otherwise, the general-purpose register 0 is set to a response code, for example, hexadecimal "0015", and the instruction completes with a condition code, for example, "1". The address follows the current addressing mode. The size of the function-specific save area depends on the function code.

[0095] If the entire function-specific save area overlaps with the program event recording (PER) storage area designation, a PER storage change event will be recognized for the function-specific save area, if applicable. If only a portion of the function-specific save area overlaps with the PER storage area designation, which of the following occurs depends on the model.

[0096] * For the entire function-specific save area, PER storage change events are recognized where applicable.

[0097] * For the portion of the function-specific save area that is stored, PER storage change events will be recognized if applicable.

[0098] If the entire parameter block overlaps with the PER storage area specification, a PER storage change event will be recognized for the parameter block, if applicable. If only a portion of the parameter block overlaps with the PER storage area specification, which of the following occurs depends on the model.

[0099] * For the entire parameter block, PER storage change events are recognized if applicable.

[0100] * For the stored portion of the parameter block, PER storage change events are recognized, if applicable.

[0101] For parameter blocks, a PER zero-address detection event is recognized where applicable. Zero-address detection does not apply to tensor addresses or function-specific save region addresses, for example.

[0102] Output tensor descriptor (e.g., 1-2) 360 / Input tensor descriptor (e.g., 1-3) 365: An example of a tensor descriptor is explained with reference to Figure 3G. In this example, tensor descriptors 360 and 365 include the following:

[0103] Data layout format 382: This field of the tensor descriptor (e.g., byte 0) specifies the data layout format. Valid data layout formats include, for example, the following (additional data layout formats, fewer data layout formats, other data layout formats, or combinations thereof are also possible).

[0104] Format Description Alignment (bytes)

[0105] 0 4-dimensional feature tensor 4096

[0106] 1. 4-dimensional kernel tensor 4096

[0107] 2-255 Reserve -

[0108] If an unsupported or alternative data layout format is specified, for example, a hexadecimal response code of "0010" is stored in general-purpose register 0, and the instruction is completed by setting a condition code of "1", for example.

[0109] Data type 384: This field (e.g., byte 1) specifies the data type of the tensor. Examples of supported data types are shown below (additional data types, fewer data types, other data types, or combinations thereof are also possible).

[0110] Value, Data Type, Data Size (bits)

[0111] 0 NNP Data Type-1 16

[0112] 1-255 Reserve -

[0113] If an unsupported or alternative data type is specified, for example, a hexadecimal response code of "0011" is stored in general-purpose register 0, and the instruction is completed by setting a condition code of "1", for example.

[0114] Dimensions 1-4 Index Size 386: Collectively, dimension index sizes 1-4 specify the shape of the 4-dimensional tensor. Each dimension index size is greater than 0 and less than or equal to the maximum dimension index size (340, Figure 3E). Otherwise, for example, a hexadecimal response code "0012" is stored in general-purpose register 0, and the instruction is completed by setting a condition code of "1". The total tensor size is less than or equal to the maximum tensor size (342, Figure 3E). Otherwise, for example, a hexadecimal response code "0013" is stored in general-purpose register 0, and the instruction is completed by setting a condition code of "1".

[0115] In one example, to determine the number of bytes (i.e., total tensor size) of a 4D feature tensor with elements of NNPA-data type 1, the following is used: Dimension index 4 * Dimension index 3 * ceil(Dimension index 2 / 32) * 32 * ceil(Dimension index 1 / 64) * 64 * 2.

[0116] Tensor address 388: This field of the tensor descriptor (e.g., bytes 24-31) contains the logical address of the leftmost byte of the tensor. The address follows the current addressing mode.

[0117] If the address is not aligned to the boundary of the associated data layout format, for example, a response code of hexadecimal "0014" is stored in general-purpose register 0, and the instruction is completed by setting a condition code of "1", for example.

[0118] In access register mode, access register 1 specifies the address space that contains all active input / output tensors in storage.

[0119] Returning to Figure 3F, the parameter block 350 further includes, in one example, function-specific parameters 1 to 5 (370) that are available by a particular function, as described herein.

[0120] Furthermore, the parameter block 350, in one example, includes a continuation state buffer field 375. This field contains data (or the location of data) used when the operation of this instruction is resumed.

[0121] As input to the calculation, the reserve field of the parameter block must contain 0. Once the calculation is complete, the reserve field may be stored as 0 or may not be changed.

[0122] An example of a parameter block for a non-query function is illustrated with reference to Figure 3F, but other formats of parameter blocks for non-query functions, including non-query functions for neural network processing assist instructions, may be used. The format may depend on the type of function being executed, in one example. Furthermore, an example of a tensor descriptor is illustrated with reference to Figure 3G, but other formats may be used. In addition, different formats may be used for input and output tensors. Other variations are also possible.

[0123] Further details regarding the various functions supported by one embodiment of the neural network processing assist instruction are described below.

[0124] Function code 0: NNPA-QAF (Available Function Inquiry)

[0125] The Neural Network Processing Assist (NNPA) query function provides a mechanism to display selected information, such as the availability of installed functions, installed parameter block format, installed data type, installed data layout format, maximum dimension index size, and maximum tensor size. Once the information is retrieved, it is placed in a selected location, such as a parameter block (e.g., parameter block 330). After the operation is complete, the parameter block's reserve fields may be stored as 0 or may remain unchanged.

[0126] In the execution of one embodiment of the query function, a processor such as a general-purpose processor 104 obtains information about a specific processor, such as a specific model of a neural network processor such as a neural network processor 105. A specific model of a processor or machine has specific capabilities. Another model of a processor or machine may have additional capabilities, fewer capabilities, or different capabilities or combinations thereof, and / or may be a different generation (e.g., current or future generation) having additional capabilities, fewer capabilities, or different capabilities or combinations thereof. The obtained information is placed in a parameter block (e.g., parameter block 330) or in another structure that is accessible to or used with one or more applications that may use this information in further processing, or both. In one example, the parameter block or the information in the parameter block, or both, is held in memory. In other embodiments, the parameter block or the information, or both, may be held in one or more hardware registers. As another example, the query function may be a privileged operation performed by the operating system that makes available an application programming interface for making this information available to applications or non-privileged programs. In yet another example, the query function is executed by a dedicated processor, such as the neural network processor 105. Other variations are also possible.

[0127] The information is obtained, for example, by the firmware of the processor executing the query function. The firmware has knowledge of the attributes of a particular model of a particular processor (e.g., a neural network processor). This information may be stored, for example, in control blocks, registers, or memory or a combination thereof, and / or may be accessible to the processor executing the query function.

[0128] The information retrieved includes model-dependent details about at least one data attribute of a specific processor, including, for example, one or more installed or supported data types, one or more installed or supported data layout formats, or one or more installed or supported data sizes, or a combination thereof, for a selected model of a particular processor. This information is model-dependent in that other models (e.g., previous models, future models, or both) may not support the same data attributes, such as the same data types, data sizes, or data layout formats, or combinations thereof. When the execution of a query function (e.g., the NNPA-QAF function) is complete, condition code 0 is set, for example. Condition codes 1, 2, and 3 are not applied to the query function, for example. Further details about the information retrieved are described below.

[0129] As shown here, in one example, the information obtained includes model-dependent information about one or more data attributes of a particular model of a neural network processor. An example of a data attribute is the installed data type of the neural network processor. For example, a particular model of a neural network processor (or other processor) may support one or more data types, such as NNP-DataType1 data type (also called Neural Network Processing-DataType1 data type) or other data or both. The NNP-DataType1 data type is a 16-bit floating-point format that offers many advantages, for example, for training and inference operations in deep learning. These advantages include, for example, maintaining the precision of deep learning networks, eliminating subnormal formats and simplifying the handling of rounding mode and corner cases, automatically rounding to the nearest value in arithmetic operations, and combining the special entities of infinity and NaN (not-a-number) into a single value (NINF) that can be accepted and handled in arithmetic operations. NINF provides better defaults for exponential overflows and invalid operations (such as division by zero). This allows many programs to continue running without hiding such errors or using dedicated exception handlers. Other model-dependent data types are also supported.

[0130] An example of the format for the NNP-DataType 1 data type is shown in Figure 4. As shown in the figure, in one example, NNP-DataType 1 data can be represented in format 400, for example. Format 400 includes, for example, the sign 402 (e.g., bit 0), the exponent +31 (404) (e.g., bits 1-6), and the decimal 406 (e.g., bits 7-15).

[0131] Examples of the format characteristics of NNP-DataType 1 are shown below.

[0132] Characteristics NNP-Data Type 1

[0133] Format length (bits): 16 bits

[0134] Biased exponential length (bits): 6 bits

[0135] 9 bits

[0136] Precision (p): 10 bits

[0137] Maximum left-units-view index (Emax) 32

[0138] Minimum left unit exponent (Emin): -31

[0139] Left-hand unit display (LUV) bias 31

[0140] TIFF0007845804000001.tif6152

[0141] TIFF0007845804000002.tif6155

[0142] Dmin -

[0143] TIFF0007845804000003.tif12167

[0144] Further details regarding the NNP-DataType 1 data type are described below.

[0145] Biased exponent: The bias used to allow the exponent to be represented as an unsigned number is shown above. Biased exponents are similar to the properties of binary floating-point formats, except that no special meaning is assigned to biased exponents of all 0s and all 1s, as will be discussed later with reference to the NNP-DataType 1 data type class.

[0146] Mantissa (significant): For NNP-data type 1 numbers, the binary point is considered to be to the left of the leftmost fractional bit. To the left of the binary point is an implicit unit bit, which is considered 1 for normal numbers and 0 for zero. The fractional part with the implicit unit bit appended to the left becomes the mantissa of the number.

[0147] The typical NNP-datatype 1 value is obtained by multiplying the mantissa by base 2 and raising the result to an unbiased exponential power.

[0148] Non-zero numbers: The values ​​of non-zero numbers are shown below.

[0149] Numeric class value

[0150] Normal numbers ±2 e-31 x (1.f)

[0151] Here, e is a biased exponent expressed in decimal, and f is a decimal number expressed in binary.

[0152] In one embodiment, NNP-DataType 1 data has three classes, including numeric entities and associated non-numeric entities. Each data item includes a sign, exponent, and mantissa. The exponent is biased such that all biased exponents are non-negative unsigned numbers, with the smallest biased exponent being 0. The mantissa includes an explicit fractional part and an implicit unit bit to the left of the binary point. The sign bit is 0 for positive and 1 for negative.

[0153] All allowed non-zero finite numbers have a unique NNP-datatype 1 representation. There are no subnormal numbers that allow multiple representations for the same value, and there are no subnormal arithmetic operations. These three classes include, for example:

[0154] Data class Sign Biased exponent Unit bit * Fractional part

[0155] Zero ± 0 0 0

[0156] Normal numbers ± 0 1 Non-zero

[0157] Normal numbers ± non-zero, not all ones 1 Any value

[0158] Normal numbers ± all 1s - not all 1s

[0159] NINF ± All 1 - All 1

[0160] Here, "-" indicates that it does not apply, "*" indicates an implicit unit bit, and NINF indicates that it is not a number or infinity.

[0161] Further details about each class are provided below.

[0162] Zero: Zero has a zero-biased exponent and a zero-fractional part. The implicit unit bit is zero.

[0163] Normalized numbers: Normalized numbers can have biased exponents of any value. If the biased exponent is 0, the fractional part is non-zero. If all biased exponents are 1, the fractional part will not be all 1. Other biased exponent values ​​can be any decimal values. The implicit unit bit is 1 for all normalized numbers.

[0164] NINF: NINF is represented by a biased exponent of all ones and a fractional part of all ones. NINF represents a value that is not within the range of representable values ​​of NNP-data type 1 (i.e., a 16-bit floating-point number designed for deep learning with 6 exponent bits and 9 fractional bits). Typically, NINF is only propagated during the operation and remains displayed even after the operation is complete.

[0165] In one example, NNP-data type 1 is supported, but other proprietary or non-standard data types, and one or more standard data types (e.g., IEEE 754 short precision, 16-bit binary floating-point, IEEE half-precision floating-point, 8-bit floating-point, 4-bit integer format, or 8-bit integer format or a combination thereof) may also be supported. These data formats have different properties for neural network processing. For example, smaller data types (e.g., fewer bits) can be processed faster and use less cache / memory. Larger data types result in higher precision in the neural network. Supported data types may have one or more allocated bits in the query parameter block (e.g., in the installed data type field 336 of parameter block 330). For example, proprietary or non-standard data types supported by a particular processor are indicated in the installed data type field, while standard data types are not. In other embodiments, one or more standard data types are also indicated. Other variations are also possible.

[0166] In one specific example, bit 0 of the installed data type field 336 is reserved for the data type NNP-data type 1, and when set to, for example, 1, it indicates that the processor supports NNP-data type 1. For example, the bit vector of installed data types is configured to represent up to 16 data types, with bits assigned to each data type. However, in other embodiments, the bit vector may support more or fewer data types. Furthermore, the vector may be configured such that one or more bits are assigned to a single data type. Many examples are possible, and / or additional data types, fewer data types, or other data types, or combinations thereof, may be supported, represented, or both in the vector.

[0167] In one example, a query function retrieves a representation of data types installed on a model-dependent processor and places this representation in a parameter block, for example, by setting one or more bits in the installed data type field 336 of the parameter block 330. Furthermore, in one example, a query function retrieves a representation of an installed data layout format (another data attribute) and places this information in a parameter block, for example, by setting one or more bits in the installed data layout format field 338. Examples of data layout formats include, for example, a 4D-feature tensor layout and a 4D-kernel tensor layout. The 4D-feature tensor layout is used, in one example, by a function described herein, and the convolution function uses the 4D-kernel tensor layout. These data layout formats arrange data in storage for tensors in a way that improves processing efficiency when executing functions of neural network processing assist instructions. For example, a neural network processing assist instruction uses input tensors provided in a specific data layout format to operate efficiently. While example layouts are provided, additional layouts, fewer layouts, other layouts, or combinations thereof may be provided for the functions described herein, other functions, or both.

[0168] The use or availability of a layout for a particular processor model is provided by a vector of the installed data layout format (for example, field 338 of parameter block 330). This vector is, for example, a bit vector of the installed data layout format, which allows the CPU to tell the application which layouts are supported. For example, bit 0 is reserved for the 4D feature tensor layout, and when set to, for example, 1, it indicates that the processor supports the 4D feature tensor layout. Bit 1 is reserved for the 4D kernel tensor layout, and when set to, for example, 1, it indicates that the processor supports the 4D kernel tensor layout. In one example, the bit vector of the installed data layout format is configured to represent up to 16 data layouts, with a bit assigned to each data layout. However, in other embodiments, the bit vector may support more or fewer data layouts. Furthermore, the vector may be configured so that one or more bits are assigned to data layouts. Many examples are possible. Details of the 4D feature tensor layout and the 4D kernel tensor layout are described later. Again, other layouts may be used now or in the future to optimize performance.

[0169] In one example, neural network processing assist instructions operate on a four-dimensional tensor, i.e., a tensor having four dimensions. These four-dimensional tensors are obtained from the general input tensors described herein, for example, in a row-major manner. That is, when enumerating the tensor elements in increasing order of memory addresses, the internal dimension called E1 is first stepped up from 0 to an E1 index size of 1, then the index of the E2 dimension is incremented, and the stepping of the E1 dimension is repeated. Finally, the index of the external dimension called E4 is incremented.

[0170] Tensors with lower dimensions (e.g., 3-dimensional or 1-dimensional tensors) are represented as 4-dimensional tensors, where one or more dimensions of the 4-dimensional tensor that exceed the dimensions of the original tensor are set to 1.

[0171] This document describes the conversion of a row-first general-purpose 4-dimensional tensor having dimensions E4, E3, E2, and E1 to a 4-dimensional feature tensor layout (also referred to herein as the NNPA data layout format 0-4 dimensional feature tensor).

[0172] The resulting tensor can be represented, for example, as a 4-dimensional tensor with 64 vector elements, or as a 5-dimensional tensor with the following dimensions.

[0173] TIFF0007845804000004.tif12167

[0174] The elements [e4][e3][e2][e1] of the generic tensor are mapped to the following elements of the resulting 5-dimensional tensor:

[0175] TIFF0007845804000005.tif25168

[0176] The resulting tensor may be larger than the generic tensor. Elements of the resulting tensor that do not correspond to elements in the generic tensor are called pad elements.

[0177] The elements [fe4][fe1][fe3][fe2][fe0] of the 0-4 dimensional feature tensor in the NNPA data layout format for 64-element vectors, or their equivalent representations, are considered as a 5-dimensional tensor of elements. These elements are either padded elements or corresponding elements of a general-purpose 4-dimensional tensor with dimensions E4, E3, E2, and E1, which can be determined by the following formula.

[0178] If fe2 ≥ E2, it is an E2 (or page) pad element.

[0179] Otherwise, if fe1 * 64 + fe0 ≥ E1, then E1 (or row) is a padded element.

[0180] Otherwise, the corresponding elements of the general-purpose 4-dimensional tensor are as follows:

[0181] [fe4][fe3][fe2][fe1 * 64 + fe0]

[0182] In the case of convolutional neural network-based artificial intelligence models, the four-dimensional meaning of the feature tensor can generally be mapped as follows:

[0183] E4: N - Mini-batch size

[0184] E3: H-3D tensor / image height

[0185] E2:W - 3D tensor / image width

[0186] E1:C - Channels or classes of 3-dimensional tensors

[0187] In the case of machine learning or recurrent neural network-based artificial intelligence models, the four-dimensional meaning of a four-dimensional feature tensor can generally be mapped as follows:

[0188] E4: T - Number of time steps or models

[0189] E3: Reserve. Normally set to 1.

[0190] E2:N mb - Mini badge size

[0191] E1:L-Features

[0192] The NNPA Data Layout Format 0 provides, for example, two-dimensional data locality for 4k-byte data blocks (pages) and 4k-byte block data alignment with respect to the external dimension of the generating tensor.

[0193] Pad element bytes are ignored in the input tensor and are unpredictable in the output tensor. Changes to PER storage on pad bytes are unpredictable.

[0194] An example of the input data layout for a 4-dimensional feature tensor layout with dimensions E1, E2, E3, and E4 is shown in Figures 5(A) to (C), and an example of the output for a 4-dimensional feature tensor layout is shown in Figures 6(A) to (C). Referring to Figure 5(A), a 3-dimensional tensor 500 with dimensions E1, E2, and E3 is shown. In this example, each 3-dimensional tensor contains multiple 2-dimensional tensors 502. The numbers within each 2-dimensional tensor 502 describe the memory offset where each element is located in memory. The input is used to lay out the data of the original tensor in memory (for example, the original 4-dimensional tensor in Figures 5(A) to (C)), as shown in Figures 6(A) to (C), which correspond to Figures 5(A) to (C).

[0195] In Figure 6(A), as an example, each memory unit (e.g., memory page) contains a pre-selected number (e.g., 32) rows 602, each identified by, for example, e2_page_idx. Each row has a pre-selected number (e.g., 64) elements 604, each identified by, for example, e1_page_idx. If a row does not contain the pre-selected number of elements, padding 606 called row padding or E1 padding is performed. If a memory unit does not have the pre-selected number of rows, padding 608 called page padding or E2 padding is performed. For example, row padding is, for example, 0 or other values, and page padding is, for example, an existing value, 0, or other values.

[0196] In one example, the output elements of a row are provided in memory (e.g., within a page) based on the element position in the E1 direction of its corresponding input. For example, referring to Figure 5(A), the element positions 0, 1, and 2 of the three matrices shown (e.g., the element positions at the same position in each matrix) are shown in row 0 of page 0 in Figure 6(A), and so on. In this example, the 4-dimensional tensor is small, and all the elements of each 2-dimensional tensor representing the 4-dimensional tensor fit on one page. However, this is just an example. A 2-dimensional tensor may contain one or more pages. If a 2-dimensional tensor is created based on reformatting a 4-dimensional tensor, the number of pages in the 2-dimensional tensor is based on the size of the 4-dimensional tensor. In one example, one or more ceil functions are used to determine the number of rows in the 2-dimensional tensor and the number of elements in each row, indicating how many pages are used. Other variations are possible.

[0197] In one example, in addition to the 4D feature tensor layout, the neural network processor may support a 4D kernel tensor, which rearranges the elements of the 4D tensor to reduce the number of memory access and data acquisition steps when performing certain artificial intelligence (e.g., neural network processing assist) operations such as convolution. In one example, a row-first general-purpose 4D tensor with dimensions E4, E3, E2, E1 is converted to an NNPA data layout format 1-4D kernel tensor (4D kernel tensor) as described herein.

[0198] The resulting tensor can be represented, for example, as a 4-dimensional tensor with 64 elements, or as a 5-dimensional tensor with the following dimensions.

[0199] TIFF0007845804000006.tif13167

[0200] The elements [e4][e3][e2][e1] of the generic tensor are mapped to the following elements of the resulting 5-dimensional tensor:

[0201] TIFF0007845804000007.tif25167

[0202] The resulting tensor may be larger than the generic tensor. Elements of the resulting tensor that do not correspond to elements in the generic tensor are called padded elements.

[0203] The elements [fe4][fe1][fe3][fe2][fe0] of the 64-element vector NNPA data layout format 1-4 dimensional feature tensor, or their equivalent representation, are considered as a 5-dimensional tensor of elements. These elements are either padded elements or corresponding elements of a general-purpose 4-dimensional tensor with dimensions E4, E3, E2, and E1, which can be determined by the following formula.

[0204] If fe2 ≥ E2, it is an E2 (or page) pad element.

[0205] Otherwise, if fe1 * 64 + fe0 ≥ E1, then E1 (or row) is a padded element.

[0206] Otherwise, the corresponding elements of the general-purpose 4-dimensional tensor are as follows:

[0207] [fe4][fe3][fe2][fe1 * 64 + fe0]

[0208] In the case of convolutional neural network-based artificial intelligence models, the four-dimensional meaning of the kernel tensor can generally be mapped as follows:

[0209] E4:H-3D tensor / image height

[0210] E3:W - 3-dimensional tensor / image width

[0211] E2: Number of channels in a C-3D tensor

[0212] E1: Number of K kernels

[0213] The NNPA data layout format 1 provides, for example, two-dimensional kernel parallelism within a 4k-byte data block (page) and 4k-byte block data alignment with respect to the external dimension of the generated tensor, for efficient processing.

[0214] Pad bytes are ignored in the input tensor. Changes to PER storage on pad bytes are unpredictable.

[0215] Again, the exemplary data layout formats include the 4D feature tensor layout and the 4D kernel tensor layout, but other data layout formats may be supported by the processor (e.g., the neural network processor 105). The representation of a supported data layout is obtained, for example, by setting one or more bits in field 338 and placing them in the query parameter block.

[0216] The query parameter block also includes other data attribute information according to one or more aspects of the present invention. This other data attribute information includes, for example, supported size information for the data. Processors such as neural network processors typically have limitations based on internal buffer size, processing units, data bus structure, firmware limitations, etc., which can limit the maximum size of the tensor dimensions, the overall size of the tensor, or both. Therefore, the query function provides fields for communicating these limitations to the application. For example, based on the execution of the query function, the processor obtains various data sizes, such as the maximum dimension index size (e.g., 65,536 elements) and the maximum tensor size (e.g., 8GB), and includes this information in fields 340 and 342 of the parameter block (e.g., parameter block 330), respectively. Additional size information, less size information, other size information, or a combination thereof may be supported by the processor (e.g., neural network processor 105) and therefore obtained and placed in the parameter block (e.g., fields 340, 342, or other fields, or both). In other embodiments, the limit may be smaller or larger, or the size may be in other units, such as bytes instead of elements, or elements instead of bytes, or both. Furthermore, in other embodiments, the maximum size may not be the same for all dimensions, but may differ for each dimension. Many variations are possible.

[0217] According to one or more aspects of the present invention, a query function is provided that conveys detailed information relating to a particular model of a selected processor (e.g., a neural network processor 105). The detailed information includes, for example, model-dependent information relating to the particular processor (the processor may also support standard data attributes such as standard data types and standard data layouts, which are implied by the query function and not necessarily presented; however, in other embodiments, the query function may display all data attributes or various selected subsets, etc.). While illustrative information is provided, other information may be provided in other embodiments. The acquired information (which may differ in different models of one processor, or different models of different processors, or both) is used to perform artificial intelligence processing or other processing or both. The artificial intelligence processing or other processing or both may employ, for example, one or more non-query functions of a neural network processing assist instruction. A particular non-query function to be employed in the processing is executed by executing a neural network processing assist instruction one or more times and specifying the particular non-query function.

[0218] Further details of examples of non-query functions supported by neural network processing assist instructions are described below (in other embodiments, additional functions, fewer functions, other functions, or combinations thereof may be supported).

[0219] Function code 16: NNPA-ADD (addition)

[0220] When the NNPA-ADD function is specified, each element of input tensor 1 described by tensor descriptor 1 is added to the corresponding element of input tensor 2 described by tensor descriptor 2, and the resulting sum is placed in the corresponding element of the output tensor described by the output tensor descriptor.

[0221] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0222] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and output tensor are identical. Otherwise, a general operand data exception is recognized.

[0223] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific save region address field are ignored.

[0224] Function code 17: NNPA-SUB (subtraction)

[0225] When the NNPA-SUB function is specified, each element of input tensor 2 described by tensor descriptor 2 is subtracted from the corresponding element of input tensor 1 described by tensor descriptor 1, and the resulting difference is placed in the corresponding element of the output tensor.

[0226] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0227] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and output tensor are identical. Otherwise, a general operand data exception will be recognized.

[0228] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific save region address field are ignored.

[0229] Function code 18: NNPA-MUL (multiplication)

[0230] When the NNPA-MUL function is specified, the product of each element (multiplier) of input tensor 1 described by tensor descriptor 1 and the corresponding element (multiplicand) of input tensor 2 described by tensor descriptor 2 is placed in the corresponding element of the output tensor.

[0231] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0232] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and output tensor are identical. Otherwise, a general operand data exception will be recognized.

[0233] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific save region address field are ignored.

[0234] Function code 19: NNPA-DIV (division)

[0235] When the NNPA-DIV function is specified, each element (dividend) of input tensor 1 described by tensor descriptor 1 is divided by the corresponding element (divisor) of input tensor 2 described by tensor descriptor 2, and the quotient is placed in the corresponding element of the output tensor.

[0236] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0237] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and output tensor are identical. Otherwise, a general operand data exception will be recognized.

[0238] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific save region address field are ignored.

[0239] Function code 20: NNPA-MIN (minimum)

[0240] When the NNPA-MIN function is specified, each element of input tensor 1 described by tensor descriptor 1 is compared with the corresponding element of input tensor 2 described by tensor descriptor 2. The smaller of the two values ​​is placed in the corresponding element of the output tensor descriptor. If both values ​​are equal, that value is placed in the corresponding element of the output tensor.

[0241] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0242] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and output tensor are identical. Otherwise, a general operand data exception will be recognized.

[0243] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific save region address field are ignored.

[0244] Function code 21: NNPA-MAX (maximum)

[0245] When the NNPA-MAX function is specified, each element of input tensor 1 described by tensor descriptor 1 is compared with the corresponding element of input tensor 2 described by tensor descriptor 2. The larger of the two values ​​is placed in the corresponding element of the output tensor descriptor. If both values ​​are equal, that value is placed in the corresponding element of the output tensor.

[0246] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0247] In one example, the shapes, data layouts, and data types of input tensor 1, input tensor 2, and output tensor are identical. Otherwise, a general operand data exception will be recognized.

[0248] In one example, the output tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific save region address field are ignored.

[0249] Function code 32: NNPA-LOG(natural logarithm)

[0250] When the NNPA-LOG function is specified, for each element of the input tensor described by tensor descriptor 1, if that element is greater than 0, the corresponding element of the output tensor described by the output tensor descriptor will be the natural logarithm of that element. Otherwise, the corresponding element of the output tensor cannot be represented numerically, and a value related to negative infinity of the target data type will be stored.

[0251] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0252] In one example, the shape, data layout, and data type of input tensor 1 and output tensor are the same. Otherwise, a general operand data exception will be recognized.

[0253] In one example, output tensor descriptor 2, input tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific save area address field are ignored.

[0254] Function code 33: NNPA-EXP (exponential)

[0255] When the NNPA-EXP function is specified, for each element of the input tensor described by tensor descriptor 1, the corresponding element of the output tensor described by the output tensor descriptor becomes the exponent of that element.

[0256] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0257] In one example, the shape, data layout, and data type of input tensor 1 and output tensor are the same. Otherwise, a general operand data exception will be recognized.

[0258] In one example, output tensor descriptor 2, input tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific save area address field are ignored.

[0259] Function code 49: NNPA-RELU (Rectified Linear Unit)

[0260] When the NNPA-RELU function is specified, for each element of the input tensor described by tensor descriptor 1, if that element is 0 or less, the corresponding element of the output tensor described by the output tensor descriptor is 0. Otherwise, the corresponding element of the output tensor is the minimum value between the element of the input tensor and the clipping value specified by function-specific parameter 1.

[0261] For example, function-specific parameter 1 defines the clipping value for the RELU operation. For instance, the clipping value is located in bits 16-31 of function-specific parameter 1. The clipping value is specified, for example, in the NNPA-data-type 1 format. A clipping value of 0 indicates that the maximum positive value is used, in other words, no clipping occurs. If a negative value is specified, a general operand data exception is recognized.

[0262] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0263] In one example, the shape, data layout, and data type of input tensor 1 and output tensor are the same. Otherwise, a general operand data exception will be recognized.

[0264] In one example, output tensor descriptor 2, input tensor descriptor 2, input tensor descriptor 3, and the function-specific save region address field are ignored. In one example, function-specific parameters 2-5 include zero.

[0265] Function code 50: NNPA-TANH

[0266] When the NNPA-TANH function is specified, for each element of the input tensor described by tensor descriptor 1, the corresponding element value of the output tensor described by the output tensor descriptor becomes the hyperbolic tangent of that element.

[0267] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0268] In one example, the shape, data layout, and data type of input tensor 1 and output tensor are the same. Otherwise, a general operand data exception will be recognized.

[0269] In one example, output tensor descriptor 2, input tensor descriptor 2, input tensor descriptor 3, function-specific parameters 1-5, and function-specific save area address field are ignored.

[0270] Function code 51: NNPA-SIGMOID

[0271] When the NNPA-SIGMOID function is specified, for each element of the input tensor described by tensor descriptor 1, the corresponding element of the output tensor described by the output tensor descriptor becomes S-shaped (sigmoidal).

[0272] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0273] In one case, the shapes, data layouts, and data types of the input tensor 1 and the output tensor are the same. Otherwise, a general operand data exception is recognized.

[0274] In one case, the output tensor descriptor 2, the input tensor descriptor 2, the input tensor descriptor 3, the function-specific parameters 1-5, and the function-specific save area address field are ignored.

[0275] Function code 52: NNPA - SOFTMAX

[0276] When the NNPA - SOFTMAX function is specified, for each vector in dimension 1 of the input tensor 1, the corresponding vector of the output tensor is calculated as described below.

[0277] * The maximum value of the vector is calculated.

[0278] * The sum of the exponents of the differences between each element in dimension 1 of the vector and the maximum value calculated above is calculated. If both the element in dimension 1 of the input vector and the maximum value calculated above are numerical and the difference is non - numerical, the exponential result of that element is forced to zero.

[0279] * For each element of the vector, an intermediate quotient is formed by dividing the exponent of the difference between that element and the maximum value calculated above by the sum calculated above. An arbitrary activation function is applied to this intermediate quotient to form the corresponding element of the output vector.

[0280] This process is repeated, for example, for all vectors of dimension 1 with dimensions 4 index size × dimension 3 index size × dimension 2 index size.

[0281] In one example, the NNPA-SOFTMAX function specific parameter 1 controls the activation function. As an example, the ACT field (e.g., bits 28 to 31) of the function specific parameter 1 specifies the activation function. Examples of activation functions include the following.

[0282] ACT activation function

[0283] 0 Do not execute the activation function

[0284] 1 LOG

[0285] 2 to 15 Reserved

[0286] If a reserved value is specified in the ACT field, for example, a response code of hexadecimal "F001" is reported, and the operation is completed with a condition code of, for example, "1".

[0287] In one example, if any of the specified data layouts of the specified tensor descriptors do not specify a 4D feature tensor (e.g., data layout = 0), or if the data type of the specified tensor descriptor does not specify NNP-data type 1 (e.g., data type = 0), response codes (e.g., hexadecimal "0010" or hexadecimal "0011" respectively) are set in general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0288] In one example, if the dimension 3 index size of the input tensor is not equal to 1, for example, a response code of hexadecimal "F000" is stored, and the instruction completes with a condition code of, for example, "1".

[0289] In one example, the shapes, data layouts, and data types of input tensor 1 and the output tensor are the same. Otherwise, a general operand data exception is recognized.

[0290] In one example, output tensor descriptor 2, input tensor descriptor 2, and input tensor descriptor 3 are ignored. In one example, function eigenparameters 2-5 include zero.

[0291] An 8KB function-specific save area may be used by this function.

[0292] In one embodiment, when obtaining a 1-dimensional vector, the elements may not be contiguous in memory depending on the specified data layout format. If all elements of the 1-dimensional vector of input tensor 1 contain negative numbers of the largest size representable by the specified data type, the accuracy of the result may be reduced.

[0293] Function code 64: NNPA-BATCHNORM (Batch Normalization)

[0294] When the NNPA-BATCHNORM function is specified, for each dimension 1 vector in the input tensor 1, the corresponding dimension 1 vector in the output tensor is calculated by multiplying each element of that vector by the corresponding element of the dimension 1 vector that makes up the input tensor 2. Then, the full-precision product is added to the corresponding element of the dimension 1 vector that makes up the input tensor 3 and rounded to the precision of the specified data type of the output tensor. This process is repeated, for example, for all dimension 1 vectors with a dimension 4 index size × dimension 3 index size × dimension 2 index size.

[0295] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0296] In one example, a general operand data exception is recognized if the following conditions are true, and otherwise.

[0297] * The shape and data layout of input tensor 1 and output tensor are the same.

[0298] * The input tensor and output tensor have the same data type.

[0299] * The input tensors 1, 2, and 3 and the output tensor have the same dimensional 1 index size.

[0300] * The input tensors 2 and 3 have dimensions 2, 3, and 4, with index sizes of 1.

[0301] In one example, the output tensor descriptor 2 and the function-specific save region address field are ignored. In another example, function-specific parameters 2-5 include zero.

[0302] Function code 80: NNPA-MAXPOOL2D Function code 81: NNPA-AVGPOOL2D

[0303] When either the NNPA-MAXPOOL2D function or the NNPA-AVGPOOL2D function is specified, the input tensor 1 described by the input tensor 1 descriptor is reduced by the specified operation and summarizes the windows of the input. The input windows are selected by moving a two-dimensional sliding window over dimensions index 2 and 3. The summarization of the window becomes an element of the output tensor. The dimensions of the sliding window are described, for example, by function-specific parameter 4 and function-specific parameter 5. When calculating adjacent output tensor elements, the amount by which the sliding window moves over the input 1 tensor is called the stride. The stride of the sliding window is specified, for example, by function-specific parameter 2 and function-specific parameter 3. When the NNPA-MAXPOOL2D operation is specified, the Max operation defined below is performed on the window. When the NNPA-AVGPOOL2D operation is specified, the AVG operation defined below is performed on the window. When the specified padding type is "Valid", all elements within the window are added to the collection used for the calculation of the output element. When the specified padding type is "Same", depending on the position of the window, only a subset of the elements from the window may be added to the collection used for the calculation of the output element.

[0304] In one example, the CollectElements operation adds one element to a collection of elements and increments the number of elements in the collection. Each time the start position of the window moves, the collection becomes empty. It is unpredictable whether access to elements not required for the execution of the operation is made.

[0305] Max operation: In one example, the maximum value of a collection of elements within a window is calculated by comparing all elements in the collection with each other and returning the largest value.

[0306] Avg (average) calculation: In one example, the average of a set of elements within a window is calculated by dividing the sum of all elements in the set by the number of elements in the set.

[0307] In one example, the fields are assigned as follows:

[0308] * The pooling function-specific parameter 1 controls the padding type. For example, bits 29-31 of function-specific parameter 1 contain the PAD field, which specifies the padding type. Examples of types include, for example, the following:

[0309] PAD Padding type

[0310] 0 Valid

[0311] 1 Same

[0312] 2-7 spare

[0313] If an alternate value is specified in the PAD field, for example, a hexadecimal response code "F000" is reported, and the calculation is completed with a condition code of, for example, "1".

[0314] In one example, bit positions 0-28 of function-specific parameter 1 are reserved and include zero.

[0315] * Function-specific parameter 2 includes, for example, a 32-bit unsigned binary integer that specifies the dimension-2 stride (D2S), which specifies the number of elements the sliding window moves across in dimension 2.

[0316] * Function-specific parameter 3 includes, for example, a 32-bit unsigned binary integer that specifies the 3-dimensional stride (D3S), which specifies the number of elements the sliding window moves across in 3 dimensions.

[0317] * Function-specific parameter 4 includes, for example, a 32-bit unsigned binary integer that specifies the dimension-2 window size (D2WS), which specifies the number of dimension-2 elements contained in the sliding window.

[0318] * Function-specific parameter 5 includes, for example, a 32-bit unsigned binary integer that specifies the dimension 3 window size (D3WS), which specifies the number of dimension 3 elements contained in the sliding window.

[0319] In one example, the specified values ​​for function-specific parameters 2-5 are less than or equal to the maximum dimension index size, and the specified values ​​for function-specific parameters 4-5 are greater than zero. Otherwise, a response code of, for example, hexadecimal "0012" is reported, and the operation is completed with a condition code of, for example, "1".

[0320] If both the Dimension 2 stride and Dimension 3 stride are 0, and either the Dimension 2 window size or the Dimension 3 window size is greater than, for example, 1024, then a response code of, for example, hexadecimal "F001" is stored. If both the Dimension 2 stride and Dimension 3 stride are greater than, for example, 0, and either the Dimension 2 window size or the Dimension 3 window size is greater than, for example, 64, then a response code of, for example, hexadecimal "F002" is stored. If both the Dimension 2 stride and Dimension 3 stride are greater than, for example, 0, and either the Dimension 2 stride or the Dimension 3 stride is greater than, for example, 30, then a response code of, for example, hexadecimal "F003" is stored. If both the Dimension 2 stride and Dimension 3 stride are greater than, for example, 0, and either the input tensor Dimension 2 index size or the input tensor Dimension 3 index size is greater than, for example, 1024, then a response code of, for example, hexadecimal "F004" is stored. For all of the above conditions, the instruction completes with a condition code of, for example, "1".

[0321] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0322] In one example, a general operand data exception is recognized if the following conditions are true, and otherwise.

[0323] * The input tensor and the output tensor have the same dimensional 4 index size and dimensional 1 index size.

[0324] * The input and output tensors have the same data layout and data type.

[0325] * If both the Dimension 2 stride and Dimension 3 stride are 0, the following additional condition is true in one example:

[0326] * The input tensor's 2-dimensional index size is equal to the 2-dimensional window size.

[0327] * The input tensor's 3-dimensional index size is equal to the 3-dimensional window size.

[0328] * The 2-dimensional and 3-dimensional index sizes of the output tensor are 1.

[0329] * The specified padding is valid.

[0330] * In one example, both strides are non-zero, if either the 2-dimensional stride or the 3-dimensional stride is non-zero.

[0331] * If both the Dimension 2 stride and the Dimension 3 stride are greater than 0, the following additional condition is true in one example:

[0332] * If the specified padding is valid, the 2D window size is less than or equal to the 2D index size of the input tensor.

[0333] * If the specified padding is valid, the 3D window size is less than or equal to the 3D index size of the input tensor.

[0334] * When the specified padding is Same, the following relationship is satisfied between the 2-dimensional index size and the 3-dimensional index size of the input and output tensors (pooling Same padding).

[0335] TIFF0007845804000008.tif14137

[0336] TIFF0007845804000009.tif12137

[0337] Herein is the case:

[0338] IxDyIS: Dimension y of the input tensor x defined by the tensor descriptor x is the index size.

[0339] OxDyIS: Dimension y of the output tensor x defined by the tensor descriptor x is the index size.

[0340] D2S Dimension 2 Stride

[0341] D3S Dimensional 3-Stride

[0342] * If the specified padding is valid, the following relationship is satisfied between the 2-dimensional index size and the 3-dimensional index size of the input and output tensors (pooling valid padding):

[0343] TIFF0007845804000010.tif13143

[0344] TIFF0007845804000011.tif12143

[0345] Here, D2WS is the 2-dimensional window size, and D3WS is the 3-dimensional window size.

[0346] The output tensor descriptor 2, input tensor descriptors 2 and 3, and the function-specific save region address field are ignored.

[0347] Function code 96: NNPA-LSTMACT (Long Short-Term Memory Activation)

[0348] When the NNPA-LSTMACT function is specified, input tensor 1, described by the input tensor 1 descriptor and divided into four subtensors for each of the four index values ​​in each dimension, along with input tensor 2, described by the input tensor 2 descriptor and divided into four subtensors for each of the four index values ​​in each dimension, and input tensor 3, described by the input tensor 3 descriptor, become the inputs to the LSTMACT operation. At the end of the LSTMACT operation, the results are written to output tensor 1, described by the output tensor 1 descriptor, and output tensor 2, described by the output tensor 2 descriptor.

[0349] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code hexadecimal "0010" or hexadecimal "0011" is set to general-purpose register 0, and the instruction is completed with a condition code of, for example, "1".

[0350] In one embodiment, a general operand data exception is recognized if the following condition is true, otherwise.

[0351] * The dimensional 4 index size of input tensor 3 and output tensors 1 and 2 is, for example, equal to 1.

[0352] * The 4-dimensional index size of input tensor 1 and input tensor 2 is, for example, equal to 4.

[0353] * For example, the dimensional 3 index size of all input tensors and the two output tensors is equal to, for example, 1.

[0354] * For example, all input tensors and the two output tensors have the same data layout and data type.

[0355] * For example, all input tensors and the two output tensors have the same dimensional 1 index size.

[0356] * For example, all input tensors and the two output tensors have the same dimensional 2 index size.

[0357] In one example, the function-specific save area address field is ignored. Function-specific parameters 1-5 include 0 in one example.

[0358] Function code 97: NNPA-GRUACT (Gated Recurrent Unit Activation)

[0359] When the NNPA-GRUACT function is specified, input tensor 1, described by the input tensor 1 descriptor and divided into three subtensors for each dimension's 4 index values, along with input tensor 2, described by the input tensor 2 descriptor and divided into three subtensors for each dimension's 4 index values, and input tensor 3, described by the input tensor 3 descriptor, become the inputs to the GRUACT operation. At the end of the GRUACT operation, the output tensor described by the output tensor descriptor is recorded.

[0360] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0361] In one embodiment, a general operand data exception is recognized if the following condition is true, otherwise.

[0362] * The dimensional 4 index size of the output tensor and the input tensor 3 is equal to, for example, 1.

[0363] * The 4-dimensional index size of input tensor 1 and input tensor 2 is, for example, equal to 3.

[0364] * For example, the dimensional 3 index size of all input and output tensors is equal to, for example, 1.

[0365] * For example, all input and output tensors have the same dimensional 1 index size.

[0366] * For example, all input and output tensors have the same dimensional 2 index size.

[0367] * For example, all input and output tensors have the same data layout and data type.

[0368] In one example, the output tensor descriptor 2 and the function-specific save region address field are ignored. Function-specific parameters 2-5 include 0 in one example.

[0369] Function code 112: NNPA-CONVOLUTION

[0370] When the NNPA-CONVOLUTION function is specified, for each output element of the output tensor described by the output tensor 1 descriptor, a 3D input 1 window consisting of dimension indices 3, 2, and 1 is selected from input tensor 1 described by the input tensor 1 descriptor. A 3D input 2 window of the same size consisting of dimension indices 4, 3, and 2 is selected from tensor 2 described by the input tensor 2 descriptor. The elements of the input 1 window are multiplied with the corresponding elements of the input 2 window, and all products are added together to create an initial summation. This initial summation is added to the corresponding elements of input tensor 3 to calculate the intermediate summation. The elements of the output tensor are the result of applying the specified activation function to the intermediate summation. If no activation function is specified, the output elements are equal to the intermediate summation.

[0371] If the specified padding type is Valid, all elements of the window are used to calculate the initial sum. If the specified padding type is Same, depending on the window's position, some elements of the Input 1 window may be implicitly set to 0 when calculating the initial sum.

[0372] It is unpredictable whether elements not necessary for the operation will be accessed.

[0373] In one example, the fields of the function-specific parameters used by the convolution function are assigned as follows:

[0374] * The NNPA-CONVOLUTION function-specific parameter 1 controls the padding type and activation function. In one example, bits 29-31 of function-specific parameter 1 contain the PAD field, which specifies the padding type. Examples of types are shown below.

[0375] PAD Padding type

[0376] 0 Valid

[0377] 1 Same

[0378] 2-7 spare

[0379] If an alternate value is specified in the PAD field, for example, a hexadecimal response code "F000" will be reported, and the operation will complete with a condition code of "1".

[0380] Furthermore, in one example, bits 24-27 of NNPA-CONVOLUTION function-specific parameter 1 contain an activation field that specifies the activation function. An example of a function is shown below.

[0381] ACT activation function

[0382] 0 The activation function is not executed.

[0383] 1 RELU

[0384] 2-15 spare

[0385] When the RELU activation function is specified, the output element values ​​are determined as follows: If the value of the intermediate sum is less than or equal to 0, the corresponding element of the output tensor will be 0. Otherwise, the corresponding element of the output tensor will be the minimum of the value of the intermediate sum and the clipping value specified by function-specific parameter 4.

[0386] If an alternate value is specified in the ACT field, a response code such as the hexadecimal "F001" will be reported, and the operation will complete with a condition code such as "1".

[0387] * Function-specific parameter 2 includes, for example, a 32-bit unsigned binary integer that specifies the 2-dimensional stride (D2S), which specifies the number of elements the sliding window moves across in 2 dimensions.

[0388] * Function-specific parameter 3 includes, for example, a 32-bit unsigned binary integer that specifies the 3-dimensional stride (D3S), which specifies the number of elements the sliding window moves across in 3 dimensions.

[0389] The specified values ​​of function-specific parameters 2 and 3 are less than the maximum dimension index size. Otherwise, a response code of, for example, hexadecimal "0012" is reported, and the operation completes with a condition code of, for example, "1".

[0390] * Function-specific parameter 4 defines the clipping value for any RELU operation. In one example, the clipping value is located in bits 16-31 of function-specific parameter 4.

[0391] In one example, if the ACT field is 0, this field is ignored. If the ACT field specifies RELU, the clipping value is specified in NNP-datatype1 format. A clipping value of 0 indicates that the maximum positive value is used, in other words, no clipping occurs. If a non-zero value is specified, a general operand data exception is recognized.

[0392] In one example, if the specified data layout of any of the specified tensor descriptors other than input tensor 2 does not specify a 4D feature tensor (e.g., data layout=0), or if the specified data layout of input tensor 2 does not specify a 4D kernel tensor (e.g., data layout=1), then a response code of, for example, hexadecimal "0010" is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1". In another example, if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), then a response code of, for example, hexadecimal "0011" is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0393] If both the Dimension 2 stride and Dimension 3 stride are 0, and the Dimension 3 index size or Dimension 4 index size of input tensor 2 is greater than, for example, 448, then a response code of, for example, hexadecimal "F002" is stored. If both the Dimension 2 stride and Dimension 3 stride are greater than 0, and either the Dimension 3 index size or Dimension 4 index size of input tensor 2 is greater than, for example, 64, then a response code of, for example, hexadecimal "F003" is stored, and the operation is completed with a condition code of, for example, "1". If either the Dimension 2 stride or Dimension 3 stride is greater than, for example, 13, then a response code of, for example, hexadecimal "F004" is stored, and the operation is completed with a condition code of, for example, "1".

[0394] In one example, a general operand data exception is recognized if the following conditions are true, and otherwise.

[0395] * The data layouts of input tensor 1, input tensor 3, and output tensor are the same.

[0396] * All input and output tensors have the same data type.

[0397] * The input 3-tensor has index sizes of 1 for dimensions 2, 3, and 4.

[0398] * The 4-dimensional index size of the output tensor is equal to the 4-dimensional index size of the input 1 tensor.

[0399] * The dimensional 1 index size of the output tensor is equal to the dimensional 1 index size of the input 2 tensor and the dimensional 1 index size of the input 3 tensor.

[0400] * The dimensional 1 index size of the input tensor 1 is equal to the dimensional 2 index size of the input tensor 2.

[0401] * If both the Dimension 2 stride and the Dimension 3 stride are 0, the following additional condition is true in one example:

[0402] * The dimensional 2 index size of an input 1 tensor is equal to the dimensional 3 index size of an input 2 tensor.

[0403] * The 3-dimensional index size of an input 1-tensor is equal to the 4-dimensional index size of an input 2-tensor.

[0404] * The 2-dimensional and 3-dimensional index sizes of the output tensor are 1.

[0405] * The specified padding is valid.

[0406] * If either the 2-dimensional stride or the 3-dimensional stride is non-zero, then both strides are non-zero.

[0407] * If both the Dimension 2 stride and the Dimension 3 stride are greater than 0, the following additional condition is true in one example:

[0408] * If the specified padding is valid, the dimensional 2 index size of input tensor 1 is greater than or equal to the dimensional 3 index size of input tensor 2.

[0409] * If the specified padding is valid, the dimensional 3 index size of the input tensor 1 is greater than or equal to the dimensional 4 index size of the input tensor 2.

[0410] * When the specified padding is Same, in one example, the following relationship is satisfied between the Dimension 2 index size and the Dimension 3 index size of the input tensor and the output tensor (convolution Same padding).

[0411] TIFF0007845804000012.tif12143

[0412] TIFF0007845804000013.tif12143

[0413] Herein is the case:

[0414] O1D2IS Output tensor dimension 2 index size

[0415] O1D3IS Output Tensor Dimension 3 Index Size

[0416] I1D2IS Input 1 Tensor Dimension 2 Index Size

[0417] I1D3IS Input 1 Tensor Dimension 3 Index Size

[0418] D2S Dimension 2 Stride

[0419] D3S Dimensional 3-Stride

[0420] * If the specified padding is valid, in one example, the following relationship is satisfied between the dimensional 2 index size and dimensional 3 index size of input tensor 1, the dimensional 3 index size and dimensional 4 index size of input tensor 2, and the output tensor (convolutional valid padding).

[0421] TIFF0007845804000014.tif13143

[0422] TIFF0007845804000015.tif12143

[0423] Herein is the case:

[0424] O1D2IS Output tensor dimension 2 index size

[0425] O1D3IS Output Tensor Dimension 3 Index Size

[0426] I1D2IS Input 1 Tensor Dimension 2 Index Size

[0427] I1D3IS Input 1 Tensor Dimension 3 Index Size

[0428] I2D3IS Input 2 Tensor Dimension 3 Index Size

[0429] I2D4IS Input 2-tensor Dimension 4 Index Size

[0430] D2S Dimension 2 Stride

[0431] D3S Dimensional 3-Stride

[0432] In one example, the output tensor descriptor 2 and the function-specific save region address field are ignored. Function-specific parameter 5 includes 0 in one example.

[0433] Function code 113: NNPA-MATMUL-OP (Matrix Multiplication Operation)

[0434] When the NNPA-MATMUL-OP function is specified, each element of the output tensor described by the output tensor descriptor is computed as shown below in one example.

[0435] * A dimension-1 vector is selected from input tensor 1, which is described by the input tensor 1 descriptor, using the dimension-1 vector acquisition operation (get-dimension-1-vector operation) described later.

[0436] * A 2-dimensional vector is selected from the input tensor 2 described by the input tensor 2 descriptor using the 2-dimensional vector acquisition operation (get-dimension-2-vector operation) described later.

[0437] * The intermediate dot product of a 1-dimensional vector and a 2-dimensional vector is calculated using the dot product operation described later.

[0438] * The operation is performed on the intermediate dot product and the elements of input tensor 3 that have the same dimensional index 4 and dimensional index 1 values ​​as the output tensor elements, as described by the input tensor 3 descriptor. The resulting elements are stored in the output tensor. The fused operation is determined by function-specific parameter 1, which is described below.

[0439] Dimension 1 vector acquisition operation: For a specified output element, a dimensional 1 vector is selected from the input tensor. Here, an input dimensional 4 index becomes an output dimensional 4 index, an input dimensional 3 index becomes an output dimensional 3 index, and an input dimensional 2 index becomes an output dimensional 2 index.

[0440] Dimension 2 vector acquisition operation: For a specified output element, a 2-dimensional vector is selected from the input 2-tensor. Here, an input 4-dimensional index becomes an output 4-dimensional index, an input 3-dimensional index becomes an output 3-dimensional index, and an input 1-dimensional index becomes an output 1-dimensional index.

[0441] Dot product operation: The intermediate dot product of two vectors of the same size and data type is calculated as the sum of the products of each element of input vector 1 and the corresponding elements of input vector 2.

[0442] Fusion Operation: Function-specific parameter 1 controls the operation performed on the intermediate dot product and the corresponding elements from input tensor 3. In one example, NNPA-MATMUL-OP function-specific parameter 1 includes an operation field in bits 24-31, for example. The operation field specifies the operation to be performed. An example of the operation is shown below.

[0443] Calculation Calculation type

[0444] Add 0

[0445] 1. Compare which dot product is high.

[0446] 2. Compare whether the dot product is low.

[0447] 3. Dot product and comparison of element equality

[0448] 4. Dot product and comparison of whether elements are equal.

[0449] 5. Compare whether the dot product is high or not.

[0450] 6. Compare whether the dot product is low.

[0451] In one example, for an addition operation, the three elements of the input tensor are added to the intermediate dot product. For a comparison operation, the intermediate dot product is compared to the three elements of the input tensor; if the comparison is true, the result is set to a value such as +1; otherwise, the output tensor is set to a value such as +0 in the specified data type.

[0452] In one example, all other values ​​in the OPERATION field are reserve. If a reserve value is specified in the OPERATION field, for example, a hexadecimal response code "F000" is reported, and the calculation is completed with a condition code of, for example, "1".

[0453] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0454] In one embodiment, a general operand data exception is recognized if the following condition is true, otherwise.

[0455] * All input and output tensors have the same 4-dimensional index size.

[0456] * The 3-dimensional index size of all input and output tensors is equal to 1.

[0457] * The size of the 2-dimensional index of the input tensor 3 is equal to 1.

[0458] * The input tensor 1 and the output tensor have the same dimensional 2 index size.

[0459] * The dimensional 1 index size of input tensor 1 is the same as the dimensional 2 index size of input tensor 2.

[0460] * The Dimension 1 index sizes of input tensor 2, input tensor 3, and output tensor are the same.

[0461] * All input and output tensors have the same data layout and data type.

[0462] In one embodiment, the output tensor descriptor 2 and the function-specific save region address field are ignored. Function-specific parameters 2-5 include 0 in one example.

[0463] Function code 114: NNPA-MATMUL-OP-BCAST23 (Matrix multiplication operation - broadcast 23)

[0464] When the NNPA-MATMUL-OP-BCAST23 function is specified, each element of the output tensor described by the output tensor descriptor is computed as shown below as an example.

[0465] * A 1-dimensional vector is selected from input tensor 1, which is described by the input tensor 1 descriptor, using the 1-dimensional vector acquisition operation described later.

[0466] * A 2-dimensional vector is selected from the input tensor 2 described by the input tensor 2 descriptor using the 2-dimensional vector acquisition operation described later.

[0467] * The dot product of a 1-dimensional vector and a 2-dimensional vector is calculated using the dot product operation described later.

[0468] * The elements of input tensor 3 that have the same dimension index 1 value as the output tensor elements, as described by the input tensor 3 descriptor, are added to the previously calculated dot product and stored in the output tensor.

[0469] Dimension 1 vector acquisition operation: For a specified output element, a dimensional 1 vector is selected from the input tensor. Here, an input dimensional 4 index becomes an output dimensional 4 index, an input dimensional 3 index becomes an output dimensional 3 index, and an input dimensional 2 index becomes an output dimensional 2 index.

[0470] Dimension 2 vector acquisition operation: For a specified output element, a dimensional 2 vector is selected from the input 2 tensor. Here, the input dimension 4 index is 1, the input dimension 3 index is the output dimension 3 index, and the input dimension 1 index is the output dimension 1 index.

[0471] Dot product: The intermediate product of two vectors of the same size and data type is calculated as the sum of the products of each element of input vector 1 and the corresponding elements of input vector 2.

[0472] In one example, if the specified data layout of any of the specified tensor descriptors does not specify a 4D feature tensor (e.g., data layout=0), or if the data type of the specified tensor descriptor does not specify NNP-datatype1 (e.g., data type=0), the response code (e.g., hexadecimal "0010" or hexadecimal "0011", respectively) is set to general-purpose register 0, and the instruction completes with a condition code of, for example, "1".

[0473] In one embodiment, a general operand data exception is recognized if the following condition is true, otherwise.

[0474] * The input tensor 1 and the output tensor have the same dimension 4 index size.

[0475] * The 4-dimensional index size of input tensor 2 and input tensor 3 is equal to 1.

[0476] * The 3-dimensional index size of all input and output tensors is equal to 1.

[0477] * The size of the 2-dimensional index of the input tensor 3 is equal to 1.

[0478] * The input tensor 1 and the output tensor have the same dimensional 2 index size.

[0479] * The dimensional 1 index size of input tensor 1 is the same as the dimensional 2 index size of input tensor 2.

[0480] * The Dimension 1 index sizes of input tensor 2, input tensor 3, and output tensor are the same.

[0481] * All input and output tensors have the same data layout and data type.

[0482] In one embodiment, the output tensor descriptor 2 and the function-specific save region address field are ignored. Function-specific parameters 1 to 5 include 0 in one example.

[0483] In one embodiment of the neural network processing assist instruction, if the output tensor overlaps with any of the input tensors or parameter blocks, the result is unpredictable.

[0484] For example, if an attempt is made to execute a neural network processing assist instruction and the parameter block is not specified, for example, on a double word boundary, a specification exception is recognized.

[0485] An attempt is made to execute the neural network processing assist instruction, and a general operand data exception is recognized, for example, if there is a tensor descriptor mismatch.

[0486] The result of the condition code for a neural network processing assist instruction may include, for example, the following: 0: Successful completion, 1: Response code is set, 2: -, 3: Amount of data processed determined by the CPU.

[0487] In one embodiment, the execution priority for neural network processing assist instructions includes, for example, the following:

[0488] 1.~7. Exceptions with the same priority as the general program interrupt conditions.

[0489] 8.A Condition code 1 due to the specification of an unassigned or uninstalled function code.

[0490] 8.B Specification exception due to parameter block not being specified on a double word boundary.

[0491] 9. Access exceptions for accessing parameter blocks

[0492] 10. Condition code 1 due to the specified format of the parameter block not being supported by the model.

[0493] 11.A Condition code 1 due to the specified tensor data layout not being supported.

[0494] 11.B General operand data exception due to different data layouts between tensor descriptors.

[0495] 12.A Condition code 1 for conditions other than those included in 8.A, 10, 11.A above and 12.B.1 below

[0496] 12.B.1 Condition code 1 due to invalid data type of output tensor for NNPA-RELU and NNPA-CONVOLUTION

[0497] 12.B.2 General operand data exception for invalid values ​​of NNPA-RELU function-specific parameter 1 and NNPA-CONVOLUTION function-specific parameter 4

[0498] 13.A Access exceptions for accessing the output tensor

[0499] 13.B Access exceptions for accessing input tensors

[0500] 13.C Access exceptions for accessing function-specific save areas

[0501] 14. Condition code 0

[0502] As described herein, a single instruction (e.g., a neural network processing assist instruction) is configured to execute multiple functions, including a query function and multiple non-query functions. Each non-query function may manipulate large amounts of data, and exception checking is provided. For example, during computation, non-numeric, unrepresentable, or out-of-range values ​​or combinations thereof may propagate through the computation. Knowing when invalid values, such as non-numeric, unrepresentable, or out-of-range values ​​or combinations thereof, are generated is useful because it may be possible to decide not to perform further computation (e.g., not to execute a neural network processing assist instruction to call another function). Detecting invalid values ​​does not result in performance degradation. Therefore, according to one or more aspects of the present invention, when performing computation, it is detected whether an input or output value is an invalid value, such as a non-numeric, unrepresentable, or out-of-range value or combination thereof. At the end of computation (e.g., after detecting an invalid input before performing the computation, or after performing one or more computations of the called functions), a summary indicator (e.g., a range exception flag) is set (e.g., to 1). In one example, no indication is provided as to which data element provided an invalid value. This allows for higher performance in arithmetic operations. Reporting whether an invalid value is an input or generated by a calculation facilitates debugging of the artificial intelligence model. While precise identification of the invalid value is not necessary, it is useful to know which calculation function generated one of these values.

[0503] One or more aspects of the present invention are closely linked to computer technology and facilitate processing within computers, improving their performance. By using a single, architecturally structured machine instruction configured to execute various functions, performance in the computing environment is improved by reducing complexity, reducing resource usage, and increasing processing speed. Furthermore, by performing exception checking and reporting that provides an exception summary, performance is not degraded, and the creation or debugging of artificial intelligence models, or both, is facilitated. Instructions or optimizations in processing, or both, can be used in many technical fields, such as computer processing, medical processing, engineering, automotive technology, and manufacturing. By achieving optimization, these technical fields are improved, for example, by reducing errors or execution time, or both, and / or by improving performance.

[0504] Further details of one embodiment that facilitates processing in a computing environment, relating to one or more aspects of the present invention, will be described with reference to Figures 7A and 7B.

[0505] Referring to Figure 7A, in one example, an indication is obtained showing that the value determined to be invalid was included in the input data for one of the calculations or in the output data obtained as a result of one or more calculations (700). This value is determined to be invalid due to one of several exceptions (702). Based on the fact that an indication has been obtained that the value has been determined to be invalid, a summary indicator is set (704). The summary indicator represents multiple exceptions collectively (706).

[0506] Providing an indication that invalid values ​​have been detected in the input or output data makes debugging artificial intelligence models easier, for example. Higher computational performance is achieved by avoiding the need to specify a particular data element with an invalid value or distinguishing which of multiple exceptions was detected.

[0507] In one example, the summary indicator represents multiple exceptions without distinguishing between them (708). In another example, the summary indicator is set regardless of which of the multiple exceptions was used to determine if the value was invalid (710).

[0508] Using a single indicator to show invalid values, regardless of the type of invalid value (e.g., non-numeric, unrepresentable, out-of-range), simplifies coding and handling, reduces complexity, and improves system performance.

[0509] In one example, a summary indicator is a range violation indicator of an exception flag placed at a location specified by an instruction issued to perform one or more calculations (712).

[0510] For example, several exceptions include non-numeric, unrepresentable numeric values, and out-of-range values ​​(714).

[0511] Referring to Figure 7B, in one example, obtaining a message indicating that a value has been determined to be invalid is based on executing instructions that perform one or more calculations (720). This facilitates the processing of instructions and the provision of invalid values.

[0512] For example, an instruction is configured to execute multiple functions that perform calculations, and these functions use a summary indicator (722). Having a single indicator used by multiple functions reduces complexity and the effort required for coding and verification.

[0513] In one example, an instruction is a neural network instruction that performs computations on an input tensor to provide an output tensor used for artificial intelligence processing (724). In one example, an instruction is configured to perform multiple functions that perform computations, and these functions use a summary indicator (726).

[0514] For example, a summary indicator is defined for a particular instruction, and another instruction uses a different summary indicator (728).

[0515] In one example, the value of a condition code set based on the execution of the particular instruction is determined (730), and the effectiveness of the summary indicator is based on the determination that the value of the condition code is the selected value (732).

[0516] Other variations and embodiments are also possible.

[0517] Aspects of the present invention can be used in many types of computing environments. Another example of a computing environment incorporating one or more aspects of the present invention is described with reference to Figure 8A. As an example, the computing environment in Figure 8A is based on the z / Architecture® instruction set architecture from International Business Machines Corporation (Armonk, New York). However, the z / Architecture instruction set architecture is only one example of an architecture. Again, the computing environment may be based on other architectures, including (but not limited to) the Intel® x86 architecture, other architectures from International Business Machines Corporation, or architectures from other companies or combinations thereof. Note that Intel is a trademark or registered trademark of Intel Corporation or its subsidiaries in the United States and other countries.

[0518] As an example, the computing environment 10 includes a central electronics complex (CEC) 11. The central electronics complex 11 includes multiple components. These multiple components include, for example, one or more processors such as one or more general-purpose processors (also known as central processing units (CPUs) 13) and one or more dedicated processors (e.g., neural network processors 31), and memory 12 (also known as system memory, main memory, main storage, central storage, storage) coupled to an input / output (I / O) subsystem 14.

[0519] For example, one or more dedicated processors may be provided separately from one or more general-purpose processors, or at least one dedicated processor may be incorporated into at least one general-purpose processor, or both. Other variations are also possible.

[0520] The I / O subsystem 14 may be part of the central electronic processing unit or it may be separate from it. The I / O subsystem 14 directs the information flow between the main storage 12 and the input / output control unit 15 and input / output (I / O) devices 16 that are coupled to the central electronic processing unit.

[0521] Many types of I / O devices can be used. One particular type is the data storage device 17. The data storage device 17 can store one or more programs 18, one or more computer-readable program instructions 19, or data or a combination thereof. The computer-readable program instructions can be configured to perform the functions of embodiments according to aspects of the present invention.

[0522] The central electronic processing unit 11 may include, or be coupled with, a removable / non-removable volatile / non-volatile computer system storage medium, or both. For example, the central electronic processing unit 11 may include, or be coupled with, a magnetic disk drive for reading and writing to a non-removable non-volatile magnetic medium (commonly called a "hard disk"), a removable non-volatile magnetic disk (e.g., a "floppy disk"), or an optical disk drive for reading and writing to a removable non-volatile optical disk, such as a CD-ROM, DVD-ROM, or other optical medium, or both. Other hardware and / or software components may be used in combination with the central electronic processing unit 11. Examples of these components, but not limited to, include microcode or millicode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0523] Furthermore, the central electronic processing unit 11 can operate with many other general-purpose or dedicated computing system environments or configurations. Examples of well-known computing systems, environments, or configurations or combinations suitable for use with the central electronic processing unit 11 include, but are not limited to, personal computer (PC) systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable home appliances, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of these systems or devices.

[0524] The central electronic processing unit 11 provides logical partitioning, virtualization support, or both in one or more embodiments. In one embodiment, as shown in Figure 8B, the memory 12 includes, for example, one or more logical partitions 20, a hypervisor 21 that manages the logical partitions, and processor firmware 22. An example of a hypervisor 21 is Processor Resource / System Manager (PR / SM) from International Business Machines Corporation (Armonk, New York). TM ) are examples. PR / SM is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction.

[0525] Each logical partition 20 can function as a separate system. That is, each logical partition can be reset independently and run a guest operating system 23 (such as the z / OS® operating system from International Business Machines Corporation (Armonk, New York)) or other control code 24 (such as a coupling facility control code (CFCC)) and operate with different programs 25. An operating system or application program running within a logical partition appears to have access to the entire system, but in reality, only a portion of it is available. Although the z / OS operating system has been used as an example, other operating systems provided by International Business Machines Corporation, other companies, or both may be used according to one or more aspects of the present invention.

[0526] Memory 12 is coupled to, for example, CPU 13 (Figure 8A). CPU 13 is a physical processor resource that can be allocated to a logical partition. For example, a logical partition 20 may contain one or more logical processors, each of which represents all or a portion of the physical processor resource 13 that can be dynamically allocated to the logical partition.

[0527] In a further embodiment, the central electronic processing unit provides virtual machine support (with or without logical partitioning support). As shown in Figure 8C, the memory 12 of the central electronic processing unit 11 includes, for example, one or more virtual machines 26, a virtual machine manager such as a hypervisor 27 that manages the virtual machines, and processor firmware 28. An example of a hypervisor 27 is the z / VM® hypervisor manufactured by International Business Machines Corporation (Armonk, New York). A hypervisor is sometimes referred to as a host. Note that z / VM is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction.

[0528] The virtual machine support of the central electronic processing unit provides the ability to operate a number of virtual machines 26, each capable of running a different program 29 and a guest operating system 30, such as the Linux® operating system. Each virtual machine 26 can function as a separate system; that is, each virtual machine can be reset independently, run a guest operating system, and operate with different programs. An operating system or application program running within a virtual machine appears to have access to the entire system, but in reality, only a portion of it is available. While z / VM and Linux have been used as examples, other virtual machine managers and / or operating systems may be used in accordance with one or more aspects of the present invention. The registered trademark Linux is used in accordance with a sublicense from the Linux Foundation, which is the exclusive licensee of Linus Torvalds, the worldwide owner of the trademark.

[0529] Another embodiment of a computing environment using one or more aspects of the present invention will be described with reference to Figure 9A. In this example, the computing environment 36 includes, for example, a native central processing unit (CPU) 37, memory 38, and one or more input / output devices 39 or input / output interfaces 39 or both, coupled to each other, for example, via one or more buses 40 or other connections or both. As an example, the computing environment 36 may include a PowerPC® processor from International Business Machines Corporation (Armonk, New York), an HP Superdome with an Intel® Itanium® II processor from Hewlett-Packard Company (Palo Alto, California), or other machines or combinations based on architectures provided by International Business Machines Corporation, Hewlett-Packard, Intel Corporation, Oracle, or other companies or combinations thereof. PowerPC is a trademark or registered trademark of International Business Machines Corporation in at least one jurisdiction. Itanium is a trademark or registered trademark of Intel Corporation or its subsidiaries in the United States and other countries.

[0530] The native central processing unit 37 includes one or more native registers 41, such as one or more general-purpose registers, one or more dedicated registers, or both, used during processing within the environment. These registers contain information representing the state of the environment at any given point in time.

[0531] Furthermore, the native central processing unit 37 executes instructions and code stored in memory 38. In one specific example, the central processing unit executes emulator code 42 stored in memory 38. This code enables a computing environment configured on one architecture to emulate another architecture. For example, the emulator code 42 enables a machine based on an architecture other than the z / Architecture instruction set architecture (such as a PowerPC processor or an HP Superdome server) to emulate the z / Architecture instruction set architecture and execute software and instructions developed based on the z / Architecture instruction set architecture.

[0532] Further details relating to the emulator code 42 are described with reference to Figure 9B. The guest instructions 43 stored in memory 38 include software instructions (e.g., those related to machine instructions) that were developed to run on architectures other than the native CPU 37 architecture. For example, the guest instructions 43 may have been designed to run on a processor based on the z / Architecture instruction set architecture, but instead are emulated on the native CPU 37 (which may be an Intel Itanium II processor). As an example, the emulator code 42 includes an instruction fetching routine 44 to retrieve one or more guest instructions 43 from memory 38 and provide local buffers for the retrieved instructions as needed. The emulator code 42 also includes an instruction translation routine 45 to determine the type of the retrieved guest instruction and translate the guest instruction into one or more corresponding native instructions 46. This translation includes, for example, identifying the function performed by the guest instruction and selecting the native instruction to perform that function.

[0533] Furthermore, the emulator code 42 includes an emulation control routine 47 for executing native instructions. The emulation control routine 47 may cause the native CPU 37 to execute a native instruction routine that emulates one or more previously obtained guest instructions, and upon completion of such execution, return control to the instruction fetch routine to emulate obtaining the next guest instruction or group of guest instructions. Execution of the native instruction 46 may include loading data from memory 38 into registers, returning data from registers to memory for storage, or performing some type of arithmetic or logical operation determined by a translation routine.

[0534] Each routine is implemented, for example, in software stored in memory and executed by the native central processing unit 37. In other examples, one or more routines or operations are implemented in firmware, hardware, software, or a combination thereof. Registers of the emulated processor may be emulated using registers 41 of the native CPU, or they may be emulated using locations in memory 38. In embodiments, the guest instruction 43, native instruction 46, and emulator code 42 may reside in the same memory, or they may be distributed across different memory devices.

[0535] The instructions that can be emulated include the neural network-assisted processing instructions described herein, in accordance with one or more aspects of the present invention. Furthermore, in accordance with one or more aspects of the present invention, other instructions, one or more aspects of neural network processing, or exception checking and reporting, or a combination thereof, may also be emulated.

[0536] The computing environments described above are merely examples of usable computing environments. Other environments (including, but not limited to, non-partitioned environments, partitioned environments, cloud environments, or emulated environments or combinations thereof) may be used, and embodiments are not limited to any one of these environments. While various examples of computing environments are described herein, one or more aspects of the present invention can be used with many types of environments. The computing environments provided herein are for illustrative purposes only.

[0537] Each computing environment can be configured to include one or more aspects of the present invention.

[0538] One or more of these embodiments may relate to cloud computing.

[0539] While this disclosure includes a detailed description of cloud computing, it should be understood that the implementations of the teachings described herein are not limited to cloud computing environments. Rather, embodiments of the present invention can be implemented in combination with any other type of computing environment that is currently known or will be developed in the future.

[0540] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and deployed with minimal administrative effort or interaction with service providers. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0541] The characteristics are as follows:

[0542] On-demand self-service: Cloud consumers can unilaterally prepare computing power, such as server time and network storage, automatically as needed, without requiring human interaction with service providers.

[0543] Broad network access: Computing power is available over the network and accessible through standard mechanisms. This facilitates utilization by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, PDAs).

[0544] Resource pooling: A provider's computing resources are pooled and delivered to multiple consumers using a multi-tenant model. Various physical and virtual resources are dynamically allocated and reallocated as needed. Generally, consumers have a sense of location independence because they do not manage or know the exact location of the resources provided. However, consumers may be able to identify the location at a higher level of abstraction (e.g., country, state, data center).

[0545] Rapid Elasticity: Computing power can be prepared quickly and flexibly, allowing it to scale out automatically and immediately, and to be quickly released and scale in immediately. To consumers, the computing power available for preparation often appears unlimited and can be purchased in any quantity at any time.

[0546] Service Measurement: Cloud systems leverage metering capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, active user accounts) to automatically control and optimize resource usage. Resource usage can be monitored, controlled, and reported, providing transparency to both service providers and consumers.

[0547] The service model is as follows:

[0548] Software as a Service (SaaS): The functionality offered to consumers is the ability to use the provider's applications running on a cloud infrastructure. These applications can be accessed from various client devices via thin client interfaces such as web browsers (e.g., webmail). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions, except for configuring a limited number of user-specific applications.

[0549] Platform as a Service (PaaS): The functionality offered to consumers is the ability to deploy applications they have created or acquired to cloud infrastructure using programming languages ​​and tools supported by the provider. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, and storage, but they can control the deployed applications and, in some cases, the configuration of their hosting environment.

[0550] Infrastructure as a Service (IaaS): The functionality provided to consumers is the provision of processors, storage, networking, and other basic computing resources that enable consumers to deploy and run any software, including operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but they can control the operating system, storage, and deployed applications, and in some cases, partially control certain network components (e.g., host firewalls).

[0551] The deployment model is as follows:

[0552] Private Cloud: This cloud infrastructure is operated exclusively for a specific organization. This cloud infrastructure can be managed by that organization or a third party and can reside on-premises or off-premises.

[0553] Community Cloud: This cloud infrastructure is shared by multiple organizations to support a specific community with common interests (e.g., mission, security requirements, policies, and compliance). This cloud infrastructure can be managed by the organization or a third party and can reside on-premises or off-premises.

[0554] Public Cloud: This cloud infrastructure is provided to a large number of people or large industry groups and is owned by organizations that sell cloud services.

[0555] Hybrid Cloud: This cloud infrastructure combines two or more cloud models (private, community, or public). While maintaining the unique entities of each model, they are bound together by standards or individual technologies to achieve data and application portability (e.g., cloud bursting for load balancing across clouds).

[0556] Cloud computing environments are service-oriented environments that emphasize statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is the infrastructure, which includes a network of interconnected nodes.

[0557] Here, Figure 10 shows an exemplary cloud computing environment 50. As shown in the figure, the cloud computing environment 50 includes one or more cloud computing nodes 52. Local computer devices used by cloud consumers (e.g., PDAs or mobile phones 54A, desktop computers 54B, laptop computers 54C, or automotive computer systems 54N, or a combination thereof) can communicate with these nodes. The nodes 52 can communicate with each other. The nodes 52 can be grouped physically or virtually (not shown) in one or more networks, such as the private, community, public, or hybrid clouds or a combination thereof. This allows the cloud computing environment 50 to provide infrastructure, platforms, or software as a service, or a combination thereof, without requiring cloud consumers to maintain resources on their local computer devices. Note that the types of computer devices 54A-N shown in Figure 10 are merely examples, and it should be understood that the computing nodes 52 and the cloud computing environment 50 can communicate with any type of electronic device via any type of network or network addressable connection (e.g., using a web browser) or both.

[0558] Here, Figure 11 shows a set of functional abstraction layers provided by the cloud computing environment 50 (Figure 10). It should be understood that the components, layers, and functions shown in Figure 11 are illustrative only, and the embodiments of the present invention are not limited to these. As illustrated, the following layers and corresponding functions are provided.

[0559] The hardware and software layer 60 includes hardware components and software components. Examples of hardware components include a mainframe 61, a reduced instruction set computer (RISC) architecture-based server 62, server 63, blade server 64, storage 65, and a network and network components 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0560] The virtualization layer 70 provides an abstraction layer. From this layer, virtual entities such as virtual servers 71, virtual storage 72, virtual networks 73 including virtual private networks, virtual applications and operating systems 74, and virtual clients 75 can be provided.

[0561] As an example, the management layer 80 can provide the following functions: Resource preparation 81 enables the dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing 82 enables cost tracking as resources are used within the cloud computing environment and billing or invoicing for the consumption of these resources. As an example, these resources may include licenses for application software. Security enables not only protection of data and other resources but also identification and verification of cloud consumers and tasks. The user portal 83 provides consumers and system administrators with access to the cloud computing environment. Service level management 84 enables the allocation and management of cloud computing resources to ensure that requested service levels are met. Service Level Agreement (SLA) planning and execution 85 enables the pre-arrangement and procurement of cloud computing resources that are expected to be needed in the future in accordance with the SLA.

[0562] Workload layer 90 provides examples of capabilities available in a cloud computing environment. Examples of workloads and capabilities available from this layer include mapping and navigation 91, software development and lifecycle management 92, virtual classroom education delivery 93, data analytics processing 94, transaction processing 95, neural network processing assistance processing or exception / check reporting processing or both 96.

[0563] Aspects of the present invention may be systems, methods, or computer program products or combinations thereof, integrated at any possible level of technical detail. The computer program product may include a computer-readable storage medium storing computer-readable program instructions for causing a processor to execute aspects of the present invention.

[0564] A computer-readable storage medium can be a tangible device capable of holding and storing instructions used by an instruction execution device. Examples of computer-readable storage media include electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or appropriate combinations thereof. More specific examples of computer-readable storage media include portable computer diskettes, hard disks, RAM, ROM, erasable programmable ROM (EPROM or flash memory), static random access memory (SRAM), CD-ROMs, DVDs, memory sticks, floppy disks, mechanically encoded devices with instructions recorded on punch cards or grooved raised structures, and appropriate combinations thereof. The computer-readable storage media used herein should not be interpreted as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0565] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing device / processing device. Alternatively, they can be downloaded to an external computer or external storage device via a network (e.g., the Internet, LAN, WAN, or wireless network, or a combination thereof). The network may include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers or edge servers, or a combination thereof. A network adapter card or network interface within each computing device / processing device receives computer-readable program instructions from the network and transfers them for storage in a computer-readable storage medium within each computing device / processing device.

[0566] The computer-readable program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk and C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions can be executed as a standalone software package, either entirely on the user's computer or partially on the user's computer. Alternatively, they can be executed partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including LANs and WANs, or it may be connected to an external computer (for example, via the Internet using an Internet service provider). In some embodiments, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), and programmable logic arrays (PLAs), can execute computer-readable program instructions by utilizing state information of computer-readable program instructions in order to customize the electronic circuits for the purpose of performing aspects of the present invention.

[0567] Aspects of the present invention are described herein with reference to flowcharts or block diagrams, or both, of methods, apparatus (systems), and computer program products according to embodiments of the present invention. Each block in a flowchart or block diagram, or both, and combinations of blocks in a flowchart or block diagram, or both, are executable by computer-readable program instructions.

[0568] These computer-readable program instructions can be provided to a processor of a computer or other programmable data processing device to produce a machine. This creates a means for these instructions, executed via such a computer or other programmable data processing device processor, to perform functions / operations identified in one or more blocks in a flowchart or block diagram, or both. These computer-readable program instructions can further be stored in a computer-readable storage medium that can be instructed to function in a particular manner to a computer, programmable data processing device, or other device, or a combination thereof. Thus, the computer-readable storage medium containing the instructions constitutes a product containing instructions for performing functions / operations identified in one or more blocks in a flowchart or block diagram, or both.

[0569] Alternatively, a computer execution process may be generated by loading computer-readable program instructions into a computer, another programmable device, or other device, and having a series of operational steps executed on that computer, other programmable device, or other device. This ensures that the instructions executed on the computer, other programmable device, or other device perform functions / operations identified by one or more blocks in a flowchart, block diagram, or both.

[0570] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions containing one or more executable instructions for performing a specific logical function. In some other implementations, the functions shown within a block may be executed in an order different from the order shown in each diagram. For example, depending on the functions involved, two consecutively shown blocks may actually be achieved as a single process, executed simultaneously or nearly simultaneously, executed in a manner that partially or entirely overlaps in time, or the blocks may be executed in reverse order. Each block in a block diagram or flowchart or both, and combinations of multiple blocks in a block diagram or flowchart or both, are executable by a dedicated hardware-based system that performs a specific function or operation, or executes a combination of dedicated hardware and computer instructions.

[0571] In addition to the above, one or more aspects may be provided, delivered, deployed, managed, or serviced by a service provider that provides management of the customer environment. For example, a service provider may create, maintain, or support computer code or computer infrastructure, or both, that implements one or more aspects for one or more customers. In return, the service provider may receive payments from customers, for example, on the basis of a subscription or fee agreement or both. In addition to or instead of this, the service provider may receive payments from the sale of advertising content to one or more third parties.

[0572] In one embodiment, an application may be deployed to implement one or more embodiments. For example, the deployment of an application may include providing a computer infrastructure capable of operating to implement one or more embodiments.

[0573] In a further embodiment, a computing infrastructure may be deployed that includes integrating computer-readable code into a computing system. In this case, the code combined with the computing system can implement one or more embodiments.

[0574] In yet another embodiment, a process for integrating computing infrastructure may be provided, which includes integrating computer-readable code into a computer system. The computer system includes a computer-readable medium. The computer medium includes one or more embodiments. The code combined with the computer system can implement one or more embodiments.

[0575] Although various embodiments have been described above, these are merely examples. For example, one or more embodiments can be incorporated, used, or both, using computing environments of other architectures. Furthermore, different instructions or operations may be used. Furthermore, different types of registers, different registers, or both may be used. Furthermore, other data formats, data layouts, or data sizes or combinations thereof may be supported. In one or more embodiments, one or more general-purpose processors, one or more dedicated processors, or a combination of general-purpose and dedicated processors may be used. Many variations are possible.

[0576] This specification describes various embodiments. Furthermore, many modifications are possible without departing from the spirit of the embodiments of the present invention. Unless otherwise inconsistent, each embodiment or feature described herein, and its variations thereof, may be combined with any other embodiment or feature.

[0577] Furthermore, other types of computing environments can also benefit from and be used with the present invention. As an example, a data processing system suitable for storing or executing program code, or both, can be used, which includes at least two processors directly or indirectly coupled to a memory element via a system bus. The memory element may include, for example, local memory used during the actual execution of the program code, bulk storage, and cache memory that provides temporary storage for at least some of the program code to reduce the number of times the code is retrieved from bulk storage during execution.

[0578] Input / output or I / O devices (including, but not limited to, keyboards, displays, pointing devices, DASDs, tapes, CDs, DVDs, thumb drives, and other memory media) can be connected to the system directly or via an I / O controller. Network adapters may also be connected to the system to allow data processing systems to be connected to other data processing systems, remote printers, or storage devices via private or public networks. Some examples of available network adapters include modems, cable modems, and Ethernet cards.

[0579] The terms used herein are intended solely to describe specific embodiments and are not intended to limit the range of embodiments. In this specification, the singular forms “a,” “an,” and “the” are intended to include the plural form unless the context makes it clear otherwise. Furthermore, where the terms “comprises,” “comprising,” or both are used herein, they specify the existence of the described features, integers, steps, operations, elements, or components or combinations thereof, but do not preclude the existence or addition of one or more other features, integers, steps, operations, elements, components, or groups or combinations thereof.

[0580] Where present in the following claims, all corresponding structures, materials, actions, and equivalents of means-plus-function elements or step-plus-function elements are intended to include any structures, materials, or actions for performing a function in combination with other specifically claimed elements. The descriptions of one or more embodiments are presented for illustrative and explanatory purposes only and are not intended to be exhaustive or limit to the disclosed forms. Many changes and modifications will be apparent to those skilled in the art. These embodiments have been selected and described to best illustrate various aspects and practical applications, and to enable others skilled in the art to understand various embodiments with various modifications suitable for their particular intended use.

Claims

1. A computer program for facilitating processing within a computing environment, The method includes one or more computer-readable storage media and program instructions for executing a method, which are stored together on the one or more computer-readable storage media. The instruction executes one of several functions defined in the instruction, and the instruction has a summary indicator that is used exclusively by the instruction and used by the several functions defined in the instruction, and the summary indicator is used to indicate that the input data used by the several functions is invalid, The process involves obtaining an indication that a value determined to be invalid based on the execution of the aforementioned instruction was included in the input data for one of the one or more calculations performed by the single function, wherein the value is determined to be invalid due to one of the multiple exceptions. Based on the acquisition of the indication that the aforementioned value has been determined to be invalid, the summary indicator that collectively represents the multiple exceptions is set, A computer program that includes [this].

2. The computer program according to claim 1, wherein the summary indicator represents the plurality of exceptions without distinguishing them from one another.

3. The computer program according to claim 1 or 2, wherein the summary indicator is set regardless of which of the plurality of exceptions was used to determine the invalidity of the value.

4. The computer program according to claim 1, wherein the summary indicator is a range violation indicator of an exception flag located at a location specified by an instruction issued to perform the one or more calculations.

5. The computer program according to claim 1, wherein the aforementioned exceptions include non-numeric, unrepresentable numerical, and out-of-range values.

6. The computer program according to claim 1, wherein the instruction is a neural network instruction, and the one function is executed on one or more input tensors to provide one or more output tensors used for artificial intelligence processing.

7. The computer program according to claim 1, wherein another instruction different from the aforementioned instruction uses another summary indicator.

8. The computer program according to claim 1, wherein the method further comprises determining a value of a condition code set based on the execution of the instruction, and the effectiveness of the summary indicator is based on the determination that the value of the condition code is a selected value.

9. The computer program according to claim 6, wherein the one or more output tensors are used for artificial intelligence processing.

10. The computer program according to claim 1, wherein the instruction is a single instruction of an instruction set architecture having a defined operation code.

11. The computer program according to claim 1, wherein the summary indicator is provided in a register defined to be used by the instruction.

12. A computer system for facilitating processing within a computing environment, Memory and The computer system is configured to perform a method, comprising at least one processor that communicates with the memory, and the method is The instruction executes one of several functions defined in the instruction, and the instruction has a summary indicator that is used exclusively by the instruction and used by the several functions defined in the instruction, and the summary indicator is used to indicate that the input data used by the several functions is invalid, The process involves obtaining an indication that a value determined to be invalid based on the execution of the aforementioned instruction was included in the input data for one of the one or more calculations performed by the single function, or in the output data obtained as a result of said one or more calculations, wherein the value is determined to be invalid due to one of the multiple exceptions. Based on the acquisition of the indication that the aforementioned value has been determined to be invalid, the summary indicator that collectively represents the multiple exceptions is set, A computer system, including a computer system.

13. The computer system according to claim 12, wherein the summary indicator represents the plurality of exceptions without distinguishing them from one another.

14. The computer system according to claim 12 or 13, wherein the summary indicator is set regardless of which of the plurality of exceptions was used to determine the invalidity of the value.

15. The computer system according to claim 12, wherein the instruction is a neural network instruction, and the one function is executed on one or more input tensors to provide one or more output tensors used for artificial intelligence processing.

16. The computer system according to claim 12, wherein another instruction different from the aforementioned instruction uses another summary indicator.

17. A computer implementation method for facilitating processing within a computing environment, The instruction executes one of several functions defined in the instruction, and the instruction has a summary indicator that is used exclusively by the instruction and used by the several functions defined in the instruction, and the summary indicator is used to indicate that the input data used by the several functions is invalid, The process involves obtaining an indication that a value determined to be invalid based on the execution of the aforementioned instruction was included in the input data for one of the one or more calculations performed by the single function, wherein the value is determined to be invalid due to one of the multiple exceptions. Based on the acquisition of the indication that the aforementioned value has been determined to be invalid, the summary indicator that collectively represents the multiple exceptions is set, Computer implementation methods, including those mentioned above.

18. The computer implementation method according to claim 17, wherein the summary indicator represents the plurality of exceptions without distinguishing them from one another.

19. The computer implementation method according to claim 17 or 18, wherein the summary indicator is set regardless of which of the plurality of exceptions was used to determine the invalidity of the value.

20. The instruction is a neural network instruction, and the function is executed on one or more input tensors to provide one or more output tensors used for artificial intelligence processing. The computer implementation method according to claim 17.

21. The computer implementation method according to claim 17, wherein another instruction different from the aforementioned instruction uses another summary indicator.

22. The computer system according to claim 15, wherein the one or more output tensors are used for artificial intelligence processing.

23. A computer program for facilitating processing within a computing environment, The method includes one or more computer-readable storage media and program instructions for executing a method, which are stored together on the one or more computer-readable storage media. An instruction is executed which executes one of several functions defined in the instruction, and the instruction has a summary indicator which is used exclusively by the instruction and used by the several functions defined in the instruction, and the summary indicator is used to indicate that the output data obtained from the execution of the instruction which includes the one function is invalid, The method involves obtaining an indication that a value determined to be invalid based on the execution of the aforementioned instruction was included in the output data obtained as a result of one or more calculations performed by the aforementioned function, wherein the value is determined to be invalid due to one of multiple exceptions. Based on the acquisition of the indication that the aforementioned value has been determined to be invalid, the summary indicator that collectively represents the multiple exceptions is set, A computer program that includes [this].

24. The computer program according to claim 23, wherein the instruction is a neural network instruction that performs one or more calculations on one or more input tensors in order to provide one or more output tensors used for artificial intelligence processing.

25. A computer implementation method for facilitating processing within a computing environment, An instruction is executed which executes one of several functions defined in the instruction, and the instruction has a summary indicator which is used exclusively by the instruction and used by the several functions defined in the instruction, and the summary indicator is used to indicate that the output data obtained from the execution of the instruction which includes the one function is invalid, The method involves obtaining an indication that a value determined to be invalid based on the execution of the aforementioned instruction was included in the output data obtained as a result of one or more calculations performed by the aforementioned function, wherein the value is determined to be invalid due to one of multiple exceptions. Based on the acquisition of the indication that the aforementioned value has been determined to be invalid, the summary indicator that collectively represents the multiple exceptions is set, Computer implementation methods, including those mentioned above.

Images