Techniques for performing outer product operations

Abstract

The invention relates to a device having: processing circuitry to perform a vector operation; an instruction decoder to decode an instruction to control the processing circuitry to perform an associated vector operation; and an array storage device comprising storage elements to store data elements, the array storage device storing at least one two-dimensional array data element. The instruction set includes a multi-outer-product instruction identifying a first source vector operand, a second source vector operand, and a given two-dimensional array data element within the array storage forming a destination operand. At least the first source vector operand identifies at least one vector of data elements to be considered to include a plurality of sub-vectors, and at least the second source vector operand identifies a plurality of vectors of data elements. In response to the multiple outer product instruction, the instruction decoder controls the processing circuitry to perform an outer product operation for each sub-vector identified by the first source vector operand. Each outer product operation includes multiplying each data element of an associated sub-vector identified by the first source vector operand by each data element of a group of data elements selected from the second source vector operand to produce a plurality of outer product results, and using each outer product result to update a value in an associated storage element held within the storage elements of the given two-dimensional array. Selection circuitry controls selection of the data elements processed by each outer product operation to switch between vectors of the second source vector operand when switching between different sub-vectors within a given vector of the first source vector operand.

Description

Background Art

[0001] The present technology relates to the field of data processing, and more particularly to the performance of outer product operations.

[0002] Some modern data processing systems may provide array storage means for storing one or more two-dimensional array data elements, which data elements may be accessed by processing circuitry of the data processing system when performing data processing operations. This may provide an efficient mechanism for performing many different types of operations, such as outer product operations. Considering two input operand vectors, the outer product of those two vectors is a matrix of data elements produced by multiplying each data element of one operand by each data element of the other operand. If the two vectors have dimensions M and N, their outer product is an M×N matrix. Providing an array storage means that may store two-dimensional array data elements may provide a useful mechanism for storing the results of such outer product operations.

[0003] Outer product operations can be used in modern data processing systems when performing various types of calculations. For example, the use of outer product operations can be used to accelerate matrix multiplication operations. However, in order to improve efficiency and performance, it is desirable to use array storage devices efficiently and improve the utilization of processing circuits / multiply-accumulate resources provided within the data processing system. Summary of the invention

[0004] In one example arrangement, a device is provided, the device comprising: processing circuitry to perform vector operations; instruction decoder circuitry to decode instructions from an instruction set to control the processing circuitry to perform the vector operations specified by the instructions; an array storage device comprising storage elements to store data elements, the array storage device being arranged to store at least one two-dimensional array data element accessible to the processing circuitry when performing the vector operations; wherein the instruction set includes a multiple outer product instruction, the multiple outer product instruction identifying a first source vector operand, a second source vector operand and a given two-dimensional array data element within the array storage device forming a destination operand, wherein at least the first source vector operand identifies at least one vector to be treated as comprising data elements of a plurality of sub-vectors, and At least the second source vector operand identifies multiple vectors of data elements; wherein the instruction decoder circuit is arranged to control the processing circuit in response to the multiple outer product instruction to perform an outer product operation for each sub-vector identified by the first source vector operand, and each outer product operation includes multiplying each data element of the associated sub-vector identified by the first source vector operand by each data element of a group of data elements selected from the second source vector operand to produce a plurality of outer product results, and using each outer product result to update the value held in the associated storage element within the storage element of the given two-dimensional array; wherein the processing circuit includes selection circuitry for controlling the selection of these data elements processed by each outer product operation to switch between vectors of the second source vector operand when switching between different sub-vectors within a given vector of the first source vector operand.

[0005] In another example arrangement, a method for performing outer product operations is provided, the method comprising: performing vector operations using a processing circuit; using an instruction decoder circuit to decode instructions from an instruction set to control the processing circuit to perform the vector operations specified by the instructions; providing an array storage device, the array storage device including storage elements for storing data elements, the array storage device being arranged to store at least one two-dimensional array data element, the processing circuit being able to access these data elements when performing the vector operations; wherein the instruction set includes a multiple outer product instruction, the multiple outer product instruction identifying a first source vector operand, a second source vector operand and a given two-dimensional array data element in the array storage device forming a destination operand, wherein at least the first source vector operand identifies at least one of the data elements to be treated as comprising a plurality of sub-vectors. a vector, and at least the second source vector operand identifies multiple vectors of data elements; controlling the processing circuitry to perform an outer product operation for each sub-vector identified by the first source vector operand in response to decoding the multiple outer product instruction by the instruction decoder circuitry, wherein each outer product operation includes multiplying each data element of an associated sub-vector identified by the first source vector operand by each data element of a group of data elements selected from the second source vector operand to produce a plurality of outer product results, and using each outer product result to update a value held in an associated storage element within a storage element of the given two-dimensional array; and controlling selection of these data elements processed by each outer product operation to switch between vectors of the second source vector operand when switching between different sub-vectors within a given vector of the first source vector operand.

[0006] In yet another example arrangement, a computer program is provided for controlling a host data processing device to provide an instruction execution environment, the computer program comprising: a processing program logic unit for performing vector operations; an instruction decoding program logic unit for decoding instructions from an instruction set to control the processing program logic unit to perform the vector operations specified by the instructions; an array storage device emulation program logic unit for emulating an array storage device, the array storage device comprising storage elements for storing data elements, the array storage device being arranged to store at least one two-dimensional array data element, the processing program logic unit having access to the data elements when performing the vector operations; wherein the instruction set includes a multiple outer product instruction, the multiple outer product instruction identifying a first source vector operand, a second source vector operand and a given two-dimensional array data element in the array storage device forming a destination operand, wherein At least the first source vector operand identifies at least one vector to be treated as comprising data elements of a plurality of sub-vectors, and at least the second source vector operand identifies a plurality of vectors of data elements; wherein the instruction decoding program logic unit is arranged to control the processing program logic unit in response to the multiple outer product instruction to perform an outer product operation for each sub-vector identified by the first source vector operand, and each outer product operation comprises multiplying each data element of the associated sub-vector identified by the first source vector operand by each data element of a group of data elements selected from the second source vector operand to produce a plurality of outer product results, and using each outer product result to update a value held in an associated storage element within a storage element of the given two-dimensional array; wherein the processing program logic unit comprises a selection program logic unit for controlling the selection of these data elements processed by each outer product operation to switch between vectors of the second source vector operand when switching between different sub-vectors within a given vector of the first source vector operand. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The present technology will be further described, by way of illustration only, with reference to examples thereof as shown in the accompanying drawings, in which:

[0008] Figure 1 is a block diagram of a device according to an example implementation;

[0009] Figure 2 shows examples of architectural registers that may be provided within a device, including vector registers for storing vector operands and array registers for storing 2D array data elements, including examples of physical implementations of array registers;

[0010] Figure 3A and Figure 3Bschematically illustrates how access may be performed to a square 2D array within an array storage device according to one example implementation;

[0011] Figure 4 is a block diagram of a device according to one example implementation, illustrating how processing circuitry may be used to perform an outer product operation;

[0012] Figure 5A and Figure 5B is a schematic diagram showing how to implement the Figure 4 a selection circuit implementing a diagram of a selection function to switch between vectors and sub-vectors to select appropriate data elements to be processed by each outer product operation when executing a multiple outer product instruction;

[0013] Figures 6 to 11 is a diagram schematically illustrating various specific examples of multiple outer product operations that may be performed in response to executing a multiple outer product instruction according to an example implementation;

[0014] Fig. 12A and Fig. 12B illustrates how outer product results generated according to an example implementation may be used to update associated storage elements within a 2D array of an array storage device;

[0015] Fig.13 schematically illustrates fields that may be provided within a multiple outer product instruction according to one example implementation;

[0016] Fig.14 is a flowchart illustrating steps performed when decoding a multiple outer product instruction according to one example implementation; and

[0017] Fig.15 A simulator implementation that can be used is shown. DETAILED DESCRIPTION

[0018] According to an example implementation discussed herein, a device is provided having: a processing circuit for performing vector operations; and an instruction decoder circuit for decoding instructions from an instruction set to control the processing circuit to perform the vector operations specified by the instructions. An array storage device is also provided, the array storage device comprising storage elements for storing data elements. The array storage device is arranged to store at least one two-dimensional array data element, which the processing circuit can access when performing the vector operations.

[0019] As previously mentioned, the use of array storage devices can provide a useful mechanism for performing certain types of operations, such as outer product calculations. Specifically, the matrix of data elements generated as a result of performing outer product operations can be stored in the associated data elements of the two-dimensional array within the array storage device. However, it is beneficial to efficiently use the available storage elements within a given two-dimensional array because this can help increase the performance of the system. It will also be beneficial to better use the hardware multiplication-accumulation resources provided in the system, which in some specific implementations may be provided in the system for each storage element in the array to support other calculations that may occur using the two-dimensional array within the array storage device.

[0020] According to the techniques described herein, the instruction set is arranged to include "multiple outer product instructions" that identify two source vector operands and a given two-dimensional array data element within the array storage device that forms a destination operand. In addition, at least one of the source vector operands (referred to herein as the "first source vector operand", but it should be noted that this first source vector operand can be either of the two source vector operands specified by the multiple outer product instruction, and it does not need to be the first input operand specified by the instruction) identifies at least one vector of data elements that will be considered to include multiple sub-vectors. In addition, the other of the source vector operands (referred to herein as the "second source vector operand", but it should be noted that this second source vector operand can be either of the two source vector operands specified by the multiple outer product instruction, and it does not need to be the second input operand specified by the instruction) identifies multiple vectors of data elements.

[0021] The instruction decoder is arranged to control the processing circuit in response to the multiple outer product instruction to perform an outer product operation for each sub-vector identified by the first source vector operand. Each outer product operation includes multiplying each data element of the associated sub-vector identified by the first source vector operand by each data element of a group of data elements selected from the second source vector operand to produce a plurality of outer product results, and using each outer product result to update the value of the associated storage element held in the storage element of the given two-dimensional array. The group of data elements selected from the second source vector operand may depend on whether the second source vector operand is also regarded as including a plurality of sub-vectors. Therefore, in an example implementation, if the second source vector operand is regarded as including a plurality of sub-vectors, the group of data elements selected from the second source vector operand may be those data elements of the selected sub-vector belonging to the second source vector operand, but if the second source vector operand is not regarded as including a plurality of sub-vectors, the group of data elements selected from the second source vector operand may be those data elements of the selected vector belonging to the second source vector operand.

[0022] The processing circuitry further comprises selection circuitry for controlling selection of the data elements processed by each outer product operation to switch between vectors of the second source vector operand when switching between different sub-vectors within a given vector of the first source vector operand.

[0023] The inventors of the present case understand that in some example use cases, when an outer product operation is performed using two source vectors, it may be the case that the size of one or both of those source vectors is smaller than the corresponding size of the two-dimensional array used to store the result of the outer product operation. This can lead to inefficient use of the storage elements of the two-dimensional array, because a large number of those storage elements are then unusable, and can also lead to inefficient use of resources of hardware components that form processing circuits (which may be able to perform calculations to produce results for each of the storage elements). However, according to the techniques described herein, a single instruction (i.e., the multiple outer product instruction discussed above) can be defined to cause multiple outer product operations to be performed by using sub-vectors within one or both of the source vector operands, wherein the result of each outer product operation is stored in an associated storage element of a two-dimensional (2D) array. By causing multiple outer product operations to be performed in response to a single instruction (in an example implementation, those multiple outer product operations can be performed in parallel), this can significantly improve throughput while also making more efficient use of available storage elements within the array storage device.

[0024] Through execution of the multiple outer product instructions described herein, one or more rows and / or columns of the 2D array may be arranged to capture results produced for more than one outer product operation by using more than one sub-vector provided within a given input vector when calculating outer product results used to update those one or more rows and / or columns.

[0025] Expressed in another way, for each vector register within a single source vector operand, a separate outer product operation is performed, and thus in one example implementation, the total number of outer products is determined by the product of the number of vectors provided by both source vector operands. In addition, each outer product operation uses a subset of the input data elements (sub-vectors) of at least one source vector operand.

[0026] Within any given vector of data elements that are to be considered to include a plurality of sub-vectors, the various sub-vectors may be considered to occupy an associated sub-vector region within the given vector of data elements. In some cases, each sub-vector may occupy the entirety of the associated sub-vector region, but in other example implementations, some data element positions within the given vector may be unused, and thus one or more of the sub-vectors may not occupy the entire associated sub-vector region. It should also be noted that any particular sub-vector region need not be provided by consecutive data element positions within a given vector, and the various data elements forming any particular sub-vector therefore need not be provided consecutively within a given vector.

[0027] The selection circuitry may take various forms, but in one example implementation, for each multiplication operation to be performed by the processing circuitry to produce a corresponding outer product result, the selection circuitry may include associated multiplexer circuitry to select data elements from the first source vector operand and data elements from the second source vector operand to be subject to the multiplication operation, wherein the selection made by the associated multiplexer circuitry depends on which outer product operation the corresponding outer product result is controlled with respect to. When a hardware implementation provides separate multiplier circuitry for each of the multiplication operations to be performed, such an approach may facilitate the various multiplication operations being performed in parallel.

[0028] Although at least the first source vector operand identifies a plurality of sub-vectors (provided in one or more vectors), and at least the second source vector operand identifies a plurality of vectors (as previously mentioned, any one of the source vector operands specified by the instruction may be considered as the first source vector operand, and the other source vector operand will be considered as the second source vector operand), both the first source vector operand and the second source vector operand may identify a plurality of sub-vectors, and indeed both the first source vector operand and the second source vector operand may also identify a plurality of vectors. In a particular example implementation, both the first source vector operand and the second source vector operand identify a plurality of vectors of data elements, and each vector will be considered as including a plurality of sub-vectors. Using such an arrangement, at least four outer product operations may be performed in response to a single multiple outer product instruction. Although the number of sub-vectors specified by each source vector operand may be different in principle in such an implementation, in a particular example arrangement, each of the first source vector operand and the second source vector operand contains N sub-vectors, and the processing circuit is arranged to perform N outer product operations.

[0029] In an example implementation, wherein the first source vector operand and the second source vector operand both identify multiple vectors of data elements, each vector comprising multiple sub-vectors, the multiple sub-vectors within each vector of the first source vector operand may be considered to have associated sub-vectors within a different vector of the second source vector operand. The selection circuitry may then be arranged to control selection of these data elements processed by each outer product operation such that when switching between different sub-vectors within a given vector of the first source vector operand, switching to a different vector of the second source vector operand causes these data elements from these associated sub-vectors within the second source vector operand to be selected.

[0030] In an alternative example implementation, it may be the case that only one of the source vector operands identifies a subvector. For example, the first source vector operand may contain a plurality of P subvectors, and the second source vector operand may include P vectors of data elements, wherein each vector in the second source vector operand is associated with one of the subvectors in the first source vector operand. Such example implementations still allow multiple outer product operations to be performed in response to a single instruction, and allow for efficient use of available storage elements within a two-dimensional array.

[0031] In one such example arrangement, the processing circuitry may be arranged to perform P outer product operations, wherein each outer product operation is performed using as input an associated sub-vector from the first source vector operand and an associated vector from the second source vector operand.

[0032] There are various ways in which sub-vectors may be specified within a given vector of a source vector operand. In one example implementation, for at least one given vector to be considered to include a plurality of sub-vectors, the data elements forming each sub-vector are provided at consecutive data element positions within the given vector.

[0033] However, the data elements forming the sub-vectors need not be provided at consecutive data element positions. In fact, for at least one given vector that will be considered to include multiple sub-vectors, the data elements forming each sub-vector may be provided at non-consecutive data element positions within the given vector. Such an approach allows for greater flexibility in how the sub-vectors are arranged within a particular vector, allowing, for example, data elements of one sub-vector to be interleaved with data elements of another sub-vector. Although in some implementations, one or more vectors with interleaved sub-vector elements and one or more vectors with consecutive sub-vector elements, in one example implementation, the same scheme is used for each of the vectors containing sub-vectors.

[0034] In some implementations, any given vector that is considered to be formed from a plurality of sub-vectors will be arranged so that each data element position contains a valid data element of one of the sub-vectors. However, this is not necessary, and in alternative implementations, for at least one given vector that is considered to include a plurality of sub-vectors, the given vector may have one or more unused data element positions that do not contain data elements of the plurality of sub-vectors. Although this may result in some storage elements within the destination 2D array being unused, it still facilitates multiple outer product operations to be performed in response to a single instruction when compared to existing schemes (which only perform a single outer product operation in response to a single instruction), thereby facilitating improved performance / throughput, and also facilitating improved use of the 2D array. There are several ways in which unused data element positions may be identified, but in one example implementation, a predicate technique is used, such as by specifying a predicate vector operand associated with one or more of the source vector operands, such predicate vector operand providing one or more vectors of predicate values ​​to identify, based on the data element position, whether the data element position contains a data element to be included in the outer product operation at the data element position.

[0035] There are various ways in which source vector operands for a multiple outer product instruction may be specified. However, in one example implementation, the apparatus further comprises a set of vector registers accessible by the processing circuitry, wherein each vector register is arranged to store a vector comprising a plurality of data elements, and the first source vector operand and the second source vector operand comprise vectors contained within a vector register of the set of vector registers. Thus, the multiple outer product instruction may identify each source vector operand by specifying that the contents of one or more vector registers of the set of vector registers are used to form the source vector operand.

[0036] In one example implementation, the vector length identifies the size of the vector registers in the set of vector registers and the size of the given two-dimensional array data element within the array storage device (e.g., the vector length may be used to specify both the x-dimension and the y-dimension of the two-dimensional array). Some architectures may support variable vector lengths, where the vector length may be fixed for any particular instance of the device, but where the vector length may vary between different instances of the device, and the same instruction may be executed on any of these different instances of the device. The techniques described herein may be particularly advantageously employed within devices where vector lengths are relatively large, as there may be more cases where outer product operations to be performed use one or more vectors / sub-vectors that are smaller than the specified vector length, and therefore more opportunities to use the present techniques to improve performance and 2D array usage.

[0037] In one example implementation, the multiple outer product instruction may be arranged to provide a sub-vector indicator that is used to determine the number of sub-vectors within each vector that is to be considered to include a plurality of sub-vectors, wherein the size of each sub-vector depends on the determined number of sub-vectors and the vector length. In a particular example implementation, the sub-vector indicator may be specified in a manner independent of the length of the vector. For example, the sub-vector indicator may be arranged to identify that the vector is divided into two sub-vector regions (e.g., by identifying that each sub-vector region is half the length of the vector), four sub-vector regions (e.g., by identifying that each sub-vector region is one-quarter the length of the vector), etc., wherein the actual size of each sub-vector region depends on the vector length.

[0038] Although in some implementations, each sub-vector may occupy the entire associated sub-vector region, this is not necessary, and in alternative implementations, the sub-vector may occupy only a portion of the associated sub-vector region, with the remainder being unused (i.e., including one or more unused data element positions). In such cases, there are various ways in which the unused data element positions may be handled. For example, the hardware may use data elements in all data element positions to compute a result, and then simply ignore later undesired results (i.e., effectively processing the input as if there were no unused data element positions). Alternatively, however, the predicate techniques mentioned earlier may be used to identify individual data element positions that should not be used when performing outer product calculations. Using such predicate techniques may avoid generating undesired results, and thus more easily facilitate merging valid result data elements with the existing contents of the 2D array.

[0039] The subvector indicator can be specified in a variety of ways. For example, the subvector indicator can be an explicit field provided in the instruction, or alternatively can be implicitly specified by the part of the opcode in order to define the instruction. Therefore, the specific form of the multi-outer product instruction can have an explicit subvector indicator field to allow the number of specified subvectors, or alternatively there can be different variants of the multi-outer product instruction for each of the different numbers of subvectors to be supported. In an example implementation in which an explicit subvector indicator field is provided, this can, for example, allow the subvector indicator to be set at run time, for example, by providing a register identifier in the subvector indicator for a register that defines the number of subvectors for its content.

[0040] The outer product operation performed in response to the multiple outer product instruction can take a variety of forms. However, in an example implementation, the multiple outer product instruction is an accumulation instruction, and each outer product result is used to update the existing value by combining the outer product result with the existing value in the associated storage element maintained in the storage element of the given two-dimensional array. The manner in which the outer product result is combined with the existing value can vary, depending on the specific implementation, but in an example implementation, it can involve adding the outer product result to the existing value or subtracting the outer product result from the existing value. When performing such an accumulation operation, the use of a two-dimensional array can be particularly advantageous because multiple iterations of the outer product operation can be performed, each of which produces a result accumulated in the two-dimensional array.

[0041] Although in one example arrangement, there will be a one-to-one correspondence between each outer product result generated and its corresponding associated storage element in the two-dimensional array, this is not necessary, and in other example implementations, the outer product operation performed can cause multiple outer product results generated to be associated with the same storage element in the two-dimensional array. By means of a specific example, the multi-outer product instruction can be the sum of the outer product instructions, generating multiple outer product results with the same associated storage element in the storage element of the given two-dimensional array, and combining those multiple outer product results to update the value maintained in the associated storage element. For example, when updating the value maintained in the associated storage element, each of the multiple outer product results associated with the same storage element can be added together. As in the earlier example, cumulative variants can be supported so that the resulting sum of the various outer product results associated with a specific storage element is subsequently added to the current value in the associated storage element or subtracted therefrom, so as to generate a new value to be stored in the storage element. When performing such summation of the outer product operation, it is generally the case that the individual data element provided in the source vector operand is less than the data element size associated with each storage element in the two-dimensional array.

[0042] The techniques described herein provide a great deal of flexibility in how various source vector operands are specified, including specifying how many sub-vectors are within those source vector operands, and allow for improved performance and use of 2D arrays in many different scenarios. By way of specific example only, in one particular use case, both the first source vector operand and the second source vector operand include two vectors, each vector being formed by two sub-vectors, and the instruction decoder circuit is arranged to control the processing circuitry to perform four outer product operations in response to the multiple outer product instruction, wherein the results of those four outer product operations are stored in storage elements within associated regions of storage elements of the given two-dimensional array. In a specific implementation where the sub-vectors fully occupy each vector, this may allow the entirety of the 2D array to be used to store the results of the four outer product operations.

[0043] Specific example implementations will now be discussed with reference to the figures.

[0044] Figure 1 A data processing system 10 is schematically shown which includes a processor 20 coupled to a memory 30 which stores data values ​​32 and program instructions 34. The processor 20 includes an instruction fetch unit 40 for fetching program instructions 34 from the memory 30 and supplying these fetched program instructions to an instruction decoder circuit 50. The decoder circuit 50 decodes these fetched program instructions and generates control signals to control the processing circuit 60 to perform processing operations on the data values ​​held in the storage elements of the register storage device 65 as specified by the decoded vector instructions. Figure 1 As shown, register storage device 65 may be formed of a plurality of different blocks. For example, a scalar register file 70 may be provided, which includes a plurality of scalar registers that can be specified by instructions, and similarly, a vector register file 80 may be provided, which includes a plurality of vector registers that can be specified by instructions.

[0045] like Figure 1 As shown, the processor 20 can access the array storage device 90. Figure 1 In the example shown, array memory device 90 is provided as part of processor 20, but this is not required. In various examples, the array memory device may be implemented as any one or more of the following: architecturally addressable registers; non-architecturally addressable registers; high-speed scratchpad memory; and cache.

[0046] In an example implementation, the processing circuit 60 may include both vector processing circuitry and scalar processing circuitry. The general difference between scalar processing and vector processing is as follows. Vector processing may involve applying a single vector processing instruction to a data element of a data vector, which has multiple data elements at corresponding positions in the data vector. The processing circuit may also perform vector processing to perform operations on multiple vectors in a two-dimensional array of data elements (which may also be referred to as subarrays) stored in the array storage device 90. Scalar processing effectively operates on a single data element, but not on a data vector. Vector processing can be used in instances where processing operations are performed on many different instances of data to be processed. In a vector processing arrangement, a single instruction may be applied to multiple data elements (of a data vector) simultaneously. Compared to scalar processing, this can improve the efficiency and throughput of data processing.

[0047] The processor 20 may be arranged to process two-dimensional array data elements stored in the array storage device 90. In at least some examples, the two-dimensional array may be accessed in multiple directions as one-dimensional vector data elements. In one example implementation, the array storage device 90 may be arranged to store one or more two-dimensional array data elements, and each two-dimensional array data element may form a square array portion of a larger or even higher dimensional array data element in memory.

[0048] Figure 2 An example of an architectural register 65 of a processor 20 that may be provided in one example implementation is shown. The architectural registers (as defined in an instruction set architecture (ISA)) may include a set of scalar integer registers 100 that function as general purpose registers for processing operations performed by scalar processing circuitry within the processing circuitry 60. For example, there may be a number of general purpose registers 100, such as 31 registers X0 to X30 provided in this example (the 32nd encoding of the scalar register field may not correspond to a register provided in hardware, as it may be considered by default to indicate a value such as zero, or may be used to indicate a special type of register that is not a general purpose register). Accessing scalar registers of different sizes mapped to the same physical storage device is possible. For example, register labels X0 to X30 may refer to a 64-bit register, but the same register may also be accessed as a 32-bit register (e.g., using the lower 32 bits of each 64-bit register provided in hardware for access), in which case register labels W0 to W30 may be used in assembly code to refer to the same register.

[0049] Furthermore, the architectural registers selectable by program instructions in the ISA supported by decoder 50 may include a number of vector registers 105 (labeled Z0 to Z31 in this example). Figure 2 The several scalar / vector registers shown in are not necessary, and other examples may provide different numbers of registers that may be specified by program instructions. Each vector register may store a vector operand including a variable number of data elements, wherein each data element may represent an independent data value. In response to a vector processing (SIMD) instruction, the processing circuit may perform vector processing on the vector operand stored in the register to produce a result. For example, vector processing may include channel-by-channel operations, wherein the corresponding operation is performed on each element channel in one or more operand vectors to produce the corresponding result of the element of the result vector. When performing vector or SIMD processing, each vector register may have a certain vector length VL, wherein the vector length refers to the number of bits in a given vector register. The vector length VL used in the vector processing mode may be fixed or variable for a given hardware implementation. The ISA supported by the processor 20 may support variable vector lengths, so that different processors may be selected to implement vector registers of different sizes, but the ISA is vector length-independent, so that the instruction is designed so that the code can function correctly regardless of the specific vector length implemented on the given CPU that executes the program.

[0050] Vector registers Z0 to Z31 may also function as operand registers for storing vector operands, which provide input to processing and accumulation operations performed by processing circuitry 60 on two-dimensional array data elements stored in array storage device 90. When a vector register is used to provide input to such an operation, the vector register has a vector length MVL, which may be the same as the vector length VL used for the vector operation, or may be a different vector length.

[0051] like Figure 2 As shown, the architecture registers also include the array storage device 90 (ZA0 to ZA0N) mentioned above. A -1)) a certain number N A Array registers 110 are provided. Each array register may be viewed as a set of register storage devices for storing a single 2D array data element (e.g., the result of a process and accumulate operation). However, process and accumulate operations may not be the only operations that may use array registers. Array registers may also be used to store square arrays while performing a row / column transposition of the array structure in memory. When a program instruction refers to one of the array registers 110, it is referenced as a single entity using an array identifier ZAi, but some types of instructions (e.g., data transfer instructions) may also select a sub-portion of the array (e.g., a horizontal / vertical group of elements) by defining an index value that selects a portion of the array.

[0052] In practice, the physical implementation of the register storage devices corresponding to these array registers may include a certain number N R There are array vector registers ZAR0 to ZAR(N R -1), such as Figure 2 As shown. The array vector registers ZAR forming the array register storage device 110 may be a set of registers different from the vector registers Z0 to Z31 used for SIMD processing and vector input to array processing. Each of the array vector registers ZAR may have a vector length MVL, so each array vector register ZAR may store a 1D vector of length MVL, which may be logically divided into a variable number of data elements. For example, if MVL is 512 bits, this may be a set of 64 8-bit elements, 32 16-bit elements, 16 32-bit elements, 8 64-bit elements, or 4 128-bit elements. It should be understood that not all of these options need to be supported in a given specific implementation. By supporting variable element sizes, this provides flexibility in handling calculations involving data structures of different precisions. In order to represent 2D array data, a group of array vector registers ZAR0 to ZAR(N R -1) can be logically viewed as assigning array register identifiers ZA0 to ZA(N A-1) such that a 2D array is formed with elements extending within a single vector register corresponding to one dimension of the array, and elements in the other dimension of the array striped across multiple vector registers.

[0053] It may be useful (although not necessary) to arrange the array registers ZA so that they store square array data, where the number of elements in the horizontal direction is equal to the number of elements in the vertical direction. This may help support on-the-fly transposition of the array, where by providing support for reading / writing the array registers 110 in either the horizontal or vertical direction, the row / column dimensions of the array structure in memory may be swapped when transferring the array structure between the array registers 110 and memory. By providing support for writing / reading data to / from a 2D array register in either the horizontal or vertical direction, this may allow data loaded from memory in one direction (e.g., row by row) to be written back to memory in the opposite direction (e.g., column by column), potentially faster than using several gather / scatter load / store or permute operations to transfer data between memory and the vector registers.

[0054] As discussed above, the processing circuitry 60 is arranged to access the scalar registers 70, the vector registers 80 and / or the array storage device 90 under the control of instructions decoded by the decoder circuitry 50. Figure 3A Further details of this latter arrangement are described, which merely provides one illustrative example of how an array storage device may be accessed, particularly considering accessing a square 2D array within the array storage device.

[0055] In the illustrated example, the square 2D array within the array storage device 90 is arranged as an array 205 of n×n storage elements / locations 200, where n is an integer greater than 1. In this example, n is 16, which means that the granularity of access to the storage locations 200 is 1 / 16 of the total storage in either the horizontal or vertical array direction.

[0056] From the perspective of the processing circuitry, the array of n×n locations is accessible as n linear (one-dimensional) vectors in a first direction (e.g., the horizontal direction as shown) and n linear vectors in a second array direction (e.g., the vertical direction as shown). Thus, from the perspective of the processing circuitry 60, the n×n storage locations are arranged or at least accessible as 2n linear vectors, each having n data elements.

[0057] The array of storage locations 200 is accessible by access circuits 210 , 220 , column select circuits 230 , and row select circuits 240 under the control of control circuitry 250 in communication with at least processing circuitry 60 and optionally decoder circuitry 50 .

[0058] refer to Figure 3B In the case of an example square 2D array designated as "A1" (note that more than one such 2D array may be provided within array storage device 90, such as A0, A1, A2, and so on, as discussed below), the n linear vectors in a first direction (the horizontal or "H" direction as shown) may each have 16 data elements 0...F (expressed in hexadecimal notation) and may be referred to in this example as A1H0...A1H15. The ... each have 16 data elements 0...F (expressed in hexadecimal notation) and may each have 16 data elements 0...F (expressed in hexadecimal notation) and may each have 16 data elements 0...F (expressed in hexadecimal notation) and may each have 16 data elements 0...F ( Figure 3B . . A1V15. It should be noted that, for example, data element 260 is referenced as item F of A1H0, rather than item 0 of A1V15. It should be noted that the use of "H" and "V" does not indicate any spatial or physical layout requirements regarding the storage of the data elements that make up array storage device 90, nor does it have any relevance as to whether the 2D arrays within the array storage device store row data or column data in any example application.

[0059] Figure 4 8 is a block diagram of a device specifically implemented according to an example, which shows how to use processing circuits to perform outer product operations. A vector register file 80 provides a plurality of vector registers that can be used to store vectors of data elements. As previously discussed, multiple outer product instructions can be arranged to identify a first source vector operand 300 and a second source vector operand 320. At least the first source vector operand 300 is arranged to identify at least one vector of data elements 305 that will be considered to include a plurality of sub-vectors, and at least the second source vector operand 320 is arranged to identify a plurality of vectors of data elements 325, 330. It should be noted that the terms "first" and "second" used herein refer to two source vector operands used purely as tags to distinguish between two source vector operands, and do not imply any particular ordering of how these operands are specified by instructions. Therefore, any one of the source operand fields of the instruction can be used to specify the first source vector operand referred to above, and the other of the source operand fields will be used to specify the second source vector operand referred to above.

[0060] Furthermore, while according to the techniques described herein, at least one of the two source vector operands will identify multiple sub-vectors, and the other source vector operand may not, it may also be the case that, in some example implementations, both source vector operands identify multiple sub-vectors. Similarly, the two source vector operands may specify multiple vectors, and thus the first source vector operand 300 may include not only the first vector 305, but also the second vector 310. Furthermore, the number of vectors specified by any particular source vector operand is not limited to one or two vectors, but in fact more vectors may be specified by a particular source vector operand (e.g., four vectors or eight vectors).

[0061] The processing circuitry 60 is controlled in dependence upon control signals received from the decoder circuitry 50, and when the decoder circuitry 50 decodes the multiple outer product instruction mentioned earlier, it sends control signals to the processing circuitry to control the processing circuitry to perform an outer product operation for each sub-vector identified by the first source vector operand. As part of this process, those control signals will control the selection circuitry 340 provided by the processing circuitry 60 to select the appropriate data element to be processed by each outer product operation. Each outer product operation comprises multiplying each data element of the associated sub-vector identified by the first source vector operand by a group of data elements (those of one of the sub-vectors or vectors) selected from the second source vector operand to produce a plurality of outer product results, and then using each outer product result to update the value in the associated storage element within the storage element of a given two-dimensional array 380 held within the array storage device 90.

[0062] The selection circuit 340 may be organized in a variety of ways, but in one example implementation includes a multiplexer circuit that provides for each multiplier used to produce an outer product result from two input data elements, the multiplexer circuit being used to select the appropriate two input data elements for each multiplier. The selection circuit controls the selection of these data elements processed by each outer product operation to switch between the vectors 325, 330 of the second source vector operand when switching between different sub-vectors within a given vector 305 or 310 of the first source vector operand.

[0063] Next, the selected input data elements are forwarded to the multiplication circuit 350, which, as mentioned above, may include a multiplier circuit for each outer product result to be generated in one example implementation. Each outer product result is generated by multiplying two input data elements provided to a corresponding multiplier within the multiplication circuit 350. The outer product results may be provided directly to the array update circuit 370, which is used to update the storage elements within the 2D array 380, each outer product result having an associated storage element within the 2D array 380 and to update the value held in the associated storage element. However, it is typically the case that the outer product operation performed is an accumulation operation, and each outer product result generated is combined with an existing value stored in an associated storage element of the 2D array 380 (e.g., by adding the outer product result to the existing value or subtracting the outer product result from the existing value) (using the optional accumulation circuit 360). Although the multiplication circuit 350 and the optional accumulation circuit 360 are shown as Figure 4 In one example implementation, they may be provided as a combined block formed by a multiply-accumulate circuit.

[0064] The array update circuit 370 is used to control access to the associated storage elements within the 2D array 380 to ensure that each value received by the array update circuit is used to update the associated storage element within the 2D array 380. In one example, the array update circuit 370 may use a reference Figure 3A The access components 210 to 250 described earlier are implemented.

[0065] Outer product operations are effectively used within data processing systems for a variety of different reasons, and thus the ability to perform multiple outer product operations in response to a single instruction can provide significant performance / throughput improvements, as well as making more efficient use of available storage resources provided by the two-dimensional array within the array storage device 90. As to how the outer product operation can be used, by way of example only, it can be used to implement a matrix multiplication operation. A matrix multiplication can, for example, involve multiplying a first M×K matrix of data elements by a K×N matrix of data elements to produce an M×N matrix of data elements as a result. This operation can be decomposed into multiple outer product operations (more specifically, K outer product operations, where K can be referred to as the depth), where each outer product operation involves performing a sequence of multiply-accumulate operations to multiply each data element of the M vectors of data elements from the first matrix by each data element of the N vectors of data elements from the second matrix to produce an M×N matrix of result data elements stored within the 2D array. The results of the multiple outer product operations can be accumulated within the same 2D array to produce the M×N matrix that has been produced by performing the matrix multiplication mentioned earlier.

[0066] Figure 5A Schematically showing the Figure 4The selection functions 415, 420 performed by the selection circuit 340 of , for an example use case, wherein a first source vector operand specifies a vector 400 formed of two sub-vectors, and a second source vector operand specifies two vectors 405, 410, wherein each of these vectors is treated as a single vector of data elements (rather than being divided into sub-vectors).

[0067] In this example implementation, execution of the multiple outer product instruction will result in two outer product operations to be performed, where the selection function 415 selects the data elements for the first outer product operation and the selection function 420 selects the data elements to be used for the second outer product operation. Figure 5A As shown, for the first outer product operation, the first sub-vector within the vector 400 of the first source vector operand is used in combination with the first vector 405 of the second source vector operand. Similarly, for the second outer product operation, the second sub-vector within the vector 400 of the first source vector operand is used in combination with the second vector 410 of the second source vector operand. Therefore, it can be seen that when switching between different sub-vectors within the first source vector operand 400, the selection circuit switches between different vectors 405, 410 of the second source vector operand.

[0068] Figure 5B Another example is shown, in which both source vector operands include two vectors, and each of these vectors is considered to include two sub-vectors. Thus, the first source vector operand includes two vectors 425, 430, and the second source vector operand includes two vectors 435, 440. In this case, four outer product operations are performed, each outer product operation having an associated selection function 445, 450, 455, 460 performed by the selection circuit 340. As can be seen, the selection function 445 associated with the first outer product operation uses the first sub-vector within the first vector 425 of the first source vector operand combined with the first sub-vector of the first vector 435 of the second source vector operand, and the selection function 450 associated with the second outer product operation uses the second sub-vector within the first vector 425 of the first source vector operand combined with the third sub-vector provided by the second vector 440 of the second source vector operand. Similarly, the selection function 455 associated with the third outer product operation uses the third sub-vector provided by the second vector 430 of the first source vector operand to combine with the second sub-vector of the first vector 435 of the second source vector operand, and the selection function 460 associated with the fourth outer product operation uses the fourth sub-vector provided by the second vector 430 of the first source vector operand to combine with the fourth sub-vector provided by the second vector 440 of the second source vector operand. Again, it can be seen that when switching between different sub-vectors within a given vector of the first source vector operand (either the first vector 425 or the second vector 430), the selection circuit switches between different vectors of the second source vector operand.

[0069] Figure 6 is a diagram schematically illustrating one specific example of multiple outer product operations that may be performed in response to an execute multiple outer product instruction. In this case, the two source vector operands specify two vectors 465, 470 and 475, 480, respectively, and each of those four vectors is considered to include two sub-vectors. This allows four outer product operations to be performed in response to a single instruction, Figure 6 These outer product operations are schematically illustrated as problems 1 to 4 in FIG. As can be schematically seen by the illustrated 2D array 490, the results of each of those four outer product operations can be stored within a single 2D array. Specifically, in this case, each outer product operation produces a 2×2 matrix that can be stored within an associated quarter portion of the 2D array 490. This therefore allows the entire 2D array to be utilized, providing particularly efficient utilization of the 2D array, and significantly increasing performance / throughput by avoiding the need to execute separate instructions to implement each of the outer product operations. In this example use case, the multiple outer product instruction is referred to as the FMOPA4 (floating point multiplication outer product and accumulation capable of performing up to 4 outer products) instruction.

[0070] Figure 7 Another example use case is shown, in which only one of the source vector operands identifies multiple sub-vectors. Specifically, one source vector operand specifies a single vector 510 formed by two sub-vectors, and the other source vector operand identifies two vectors 500, 505, each of which is considered to be a single vector including data elements. In this case, it can be seen that two outer product operations are performed, and the entire 2D array 515 is utilized again, so a particularly efficient and efficient method for implementing two outer product operations is provided. The multi-outer product instruction is again referred to as the FMOPA4 instruction. This particular instruction identifies array register ZA0 as a 2D array 515, identifies that one of the source vector operands is formed by two vector registers z0 and z1, and identifies that the other source vector operand is formed by vector register z4. The suffix ".s" indicates that the processed data is a single precision floating point data. In this particular example, the instruction can also specify the predicates of both the first source vector operand and the second source vector operand, allowing outer product operations to be performed on selected data elements in various vectors forming the two source vector operands, while excluding other data elements. The " / M" suffix indicated next to the predicate stands for "Merge", meaning that when the outer product operation has been performed, the current values ​​stored in any storage elements of the 2D array associated with the predicated (i.e., unused) data elements will remain unmodified. This is in contrast to the " / Z" suffix, which would indicate that such values ​​should be set to zero.

[0071] Figure 8Still another example of multiple outer product operations that may be performed in response to a multiple outer product instruction is shown, in this case the instruction is referred to as the FMOPA16 (Floating Point Multiplication Outer Product and Accumulation Capable of Performing Up to 16 Outer Products) instruction. In this case, one of the source vector operands specifies a single vector 528 consisting of four sub-vectors, while the other source vector operand identifies four vectors 520, 522, 524, 526, each of which is considered to be a single vector comprising data elements. As can be seen, four outer product operations are performed in this example, each producing a 2×8 matrix, and all outer product results are accommodated within a single 2D array 530.

[0072] Fig. 9 Another example scenario is shown, where each of the source vector operands is formed of two vectors 536, 538 and 532, 534, respectively, and each of the vectors is considered to include two sub-vectors. In this example, the sub-vectors do not actually occupy the entirety of the sub-vector region. Specifically, in this example use case, each sub-vector includes three data elements, and the associated sub-vector region provides four data element positions within the associated vector. Again, four outer product operations are performed, in which case each outer product operation produces a 3×3 matrix that can be stored within the associated region of a single 2D array 540.

[0073] Fig.10 Another example use case is shown where each of the source vector operands comprises two vectors 546, 548 and 542, 544, respectively, and each vector is treated as comprising two sub-vectors. However, in this case, the data elements forming each sub-vector are not located consecutively within its associated vector, but instead, the data elements of different sub-vectors are interleaved. This can be used to support such flexibility because it can avoid the need to rearrange the data elements within the vectors before performing the required outer product operation. Again, as Fig. 9 , it can be seen that each sub-vector includes three data elements, and each outer product operation produces a 3×3 matrix that can be accommodated within a single 2D array 550. However, in this example, the constituent data elements of each matrix are not stored in adjacent storage elements of the 2D array 550, but instead, the storage elements associated with the results of each outer product operation are separated from each other within the 2D array.

[0074] In reference Figures 6 to 10 In the example discussed, each outer product result generated when each outer product operation is performed has an associated storage element, and only one outer product result is used to update each associated storage element. However, the techniques described herein can also be used when performing other types of outer product operations (such as summation of outer product operations). Such use cases are described in Fig.11, where a multiple outer product instruction (in this case, multiple summations of outer product instructions) is used to perform four summations of outer product operations. As can be seen, each of the two source vector operands specifies two vectors 556, 558 and 552, 554, respectively, each of which is considered to be formed by two sub-vectors. However, the size of the data elements within each sub-vector is reduced, and the two outer product results are associated with each storage element within the 2D array 560. Therefore, by way of a specific example, the first outer product result calculated by multiplying a0 by i0 and the second outer product result calculated by multiplying b0 by j0 are both added together to produce a value for updating the associated storage element 562. In this example, it can be seen that four summations of outer product operations (eight original outer product operations) are performed, each producing a 4×4 matrix, and the results of each of those outer product operations can all be accommodated within a single 2D array 560. In this example, the instruction is called a BFMOPA4 instruction, where the "B" in "BFMOPA4" indicates a BFloat16 data type.

[0075] Fig. 12A shows how the outer product result can be associated with a specific storage element in a 2D array. Fig. 12A In the example, a data element 570 from a first source vector operand is multiplied by a data element 572 from a second source vector operand using a multiplication function 574 to produce an outer product result, which is then subjected to an accumulation operation by an accumulation function 576 to add the outer product result to a current value stored in an associated storage element 578 (or to subtract the outer product result from the current value stored in the associated storage element 578) to produce an updated value that is then stored in the associated storage element.

[0076] Fig. 12B The summation of outer product operations is shown, wherein two outer product results are associated with the same storage element in a 2D array. In this example, a data element 580 from a first source vector operand is multiplied by a data element 582 from a second source vector operand using a multiplication function 584 to produce a first outer product result. Similarly, a data element 586 from a first source vector operand is multiplied by a data element 588 from a second source vector operand by a multiplication function 590 to produce a second outer product result. Then, an addition function 592 is used to add the two outer product results together, and an accumulation function 594 is performed to produce an updated data value for storage in an associated storage element 596. Therefore, it should be understood that in some implementations, more than one outer product result associated with the same storage element in a 2D array may exist.

[0077] Fig.13605 is a diagram schematically illustrating a field that can be provided in a multiple outer product instruction according to an example implementation. The opcode field 605 is used to identify the type of instruction, in this case, to identify that the instruction is a multiple outer product instruction. The subvector indicator field 610 can then be used to identify the number of subvectors in each of one or more vectors that are considered to include subvectors. As previously discussed, in an example implementation, the subvector indicator value in field 610 can be specified in a manner independent of the vector length, such as by specifying each subvector region as a specific fraction of the vector length (so that the actual size of each subvector region depends on the vector length, and the vector length does not need to be known when specifying the subvector indicator value). Also as previously discussed, in an alternative example implementation, an explicit subvector indicator field 610 may not be provided, and instead, an indication of the subvector size may be directly encoded in the opcode field 605, effectively providing different variants of instructions for different subvector sizes.

[0078] One or more other control information fields 615 may be provided, for example, to identify one or more predicates as previously mentioned. Field 620 is then used to identify one of the source vector operands, for example, by specifying one or more vector registers in the vector register file 80, which is used to provide the data elements of the source vector operand. Similarly, field 625 may be used to identify another source vector operand, again for example, by specifying one or more vector registers in the vector register file 80, which is used to provide the data elements of the source vector operand. As previously discussed, any one of fields 620, 625 may be used to specify the first source vector operand mentioned earlier, wherein another field specifies the second source vector operand. Finally, field 630 may be used to identify a destination 2D array in the array storage device 90, which is used to store a matrix generated as a result of performing a plurality of outer product operations specified by a plurality of outer product instructions.

[0079] Fig.14 6 is a flow chart showing the steps performed to decode a multiple outer product instruction according to an example implementation. At step 650, it is determined whether a multiple outer product instruction has been encountered. If not, a standard decode of the relevant instruction is performed at step 655, where the processing circuit is then controlled to perform the required operation defined by the instruction.

[0080] However, if a multiple outer product instruction is encountered, then at step 660, the instruction is decoded to identify both the source vector operands, the destination 2D array, the required sub-vector information, and the form of the outer product to be performed (e.g., whether an accumulate outer product is being performed, or a non-accumulate variant is being performed, and also, for example, whether a normal outer product operation is being performed or a sum of outer product operations is being performed).

[0081] Next, at step 665, the processing circuitry is controlled to perform the required outer product operations and to perform the required updates to the 2D array storage elements. As part of this process, the selection circuitry is controlled to select the data elements for each multiplication operation depending on the identified subvectors. As previously discussed, this involves switching between vectors of the second source vector operand when switching between different subvectors in any given vector of the first source vector operand.

[0082] Fig.15 The simulator that can be used is shown to be implemented. Although the examples described previously implement the present invention with the apparatus and method for operating the specific processing hardware supporting the technology of interest, it is also possible to provide an instruction execution environment according to the examples described herein, which is implemented by using a computer program. Such computer programs are often referred to as simulators because they provide a software-based implementation of a hardware architecture. The types of simulator computer programs include emulators, virtual machines, models, and binary converters (including dynamic binary converters). In general, the simulator is implemented to run on a host processor 715 that optionally runs a host operating system 710 and supports the simulator program 705. In some arrangements, there may be multiple layers of simulation between the hardware and the provided instruction execution environment and / or multiple different instruction execution environments provided on the same host processor. Historically, powerful processors have been required to provide simulator implementations that are executed at reasonable speeds, but such methods may be justified in some cases, such as when another processor native code needs to be executed because of compatibility or reuse reasons. For example, the simulator may provide an instruction execution environment with additional functionality that is not supported by the host processor hardware, or provide an instruction execution environment that is generally associated with different hardware architectures. An overview of simulation is given in "Some Efficient Architecture Simulation Techniques" (Robert Bedichek, Winter 1990 USENIX Conference, pages 53-63).

[0083] Where implementations have previously been described with reference to specific hardware architectures or features, in a simulated implementation, equivalent functionality may be provided by a suitable software architecture or feature. For example, a specific circuit may be provided as a computer program logic component in a simulated implementation. Similarly, memory hardware (e.g., registers or caches) may be provided as software data structures in a simulated implementation. Furthermore, the physical address space used to access the memory 30 in the hardware device 10 may be simulated as a simulated address space mapped to a virtual address space used by the host operating system 710 by the simulator 705. In an arrangement where one or more of the hardware elements mentioned in the previously described examples are present on the host hardware (e.g., the host processor 715), some simulated implementations may utilize the host hardware (where appropriate).

[0084] The simulator program 705 may be stored on a computer-readable storage medium (which may be a non-transitory medium) and provide a virtual hardware interface (instruction execution environment) to the target code 700 (which may include an application, an operating system, and a hypervisor) that is the same as the hardware interface of the hardware architecture modeled by the simulator program 705. Therefore, the program instructions of the target code 700 may be executed within the instruction execution environment using the simulator program 705, so that a host computer 715 that does not actually have the hardware features of the device 10 discussed above may simulate these features. The simulator program may include: a processing program logic unit 720, which is used to simulate the behavior of the processing circuit 60; an instruction decoding program logic unit 725, which is used to simulate the behavior of the instruction decoder circuit 50; and an array storage device simulation program logic unit 722, which is used to maintain data structures to simulate the array storage device 90. Therefore, the technology described herein is Fig.15 In the example, the simulator program 705 can be used to execute the program in software.

[0085] As will be apparent from the earlier discussion of example implementations of the present technology, the technology described herein provides a single instruction (i.e., the multiple outer product instruction discussed above) that causes multiple outer product operations to be performed using sub-vectors within one or both of the source vector operands, wherein the result of each outer product operation is stored within an associated storage element of a selected two-dimensional array within an array storage device. By causing multiple outer product operations to be performed in response to a single instruction, this can significantly improve throughput while also making more efficient use of available storage elements within the array storage device.

[0086] In this application, the phrase "configured to..." is used to mean that the elements of a device have a configuration that enables the defined operation to be performed. In this context, "configuration" means the arrangement or manner of interconnection of hardware or software. For example, the device may have dedicated hardware that provides the defined operation, or a processor or other processing device may be programmed to perform the function. "Configured to" does not mean that the elements of the device need to be changed in any way to provide the defined operation.

[0087] Although illustrative embodiments of the present invention have been described in detail herein with reference to the accompanying drawings, it should be understood that the invention is not limited to these precise embodiments and that various changes, additions and modifications may be effected therein by those skilled in the art without departing from the scope and spirit of the invention as defined in the accompanying claims. For example, various combinations of features of the independent claims with features of the dependent claims may be made without departing from the scope of the invention.

Claims

1. A device, comprising: A processing circuit, the processing circuit is used to perform vector operations; an instruction decoder circuit, the instruction decoder circuit being used to decode an instruction from an instruction set to control the processing circuit to perform the vector operation specified by the instruction; an array storage device comprising storage elements for storing data elements, the array storage device being arranged to store at least one two-dimensional array of data elements accessible to the processing circuitry when performing the vector operation; wherein the instruction set includes a multiple outer product instruction that identifies a first source vector operand, a second source vector operand, and a given two-dimensional array of data elements within the array storage device that form a destination operand, wherein at least the first source vector operand identifies at least one vector of data elements to be treated as comprising a plurality of sub-vectors, and at least the second source vector operand identifies a plurality of vectors of data elements; wherein the instruction decoder circuitry is arranged to control the processing circuitry in response to the multiple outer product instruction to perform an outer product operation for each sub-vector identified by the first source vector operand, and each outer product operation comprises multiplying each data element of the associated sub-vector identified by the first source vector operand by each data element of a group of data elements selected from the second source vector operand to produce a plurality of outer product results, and using each outer product result to update a value held in an associated storage element within the storage element of the given two-dimensional array; Wherein the processing circuitry includes selection circuitry to control selection of the data elements processed by each outer product operation to switch between vectors of the second source vector operand when switching between different sub-vectors within a given vector of the first source vector operand.

2. An apparatus according to claim 1, wherein for each multiplication operation to be performed by the processing circuit to produce a corresponding outer product result, the selection circuit includes an associated multiplexer circuit to select data elements from the first source vector operand and select data elements from the second source vector operand to undergo the multiplication operation, wherein the selection made by the associated multiplexer circuit depends on which outer product operation the corresponding outer product result is controlled.

3. An apparatus according to claim 1 or claim 2, wherein the first source vector operand and the second source vector operand both identify multiple vectors of data elements, and each vector is to be regarded as comprising multiple sub-vectors.

4. An apparatus according to claim 3, wherein each of the first source vector operand and the second source vector operand contains N sub-vectors, and the processing circuit is arranged to perform N outer product operations.

5. The apparatus of claim 3 or claim 4, wherein: The plurality of sub-vectors within each vector of the first source vector operand have associated sub-vectors within a different vector of the second source vector operand; and The selection circuitry is arranged to control selection of the data elements processed by each outer product operation such that when switching between different sub-vectors within a given vector of the first source vector operand, switching to a different vector of the second source vector operand causes the data elements from the associated sub-vector within the second source vector operand to be selected.

6. An apparatus according to any one of claims 1 to 3, wherein the first source vector operand contains a plurality of P sub-vectors, and the second source vector operand comprises P vectors of data elements, wherein each vector in the second source vector operand is associated with one of the sub-vectors in the first source vector operand.

7. An apparatus according to claim 6, wherein the processing circuit is arranged to perform P outer product operations, wherein each outer product operation is performed using an associated sub-vector from the first source vector operand and an associated vector from the second source vector operand as input.

8. Apparatus according to any preceding claim, wherein for at least one given vector to be regarded as comprising a plurality of sub-vectors, the data elements forming each sub-vector are provided at consecutive data element positions within the given vector.

9. Apparatus according to any preceding claim, wherein for at least one given vector to be regarded as comprising a plurality of sub-vectors, the data elements forming each sub-vector are provided at non-consecutive data element positions within the given vector.

10. Apparatus according to any preceding claim, wherein for at least one given vector to be regarded as comprising a plurality of sub-vectors, the given vector can have one or more unused data element positions containing no data elements of the plurality of sub-vectors.

11. The apparatus according to any preceding claim, further comprising: A set of vector registers accessible by the processing circuitry, wherein each vector register is arranged to store a vector comprising a plurality of data elements, and the first source vector operand and the second source vector operand comprise vectors contained within a vector register in the set of vector registers.

12. The apparatus of claim 11, wherein: a vector length identifying a size of the vector register within the set of vector registers and a size of the given two-dimensional array data element within the array storage device; and The multiple outer product instruction is arranged to provide a subvector indicator for determining the number of subvectors within each vector to be treated as comprising a plurality of subvectors, wherein the size of each subvector depends on the determined number of subvectors and the vector length.

13. The apparatus of claim 12, wherein the sub-vector indicator is specified in a manner independent of the length of the vector.

14. An apparatus according to any preceding claim, wherein the multiple outer product instructions are accumulate instructions and each outer product result is used to update an existing value held in the associated storage element within the given two-dimensional array by combining the outer product result with the existing value.

15. An apparatus according to any preceding claim, wherein the multiple outer product instruction is a sum of outer product instructions, multiple outer product results have the same associated storage elements within the storage elements of the given two-dimensional array, and those multiple outer product results are combined to update the values ​​held in the associated storage elements.

16. An apparatus according to any preceding claim, wherein the first source vector operand and the second source vector operand both comprise two vectors, each vector being formed from two sub-vectors, and the instruction decoder circuit is arranged to control the processing circuit to perform four outer product operations in response to the multiple outer product instruction, wherein the results of those four outer product operations are stored in storage elements within associated regions of storage elements of the given two-dimensional array.

17. A method for performing an outer product operation, the method comprising: performing vector operations using processing circuits; utilizing an instruction decoder circuit to decode an instruction from an instruction set to control the processing circuit to perform the vector operation specified by the instruction; providing an array storage device comprising storage elements for storing data elements, the array storage device being arranged to store at least one two-dimensional array of data elements, the data elements being accessible to the processing circuitry when performing the vector operation; wherein the instruction set includes a multiple outer product instruction that identifies a first source vector operand, a second source vector operand, and a given two-dimensional array of data elements within the array storage device that form a destination operand, wherein at least the first source vector operand identifies at least one vector of data elements to be treated as comprising a plurality of sub-vectors, and at least the second source vector operand identifies a plurality of vectors of data elements; controlling the processing circuitry to, in response to decoding the multiple outer product instruction by the instruction decoder circuitry, perform an outer product operation for each sub-vector identified by the first source vector operand, wherein each outer product operation comprises multiplying each data element of an associated sub-vector identified by the first source vector operand by each data element of a group of data elements selected from the second source vector operand to produce a plurality of outer product results, and using each outer product result to update a value held in an associated storage element within the storage element of the given two-dimensional array; as well as Selection of the data elements processed by each outer product operation is controlled to switch between vectors of the second source vector operand when switching between different sub-vectors within a given vector of the first source vector operand.

18. A computer program for controlling a host data processing device to provide an instruction execution environment, the computer program comprising: A processor logic unit, the processor logic unit is used to perform vector operations; An instruction decoding program logic unit, the instruction decoding program logic unit is used to decode an instruction from an instruction set to control the processing program logic unit to perform the vector operation specified by the instruction; an array storage device emulation program logic unit, the array storage device emulation program logic unit being configured to emulate an array storage device, the array storage device comprising storage elements for storing data elements, the array storage device being arranged to store at least one two-dimensional array data element, the data elements being accessible to the processing program logic unit when performing the vector operation; wherein the instruction set includes a multiple outer product instruction that identifies a first source vector operand, a second source vector operand, and a given two-dimensional array of data elements within the array storage device that form a destination operand, wherein at least the first source vector operand identifies at least one vector of data elements to be treated as comprising a plurality of sub-vectors, and at least the second source vector operand identifies a plurality of vectors of data elements; wherein the instruction decode program logic unit is arranged to control the processing program logic unit in response to the multiple outer product instruction to perform an outer product operation for each sub-vector identified by the first source vector operand, and each outer product operation comprises multiplying each data element of the associated sub-vector identified by the first source vector operand by each data element of a group of data elements selected from the second source vector operand to produce a plurality of outer product results, and using each outer product result to update a value held in an associated storage element within the storage element of the given two-dimensional array; Wherein the processing program logic unit includes a selection program logic unit for controlling the selection of the data elements processed by each outer product operation to switch between vectors of the second source vector operand when switching between different sub-vectors within a given vector of the first source vector operand.

Images