Data processing

Abstract

Data processing apparatus comprising: processing circuitry to apply a processing operation to one or more data items of a linear array, the linear array comprising a plurality of n data items at respective positions in the linear array, the processing circuitry configured to access an array of n x n storage locations, where n is an integer greater than one, the processing circuitry comprising: instruction decoder circuitry to decode program instructions; and instruction processing circuitry to execute instructions decoded by the instruction decoder circuitry; wherein the instruction decoder circuitry, in response to an array access instruction, controls the instruction processing circuitry to access a set of n storage locations arranged in an array direction as a linear array, the array direction being selected from a set of candidate array directions, the set of candidate array directions comprising at least a first array direction and a second array direction different from the first array direction, under control of the array access instruction.

Description

BACKGROUND

[0001] The present disclosure relates to data processing.

[0002] Some data processing arrangements allow vector processing operations which involve applying a single vector processing instruction to data items of a data vector which has a plurality of data items at respective positions in the data vector. By contrast, scalar processing operates on a single data item rather than a data vector. SUMMARY

[0003] In one example arrangement there is provided data processing apparatus comprising:

[0004] processing circuitry to apply a processing operation to one or more data items of a linear array comprising a plurality (n) of data items at respective positions in the linear array, the processing circuitry being configured to access an array of n x n storage locations, where n is an integer greater than one, the processing circuitry comprising:

[0005] instruction decoder circuitry to decode program instructions; and

[0006] instruction processing circuitry to execute instructions decoded by the instruction decoder circuitry;

[0007] wherein the instruction decoder circuitry is responsive to an array access instruction to control the instruction processing circuitry to access a set of n storage locations arranged in an array direction as a linear array, the array direction being selected from a set of candidate array directions under control of the array access instruction, the set of candidate array directions comprising at least a first array direction and a second array direction different from the first array direction.

[0008] In another example arrangement there is provided data processing apparatus comprising:

[0009] processing means for applying a processing operation to one or more data items of a linear array comprising a plurality (n) of data items at respective positions in the linear array, the vector processing means being operable to access an array of n x n storage locations, where n is an integer greater than one, the processing means comprising:

[0010] instruction decoder means for decoding program instructions; and

[0011] instruction processing means for executing instructions decoded by the instruction decoder means;

[0012] wherein the instruction decoder means, in response to an array access instruction, controls the instruction handling means to access a set of n storage locations arranged in an array direction as a linear array, the array direction being selected from a set of candidate array directions under the control of the array access instruction, the set of candidate array directions comprising at least a first array direction and a second array direction different from the first array direction.

[0013] In another example arrangement, there is provided a data processing method, the method comprising the steps of:

[0014] applying a processing operation to one or more data items of a linear array comprising a plurality (n) of data items at respective positions in the linear array, n being an integer greater than one, by accessing an array of n x n storage locations, the applying step comprising the steps of:

[0015] decoding program instructions; and

[0016] executing instructions decoded by the decoding step;

[0017] wherein the decoding step, in response to an array access instruction, controls the executing step to access a set of n storage locations arranged in an array direction as a linear array, the array direction being selected from a set of candidate array directions under the control of the array access instruction, the set of candidate array directions comprising at least a first array direction and a second array direction different from the first array direction.

[0018] In another example arrangement, there is provided a virtual machine, the virtual machine comprising a data processor to execute a computer program, the computer program comprising machine readable instructions, wherein execution of the computer program causes the data processor to operate as a data processing apparatus, the data processing apparatus comprising:

[0019] processing circuitry to apply a processing operation to one or more data items of a linear array comprising a plurality (n) of data items at respective positions in the linear array, the processing circuitry being configured to access an array of n x n storage locations, n being an integer greater than one, the processing circuitry comprising:

[0020] instruction decoder circuitry to decode program instructions; and

[0021] instruction handling circuitry to execute instructions decoded by the instruction decoder circuitry;

[0022] wherein the instruction decoder circuitry is responsive to an array access instruction to control the instruction processing circuitry to access a set of n storage locations arranged in an array direction as a linear array, the array direction being selected from a set of candidate array directions under control of the array access instruction, the set of candidate array directions comprising at least a first array direction and a second array direction different from the first array direction.

[0023] Further respective aspects and features of the present disclosure are defined by the appended claims. BRIEF DESCRIPTION OF DRAWINGS

[0024] The technology will be further described by way of example only with reference to embodiments thereof illustrated in the accompanying drawings, wherein:

[0025] Figure 1 A data processing apparatus is schematically illustrated;

[0026] Figure 2a A storage array is schematically illustrated;

[0027] Figures 2b-4 Access to a storage array is schematically illustrated; Figure 2b

[0028] Figures 5a-5e An example of vector multiplication is schematically illustrated;

[0029] Figures 6a-6e An example of matrix multiplication is schematically illustrated;

[0030] Figures 7a-7f Use of a matrix multiplication accumulation (MMLA) instruction is schematically illustrated;

[0031] Figure 8 A virtual machine is schematically illustrated; and

[0032] Figure 9 is a schematic flow diagram representing a method. DETAILED DESCRIPTION

[0033] Before the embodiments are discussed with reference to the drawings, the following explanations of the embodiments are provided.

[0034] An example embodiment provides a data processing apparatus comprising:

[0035] processing circuitry to apply a processing operation to one or more data items of a linear array, the linear array comprising a plurality of n data items at respective positions in the linear array, the processing circuitry being configured to access an array of n x n storage locations, where n is an integer greater than one, the processing circuitry comprising:

[0036] ​instruction decoder circuitry to decode program instructions; and

[0037] instruction processing circuitry to execute instructions decoded by the instruction decoder circuitry;

[0038] wherein the instruction decoder circuitry, in response to an array access instruction, controls the instruction processing circuitry to access a set of n storage locations arranged in an array direction as a linear array, the array direction being selected from a set of candidate array directions, the set of candidate array directions including at least a first array direction and a second array direction different from the first array direction, under control of the array access instruction.

[0039] Example implementations represent techniques that can provide particular efficiencies and conveniences in array processing. One example relates to matrix processing, but the techniques can be used in other applications.

[0040] Multiplication or other matrix manipulation is common in digital signal processing used by many application domains. Also, in recent years, with machine learning becoming more prevalent in which matrix multiplication is found in training and inference algorithms, interest in accelerating matrix multiplication processing is significant.

[0041] For example, for two matrices A and B with matrix A having dimensions [M, K] and matrix B having dimensions [K, N], matrix C = A * B has dimensions [M, N], where M is the number of columns of A and C, K is the number of rows of A and columns of B, and N is the number of rows of B and C.

[0042] Even though matrix multiplication computation is fairly simple (i.e., each element of C is obtained by a K dot product of a column of A and a row of B), and this computation can be viewed as a K multiply-accumulate operation, the challenge is to deliver to instruction inputs in a particular order delegated by an instruction that will use this data to compute a result, a sequence of particular elements of matrices A and B that contribute to a particular result. The present implementations relate to delivering appropriately organized input matrix elements when performing computations, for example, but not exclusively, on a vector processing CPU.

[0043] Typically, in a general purpose application architecture that is not tailored for a specific implementation, the memory layout of the three matrices A, B and C is the same, either all three matrices are in row-major memory layout or all three matrices are in column-major memory layout. The challenge of efficiently delivering the elements of the A matrix and the B matrix that contribute to the result of a particular pair as input to a CPU instruction that will process such input is that the data arrangement in memory can be the same, while what is needed is a column of A and a row of B. If the A and B matrices have the same memory layout, then always one of A or B will have data that needs to be delivered contiguously from memory, while the data of the other matrix will need to be collected from non-contiguous locations in memory.

[0044] In examples of the present technology, the challenges of matrix multiplication found in general, non-tailored application architectures are addressed, at least in part, by providing an efficient mechanism for delivering input matrix elements as linear arrays for any combination of memory layouts of the input matrices. There are four combinations: both A and B are in row-major memory layout, both A and B input matrices are in column-major memory layout, A is in row-major memory layout and B is in column-major memory layout, A is in column-major memory layout and B is in row-major memory layout.

[0045] These features can be implemented by example embodiments in which data can be written to and read from n x n arrays of storage locations as linear arrays of n data items in one of at least two different directions, e.g. orthogonal directions.

[0046] Example embodiments can provide storage arrays, each of which can be viewed as n horizontal linear arrays and n vertical linear arrays. Processing circuitry that extracts input data from the storage arrays can read the arrays in two directions: in the same direction as the load that populated the array, or in the other direction, e.g. a second direction that is orthogonal to the direction. If the instruction is used such that it reads the array in the same direction as the load that populated the array, then the instruction consumes the input in its native layout in memory. On the other hand, if the instruction is used such that it reads the array in a direction that is orthogonal to the direction of the load that populated the array, then the instruction consumes the input in a transposed form compared to its native layout in memory. This results in allowing a "true-time transpose capability" with the proposed technology.

[0047] In examples as described above, the second array direction is orthogonal to the first array direction.

[0048] In some examples, the instruction processing circuitry is configured to store input vectors to the array of storage locations as groups of n storage locations arranged in the first array direction; and to retrieve, in response to a data fetch instruction, sets of n storage locations arranged in an array direction as linear arrays, the array direction being selected from the set of candidate array directions under control of the data fetch instruction; and the first array direction being a predetermined array direction.

[0049] In general, while array directions can be implied, or a separate instruction can be used for each access direction, the data fetch instruction can also have an associated instruction parameter to specify an array direction by which the instruction processing circuitry is to access (in examples, fetch) a set of n storage locations as a linear array.

[0050] Conveniently, for example to allow processing operations to be performed therebetween, the data processing apparatus can comprise two or more arrays of n x n storage locations.

[0051] In some examples, the present techniques are particularly useful for array access instructions, including matrix processing instructions, the matrix processing instructions defining a matrix processing operation to produce output data dependent on respective input matrices of data items stored in respective ones of the two or more arrays of n x n storage locations. For example, the matrix processing operation can define an operation between columns or rows of data items of one of such input matrices and columns or rows of data items of another of such input matrices; and the matrix processing instructions can have associated instruction parameters to specify, for each of such respective ones of the arrays of n x n storage locations storing data items of such input matrices, an array direction from which the instruction decoder circuitry is to control the instruction processing circuitry to fetch a linear array from the array of storage locations by the array direction.

[0052] In an example arrangement, the matrix processing operation is a matrix multiplication operation to produce elements of an output matrix dependent on an operation between elements of one of columns and rows of data items of one of such input matrices and elements of another of columns and rows of data items of another of such input matrices.

[0053] Conveniently, the array of n x n storage locations comprises an array of storage elements that can be accessed by the instruction processing circuitry as 2n linear arrays, the 2n linear arrays comprising n linear arrays in the first array direction and n linear arrays in the second array direction, each linear array comprising n data items. For example, the array access instruction can specify one or more of the 2n linear arrays. In some example implementations, although not exclusively, linear arrays can be accessed as vector registers.

[0054] The quantity n can be any value and can be related to the overall size of the storage allocated to the array and the granularity of access, but in one example, n equals 16.

[0055] Another example implementation provides a data processing apparatus that includes:

[0056] A processing component for applying processing operations to one or more data items of a linear array, the linear array comprising a complex number (n) data items at corresponding positions in the linear array, the processing component being operable to access an array of n×n storage locations, where n is an integer greater than one, the processing component comprising:

[0057] Instruction decoder components are used to decode program instructions; and

[0058] An instruction processing component is used to execute instructions decoded by the instruction decoder component;

[0059] The instruction decoder component responds to the array access instruction and controls the instruction processing component to access the set of n storage locations arranged in a certain array direction as a linear array. Under the control of the array access instruction, the array direction is selected from the set of candidate array directions, which includes at least a first array direction and a second array direction different from the first array direction.

[0060] Another example implementation provides a data processing method that includes the following steps:

[0061] By accessing an array of n×n storage locations, a processing operation is applied to one or more data items of a linear array, the linear array comprising a complex number (n) data items at the corresponding positions in the linear array, where n is an integer greater than one. The application step includes the following steps:

[0062] Decode program instructions; and

[0063] Execute the instructions decoded by this decoding step;

[0064] The decoding step responds to an array access instruction, which controls the execution step to access a set of n storage locations arranged in a certain array direction as a linear array. Under the control of the array access instruction, the array direction is selected from a set of candidate array directions, which includes at least a first array direction and a second array direction that is different from the first array direction.

[0065] An example embodiment can be implemented as computer software which, when executed by a computer, causes the computer to implement the method defined above. Such software can be provided through a non-transitory machine-readable storage medium storing such computer software.

[0066] Another example embodiment provides a virtual machine comprising a data processor to execute a computer program comprising machine-readable instructions, wherein execution of the computer program causes the data processor to operate as a data processing apparatus comprising:

[0067] processing circuitry to apply a processing operation to one or more data items of a linear array comprising n data items at respective positions in the linear array, the processing circuitry being configured to access an array of n x n storage locations, where n is an integer greater than one, the processing circuitry comprising:

[0068] instruction decoder circuitry to decode program instructions; and

[0069] instruction processing circuitry to execute instructions decoded by the instruction decoder circuitry;

[0070] wherein the instruction decoder circuitry, in response to an array access instruction, controls the instruction processing circuitry to access a set of n storage locations arranged in an array direction as a linear array, the array direction being selected from a set of candidate array directions, the set of candidate array directions comprising at least a first array direction and a second array direction different from the first array direction, under control of the array access instruction.

[0071] Reference will now be made, by way of example, to the accompanying drawings, Figure 1 A data processing system 10 is shown schematically, comprising a processor 20 coupled to a memory 30 storing data values 32 and program instructions 34. The processor 20 comprises an instruction fetch unit 40 for fetching program instructions 34 from the memory 30 and supplying the fetched program instructions to decoder circuitry 50. The decoder circuitry 50 decodes the fetched program instructions and generates control signals to control processing circuitry 60 to perform processing operations on data stored in register circuitry 70 as specified by decoded vector instructions.

[0072] The processor 20 can have access to a storage array 90. This storage array is drawn with dashed lines to illustrate that it can or can not be provided as part of the processor 20. In various examples, the storage array can be implemented as any one or more of the following: an architecturally addressable register; an architecturally non-addressable register; a scratchpad memory; and a cache.

[0073] The processing circuitry 60 can for example be vector processing circuitry and / or scalar processing circuitry. A general distinction between scalar and vector processing is as follows. Vector processing involves applying a single vector processing instruction to data items of a data vector, which has a plurality of data items at respective positions in the data vector. Scalar processing operates on a single data item, in effect, rather than on a data vector. Vector processing can be useful in situations where a processing operation is to be implemented on many different instances of data. In a vector processing arrangement, a single instruction can be applied to multiple data items (of a data vector) simultaneously. This can improve the efficiency and throughput of data processing compared to scalar processing.

[0074] While the present embodiments can be related to vector processing, it is not a necessary condition that a vector processor is used. Rather, the present techniques are related to processing arrays of two-dimensional data items stored in, for example, the storage array 90. In at least some examples, a two-dimensional storage array can be accessed as a one-dimensional array of data items in a plurality of candidate directions. While such a one-dimensional array of data items can be treated as a data vector in some examples, this is not a necessary condition of the present techniques.

[0075] In an example embodiment, the storage array 90 can store square array portions of larger or even higher dimensional arrays or matrices of data items in memory.

[0076] Multiple instances of the storage array 90 can be provided in order to store multiple respective arrays of data items.

[0077] The following discussion is related to example program instructions 34. Embodiments of the present disclosure include apparatuses of the type shown in, for example, Figure 1 which are operable or configured to decode and execute such program instructions. Thus, Figure 1 An example is provided of processing circuitry to selectively apply processing operations to data items at positions within linear arrays, each linear array comprising a plurality of data items having an order of data items; and instruction processing circuitry to execute instructions decoded by the instruction decoder circuitry; wherein the instruction decoder circuitry is responsive to instructions to control the instruction processing circuitry to implement functionality to be discussed below.

[0078] Optionally, if a vector processor is used, the vector processing operations can be conducted under control of so-called predicates. Here, a respective predicate can control whether a particular vector function is applied for one of the data item positions within the linear array, which in this example can be processed as a data vector.

[0079] As discussed above, under control of instructions decoded by the decoder circuitry 50, the processing circuitry 60 is arranged to access the registers 70 and / or the storage array 90. Reference will now be made to Figure 2a Further details of this latter arrangement will now be described.

[0080] In the present example, the storage array 90 is arranged as an array 205 of n x n storage locations 200, where n is an integer greater than 1. In the present example, n is 16, which implies that the granularity of access to the storage locations 200 is 1 / 16 of the total storage in either of the horizontal or vertical array directions. This aspect will be discussed further below.

[0081] From the perspective of the processing circuitry, the array of n x n locations can be accessed as n linear (one-dimensional) arrays in a first direction (e.g. the drawn horizontal direction) and n linear arrays in a second array direction (e.g. the drawn vertical direction). Each linear array has n elements, so that each of the storage arrays stores a linear array of n data items. In other words, from the perspective of the processing circuitry 60, the n x n storage locations are arranged as, or at least can be accessed as, 2n linear arrays, each of n data items.

[0082] Thus, this provides the example that the array of n x n storage locations comprises an array of storage elements that can be accessed by the instruction processing circuitry as 2n linear arrays, including n linear arrays in a first array direction and n linear arrays in a second array direction, each of n data items (e.g. as a data vector register, although this is not a necessary condition). The example instructions discussed below can specify one or more of the 2n linear arrays.

[0083] Under control of control circuitry 250, which is in communication with the processing circuitry and optionally with the decoder circuitry 50, the array of storage locations 200 can be accessed by the access circuitry 210, 220, row selection circuitry 230 and column selection circuitry 240.

[0084] Reference will now be made to Figure 2bIn the case where the example storage array 90 is labeled "Al" (note that more than one such storage array 90 can be implemented, e.g., A0, Al, A2, etc., as discussed below), the n linear arrays in the first direction (as drawn, the horizontal or "H" direction) are each of the 16 data items 0...F (in hexadecimal) and can be referred to in this example as AlH0...AlH15. The same underlying data stored in the 256 entries (16x16 entries) of the storage array 90 Al can be referred to oppositely in the second direction (as drawn, the vertical or "V" direction) as AlV0...AlV15. Note, for example, that data item 260 is item F referred to as AlH0, but item 0 referred to as AlV15. Note that the use of "H" and "V" does not imply any spatial or physical layout requirements related to storage of the data elements making up the storage array 90, nor does it relate at all to whether the storage array is storing column data or row data in an example application involving matrix processing. Figure 2b

[0085] According to the techniques of Figure 2b Figure 1 and Figure 2a An apparatus providing an example of a data processing apparatus 10, the data processing apparatus comprising:

[0086] processing circuitry 20 to apply a processing operation to one or more data items of a linear array comprising n data items at respective positions in the linear array, the processing circuitry being configured to access an array 205 of n x n storage locations 200, where n is an integer greater than one, the processing circuitry comprising:

[0087] instruction decoder circuitry 50 to decode program instructions; and

[0088] instruction processing circuitry 60 to execute instructions decoded by the instruction decoder circuitry;

[0089] wherein the instruction decoder circuitry, in response to an array access instruction, controls the instruction processing circuitry to access a set of n storage locations arranged in an array direction as a linear array, the array direction being selected from a set of candidate array directions, the set of candidate array directions comprising at least a first array direction and a second array direction different from the first array direction, under control of the array access instruction.

[0090] In an example arrangement, the second array (as drawn vertically in Figure 2a is orthogonal to the first array direction (as drawn horizontally).

[0091] ​​To access one of the linear arrays A1H0...A1H15 in the first direction (e.g. the horizontal direction as drawn), the row selection circuitry 230 is controlled by the control circuitry 250 to select the row of storage locations corresponding to the linear array 300, and the access circuitry 210 controls the access (input or output) to the individual data items 310 of the linear array 300, to be provided to the processing circuitry via the interface 320. Figure 3 where an arbitrary linear array A1Hm 300 is being accessed (where m is an arbitrary number between 0 and 15 in this example). Here, the row selection circuitry 230 is controlled by the control circuitry 250 to select the row of storage locations corresponding to the linear array 300, and the access circuitry 210 controls the access (input or output) to the individual data items 310 of the linear array 300, to be provided to the processing circuitry via the interface 320.

[0092] Similarly, to access an arbitrary linear array in the other direction as drawn (e.g. the linear array A1Vm 400), the row selection circuitry 230 selects the row of storage elements corresponding to the linear array 400, and the data is read (output) via the access circuitry 220 to be interfaced with the processing circuitry via the interface 410. Figure 4

[0093] The so-called granularity of the arrangement of Figures 2a-4 will now be discussed. The linear array A1Hm represents 16 data items, each having 32 bits. There are 16 such linear arrays, and each linear array A1Vm in the second array direction also has 16 entries of 32 bits. However, conversely, this storage can be arranged in each direction as a vector of e.g. 64 data items of 8 bits. In other words, the granularity of the access to the storage providing the storage array 90 can be of 8 bits rather than 32 bits. However, in the present example, the granularity and the number of data items in each linear array in the first and second directions should be the same (16 in the first example, 64 in the second example).

[0094] In the example arrangement, the instruction processing circuitry 60 is configured to store an input vector or linear array as a group of n storage locations arranged in the first array direction (A1Hm) to the array of storage locations; and to retrieve a set of n storage locations arranged in a certain array direction as a linear array (e.g. A1Hm or A1Vm) in response to a data retrieval instruction, under control of which the array direction is selected from the set of candidate array directions; and the first array direction is a predetermined array direction (e.g. horizontal as drawn). In other words, data writing is limited to the first direction, whereas data reading is allowed in either direction. But of course another example arrangement can be provided in which data writing and data reading is allowed in either direction.

[0095] may be used to access a linear array as by Figures 2a-4 ​Examples of processing instructions for the represented storage array 90 may include one or more of the following:

[0096] (a) Load and save instructions, specifying items such as the following:

[0097] • Store a linear array (or a collection of linear arrays) in one of the arrays 90 (A1Hm in the example above);

[0098] • A block of memory location, serving as the source or destination of an operation;

[0099] • Optional (in the example case of a vector processing circuit system) predicate register, which limits whether operations are performed for each memory location;

[0100] • Optional parameters that limit the granularity of access (see above).

[0101] (b) Move instructions, providing copy or transpose functionality, and specifying items such as the following:

[0102] • Store the source linear array (or source set of linear arrays) in one of the arrays 90 (A1Hm or A1Vm in the example above), or the initial value;

[0103] • The target linear array (or the target set of linear arrays) in another (different) such array (A1Hm or A1Vm in the above example);

[0104] • Optional parameters that limit the granularity of access (see above).

[0105] (c) Matrix multiplication command, specifying:

[0106] • Store the left input linear array (or set of linear arrays) in one of the arrays 90 (A1Hm or A1Vm in the example above);

[0107] • The right input linear array (or set of linear arrays) in another (different) of this type of storage array 90 (A1Hm or A1Vm in the example above);

[0108] • Outputs a linear array (or a collection of linear arrays) to store the result of the multiplication operation (or, in the specific case of multiplying a single linear array by another single linear array, a scalar register can be specified);

[0109] • Optional parameters that limit the granularity of access (see above).

[0110] (d) Matrix multiplication-accumulation instructions (described further below)

[0111] At least some of these instructions provide an example of a data fetch instruction having associated instruction parameters to specify an array direction through which the instruction processing circuitry is to access (in examples, fetch) a set of n storage locations as a linear array.

[0112] At least some of these instructions provide an example of a matrix processing instruction defining a matrix processing operation to produce output data dependent on respective input matrices of data items stored in respective ones of two or more arrays of n x n storage locations. In such examples, the matrix processing operation can define an operation between columns or rows of data items of one of the input matrices and columns or rows of data items of another of the input matrices; and the matrix processing instruction can have associated instruction parameters to specify, for each of the respective arrays of n x n storage locations of the input matrices of data items, an array direction through which the instruction decoder circuitry is to control the instruction processing circuitry to fetch a linear array from the array of storage locations. In some examples, the matrix processing operation can be a matrix multiplication operation to produce elements of an output matrix dependent on an operation between elements of one of the columns and rows of data items of one of the input matrices and elements of another of the columns and rows of data items of another of the input matrices.

[0113] An example use of such techniques is in relation to matrix processing operations. Figures 5a-5e A simplified and specific example (i.e. vector multiplication) is illustratively shown, Figures 6a-6e More general cases of matrix processing functions are represented.

[0114] Reference is made to Figure 5a In order to produce a result 500 of a vector multiplication of one input vector 510 and another input vector 520, a so-called dot product is used, whereby consecutive elements of the input vector 510 (e.g. proceeding from left to right as drawn) are multiplied by corresponding elements of the input vector 520 (e.g. proceeding from top to bottom as drawn), with the result being a sum of each of those individual products.

[0115] Such an arrangement can be implemented using the storage arrays 90 described with reference to Figure 2- Figure 4 For example, two or more storage arrays 90 of n x n storage locations can be provided, with a pair of such arrays 530, 540 being used in the present example.

[0116] In Figure 5b In the present example, the input vector 510 is fetched from any one of the linear arrays 532 in the first direction, and the input vector 520 is fetched from any one of the linear arrays 542 in the second direction. A scalar result of the dot product operation can be stored using a scalar register 550.

[0117] However, even Figure 5a The vector multiplication illustrated schematically involves multiplying column 510 by row 520, but the flexibility provided by the storage array arrangement in this example also allows for other permutations to be implemented in the array of storage locations. For example, in array 530, linear array 510 can in fact be represented by a linear array in the second direction 534, and row 520 can also be represented by a linear array 544 in the second direction.

[0118] exist Figure 5d In the example, linear arrays 510 and 520 can both be represented in the first direction, while... Figure 5e In the illustrative example, column 510 is represented by a linear array 538 in the second direction, while row 520 is represented by a linear array 548 in the first direction.

[0119] These techniques can be particularly useful in example instances of matrix multiplication, in which ( Figure 6a In the above, the left matrix 600 is multiplied by the right matrix 610 to produce an output or product matrix 620. Each element 622 of the output matrix 620 is generated by the dot product of column 602 of the left matrix and row 612 of the right matrix. The dot product of this type is implemented between column 602 and row 612 to produce a single element 622, which is filled into the position in the output matrix represented by the intersection of column 602 and row 612.

[0120] From the perspective of implementing this matrix processing function using the first input array 630 and the second input array 640, the manner in which each of the input matrices 600 and 610 is written into the storage arrays 630 and 640 can depend on how the underlying data representing the matrices is stored in memory, such as in a so-called column-major format (column-by-column) or a so-called row-major format (row-by-row). In some examples, regardless of how the memory is formatted, data is written into the storage arrays 630 and 640 in the first direction (such as horizontally oriented linear arrays as drawn), but in other examples, the choice of which of the 2n linear arrays should be used to load the input matrices 600 and 610 can be entirely flexible (such as between the first and second directions).

[0121] The result is, as Figures 6b-6e As shown in the arrangement, the input matrix 600 can be represented such that each column is stored in a linear array in the first direction ( Figure 6b , Figure 6c ) or stored as a linear array in the second direction ( Figure 6d , Figure 6e ), and the input matrix 610 is represented such that each row is stored as a linear array in the second direction ( Figure 6b , Figure 6d) or stored as a linear array in the first direction Figure 6c and Figure 6e ).

[0122] Note that the storage array 650 used to represent the output or product matrix 620 need not necessarily provide the multi-directional access functionality discussed above with respect to the storage arrays 630, 640, but in some examples they can provide this functionality.

[0123] In other examples, so-called matrix multiply-accumulate (MMLA) instructions can be used.

[0124] Figure 7a An illustrative explanation of this technique is provided. Such instructions can produce an output matrix C by adding the results of the outer product of an input matrix A row and an input matrix B column to previous values in the output matrix in order to provide a matrix multiply and accumulate operation (e.g., Cm' = Cm + AiVk x BjHk).

[0125] A set of input matrices (or portions of larger matrices) can be loaded from memory into input arrays Ai, Bj using any of the techniques described herein. In operation, one MMLA instruction takes one AiVk and one BjHk as sources and produces n x n products accumulated in an output array Cm. For example, if n = 16 and the two input arrays are A0, B0, then for a particular instance of the MMLA instruction, 256 (16 x 16) multiply-accumulate operations are implemented. For example, for each n and m = 0,..., 15, the output array element C0'(n, m) = C0(n, m) + (A0Vk(n) * B0Hk(m)).

[0126] These MMLA operations provide 256 products for two input 16-element linear arrays (in the 16 x 16 example), so can be efficient in cases where the load bandwidth is limited, or high latency is associated with the load operation to retrieve the 16 x 16 storage array of linear arrays.

[0127] The results of the MMLA instruction can themselves be useful. That is, it is not necessarily a requirement that the MMLA operations be subjected to multiple iterations in a looped fashion. However, in other examples, the reference Figures 6a-6e Mathematical matrix multiplication of the type described.

[0128] Figures 7b-7f An analogous representation is provided to Figures 6a-6e .

[0129] Reference is made to Figure 7b, the data AjVk is represented by a row 702 of the input matrix 700, and the data BjHk is represented by a column 712 of the input matrix 710. The multiply-accumulate product is represented by an output matrix Cm 722.

[0130] In a manner analogous to that of Figures 6b-6e , the data forming the inputs to the MMLA instruction can be represented in any permutation (permitted by the true-time transposition provided by the functionality described herein) of the respective columns or rows for the data arrays 730, 740. Figures 7c-7f

[0131] In various embodiments, the arrays can be implemented as any one or more of the following: architecturally addressable registers; architecturally non-addressable registers; scratchpad memory; and cache.

[0132] Figure 8 A virtual machine is schematically illustrated by which some or all of the functionality discussed above can be provided. The virtual machine comprises a central processing unit (CPU) as an example of data processing circuitry 800, a non-volatile memory 810, a control interface 820 and an input / output (IO) interface 830, all interconnected by a bus arrangement 840. A random access memory (RAM) 850 stores program instructions providing software 860 to control the operation of the CPU 800. Under the control of the software 860, the CPU 800 provides or emulates the functionality of one or more of the processing instructions discussed above. The RAM 850 also stores program instructions 870 and data 880, where the program instructions 870 are instructions applicable to the processor 20 of Figure 1 , and such program instructions are interpreted, emulated, or otherwise executed by the CPU 800 acting as a virtual machine. The data 880 is data corresponding to the data 32 of Figure 1 , to be acted upon by the (virtual) execution of the program instructions 870. Figure 8 The arrangement of Figure 9 thus provides an example of a virtual machine comprising a data processor (e.g. the CPU 800) to execute a computer program (e.g. the software 860) comprising machine-readable instructions, wherein execution of the computer program causes the data processor to operate as a data processing apparatus of the type described above. Example embodiments are also represented by computer software and by a non-transitory machine-readable storage medium storing such computer software, which computer software, when executed by a computer, causes the computer to implement one or more of the techniques described herein, including the method of

[0133] In general, Figure 9 is a schematic flow diagram illustrating a data processing method, the method comprising the steps of:

[0134] ​(At step 900) applying a processing operation to one or more data items of a linear array comprising complex (n) data items at respective positions in the linear array by accessing an array of n x n storage locations, where n is an integer greater than one, the applying step comprising the steps of:

[0135] (At step 910) decoding program instructions; and

[0136] (At step 920) executing instructions decoded by the decoding step;

[0137] wherein the decoding step, in response to an array access instruction, controls the executing step to access a set of n storage locations arranged in an array direction as a linear array, the array direction being selected from a set of candidate array directions under control of the array access instruction, the set of candidate array directions comprising at least a first array direction and a second array direction different from the first array direction.

[0138] In this patent application, the word "configured" is used to mean that an apparatus has a component that is capable of implementing the operation in question. In this context, "configured" means that the hardware or software of the component is arranged or programmed in a certain way to perform the function in question. "Configured" does not imply that the component needs to be changed in any way in order to provide the defined operation.

[0139] While illustrative implementations of the technology have been described in detail herein, it is to be understood that the technology is not limited to the precise implementations described, and various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the technology as defined by the appended claims. For example, features of one dependent item can be combined with features of another dependent item.

Claims

1. Data processing apparatus comprising: processing circuitry to apply a processing operation to one or more data items of a linear array, the linear array comprising complex n data items at respective positions in the linear array, the processing circuitry configured to access an array of n x n storage locations, where n is an integer greater than one, the processing circuitry comprising: instruction decoder circuitry to decode program instructions; and instruction processing circuitry to execute instructions decoded by the instruction decoder circuitry; wherein the instruction decoder circuitry, in response to a array access instruction, controls the instruction processing circuitry to access a set of n storage locations arranged in an array direction as a linear array, the array direction being selected from a set of candidate array directions under control of the array access instruction, the set of candidate array directions comprising at least a first array direction and a second array direction different from the first array direction.

2. The data processing apparatus according to claim 1, wherein the second array direction is orthogonal to the first array direction.

3. The data processing apparatus according to claim 1 or claim 2, wherein: the instruction processing circuitry is configured to store an input vector to the array of storage locations as a group of n storage locations arranged in the first array direction; and in response to a data fetch instruction, to fetch a set of n storage locations arranged in an array direction as a linear array, the array direction being selected from the set of candidate array directions under control of the data fetch instruction; and the first array direction is a predetermined array direction.

4. The data processing apparatus according to claim 3, wherein the data fetch instruction has an associated instruction parameter to specify an array direction by which the instruction processing circuitry is to access a set of n storage locations as a linear array.

5. The data processing apparatus according to the preceding claim 1 or claim 2, comprising two or more arrays of n x n storage locations.

6. The data processing apparatus according to claim 5, wherein the array access instruction comprises a matrix processing instruction, the matrix processing instruction defining a matrix processing operation to produce output data dependent on respective input matrices of data items stored in respective ones of the two or more arrays of n x n storage locations.

7. The data processing apparatus according to claim 6, wherein: the matrix processing operation defines an operation between columns or rows of data items of one of the input matrices and columns or rows of data items of another of the input matrices; and the matrix processing instruction has an associated instruction parameter to specify an array direction for each of the respective ones of the n x n storage locations of the input matrices of stored data items by which the instruction processing circuitry is to control the instruction processing circuitry to fetch linear arrays from the array of storage locations.

8. The data processing apparatus of claim 7, wherein the matrix processing operation is a matrix multiplication operation to produce an element of an output matrix from an operation between an element of one of the data items of one of the input matrices and an element of another of the data items of another of the input matrices.

9. The data processing apparatus of claim 2, wherein the array of n x n storage locations comprises an array of storage elements that is accessible by the instruction processing circuitry as 2n linear arrays, the 2n linear arrays comprising n linear arrays in the first array direction and n linear arrays in the second array direction, each linear array comprising n data items.

10. The data processing apparatus of claim 9, wherein the array access instruction specifies one or more of the 2n linear arrays.

11. The data processing apparatus of any preceding claim 1 or claim 2, wherein the linear arrays are implemented as at least one of: an architecturally addressable register; an architecturally non-addressable register; a scratchpad memory; and a cache.

12. The data processing apparatus of any preceding claim 1 or claim 2, wherein n is equal to 16.

13. A data processing apparatus comprising: processing means for applying a processing operation to one or more data items of a linear array, the linear array comprising a plurality of n data items at respective positions in the linear array, the processing means being operable to access an array of n x n storage locations, where n is an integer greater than one, the processing means comprising: instruction decoder means for decoding program instructions; and instruction processing means for executing instructions decoded by the instruction decoder means; wherein the instruction decoder means, in response to an array access instruction, controls the instruction processing means to access a set of n storage locations arranged in an array direction as a linear array, the array direction being selected from a set of candidate array directions, the set of candidate array directions comprising at least a first array direction and a second array direction different from the first array direction, under control of the array access instruction.

14. A data processing method comprising the steps of: applying a processing operation to one or more data items of a linear array by accessing an array of n x n storage locations, the linear array comprising a plurality of n data items at respective positions in the linear array, where n is an integer greater than one, the applying step comprising the steps of: decoding program instructions; and executing instructions decoded by the decoding step; wherein said instruction decoding step, in response to an array access instruction, controls said execution step to access a set of n storage locations arranged in an array direction as a linear array, said array direction being selected from a set of candidate array directions, said set of candidate array directions comprising at least a first array direction and a second array direction different from said first array direction, under control of said array access instruction.

15. A computer program product comprising computer-executable instructions that, when executed by a computer, cause the computer to implement the method of claim 14.

16. A non-transitory machine-readable storage medium storing the computer program product of claim 15.

17. A virtual machine comprising a data processor to execute a computer program, said computer program comprising machine-readable instructions, wherein execution of said computer program causes said data processor to operate as a data processing apparatus, said data processing apparatus comprising: processing circuitry to apply a processing operation to one or more data items of a linear array, said linear array comprising a plurality of n data items at respective positions in said linear array, said processing circuitry being configured to access an array of n x n storage locations, where n is an integer greater than one, said processing circuitry comprising: instruction decoder circuitry to decode program instructions; and instruction processing circuitry to execute instructions decoded by said instruction decoder circuitry; wherein said instruction decoder circuitry, in response to an array access instruction, controls said instruction processing circuitry to access a set of n storage locations arranged in an array direction as a linear array, said array direction being selected from a set of candidate array directions, said set of candidate array directions comprising at least a first array direction and a second array direction different from said first array direction, under control of said array access instruction.

Images