Tensor processor units

Abstract

Examples relate to an apparatus, for example, a Tensor Processor Unit, for processing, for example, multiplying, first and second Matrix Product States, the apparatus comprising circuitry to: a. receive, or store in memory, first matrix product state data (MPS 1) associated with a first matrix product state; the first matrix product state data comprising first tensor component data associated with the tensor components of the first matrix product state; b. receive, or store in memory, second matrix product state data (MPS 1) associated with a second matrix product state; the second matrix product state data comprising second tensor component data associated with the tensor components of the second matrix product state; c. processing corresponding tensor component data of the first and second matrix product state data to produce tensor contraction data representing a contraction of the tensor components associated with the corresponding tensor component data of the first and second matrix product state data; d. processing copy tensor data representing a copy tensor and the tensor contraction data to produce respective tensor component data; the respective tensor component data representing a contraction of the copy tensor and the contraction of the tensor components associated with the corresponding tensor component data of the first and second matrix product states; and e. associating, or storing in memory, the respective tensor component data with a corresponding tensor component data position of result data representing a resultant matrix product state associated with the processing / multiplying the first and second Matrix Product States.

Description

OUI Project 21872; HGF Ref: P361170WO 1 TENSOR PROCESSOR UNITS BACKGROUND

[0001] Computational Fluid Dynamics (CFD) is the mathematical simulation and study of fluid mechanics that uses numerical methods to analyse and solve problems that involve fluid flows. Fluid flows are simulated using Direct Numerical Solutions (DNSs) that accommodate the interactions of a fluid with surfaces that define boundary conditions. CFD is applied to an extensive range of research and engineering problems across an eclectic range of technical disciplines comprising, for example, aerodynamics, weather simulation, industrial systems design and analysis, biological engineering, fluid flows and heat transfers, etc.

[0002] The basis of CFD problems is the Navier-Stokes Equation. DNS to a CFD problem typically requires a description of the domain occupied by the fluid, equations that represent the fluid behaviour in terms of specified characteristics such as, for example, pressure and velocity, and a description of any associated boundary conditions. The foregoing are typically represented in CFD using a computational mesh for the fluid, discrete equations and algorithms to calculate pressure and velocity and boundary and initial conditions for pressure and velocity.

[0003] Such DNS are complex even under stable flow conditions, but become more so when modelling turbulent conditions. Turbulence can phenomenologically be described by mutually interacting eddies spanning a wide ranges of length and time scales that range from the largest size of energy containing eddies (or object size), known as the integral scale, ^, to the smallest scales, known as the Kolmogorov microscale, ^. The Kolmogorov microscale, ^, is related to theobject size, ^, by ^where ^^ is the Reynolds number.

[0004] Direct Numerical Simulation of the Navier-Stokes equations for large Reynolds numbers is currently beyond even current supercomputing capabilities due to the need for the number of grid points in each spatial direction being given by ^^> ^^^ / ^. Computationally efficient devices and processors are being developed, together with mathematical techniques, to support DNS the Navier-Stokes equations using increasingly high resolution grid points. An example of such devices, processors or a mathematical techniques are those that use Tensor Networks (TNs) as can be seen from, for example, N. Gourianov, M. Lubasch, S. Dolgov, Q. van Berg, H. Babaee, P. Givi, MK and D. Jaksch, “A quantum-inspired approach to exploit turbulence structures”, which presents an algorithm by which the incompressible Navier-Stokes equations can be accurately solved even when reducing the number of parameters required to represent a velocity field by more than one order of magnitude compared to direct numerical simulation. Nevertheless, the computation resources required, in terms of memory resources and number of mathematical operations, scales in non-linearly with dimension and, consequently, evermore efficient devices, processors and techniques are required.OUI Project 21872; HGF Ref: P361170WO 2 BRIEF INTRODUCTION OF THE DRAWINGS

[0005] Embodiments will be described with reference to the accompanying drawings in which:

[0006] Figure 1 shows a tensor and a log-log graph indicating how memory cost to store the tensor scales with tensor dimension;

[0007] Figure 2 illustrates a tensor network representing a Matrix Product State (MPS);

[0008] Figure 3 depicts a graph showing the non-linear scaling of MPS memory storage requirements with tensor rank;

[0009] Figure 4 illustrates a pair of MPSs together with their product represented as tensor networks;

[0010] Figure 5 shows a known technique for multiplying MPSs;

[0011] Figures 6 to 10 depict evaluating a product of MPSs according to examples;

[0012] Figure 11 illustrates a flow chart of processor operations for evaluating the product of MPSs depicted in figures 6 to 10 according to examples;

[0013] Figure 12 shows a processor for efficiently evaluating the product of MPSs depicted in figures 6 to 11 taken jointly and severally in any and all permutations;

[0014] Figure 13 depicts circuitry for implementing a Matrix Product Operator for use in evaluating the above product of the MPSs; and

[0015] Figure 14 illustrates machine readable storage storing machine readable instructions for evaluating a product of MPSs according to examples.

[0016] DETAILED DESCRIPTION

[0017] Figure 1 shows a view 100 of a graphical representation 102 of a tensor 104 and a log- log graph 106 indicating how the memory cost to store the tensor 104 scales with tensor dimension. The tensor 104 is a rank N tensor comprising N indices, ^^, where ^ = {1, … , ^} in which each index ^^∈ {1, ... , ^^}, where ^^is the dimension of leg ^. It will be appreciated that the graphical representation 102 of the tensor 104 comprises a number of lines, known as legs; each of which represents a respective index, ^^, of the tensor 104.

[0018] The log-log graphs 106 shows a number of plots 124 to 130 depicting the variation in memory requirements, or memory cost 124, to store various tensors of various ranks and dimensions over a small range; the abscissa shows dimension. It can be seen that the memory required to store a tensor scales the dimension exponentially with rank, that is, ≈ ^^.

[0019] Matrix Product States were developed to mitigate the significant memory requirements to store tensors. A Matrix Product State is also known as a Tensor Train. Matrix Product States are a class of tensor that can be written as a product of a number of rank-3 tensors. Figure 2 shows a view 200 of a graphical representation 202 of a MPS representation of the tensor 104. The graphical representation 202 of the MPS comprises a number of tensors or components; the components can also be known as tensor carriages, the terms component and tensor carriageOUI Project 21872; HGF Ref: P361170WO 3 will be used synonymously. Each square 204 to 210 represents a rank-3 tensor, ^^^, but for the left-most 204 and right-most 210 tensors, or the left and right boundaries, which are rank-2 tensors.

[0020] Figure 2 shows a more detailed graphical representation 212 of a generic tensor component of the MPS. As indicated above, the vertical line 214 represent a physical index, ^^, 216. The horizontal lines 218 and 220 are, or represent, ancillary indicates, ^^, 222 and 224 respectively. The connecting lines 218 and 220 to adjacent components represents a contraction over a common index or over common indices. It can be seen that the connecting lines 218 and 220 have or represent respective dimensions 226 and 228 such as bond dimensions ^^and ^^^^depicted. The dimensions ^^and ^^^^can be different or the same.

[0021] Referring to the concept of contraction, suppose a given (2,2) tensorhas contravariant indices ^^and ^^and covariant indices ^^and ^^, the contraction of ^^^^^^^^^wrt ^^and ^^, for instance, would be:= ^^^^^^^^^. Assuming ℝ^, the contraction gives a sum of three mixed tensors of rank 2, which would give a (1,1) tensor, since contraction of a (p,q)-type tensor over a contravariant index and a covariant index results in a (p-1,q-1)-type tensor. Consequently,

[0023] Therefore, in contracting ^ to ^^, the contravariant rank is reduced by 1, and the covariant rank is reduced by 1. This follows from contracting over the common or dummy index and summing over that common or dummy index.

[0024] Within the application the terms contraction and index contraction are synonymous. Contraction can be applied to indices of a tensor, or to at least one, or more than one, common index of at least two tensors. Consequently, an index contraction is the sum over all possible values of the repeated indices of a set of tensors. For instance, the tensor product

[0025] ^^^^^^

[0026] contracted over index ^ would give

[0028] In the examples described herein, tensor ^ can represent, for example, a copy tensor and tensor ^ can represent a tensor component or tensor carriage of an MPS.

[0029] Figure 2 also shows a more detailed graphical representation 212’ of a generic tensor component of the MPS within a plateau (not shown) of the MPS. As indicated above, the vertical line 214’ represent a physical index, ^^, 216’. The horizontal lines 218’ and 220’ are, or represent, ancillary indices, ^^, 222’ and 224’ respectively. The connecting lines 218’ and 220’ to adjacent components represent a contraction over a common index or over common indices. It can beOUI Project 21872; HGF Ref: P361170WO 4 seen that the connecting lines 218’ and 220’ have or represent the maximal respective bond dimensions 226’ and 228’,of the tensor components of the MPS in the plateau.

[0030] In general, a MPS with bond dimension ^ is defined as

[0032] where the matrices, or tensor components, ^^^have dimensions ^(^ − 1). ^(^), where

[0033] ^(^) = ^^^(2^, 2^^^, χ) (2)

[0034] are the internal bonds that are summed over in the product of the tensors components of (1).

[0035] Figure 3 depicts a view 300 of a log-log graph 302 showing the variation in memory required to store an MPS with tensor rank. The log-log graphs 302 shows a number of plots 304 to 310 depicting the variation in memory requirements, or memory cost 312, to store various MPSs of various ranks and dimensions over a small range; the abscissa shows tensor rank. It can be seen that the memory required to store an MPS scales with tensor rank as ≈ ^, which is a significant improvement on the above ≈ ^^.

[0036] However, evaluating, that is, calculating, the product of two tensors, even using respective MPSs to reduce the tensor memory storage requirement, need a significant amount of memory and a significant number of processing operations as will be described below with respect to figures 4 and 5.

[0037] Figure 4 illustrates a view 400 of a first MPS 402 and a second MPS 404 represented as tensor networks. The first MPS 402 of comprises N components. The second MPS comprises M states. In the example depicted N=M.

[0038] Figure 5 shows a view 500 of a known technique for multiplying a first MPS 502 and a second MPS 504. It will be appreciated that the product of multiplying the first MPS 502 and the second MPS 504 will also be a tensor, ^, such that, for a set of indices, ^^, gives

[0039] ^(^^) = ^1(^^) ∗ ^2(^^), where

[0040] ^1(^^) is the first MPS 502 and ^2(^^) is the second MPS 504. It will be appreciated that the tensor ^(^^) comprises values of the first and second MPSs 502 and 504 that are multiplied pointwise.

[0041] The first MPS 502 comprises N rank-3 tensor components 506 to 512 with maximum bond dimension ^. The first MPS 502 is an example of the above-described MPS that corresponds to tensor 102. Similarly, the components 506 to 512 of the first MPS 502 have a structure that is the same as the above-described generic tensor components 212 and 212’.

[0042] The second MPS 504 comprises N rank-3 tensor components 514 to 520 with maximum bond dimension ^. The second MPS 504 is an example of the above-described MPS that corresponds to tensor 102. Similarly, the components 514 to 520 of the second MPS 504 have a structure that is the same as the above-described generic tensor component 212 and 212’.OUI Project 21872; HGF Ref: P361170WO 5

[0043] The product of the two MPSs 502 and 504 is realised by, firstly, creating a Matrix Product Operator (MPO) 522. The MPO 522 is derived from the first MPS 502 and a copy tensor 524. The copy tensor 524 is a three-legged tensor comprising three legs 526 to 530. The copy tensor 524 is operable to copy the value of a physical leg, in a given basis, to which one of the legs 526 of the copy tensor’s is connected to the two other legs 528 and 530 of the copy tensor. In total, the physical leg dimension is eight such that the total number of elements is eight. The copy tensor 524 has two non-zero entries. The two non-zero elements are equal to one if all three legs are indexed by zero or all three legs are indexed by 1, that is, ^^^^= ^^^^= 1, which is a three dimensional Kronecker delta. The copy tensor 524 acts to break up the bonds between the tensors while ensuring that neighbouring tensors are contracted according to the same physical index on a pair of legs. Therefore, the physical indices 532 to 538 are copied to two further tensors that couple without the legs.

[0044] The resulting MPO 522 is contracted with the second MPS 504 along the vertical indices, or physical dimensions, which results in a MPS representing the product of the first 502 and second 504 MPSs in which each component or tensor carriage 540 has an index or physical leg and maximal bond dimensions of χ^. The contraction has a respective cost of, or of the order of, or related to, or that is otherwise associated with or derived from χ^. ^^.

[0045] It will be appreciated that the contraction along the physical legs, ^^, has a cost associated with respective bond dimensions of those legs, and, in the maximal case, a cost associated with χ^, Examples can be realised in which the contraction along the physical legs, ^^, has a maximal cost associated with χ^^^per contraction, which gives a total cost that varies with χ^. ^^.

[0046] The amount of memory required to perform the foregoing calculations is significant. The amount of disc storage scales with χ^. More particularly, examples can be realised in which the amount of disc storage scales with ^. χ^. Similarly, the amount of RAM usage scales with χ^, which can be further reduced to χ^is a compression technique applied such as described in U. Schollwöck, “The density-matrix renormalization group in the age of matrix product states” Ann. Phys.326, 96–192 (2011). The number of floating point addition and multiplication operations needed to realise the above product of the first and second MPS scales with χ^.

[0047] Figure 6 shows a view 600 of calculating the product of first 602 and second 604 matrix product states according to examples.

[0048] The first MPS 602 comprises ^ rank-3 tensor components 606 to 612 of maximal bond dimension ^. Each component 606 to 612 has a respective index ^^614 to 620. The first MPS 602 is an example of the above-described MPS that corresponds to tensor 102. Similarly, the components 606 to 612 of the first MPS 602 have a structure that is the same as the above- described generic tensor component 212.OUI Project 21872; HGF Ref: P361170WO 6

[0049] The second MPS 604 comprises ^ rank-3 tensor components 622 to 628 of maximal dimension ^. Each component 622 to 628 has a respective index ^^630 to 636. The second MPS 604 is an example of the above-described MPS that corresponds to tensor 102. Similarly, the components 622 to 628 of the second MPS 604 have a structure that is the same as the above-described generic tensor component 212.

[0050] The first 602 and second 604 MPSs are concatenated to form an auxiliary MPS 638 that is represented by the respective tensor network 640 shown. The auxiliary MPS 638 comprises 2^ rank-3 tensor components 642 to 656.

[0051] The first 602, second 604 and auxiliary MPSs are stored in respective memory 658 to 662. The memory 658 to 662 can be memory associated with, or accessible to, a processor, which will be described below with reference to figures 9 to 11.

[0052] Corresponding pairs of tenor components are swapped using a Matrix Product Operator (MPO) 664 that is described below with reference to figure 11. Following such a swap, adjacent pairs of tensor comprising a swapped tensor component are contracted with one another and the result of that contraction is further contracted with a copy tensor as shown in figure 7.

[0053] Figure 7 shows a view of contracting adjacent tensor components of the auxiliary MPS 640 followed by contracting the contraction of the adjacent tensor components with a copy tensor.

[0054] In general, selected tensor components are swapped to form adjacent tensor component pairs. The adjacent tensor component pairs are subsequently contracted with one another. An example of realising such adjacent tensor component pairs can be seen in figure 7, in which selected pairs of tensor components 644 and 650 have been swapped using the MPO 664 to form first and second sets of adjacent tensor component pairs; namely, a first set of adjacent tensor component pairs that comprises tensor components in positions 1 and 2 of the auxiliary MPS 662 and a second set of adjacent tensor component pairs that comprises tensor components in positions L and L+1 of the auxiliary MPS 662. The selected pairs of tensor components of the auxiliary MPS 640 are: the first tensor component 642 and the (^ + 1)^^tensor component 650, the second tensor component 644 and the (^ + 2)^^tensor component 652, the third tensor component 646 and the (^ + 3)^^tensor component 654, … , thetensor component 648 and the (2^)^^tensor component 656. In general, the ^^^tensor component 702 and the (^ + ^ − 1)^^tensor component 704 are swapped using the MPO 664.

[0055] The swapped components are contracted with their newly adjacent tensor components. For example, following swapping the second tensor component 644 and the(^ + 1)^^tensor component 650, the newly adjacent first tensor component 642 and new second tensor component of the MPS-A 640 that was derived from the(^ + 1)^^tensor component 650, are contracted, which results in a respective or first contracted tensor component 706 having bond dimensions of ^. The cost of producing such a first contracted tensor component 706 isOUI Project 21872; HGF Ref: P361170WO 7 associated with the number of bonds, ^, and the bond dimension, ^, of those bonds. Examples can be realised in which the cost is ^^χ^.

[0056] Similarly, following swapping the second tensor component 644 and the (^ + 1)^^tensor component 650, the newly adjacent(^ + 1)^^tensor component that was derived from second tensor component 644 and the(^ + 2)^^tensor component 652 are contracted, which results in a respective or second contracted tensor component 708 having bond dimensions of ^. The cost of producing such a second contracted tensor component 708 is associated with the number of bonds, ^, and the bond dimension, ^, of those bonds. Examples can be realised in which the cost is ^^χ^.

[0057] It will be noted that each contraction reduces the length of the auxiliary MPS 640 by 1.

[0058] The first contracted tensor component 706 is contracted, by a copy tensor processor 707, with the copy tensor 710. The copy tensor processor 707 is arranged to perform a contraction between two presented tensors. In the example depicted, the copy tensor processor 707 performs a contraction between the copy tensor 710 and the first contracted tensor component 706 over the physical legs of the pair of tensor components. The copy tensor 710 is an example of the above described copy tensor 524, which results in the first tensor component of the product of the first 602 and second 604 MPSs. The second contracted tensor component 708 is contracted with the same copy tensor 710, which results in an intermediate tensor component, 2^′ , 714 of the product of the first 602 and second 604 MPSs.

[0059] Referring to figure 8, there is shown a view 800 of a partial result 802 of the product of the first 602 and second 604 MPSs. Following calculating the first tensor component, 1^, 712 of the product and the intermediate tensor component, 2^′ , 714 of the product, the intermediate tensor component, 2^′ , 714 is swapped with the third tensor component 646 of the auxiliary MPS 640 using the Matrix Product Operator 664, which results in the second tensor component, 2^, 803 of the product of the two MPSs.

[0060] The third tensor component 646, following the swap, is positioned adjacent to the (^ + 3)^^tensor component of the auxiliary MPS 640 forming a tensor component pair 804. The tensor component pair 804 is contracted, by a contraction processor 805 to produce a contracted tensor component 806. The contraction processor 805 is arrange to contract a pair of tensor components over at least one common index or over a number of common indices. In the example depicted, the contraction processor 805 is arranged to contract adjacent tensor components of the auxiliary MPS 640. The contracted tensor component 806 is further contracted, by the copy tensor processor 707 as described above, with the copy tensor 710 to produce the 3^^or further intermediate tensor component, 3^′, 808 of the product of the two MPSs 602 and 604.OUI Project 21872; HGF Ref: P361170WO 8

[0061] Figure 9 shows a view 900 of the processor operations subsequent to the contraction by the copy tensor contraction processor 707. Following contraction by the copy tensor contraction processor 707, the 3^^or further intermediate tensor component, 3^′ , 808 is swapped, using the MPO 664, with the 4^^tensor component of the auxiliary MPS 640. Swapping the 3^^or further intermediate tensor component, 3^′, 808, using the MPO 664, with the 4^^tensor component of the auxiliary MPS 640 results in the auxiliary MPS comprising three tensor components 1^, 2^, 3^as part of a partial result 902 of the product of the two MPSs 602 and 604 in which the 3rdtensor component 3^of the product of the two MPSs results from swapping the 3rdor further intermediate tensor component 3^′ from its pre-swap position to the 3rdposition in the MPS.

[0062] The processor operations continue as indicated above, that is, the processor repeatedly executes the following processing steps: a. swap selected pairs of tensor components comprising an ^^^tensor component of the auxiliary MPS 640 and a most recently calculated (^^− 1)^^intermediate tensor component of the product of the two MPSs 602 and 604, b. contract adjacent pairs of tensor components comprising the recently swapped ^^^tensor component of the auxiliary MPS and a next adjacent (^ + ^)^^tensor component of the auxiliary MPS 640, c. contract the result of the above contraction using the copy tensor to give the next ^^′^^intermediate tensor component of the product of the two MPSs 602 and 604, and d. repeat (a) to (c) above to swap, using the swap MPO, the ^^′^^intermediate tensor component, which is the most recently calculated (^^− 1)^^intermediate tensor component, and the next tensor component of the auxiliary MPS,

[0063] until all tensor components 606 to 612 and 622 to 628 have been processed.

[0064] Figure 10 shows a view 1000 of the final product 1002 of the two MPSs 602 and 604. The final product 1002 comprises half the number of tensor components 1004 to 1010 as compared to the auxiliary MPS 640.

[0065] Figure 11 depicts a view 1100 of a flow chart 1102 of processor operations for evaluating the product of MPSs depicted in, and described with reference to, figures 6 to 10 according to examples.

[0066] The first 602 and second 604 MPSs are received or accessed from the memory 658 and 660 at steps 1104 and 1106.

[0067] The auxiliary MPS 640 is formed from the first 602 and second 604 MPSs and stored in memory 662 at 1108.

[0068] At 1110, selected pairs of tensor components of the auxiliary MPS 640 are swapped using the MPO 664.OUI Project 21872; HGF Ref: P361170WO 9

[0069] Following such swapping, adjacent pairs of tensor components comprising a swapped tensor component and an immediately adjacent, preceding or subsequent, component, are contracted over at least one index, or over a number of common indices, to produce first tensor contraction data at 1112.

[0070] The first tensor contraction data is further contracted using a copy tensor to produce second tensor contraction data at 1114 to establish respective second tensor component data.

[0071] The established respective second tensor component data is swapped using the MPO 664 with the next tensor component of the auxiliary MPS 640 at 1116. It will be appreciated that 1116 can be omitted when contracting the tensor components in the first and second positions of the auxiliary MPS 640 following swapping of the 2^^and (^ + 1)^^tensor components.

[0072] At 1118, a determination is made regarding whether or not all tensor components of the auxiliary MPS 640 have been processed. If the determination at 1118 is negative, processing resumes at 1110. If the determination at 1118 is positive, processing terminates since the product of the two MPSs 602 and 604 will have been calculated.

[0073] Figure 12 shows a view 1200 of a processor 1202 configured to efficiently evaluate the product of MPSs as depicted in, and described with reference to figures 6 to 11, and figures 13 and 14.

[0074] The processor 1202 comprises memory 1204. The memory 1204 is arranged to store MPSs to be processed such as, for example, the above-described first 602 and second 604 MPSs to be multiplied, together with the resulting product and any intermediate data used to realise the product.

[0075] The processor 1202 comprises circuitry 1206 for implementing and coordinating processing operations. The circuitry 1206 can be application specific hardware or a combination of hardware and software; the latter being executable on the hardware. The application specific hardware can be configured to implement the processing for generating the product of the first 602 and second 604 MPSs. Similarly, the software can comprise instructions arranged, when processed by the hardware, to implement the processing for generating the product of the first 602 and second 604 MPSs.

[0076] The memory 1204 can be realised as individual registers or specifically allocated memory, or be realised as a generic memory addressable using a memory controller 1208 comprising address 1210 and data 1212 buses. Although the memory controller 1208 has been shows a part of the circuitry 1206, examples can be realised in which the memory controller is separate from the circuitry 1206.

[0077] The processor 1202 comprises a set of contraction circuitry. The set of contraction circuitry can comprise at least one instance of contraction circuitry 1218. Examples can be realised in which the set of contraction circuity 1202 comprises multiple instances of contractionOUI Project 21872; HGF Ref: P361170WO 10 circuitry. The contraction circuitry is arranged to perform tensor contractions of tensors over one common index or multiple common indices. In the example depicted, four sets of contraction circuitry 1220 to 1226 are provided that have respective sets of contraction indices 1228 to 1234.

[0078] A first pair of contraction circuity 1220 and 1224 of the four instances of the contraction circuity 1220 to 1226 perform the above-described tensor component contraction of adjacent tensor components of the auxiliary MPS 640 in which any set of adjacent tensor components comprises a swapped tensor component of the MPS and a non-swapped tensor component of auxiliary MPS.

[0079] Having contracted the adjacent tensor components, the contracted tensor components are contracted with a copy tensor 1236 via at least one of the contraction circuitry of the set of contraction circuitry. The copy tensor 1236 is an example of the above described copy tensor 710. In the example depicted, the copy tensor 1236 is contracted with the contractions output by the first pair of contraction circuitry 1220 and 1224 to produce intermediate tensor components 1238 and 1240. The intermediate tensor components 1238 and 1240 are then swapped, using the swap MPO, to form part of the partial result of the product of the first 602 and second 604 MPSs.

[0080] Although examples have been described comprising four instances of contraction circuity 1220 to 1226, examples are not limited to such an arrangement. Examples can be realised that use fewer than four instances or more than four instances. Examples can be realised in which a single instance is implemented in which pairs of tensor components are presented to the single instance of the contraction circuitry and contracted over the at least one common index, which is known a dummy index, or over a common set of indices, which comprises a set of dummy indices.

[0081] Figure 13 depicts a view 1300 of circuitry 1302 for implementing the Matrix Product Operator that swaps tensor component positions as described above. The circuitry 1302 implements a tensor component swap operation using tensor swap circuitry 1304 that is responsive to first 1306 and second 1308 inputs that identify the tensor components of the auxiliary tensor 640 to be swapped. The circuitry 1302 has an overall input that receives, or at least has access to, the auxiliary MPS 640 and produces a swapped MPS 640.

[0082] As indicated above, in general, a MPS with bond dimension ^ is defined as

[0084] where the matrices, or tensor components, ^^^have dimensions ^(^ − 1). ^(^), where

[0085] ^(^) = ^^^(2^, 2^^^, χ) (2)

[0086] are the internal bond that are summed over in the product of the tensors components of (1), with χ being the maximal bond dimension of tensor components or carriage of a tensor train. The MPSs are examples of such tensor trains.OUI Project 21872; HGF Ref: P361170WO 11

[0087] In the following, let the above described Matrix Product Operator be denoted by ^ and its contraction with an MPS, ^, as ^^. A generic MPO with bond dimension ^can be written as

[0088] ^ = ^^[1]... ^[^]^[^ + 1]... ^[2^]^, (3)

[0089] where ^ is a 1^^ row vector, ^ is a ^^1 column vector, and ^[^] with ^ ∈ {1, ... , 2^} are ^^^ matrices whose matrix elements are 2^2 matrices. Any 2^2 matrix can be expanded in terms of the following four operators

[0094] Also, let the identify matrix be

[0095] ^ = ^1 0^ (5) 0 1

[0096] Multiplying the matrices ^[^]in equation (3) is realised by taking the outer product of the matrix-valued matrix elements. To illustrate the notation, consider the following example for ^ = 2

[0097] ^ =(1,0)(6)

[0102] where ⊗ denotes the outer product.

[0103] Therefore, the swap MPO, denotes as ^^^^^, is realised as follows:

[0104] Given an MPS of length ^, and assume that a pair of sites ^, ^ are to be swapped with 1 ≤ ^ ≤ ^ ≤ ^ ≤ ^. Let the resulting (post-swap) swapped MPS be denoted by

[0105] ^^= ^^^^^^ (11a)

[0106] where the swap MPO ^^^^^is defined by

[0107] ^ = (1,1,1,1)OUI Project 21872; HGF Ref: P361170WO 12

[0108] ^[^]= ^, 1 ≤ ^ ≤ ^ (11c)

[0116] Examples can be realised in which the dimension, ^ , is always 2. It will be appreciated that applying the MPO to the MPS can be combined with the compression algorithm described in U. Schollwöck, “The density-matrix renormalization group in the age of matrix product states,” Ann. Phys.326, 96–192 (2011) to give an MPS of bond dimension of χ such that the bond dimension does not increase.

[0117] Figure 14 illustrates machine readable storage storing machine readable instructions for evaluating a product of MPSs according to examples.

[0118] The functionality of the processor 1202 can be realised in the form of machine instructions that can be processed by a machine comprising or having access to the instructions. The machine can comprise a computer, processor, processor core, DSP, a special purpose processor implementing the instructions such as, for example, an FPGA or an ASIC, circuitry or other logic, compiler, translator, interpreter or any other instruction processor. Processing the instructions can comprise interpreting, executing, converting, translating or otherwise giving effect to the instructions. The instructions can be stored on a machine readable medium, which is an example of machine-readable storage. The machine-readable medium can store the instructionsOUI Project 21872; HGF Ref: P361170WO 13 in a non-volatile, non-transient, manner or in a volatile, transient, manner. The instructions can be arranged to give effect to any and all operations described herein taken jointly and severally in any and all permutations. The instructions can be arranged to give effect to any and all of the operations, devices, flowcharts, protocols or methods described herein taken jointly and severally in any and all permutations. In particular, the machine instructions can give effect to, or otherwise implement, the operations of the flowcharts depicted in, or described with reference to, figures 6 to 13, taken jointly and severally in any and all permutations.

[0119] Therefore, figure 14 shows a view 1400 of machine instructions 1402 stored using machine readable storage 1404 for implementing the examples described herein. The machine instructions 1402 can be processed by, for example, a processor 1406 or other processing entity, such as, for example, an interpreter, as indicated above.

[0120] The machine instructions 1402 comprise at least one or more than one of:

[0121] Instructions 1408 to access first matrix product state data as described above,

[0122] Instructions 1410 to access second matrix product state data as described above,

[0123] Instructions 1412 to concatenate first and second matrix product state data to form auxiliary MPS,

[0124] Instructions 1414 to swap selected pairs of tensor components of the concatenated auxiliary MPS using MPO as described above,

[0125] Instructions 1416 to process corresponding swapped tensor component data to establish respective tensor contraction data as described above,

[0126] Instructions 1418 to process copy tensor data and tensor contraction data to establish respective second tensor component data,

[0127] Instructions 1420 to swap the respective second tensor component data in with the next tensor component of the auxiliary MPS to form a partial result of the product of the first and second matrix product states as described above, and

[0128] Instructions 1422 to determine whether or not all tensor components of the auxiliary MPS have been processed as described above,

[0129] the foregoing instructions 1408 to 1422 being taken jointly and severally in any and all permutations.

[0130] The foregoing processor and / or machine-readable storage can be used to process or generate data representing fluid flow as part of, for example, a DNS to computational fluid dynamics modelling or simulation.

[0131] Examples can be realised in which the processor 1202 can be realised in the form of hardware, software or a combination of hardware and software. Furthermore, examples can be realised in which the functionality of such software, hardware, or combination of hardware and software, can be provided by a web-interface, or any other network-accessible interface. TheOUI Project 21872; HGF Ref: P361170WO 14 interface can be arranged to receive the first and second MPS, or an auxiliary MPS, and return any of the above-described products.

[0132] It can be appreciated that any such processor and / or instructions uses significantly fewer computational resources as compared to the prior art. The savings in terms of memory usage, disc storage and number of floating point operations also consumes significantly less power than as compared to the prior art.

Claims

OUI Project 21872; HGF Ref: P361170WO 15 CLAIMS 1. An apparatus for processing first and second Matrix Product States, the apparatus comprising circuitry to: a. receive, or store in memory, first matrix product state data (MPS 1) associated with a first matrix product state; the first matrix product state data comprising first tensor component data associated with the tensor components of the first matrix product state; b. receive, or store in memory, second matrix product state data (MPS 2) associated with a second matrix product state; the second matrix product state data comprising second tensor component data associated with the tensor components of the second matrix product state; c. process corresponding tensor component data of the first and second matrix product state data to produce tensor contraction data representing a contraction of the tensor components associated with the corresponding tensor component data of the first and second matrix product state data; d. process copy tensor data representing a copy tensor and the tensor contraction data to produce respective tensor component data; the respective tensor component data representing a contraction of the copy tensor and the contraction of the tensor components associated with the corresponding tensor component data of the first and second matrix product states; and e. associate, or store in memory, the respective tensor component data with a corresponding tensor component data position of result data representing a resultant matrix product state associated with the processing / multiplying the first and second Matrix Product States.

2. The apparatus of claim 1, comprises circuitry to establish in a memory a data structure for storing the result data representing the resultant matrix product state.

3. The apparatus of any preceding claim, in which the circuitry to process corresponding tensor component data of the first and second matrix product state data comprises circuitry to process adjacent tensor component data of the first and second matrix product state data.

4. The apparatus of claim 3, in which the circuitry to process adjacent tensor component data of the first and second matrix product state data comprises circuitry to swap selectable tensor component data pairs to form the adjacent tensor component data of the first and second matrix product state data.

5. The apparatus of any preceding claim, comprising circuitry to construct auxiliary Matrix Product State data representing an auxiliary Matrix Product State; the auxiliary MatrixOUI Project 21872; HGF Ref: P361170WO 16 Product State data being a concatenation of the first matrix product state data and the second matrix product state data.

6. Machine-readable instructions arranged, when processed, to process or multiply first and second Matrix Product States, the instructions comprising instructions to: a. receive, or store in memory, first matrix product state data (MPS 1) associated with a first matrix product state; the first matrix product state data comprising first tensor component data associated with the tensor components of the first matrix product state; b. receive, or store in memory, second matrix product state data (MPS 2) associated with a second matrix product state; the second matrix product state data comprising second tensor component data associated with the tensor components of the second matrix product state; c. process corresponding tensor component data of the first and second matrix product state data to produce tensor contraction data representing a contraction of the tensor components associated with the corresponding tensor component data of the first and second matrix product state data; d. process copy tensor data representing a copy tensor and the tensor contraction data to produce respective tensor component data; the respective tensor component data representing a contraction of the copy tensor and the contraction of the tensor components associated with the corresponding tensor component data of the first and second matrix product states; and e. associate, or storing in memory, the respective tensor component data with a corresponding tensor component data position of result data representing a resultant matrix product state associated with the processing / multiplying the first and second Matrix Product States.

7. The machine-readable instructions of claim 6, comprising instructions to establish in a memory a data structure for storing the result data representing the resultant matrix product state.

8. The machine-readable instructions of any of claims 6 to 7, in which the instructions to process corresponding tensor component data of the first and second matrix product state data comprise instructions to process adjacent tensor component data of the first and second matrix product state data.

9. The machine-readable instructions of claim 8, in which the instructions to process adjacent tensor component data of the first and second matrix product state data comprise instructions to swap selectable tensor component data pairs to form the adjacent tensor component data of the first and second matrix product state data.OUI Project 21872; HGF Ref: P361170WO 17 10. The machine-readable instructions of any of claims 6 to 9, comprising instructions to construct auxiliary Matrix Product State data representing an auxiliary Matrix Product State; the auxiliary Matrix Product State data being a concatenation of the first matrix product state data and the second matrix product state data.

11. Machine-readable storage storing machine-readable instructions of any of claims 6 to 10.

12. The apparatus of any of claims 1 to 5, in which said processing comprises multiplying said first and second Matrix Product States.

13. A tensor processor comprising the apparatus of any of claims 1 to 5, and claim 12.

14. A tensor processor comprising machine-readable instructions of any claims 6 to 10.

Images