Computer computing device with cache memory optimized for matrix calculations

Abstract

Computer computing device with optimized cache memory for matrix calculation Computer computing device (100), comprising: - a computing unit (102); - a main memory (104); - a cache memory (106) comprising a computing circuit (108) for reduction operations between partial products from values ​​of a sparse matrix (118) and an input vector, and an output vector; in which the cache memory has a first N-way associative memory region (112) storing, with a first granularity of TD words, results of reduction operations carried out from partial products from a dense region (120) of the matrix, and a second fully associative or M-way associative memory region (114) storing, with a second granularity of TS words, results of reduction operations carried out from partial products from a sparse region (124) of the matrix. Figure for the abridged version: Fig. 1

Description

Title of the invention: Computer computing device with optimized cache memory for matrix calculations. Technical field

[0001] This description relates generally to the field of computer computing devices, or computers, used in particular for the implementation of matrix calculations. Previous technique

[0002] Many high-performance computing (HPC) tasks involve the implementation of matrix calculations, such as solving systems of partial differential equations or semantic graphs. It is common for the matrices involved in these calculations to be sparse matrices, also called low-density or sparse matrices, containing a significant number of null values, or zeros, relative to the total number of values. These calculations may involve, in particular, the execution of algorithms that multiply a sparse matrix A by a vector b (an operation called "SpMV"), which corresponds to the calculation of an output vector c = Ab. Therefore, it is important to optimize systems that perform calculations with such matrices, especially when dealing with very large matrices containing, for example, millions of non-zero values.

[0003] There are storage formats for sparse matrices that avoid storing all the zeros of these matrices. These formats correspond, for example, to the CSR (Compressed Sparse Row) or CSC (Compressed Sparse Column) formats. These formats reduce the amount of memory required for their storage. However, in return, they require traversing tables that contain indices, i.e., locations, with non-zero values. Now, the memory and cache systems used in conventional computers are not well suited to handling the irregular memory access sequences that result from processing the sparse data of sparse matrices.

[0004] There are different ways to perform a SpMV-type calculation, such as inner-product or outer-product algorithms. Outer-product algorithms require fewer memory accesses than inner-product algorithms, but they are less used because they involve updating the output vector c such that ci = Ci + A tj * bj, with ct corresponding to the values ​​of the output vector c, A ij corresponding to the values ​​of the sparse matrix A, and bj corresponding to the values ​​of the input vector b. These updates of the values ​​of the output vector c imply Irregular memory accesses occur because the positions of these values ​​depend on those of the non-zero values ​​in the sparse matrix A. These updates to the values ​​of the output vector c are called "reductions" and consist of adding a partial product (A *bj ) to the previous value of the vector c. These reductions made at non-regular addresses cause a lot of data traffic to main memory, or external memory, and for this reason, "inner-product" type algorithms are by default the most used.

[0005] It should be noted that a matrix can be decomposed into several regions, and that each region of a matrix can be treated independently. It is therefore possible, when evaluating a large matrix, to use both "inner-product" and "outer-product" type algorithms.

[0006] In many application areas, such as the solving of differential equations, sparse matrices often have non-zero elements located relatively densely around the matrix diagonal, with this distribution becoming sparse around the diagonal. Furthermore, sparse matrices that do not have this so-called "banded" structure, in which non-zero values ​​are predominantly located in the region of the matrix diagonal, can be transformed into matrices with a diagonal dense in non-zero values, at a reasonable computational cost.

[0007] US patent 2018 / 189239 A1 describes a hardware acceleration architecture for performing computations on sparse matrices. In this architecture, the matrix being processed can be decomposed into two regions, one of which is denser in non-zero values ​​than the other. These two regions are processed by separate computing units and memories. In this architecture, the system is highly parallelized with a physical separation of transfers between sparse and very sparse data. Furthermore, the decomposition of the matrix for the different regions requires preprocessing of the matrix. Summary of the invention

[0008] There is a need to propose a computer computing device optimized for the implementation of outer-product calculation algorithms with a reduced number of accesses to the main memory, or external memory, of the device.

[0009] An embodiment overcomes all or part of the drawbacks of existing solutions and proposes a computer computing device, comprising at least:

[0010] - a computing unit;

[0011] - a main memory;

[0012] - a cache memory configured to exchange data with the processing unit and with the main memory, and including a computing circuit configured to perform reduction operations between partial products from values ​​of at least one sparse matrix and at least one input vector, and at least one output vector;

[0013] in which the cache memory comprises at least a first N-way associative memory region configured to store, with a first word granularity Td, results of reduction operations performed by the computing circuit from partial products from values ​​of at least one dense region of the sparse matrix, and at least a second fully associative or M-way associative memory region configured to store, with a second word granularity Ts, results of reduction operations performed by the computing circuit from partial products from values ​​of at least one sparse region of the sparse matrix, with M, N, TD and Ts corresponding to integers such that M > N, TD > Ts, and also such that M > N if TD = Ts and such that TD > Ts if M = N.

[0014] According to a particular embodiment, the cache memory includes an interface coupled to the computing unit and configured to receive reduction operations required by the computing unit and intended to be implemented by the computing circuit, and to send corresponding data into the first memory region when the partial products of the reduction operations are from values ​​in the dense region of the sparse matrix or into the second memory region when the partial products of the reduction operations are from values ​​in the sparse region of the sparse matrix.

[0015] According to a particular embodiment, the main memory and the cache memory are configured such that exchanges between the main memory and the first memory region correspond to read and write operations, and / or in which exchanges between the main memory and the second memory region correspond to RMW type atomic operations.

[0016] According to a particular embodiment, the cache memory further comprises at least one third associative memory region per set configured to store data sent from the main memory.

[0017] According to a particular embodiment, the cache memory further comprises at least one FIFO memory region configured to temporarily store data sent from the second memory region to the main memory.

[0018] According to a particular embodiment, the second memory region is configured such that if the implementation of a reduction operation by the computing circuit involves the eviction of data stored in the second memory region, said The reduction operation is implemented in the main memory or in another cache memory interposed between the cache memory and the main memory.

[0019] According to a particular embodiment, the cache memory is configured to implement, upon receiving a reduction operation requested by the computing unit and whose result involves a modification of a result value:

[0020] - search for the presence of the result value in the first memory region;

[0021] - update the result value in the first memory region if this value is present in the first memory region, or sending the reduction operation requested by the computing unit to the first memory region or the second memory region if this value is absent from the first memory region.

[0022] According to a particular embodiment, the first memory region is configured such that each line of values ​​stored in the first memory region has at least one address field, one line status field and several value fields, and / or the second memory region is configured such that each part of the second memory region intended to store a value has at least one bit representing the state of said part.

[0023] According to a particular embodiment, the first memory region is configured to implement, during reduction operations performed by the calculation circuit from values ​​in the dense region of the sparse matrix:

[0024] - allocation of a row of null values ​​from the first memory region during a access to an address not in the cache;

[0025] - writing the results of said reduction operations in said line of the first memory region;

[0026] - when said line of the first memory region is selected to be ousted, reading of values ​​stored in main memory and combination, in said line of the first memory region, of the values ​​read from main memory with those written in said line of the first memory region;

[0027] - removal of said line from the first memory region, including a writing of the values ​​of said line of the first memory region in main memory.

[0028] According to a particular embodiment, the cache memory is configured such that when the density of non-zero values ​​of a part of the sparse region of the sparse matrix is ​​greater than a first threshold value, results of reduction operations implemented from the values ​​of said part of the sparse region of the sparse matrix are stored in the first memory region.

[0029] According to a particular embodiment, the second memory region is configured to implement the eviction of at least one of the values ​​stored in the second memory region to main memory when the number of values stored in the second memory region exceeds a predefined storage capacity threshold.

[0030] According to a particular embodiment, the cache memory further comprises an interface block configured to determine the size of each of the exchanges to and from the main memory and a circuit for implementing a leaky bucket type algorithm delivering at least one variable representative of a bandwidth of access to the main memory.

[0031] According to a particular embodiment, the first memory region is configured to synchronize read request sends to main memory based on a value of the variable representing the main memory access bandwidth, and / or the second memory region and the interface block are configured to implement evictions of values ​​stored in the second memory region to main memory based on a number of values ​​stored in the second memory region and a value of the variable representing the main memory access bandwidth.

[0032] According to a particular embodiment, the cache memory further comprises a fourth multi-way associative memory region smaller than that of the first memory region, and a buffer memory block configured to temporarily store result values ​​of reduction operations performed by the computing circuit from partial products from values ​​of at least a second dense region of the localized sparse matrix and to transfer said values ​​into the second memory region or into the fourth memory region.

[0033] According to a particular embodiment, the sizes of the cache memory regions are defined according to characteristics of the sparse matrix. Brief description of the drawings

[0034] These features and advantages, as well as others, will be described in detail in the following description of particular embodiments, given by way of non-limiting example, in relation to the accompanying figures, among which:

[0035] - [Fig. 1] schematically represents a computer computing device according to a first embodiment;

[0036] - [Fig.2] schematically represents an association made between different regions of a sparse matrix used in a calculation implemented by the computing device and different regions of the cache memory of the computing device;

[0037] - Figure 3 shows examples of reduction operations carried out sequentially in a first memory region of a cache memory of the computer computing device;

[0038] - Figure 4 shows examples of reduction operations performed sequentially in a second memory region of a cache memory of the computing device;

[0039] - [Fig. 5] schematically represents a computer computing device according to a second embodiment;

[0040] - Figure 6 schematically represents a computer computing device according to a third embodiment;

[0041] - Figure 7 schematically represents an example of a sparse matrix used in a calculation implemented by the computer computing device according to the third embodiment. Description of the implementation methods

[0042] The same elements have been designated by the same reference numerals in the different figures. In particular, the structural and / or functional elements common to the different embodiments may have the same reference numerals and may have identical structural, dimensional and material properties.

[0043] For the sake of clarity, only the steps and elements necessary for understanding the described embodiments have been shown and are detailed. In particular, various components (processor, main memory, cache memory, memory controller, data bus, etc.) of the computing device are not detailed. A person skilled in the art will be able to implement these components in detail from the functional description given here.

[0044] Unless otherwise specified, when referring to two elements connected together, this means directly connected without intermediate elements other than conductors, and when referring to two elements coupled together, this means that these two elements can be connected or linked through one or more other elements.

[0045] In the following description, when reference is made to absolute position qualifiers, such as the terms "front", "back", "top", "bottom", "left", "right", etc., or relative position qualifiers, such as the terms "above", "below", "superior", "inferior", etc., or to orientation qualifiers, such as the terms "horizontal", "vertical", etc., reference is made, unless otherwise specified, to the orientation of the figures in a normal position of use.

[0046] Unless otherwise specified, the expressions "approximately", "roughly", and "on the order of" mean at 10%, preferably at 5%.

[0047] Throughout the document, the term "vector" is used to refer to a row matrix or a column matrix.

[0048] An example of a computing device 100 according to a first embodiment is described below with reference to [Fig. 1]. In the example described, the device 100 is configured to implement algorithms for multiplying a sparse matrix A by an input vector b (SpMV operation computing the output vector c = Ab). According to one embodiment, the computed data can be floating-point numbers, typically stored in double format, and the operation implemented for reductions by the cache memory 106 can be an addition. Alternatively, or complementaryly, the device 100 can be configured to implement "SpMSpM" type operations, each corresponding to a computation of an output matrix C = A, B, A, and B, which correspond to two sparse matrices and can be seen as a sequence of several consecutive SpMV operations.

[0049] The device 100 includes a computing unit 102 which corresponds for example to a processor or any other circuit suitable for implementing the algorithms intended to be executed.

[0050] The device 100 also includes a main memory, or external memory, 104, for example of the RAM (Random Access Memory) type and typically a DRAM (Dynamic Random Access Memory). The sparse matrix A and the input vector b are stored here in the main memory 104. The sparse matrix A can be stored in a format suitable for implementing an external computing algorithm (outer-product), for example a CSC format.

[0051] The device 100 also includes a cache memory 106 to which the processing unit 102 and the main memory 104 are coupled. The cache memory 106 is configured to exchange data with the processing unit 102 and with the main memory 104, and more specifically with a memory controller of the main memory 104.

[0052] The cache memory 106 includes a calculation circuit 108 configured to perform additive reduction operations between partial products and values ​​of the output vector c. Each partial product corresponds to an operation performed by the calculation unit 102 between one of the values ​​of the sparse matrix A and one of the values ​​of the input vector b. For example, the calculation circuit 108 can perform a reduction corresponding to an addition between a value ci of the output vector c and a partial product Ai,j * bj (Ai,j corresponding to one of the values ​​of the sparse matrix A and bj corresponding to one of the values ​​of the input vector b) such that ci = ci + Ai,j * bj. For example, the calculation unit 108 can correspond to an arithmetic logic unit, or ALU. In the embodiment example As described, the computing unit 108 is configured to implement reduction operations on values ​​of an output vector c stored in cache memory 106.

[0053] In the example of [Fig.1], the cache memory 106 has an interface 110 coupled to the calculation unit 102 and configured to receive operations, corresponding to reductions in the example described, sent by the calculation unit 102 and intended to be implemented by the calculation circuit 108 of the cache memory 106.

[0054] The cache memory 106 further comprises at least a first N-way set associative memory region 112 configured to store, with a first granularity of TD words, values ​​of results of reduction operations carried out by the computing circuit 108 from partial products from, or calculated from, values ​​of a dense region of the sparse matrix A.The cache memory 106 also includes at least one second region 114 fully associative or M-way set associative configured to store, with a second granularity of words Ts, results of reduction operations performed by the computing circuit 108 from partial products from, or calculated from, values ​​of a sparse region of the sparse matrix A, with M, N, TD and Ts corresponding to integers such that M > N, TD > Ts, and also such that M > N if TD = Ts and such that TD > Ts if M = N. .

[0055] A memory region can be considered to have a word granularity T if the size of the smallest transaction of the memory region with external memory is T words. For example, a cache memory having a granularity corresponding to a line size T = 8 can perform writes and reads of a minimum of 8 words. The granularity can be seen as corresponding to the number of consecutive elements stored in the cache (cache line).

[0056] The first memory region 112 of the cache memory 106 is optimized to handle reduction operations performed with partial products from values ​​in the dense region of the sparse matrix A and is organized as a set-associative cache in which complete cache lines, for example of 64 bytes (i.e. 8 values, or 8 words, when these values ​​are stored in a double format and the first memory region 112 is 4-way associative), are stored at each write.

[0057] According to one embodiment, the first memory region 112 can be configured to implement, during reduction operations performed by the circuit Calculation 108 from partial products derived from values ​​in a dense region of the sparse matrix A, the following steps:

[0058] - allocation of a row of null values ​​from the first memory region 112 when accessing an address not yet in cache memory 106;

[0059] - writing the results of said reduction operations in said line of the first memory region 112;

[0060] - when said line of the first memory region 112 is selected to be ousted, reading of values ​​stored in main memory 104 and combination, in said line of the first memory region 112, of the values ​​read in main memory 104 with those written in said line of the first memory region 112;

[0061] - removal of said line from the first memory region 112, comprising a writing the values ​​of said line from the first memory region 112 into main memory 104.

[0062] The cache memory 106 can be configured, in order to avoid or reduce blocking due to the eviction of lines from the cache memory 106, to anticipate reads from the main memory 104, and thus balance the data traffic to and from the main memory 104 and avoid blocking situations in data traffic to / from the main memory 104. Examples of implementation of features enabling these anticipations are described later.

[0063] The second memory region 114 of the cache memory 106 is optimized to handle reduction operations performed with partial products from values ​​in the sparse regions of the sparse matrix A and is, for example, organized as a fully associative cache storing a value individually (for example, in double format, i.e., 8 bytes) at each write, and not complete cache lines as the first memory region 112 does. Indeed, since the data is not dense in the sparse regions of the sparse matrix A, it is not practical to manage, in reading and writing, entire cache lines since most of the cells of the lines would be empty (given the majority presence of zero values ​​in the sparse regions of the sparse matrix A).Managing data at a scale smaller than entire cache lines is therefore advantageous when reduction operations are implemented from partial products of values ​​from sparse regions of the sparse matrix A. Such an advantage is also found when the second memory region 114 forms an associative cache with M paths, with M > N, i.e., in which data management is performed at the scale of smaller cache lines than those manipulated in the first memory region 112, or when the two memory regions 112, 114 are configured to work with different word granularities.

[0064] In the described embodiment, the interface 110 is configured to receive the reduction operations to be performed sent by the computing unit 102 (symbolized by the expression "RED" on [Fig.1]) and, depending on the location of the data concerned in the sparse matrix A (dense or sparse region), send the data to the first memory region 112 or the second memory region 114. The destination region can correspond to a parameter of each reduction operation to be performed.

[0065] For example, considering i and j corresponding to the indices of the rows and columns of the sparse matrix A, and B corresponding to the bandwidth of the dense region, this dense region is that for which the values ​​of A are such that \ij\ < B, the other values ​​belonging to the sparse region of the sparse matrix A. The value of B can be determined as a function of the size of the cache rows of the first memory region 112, or as a function of a density difference between the dense and sparse regions of the sparse matrix A.

[0066] Furthermore, in the described embodiment, the exchanges between the first memory region 112 and the main memory 104 correspond to read and write operations of entire cache lines (symbolized by the expression "R, W" in [Fig. 1]). In addition, the exchanges between the second memory region 114 and the main memory can correspond to atomic "Read-Modify-Write" or RMW operations.

[0067] In the example embodiment of [Fig. 1], the cache memory 106 also includes a third cache memory region 116 operating as a standard cache memory, i.e., configured to store data sent from the main memory 104 and write operations from the arithmetic unit 102. This third memory region 116 can serve as a cache during memory accesses that do not involve the calculation of the output vector c, including reading data from the sparse matrix A or the input vector b, and thus reduce the access latency from the arithmetic unit 102 to data from the sparse matrix A or the input vector b stored in the main memory 104. The third region 116 can operate as a set-associative cache in which complete cache lines are stored at each write and read operation.Unlike the first and second regions 112, 114, the third region 116 is not configured to perform reduction operations on the output vector c. Furthermore, in the described embodiment, the exchanges between the third memory region 116 and the main memory 104 can correspond to read and write operations of entire cache lines.

[0068] In the embodiment shown in [Fig. 1], the cache memory 106 also includes a configured FIFO memory region 117 (“First In First Out”) to operate with the second memory region 114 and the main memory when implementing RMW atomic operations, the data corresponding to these operations being temporarily stored in the FIFO memory region 117 to avoid possible congestion problems in the main memory 104 and to avoid blocking cases in the cache memory 106.

[0069] Figure 2 schematically represents the association made between different regions of the sparse matrix A processed in a computation and different regions of the cache memory 106. In this figure, the sparse matrix A (represented symbolically) is designated by reference 118. Reference 120 designates data belonging to the dense region of the sparse matrix A, that is, located in the region of the diagonal of the sparse matrix A, and which are the subject of an operation whose result is written in a row 122 of the first memory region 112. Reference 124 designates data belonging to the sparse region of the sparse matrix A and which are the subject of an operation whose result is written in the form of individual words 126 in the second memory region 114. In Figure 2, the value of each word is symbolically represented by a "val" cell. In the example of Figure 118, the value of each word is represented symbolically by a "val" cell.[2], the granularity of the first memory region 112 is a line of words (symbolically circled in bold) and that of the second memory region 114 is a single word (symbolically circled in bold).

[0070] The cache memory 106 is configured to store at least some of the values ​​of the output vector c resulting from the operation performed between the sparse matrix A and an input vector b. During a SpMV operation, the matrix A is traversed and the values ​​of the output vector c are updated by the reductions performed. The cache memory 106 is adapted here so that this operation is implemented by an "outer-product" type algorithm.

[0071] Reduction operations performed on values ​​in the dense region(s) of the sparse matrix A result in non-zero values ​​of the output vector c that are close to each other. The first memory region 112 of the cache memory 106 is well suited to handle operations performed on this dense region of the sparse matrix A, by implementing reduction operations on the values ​​in this region, since it is efficient in this case to work with entire cache lines. On the other hand, operations performed on values ​​in the sparse region(s) of the sparse matrix A are advantageously handled individually in the second memory region 114, rather than on the scale of entire cache lines.

[0072] In one embodiment, this distribution of the processing of operations according to the location in the dense and sparse regions of the sparse matrix A of the data used, between the first and second memory regions 112, 114 of the memory Cache 106 can be adapted according to the results of the operations performed. Thus, when the density of non-zero values ​​in a portion of the sparse region of the sparse matrix A exceeds a certain threshold value that can be arbitrarily chosen (for example, when there are at least 3 non-zero values ​​in a cache row of 8 values), the results of operations performed on the values ​​in said portion of the sparse region of the sparse matrix A can be stored in entire cache rows in the first memory region 112 and not individually in the second memory region 114. For example, if a word row in the first memory region 112 is sparsely filled at the time of an eviction, it is possible to move the writing of the results of calculations initially planned in the first memory region 112 to the FIFO memory region 117 to be subsequently evicted by the implementation of RMW-type atomic operations.This avoids bringing a line from main memory 104 into the first memory region 112 to write a small number of non-zero values.

[0073] Each line of values ​​in the first memory region 112 may also include a field, for example called "tag", corresponding to the address of the corresponding line of values ​​in the main memory 104. Such a field can make it possible to determine whether the line of values ​​is already present in the first memory region 112 ("hit" of this line) or whether it is absent from the first memory region 112 ("miss" of this line).

[0074] In the cache memory 106, upon receiving a reduction operation requested by the computing unit 102 and received by the cache memory 106, the value to be updated can be sought in the first memory region 112. If this value is present in the first memory region 112 ("hit" of this value), the value can be accumulated in the first memory region 112. If this value is absent from the first memory region 112 ("miss" of this value), the reduction to be performed is sent to the first or second memory region 112, 114, depending on the location in the sparse matrix A of the data which enabled obtaining this intermediate result.

[0075] According to a particular embodiment, in the first memory region 112, each value line, or cache line, may include a field, for example called "State", indicating the state of the value line, this state being able to correspond to one of the following four possible states:

[0076] - NOV: Invalid state, indicating a free cache line;

[0077] - VNU: Valid but not updated state, indicating that the values ​​in the line of caches are not consistent with those present in main memory 104;

[0078] - VIP: Valid status but update in progress, indicating that the values ​​of the line cache values ​​are not consistent with those of main memory 104, but the update of this cache line is in progress;

[0079] - VUP: Valid and up-to-date state, indicating that the cache line values ​​are consistent with those of main memory 104 (meaning that during a write (eviction), the cache line could overwrite the line in main memory 104 without data loss).

[0080] According to an exemplary embodiment, in an initial state, all lines in the first memory region 112 are in the NOV state. As soon as a line is allocated, following a requested reduction, the values ​​in that line are set to zero, and then the reduction value is stored in the correct location within that line. At this stage, the line's contents contain this new update but are not consistent with the contents of main memory 104. The line is then in the VNU state. The cache memory 106 can be configured to then search for the corresponding values ​​of the line in main memory 104 to make the line consistent. When the read request for main memory 104 is sent, the line transitions from the VNU state to the VIP state.When the read data is sent from main memory 104 to the first memory region 112, the contents of the line in the first memory region 112 are updated by accumulating the line in the first memory region 112 with the corresponding line in main memory 104, and the line passes into the VUP state.

[0081] When a line is in the VUP state, it can be evicted, and eviction can be performed by writing the line in question to main memory 104. Such eviction is not blocking for cache memory 106 because as soon as the write operation on the line to be evicted is sent to main memory 104, the line in question in the first memory region 112 can be reused for another address. Furthermore, since eviction can be performed quickly, the arithmetic unit 102 is not blocked while waiting for the line to be freed in cache memory 106. A relevant management practice for the state of the value lines in the first memory region 112 could be to transition them to the VUP state as soon as possible.

[0082] Figure 3 shows examples of reduction operations performed on value lines in the first memory region 112. These reduction operations are implemented sequentially in time, from configuration a) to configuration i). These examples illustrate the state transition possibilities of the value lines depending on the behavior of the first memory region 112 ("hit", "miss", and "miss and eviction"). For each of these configurations, the reduction operation performed is indicated in Figure 3 as "RED address value".

[0083] Configuration a) represents the initial state of the first memory region 112, which, in this example, forms a cache with 2 sets (with a value of 0 or 1 in this example), 2 paths (i.e., 2 lines of values ​​per set, for a total of 4 lines of values), each line of values ​​having a size of 2 words. Each line of values ​​has a "tag" field that allows the address of the corresponding line in main memory 104 to be determined, and thus whether the requested line of values ​​is already present in the first memory region 112. In addition, each line of values ​​has a "State" field indicating the state of the line (corresponding to one of the four states "NOV", "VNU", "VIP", and "VUP" described previously). For each of the configurations b) to i), the reduction operation performed is indicated above the table of values ​​of the cache lines.

[0084] In configuration b), the data affected by the reduction operation performed (at address "0x18" in this example) is present in the first memory region 112 ("hit") and it is not necessary to allocate a new line or perform an eviction. The data is simply combined with that present in the relevant line (the value "2" changes to the value "7") and the "VUP" state of this line is not modified.

[0085] In configuration c), the data affected by the reduction operation is present in the first memory region 112 ("hit") and again, it is not necessary to allocate a new line or perform an eviction. The data is simply combined with that present in the line concerned (the value "3" changes to the value "6") and the "VIP" state of this line is not modified.

[0086] In configuration d), a "miss" occurs because no "tag" field value corresponds to the address of the reduction operation. The only row in the "NOV" state is allocated for this operation, and the value of the reduction operation is inserted there. The state of the modified value row changes from "NOV" to "VNU". This case does not block subsequent operations.

[0087] Configuration e) is similar to configuration d) (when a "miss" occurs), except that no line in the first memory region 112 is free, i.e., in the "NOV" state. One of the lines in the first memory region 112 is evicted. Since one of them is in the "VUP" state, it is chosen first for this eviction, and this line is written to main memory 104 (written in the form "WRITE address value" in [Fig. 3]) before the write operation in the first memory region 112. This operation does not block subsequent operations. The line written to the first memory region 112 is set to the "VNU" state.

[0088] Configuration f) is similar to configuration e), but this time, a row in the VNU state containing few elements is chosen. Eviction is performed by implementing RMW-type atomic operations on individual elements. of the line and not on the entire line. This operation is not blocking because it only occurs when the FIFO 117 memory region handling RMW type operations is not full.

[0089] Configuration g) corresponds to a "miss" occurrence. In the absence of a line in the "VUP" state, one of the lines in the "VIP" state is chosen for eviction. In this configuration, a read operation from main memory for the line targeted for eviction is in progress. Thus, before writing to the first memory region 112, the cache memory 106 is configured, for example, to wait for the response to this read operation, then perform the shrink operation, and finally the write operation to main memory 104. This operation corresponds to a blocking case because the cache memory 106 must wait for the eviction to complete before performing its write operation.

[0090] In configuration h), the data affected by the reduction operation is present in the first memory region 112 ("hit") and it is not necessary to allocate a new line or perform an eviction. The data is simply accumulated with that present in the relevant line and the "VNU" state of that line is not modified.

[0091] Configuration i) corresponds to a "miss" occurrence. In the absence of a line in the "VUP" or "VIP" state, one of the lines in the "VNU" state is chosen for eviction. In this configuration, a read operation from main memory for the line targeted for eviction is initiated. The shrinking operation and the write operation to main memory 104 are then performed. This operation corresponds to a blocking case because the cache memory 106 must initiate and wait for the eviction to complete before performing its write operation.

[0092] The structure of the second memory region 114 is similar to that of an array of values, for example, 8 bytes each, which can be fully associative, that is, each value of which is individually accessible. A Content-Addressable Memory (CAM) is one possible way to perform a search in such a memory structure. Each part of the second memory region 114 intended to store a value may include a status bit called a "dirty bit" to indicate whether that part is free or not, and may have the ability to perform "hit," "miss," and "miss and eviction" operations like the first memory region 112. To communicate with the main memory 104, the second memory region 114 may only perform RMW atomic operations on a value and its address.During an eviction, the value and address are sent to the FIFO 117 memory region, which initiates the atomic RMW operations as soon as possible.

[0093] Figure 4 shows examples of reduction operations performed in the second memory region 114. These reduction operations are implemented sequentially in time, from configuration a) to configuration j). These examples illustrate the possible state transitions of the different values ​​depending on the behavior of the second memory region 114 ("hit", "miss", and "miss and eviction"). For each of these configurations, the reduction operation performed is indicated in Figure 3 in the form "RED address value".

[0094] Configuration a) represents the initial state of the second memory region 114, which, in this example, forms a 4-word cache. A key / address field ("key" in [Fig. 4]) is associated with each word. Furthermore, in this example, the second memory region 114 is associated with the FIFO memory region 117, which has a size of 1 word and is represented below the second memory region 114. A storage capacity threshold called aregion4_full, beyond which the second memory region 114 is considered too full to perform a reduction operation, is chosen, for example, to be equal to 3.

[0095] Configuration b) corresponds to a reduction operation where a "hit" occurs, the received value being accumulated in the second memory region 114 without soliciting the FIFO memory region 117.

[0096] Configuration c) corresponds to a reduction operation where a "miss" occurs when the second memory region 114 is not considered too full (quantity of stored values ​​not exceeding the value of the storage capacity threshold), the received value then being inserted into a free part of the second memory region 114.

[0097] Configuration d) corresponds to a reduction operation where a "miss" occurs and the second memory region 114 becomes too full (number of stored values ​​> "regioniKpieine"). An early eviction is therefore performed to keep the fill level of the second memory region 114 below the threshold value of regionn4_pieine. The FIFO memory region 117 receives the value to be evicted.

[0098] Configuration e) corresponds to a reduction operation where a "miss" occurs. The received value is written to the second memory region 114 without performing an eviction because the FIFO memory region 117 is full.

[0099] Configurations f) and g) illustrate blocking cases where the second memory region 114 must wait (corresponding to the indication "Miss Stall" on [Fig.4]) for the FIFO memory region 117 to be emptied, for example, of at least one value (operation symbolized by the indication "RMW MEM" on [Fig.4]) in order to perform the new reduction that has just arrived.

[0100] Finally, configurations h), i) and j) illustrate situations where "hits" occur, allowing the restoration of the quantity of values ​​stored in the second memory region 114 below the fill threshold aregioni i4_pieine via the implementation of eviction of values ​​stored in the second memory region 114 and the transfer of the value stored in the FIFO memory region 117 to main memory 104.

[0101] The example described in relation to [Fig. 4] illustrates the advantage of anticipating the evictions to be performed from the second memory region 114 when it fills up in order to avoid the occurrence of blocking cases by using the storage capacity threshold C^regional 14_full*

[0102] Another aspect of device 100 concerns the management of congestion of accesses to main memory 104. Indeed, during the operation of device 100, accesses to main memory 104 are not distributed uniformly over time and there are times when main memory 104 is little used and others when the number of accesses to main memory 104 is very high.

[0103] To manage congestion in the main memory 104, the device 100, according to a second embodiment and represented for example in [Fig. 5], may include a cache memory 106 with an interface block 130 configured to determine the size (for example, in bytes) of each exchange to and from the main memory 104. Furthermore, to determine the usage state of the main memory 104, the interface block 130 is coupled to a circuit 132 implementing a leaky-bucket algorithm. At each unit of time, for example, at each clock cycle, the circuit 132 can receive a token corresponding to the actual available memory bandwidth between the cache memory 106 and the main memory 104.For example, if the memory interface has a bandwidth of 100 Gbit / sec and the clock period is 1 ns, then, at each clock cycle, the 132 circuit can receive a token that corresponds to 100 bits.

[0104] Each time cache memory 106 sends or receives data to main memory 104, from or to any memory region 112, 114, or 116, the number of bits corresponding to that memory access can be counted by circuit 132. Circuit 132 can also have a maximum fill level, and the number of tokens is capped at this value. Based on the state of circuit 132, the controllers of each of the memory regions 112, 114, and 116 can determine if there is available memory bandwidth. Thus, memory regions 112 and 114 can be configured to anticipate their accesses and send them at the optimal time, that is, when main memory 104 is underutilized, thereby avoiding congestion in main memory 104.

[0105] Other embodiments of the interface block 130 and / or the circuit 132 allowing to limit congestion of accesses to / from the main memory 104 are possible.

[0106] According to a particular embodiment, the cache memory 106 can be configured to define, during an update of a row in the first memory region 112, a time at which a read request from main memory 104 is sent. Indeed, in the first memory region 112, the transition of a row from the "VNU" state to the "VIP" state implies sending a read request to main memory 104. One contribution to optimizing access to main memory 104 can consist of choosing when to perform this read. It can be initiated as soon as the cache row transitions from the "NOV" state to the "VNU" state, or later when the row needs to be evicted from the first memory region 112.By choosing the timing of read operations to transition the lines of the first memory region 112 from the "VNU" to the "VIP" state, based on the state of circuit 132, it is possible to perform these reads at the optimal time, i.e., when the main memory 104 is not heavily used. In this configuration, the first memory region 112 is therefore configured to synchronize read request transmissions to the main memory 104 based on a value of the variable representing the access bandwidth to the main memory 104. Thus, the cache memory 106 can limit simultaneous memory accesses and reduce the average access latency to the main memory 104.

[0107] As an alternative or complement to the above configuration, it is possible that the second memory region 114 and the interface block 130 are configured so as to perform accesses to the main memory 104 depending on the state of the circuit 132. For example, when the second memory region 114 is not very full, for example with a number of stored values ​​less than a threshold aregion4, the interface block 130 and the second memory region 114 can be configured so that they do not launch a request to access the main memory 104.When the second memory region 114 starts to get a little fuller, for example with a number of stored values ​​between the threshold aregionii4 and the storage capacity threshold aregionii4_pieine described earlier, the second memory region 114 and the interface block 130 can be configured to implement evictions of data stored in the second memory region 114, provided that the main memory 104 is not too heavily used, i.e., depending on the state of the circuit 132. The evictions can be triggered by the interface block 130. If the second memory region 114 is almost full, for example with a number of stored values ​​exceeding the storage capacity threshold aregionii4_Pieine, the second region. Memory 114 performs data evictions to main memory 104, unless the FIFO memory region 117 is already full, as previously described. This progressive threshold system, which also takes into account the load on main memory 104 via the state of circuit 132, prevents congestion of main memory 104 that would otherwise occur due to the second memory region 114.

[0108] Updates to the data stored in the second memory region 114 are performed on memory addresses that are spatially isolated from each other. Therefore, it is not efficient to retrieve an entire cache line from main memory 104 in this second memory region 114 to update a single word. Thus, in a particular configuration of the device 100, these updates can be offloaded to a cache controller closer to main memory 104, for example to an L2 or L3 cache when cache 106 forms an L1 cache, or alternatively, offloaded to main memory 104.In other words, the second memory region 114 can be configured such that if an operation performed by the arithmetic circuit 108 involves evicting data stored in the second memory region 114, that operation is performed in main memory 104 or in another cache memory interposed between cache memory 106 and main memory 104. This configuration avoids having to bring a cache line from main memory 104 to cache memory 106 just to modify a single value, for example, a single word. This configuration reduces data traffic between main memory 104 and cache memory 106, avoids the need for sparse cache lines, and also allows the system to take advantage of the access granularity of main memory 104, which is often smaller than that of a cache line.It is possible, in particular, to find a balance in such a relocation of data update operations so as not to saturate the computing power of the memory element in which the updates are relocated.

[0109] Fig. 6 schematically represents device 100 according to a third embodiment.

[0110] For the first memory region 112 to be efficient, it is preferable that the region of the output vector c stored in this first memory region 112 not exceed the stripe height of the sparse matrix A corresponding to the dense region of this matrix (considering a stripe height such as that shown in [Fig. 2]). If the region of the output vector c stored in the first memory region 112 exceeds this size, during the traversal of a column of the sparse matrix A, all the data in the first memory region 112 could be evicted, which would not allow for a reduction in accesses to main memory 104. However, the size, or memory capacity, of the first memory region 112 is limited. Therefore, if the stripe width of the matrix corresponding to the dense region is too large, This constraint cannot be met. Device 100, according to the third embodiment, proposes defining the bandwidth, and therefore the boundaries between the dense and sparse regions of the sparse matrix, as a function of the size of the first memory region 112, in order to meet this constraint (this configuration is also feasible for the embodiments described previously). It is therefore possible that some parts of the dense region(s) of the sparse matrix A may remain unaccounted for by the first memory region 112.

[0111] This problem can be at least partially solved by equipping the cache memory 106 with a buffer block 134 called a "coalescing buffer," or CB, which is a buffer whose size is equal to that of a cache line and whose function is to handle reductions that cannot be processed by the first memory region 112. If consecutive reductions fall within the same line of values, the buffer block 134 merges them. Then, when a reduction arrives whose address is not that of the line of values ​​stored in the buffer block 134, the line of values ​​stored in the buffer block 134 is transferred either to the second memory region 114 if it contains few reductions, or to a fourth memory region 136 otherwise.The operation, in terms of associativity, of this fourth memory region 136 is similar to that of the first memory region 112 (although the number of paths may differ), its size, or memory capacity, being however smaller than that of the first memory region 112. This fourth memory region 136 is configured to process data from dense regions that are not in the main dense band but between this main dense band and the sparse regions of the sparse matrix A. The management of this fourth memory region 136, which has a smaller memory capacity than that of the first memory region 112, requires the implementation of accesses to main memory 104 to update the rows of values ​​and then discard them.

[0112] The device 100 according to the third embodiment has the advantage of protecting the data stored in the first memory region 112. The first memory region 112 allows the elements of the densest region of the matrix to always be cached, without them being disturbed by other accesses. In addition, the fourth memory region 136 allows the local spatial location to be captured for dense regions that do not fit in the first memory region 112, while protecting it. Since the fourth memory region 136 is small, its data will be overwritten when moving from one column to the next.

[0113] Figure 7 schematically represents the sparse matrix 118 used in calculations implemented by the device 100 according to the third embodiment. As in the first and second embodiments, the data 120 of the dense region data from matrix 118 are intended to be processed in the first memory region 112 of cache memory 106, and data 124 from sparse regions of matrix 118 are intended to be processed in the second memory region 114 of cache memory 106. In addition, reference 138 designates data located in dense regions of matrix 118 situated between the dense data region 120 and the sparse regions of matrix 118, as well as dense data situated within the sparse regions of matrix 118.

[0114] In the different embodiments, the size, or memory capacity, of the different memory regions of the cache memory 106 may depend in particular on the sparse matrix A from which the calculations are carried out.

[0115] According to a first configuration, the sizes of these different regions can be defined statically before launching the matrix calculations, and for example calculated by software by analyzing the sparse matrix A.

[0116] According to a second configuration, the sizes of these different regions can be defined statically and calculated in hardware following previous calculations performed on the same sparse matrix. For example, during a first pass through the sparse matrix A, the cache memory 106 can extract statistical parameters from the sparse matrix A and then, before launching the next pass through the sparse matrix A, the sizes of the different memory regions of the cache memory 106 can be redefined according to the previously extracted statistical parameters.

[0117] According to a third configuration, the sizes of the different memory regions of the cache 106 can be defined dynamically and calculated in hardware. For example, the cache 106 can analyze, during a matrix calculation, the data exchanges carried out by each of the memory regions, and adjust the size of the memory regions on the fly in order to balance the traffic level of the different memory regions of the cache 106.

[0118] The device 100 may include a single-core or multi-core processing unit 102. In the case of a multi-core system, the device 100 may include several cache memories 106 operating in parallel with each other and coupled to the same main memory 104.

[0119] Thus, in all the embodiments and examples described above, the cache memory 106 forms a processor-close data cache (generally called the "L1 data cache") which, thanks to the arithmetic circuit 108, allows reductions to be made within the cache memory 106 itself and is well suited to the access sequence typical of the execution of SpMV operations on a sparse matrix with a dense region arranged diagonally across the matrix. Such a configuration of the cache memory 106 allows the arithmetic unit 102 not to be blocked in a state of waiting for data from main memory 104 since the reduction operations are implemented within the cache memory 106 itself. In addition, the device 100 thus proposed can minimize "miss" type results when implementing reduction operations, and also minimize accesses to main memory 104.

[0120] In the various modes and embodiments described above, the cache memory 106 comprises at least one region optimized for storing dense data and at least one other region optimized for storing sparse data. Furthermore, the cache memory 106 can exploit the structure of a sparse matrix to send data to the memory region appropriate to the data region. In addition, the cache memory 106 can be capable of moving data from the sparse data memory region to the dense data memory region if the density increases, and vice versa. Furthermore, the cache memory 106 can contain several memory regions whose operating rules prevent address duplication between regions. Moreover, the cache memory 106 can anticipate data evictions in order to avoid congestion of the main memory 104.In addition, the cache memory 106 can be configured to offload reduction operations closer to the main memory 104.

[0121] The device 100 allows the same calculation unit to be used for regions of the matrix with different densities, and the cache memory 106 is adapted to the type of data processed.

[0122] In addition, the proposed cache memory 106 can enable an acceleration of sparse matrix processing, a reduction in the number of accesses to the main memory 104, as well as better use of the memory system by avoiding bursts of accesses to the main memory 104.

[0123] In the described embodiments, only the cache memory 106 is interposed between the processing unit 102 and the main memory 104. As an alternative to the various embodiments previously described, the device 100 may include one or more other cache memories (L2, L3, etc.) interposed for example between the cache memory 106 and the main memory 104.

[0124] In one particular embodiment, the cache memory 106 can form a data cache for a generic processor corresponding to the processing unit 102, thereby accelerating the matrix calculation algorithms implemented by the processor. According to another particular embodiment, the cache memory 106 can be an element integrated into an accelerator dedicated to calculating SpMV and / or SpMSpM operations using an outer-product type algorithm. In the case of implementing SpMSpM type operations, the device 100 can accelerate a Gustavson-type algorithm implemented for such operations.

[0125] Device 100 may correspond in particular to a high-performance computing device, or HPC.

[0126] Various embodiments and variations have been described. A person skilled in the art will understand that certain features of these various embodiments and variations could be combined, and other variations will become apparent to a person skilled in the art.

[0127] Finally, the practical implementation of the embodiments and variants described is within the reach of a person skilled in the art, based on the functional indications given above.

Claims

Demands

1. A computing device (100), comprising at least: - a calculation unit (102); - a main memory (104); - a cache memory (106) configured to exchange data with the processing unit (102) and with the main memory (104), and comprising a processing circuit (108) configured to perform reduction operations between partial products from values ​​of at least one sparse matrix (118) and at least one input vector, and at least one output vector; in which the cache memory (106) includes at least one first N-way associative memory region (112) configured to store, with a first granularity of TD words, results of reduction operations performed by the computing circuit (108) from partial products from values ​​of at least one dense region (120) of the sparse matrix (118), and at least one second fully associative or M-way associative memory region (114) configured to store, with a second granularity of Ts words, results of reduction operations performed by the computing circuit (108) from partial products from values ​​of at least one sparse region (124) of the sparse matrix (118), with M, N, TD and Ts corresponding to integers such that M > N, TD > Ts, and also such that M > N if TD = Ts and such that TD > Ts if M = N.

2. A computing device (100) according to claim 1, wherein the cache memory (106) has an interface (110) coupled to the computing unit (102) and configured to receive reduction operations required by the computing unit (102) and intended to be implemented by the computing circuit (108), and to send corresponding data into the first memory region (112) when the partial products of the reduction operations are from values ​​in the dense region (120) of the sparse matrix (118) or into the second memory region (114) when the partial products of the reduction operations are from values ​​in the sparse region (124) of the sparse matrix (118).

3. Computing device (100) according to any one of the preceding claims, wherein the main memory (104) and the cache memory (106) are configured such that exchanges between the main memory (104) and the first memory region (112) correspond to read and write operations, and / or wherein exchanges between the main memory (104) and the second memory region (114) correspond to RMW-type atomic operations.

4. Computer computing device (100) according to any one of the preceding claims, wherein the cache memory (106) further comprises at least one third set-associative memory region (116) configured to store data sent from the main memory (104).

5. Computer computing device (100) according to any one of the preceding claims, wherein the cache memory (106) further comprises at least one FIFO memory region (117) configured to temporarily store data sent from the second memory region (114) to the main memory (104).

6. A computing device (100) according to any one of the preceding claims, wherein the second memory region (114) is configured such that if the implementation of a reduction operation by the computing circuit (108) involves the eviction of data stored in the second memory region (114), said reduction operation is implemented in the main memory (104) or in another cache memory interposed between the cache memory (106) and the main memory (104).

7. A computing device (100) according to any one of the preceding claims, wherein the cache memory (106) is configured to perform, upon receiving a reduction operation requested by the computing unit (102) and whose result involves a modification of a result value: - searching for the presence of the result value in the first memory region (102); - updating the result value in the first memory region (112) if this value is present in the first memory region (112), or sending the reduction operation requested by the computing unit (102) in the first memory region (112) or the second memory region (114) if this value is absent from the first memory region (112).

8. A computing device (100) according to any one of the preceding claims, wherein the first memory region (112) is configured such that each row of values ​​stored in the first memory region (112) has at least one address field, one row status field and several value fields, and / or wherein the second memory region (114) is configured such that each part of the second memory region (114) intended to store a value has at least one bit representing the state of said part.

9. A computing device (100) according to any one of the preceding claims, wherein the first memory region (112) is configured to implement, during reduction operations performed by the computing circuit (108) from values ​​in the dense region (120) of the sparse matrix (118): - allocation of a row of null values ​​in the first memory region (112) when accessing an address not present in the cache memory (106); - writing results of said reduction operations into said row of the first memory region (112); - when said row of the first memory region (112) is selected to be evicted, reading values ​​stored in the main memory (104) and combining, in said row of the first memory region (112), the values ​​read from the main memory (104) with those written into said row of the first memory region (112);- removal of said line from the first memory region (112), including writing the values ​​of said line from the first memory region (112) into main memory (104).;

10. A computing device (100) according to any one of the preceding claims, wherein the cache memory (106) is configured such that when the density of non-zero values if part of the sparse region (124) of the sparse matrix (118) is greater than a first threshold value, results of reduction operations implemented from the values ​​of said part of the sparse region (124) of the sparse matrix (118) are stored in the first memory region (112).

11. A computing device (100) according to any one of the preceding claims, wherein the second memory region (114) is configured to implement the eviction of at least one of the values ​​stored in the second memory region (114) to main memory (104) when the number of values ​​stored in the second memory region (114) exceeds a predefined storage capacity threshold.

12. A computing device (100) according to any one of the preceding claims, wherein the cache memory (106) further comprises an interface block (130) configured to determine the size of each of the exchanges to and from the main memory (104) and a circuit (132) for implementing a leaky bucket type algorithm delivering at least one variable representative of an access bandwidth to the main memory (104).

13. A computing device (100) according to claim 12, wherein the first memory region (112) is configured to synchronize read request sends to main memory (104) based on a value of the variable representing the access bandwidth to main memory (104), and / or wherein the second memory region (114) and the interface block (130) are configured to implement evictions of values ​​stored in the second memory region (114) to main memory (104) based on a number of values ​​stored in the second memory region (114) and a value of the variable representing the access bandwidth to main memory (104).

14. A computing device (100) according to any one of the preceding claims, wherein the cache memory (106) further comprises a fourth multi-channel associative memory region (136) smaller in size than the first memory region (112), and a buffer memory block (134) configured to temporarily store result values ​​of reduction operations performed by the computing circuit (108) from products

15. partials from values ​​of at least a second dense region (138) of the sparse matrix (118) localized and to transfer said values ​​into the second memory region (114) or into the fourth memory region (136). Computer computing device (100) according to any one of the preceding claims, wherein the sizes of the memory regions (112, 114, 116, 136) of the cache memory (106) are defined as a function of characteristics of the sparse matrix (118).

Images