Computer computing device with memory optimized for matrix calculations

Abstract

Computer computing device with memory optimized for matrix calculation. Computer computing device (100) comprising a main memory (104) configured to store a sparse matrix in a dense vector format (106, 108, 110) and to store a second vector (112) or a second matrix, a computing unit (102) configured to multiply the sparse matrix by the second vector or by the second matrix, and a loading circuit (114) comprising: - an indexed loading block (116) having a secondary memory (118) and a FIFO query memory (120) for sending values ​​stored in the secondary memory to the computing unit; - an indexed loading engine (122) configured to sequentially generate and store the queries in the FIFO query memory in an order in which the values ​​are intended to be sent to the computing unit;the order in which queries are stored is calculated and stored in the form of a microprogram (124). Figure for the abbreviation: Fig. 1;

Description

Title of the invention: Computer computing device with optimized memory for matrix calculations. Technical field

[0001] This description relates generally to the field of computer computing devices, or computers, used in particular for the implementation of matrix calculations. Previous technique

[0002] Many high-performance computing (HPC) tasks involve the implementation of matrix calculations, such as solving systems of partial differential equations or semantic graphs.

[0003] It is common for the matrices involved in these calculations to be sparse matrices, also called low-density or sparse matrices, containing a significant number of null elements, or zeros, relative to the total number of elements. These calculations may involve, in particular, the execution of algorithms for solving linear problems formulated as equations of the type Ax = y (an operation called "SpMV"), where A is a sparse matrix, x is a dense vector, and y is the result vector of this multiplication, or of the type AX = Y, where X is a dense matrix (an operation called "SpMM") or not (an operation called "SpMSpM"), and Y is the result matrix of this multiplication. It is therefore important to optimize the systems performing calculations with such matrices, especially when dealing with very large sparse matrices containing, for example, millions of null elements.

[0004] Algorithms used to solve equations involving sparse matrices are called "linear solvers". There are two main families of linear solvers: so-called "direct" solvers, which invert the matrix A in order to perform the operation x = A 7. y, and so-called "Krylov" solvers, which are based on an iterative algorithm that modifies the vectors at each iteration until the solution is found.

[0005] Iterative solvers perform several matrix-vector multiplications at each iteration, using the same sparse matrix A each time. For equations of the form Ax = y, only the vector x changes at each iteration. Thus, it is relevant to optimize this operation, which represents the majority of the execution time.

[0006] There are different ways to perform a SpMV or SpMM calculation. Thus, inner-product algorithms traverse matrix A row by row, and outside-product algorithms (Outer-product algorithm) traverse it column by column. In an inner-product algorithm, the reading of the vector x or the matrix X is done in a disordered manner and is dependent on the structure of the matrix A. In an outer-product algorithm, the accumulation performed in the result vector y or the result matrix Y is done in a disordered manner.

[0007] There are certain storage formats for sparse matrices that avoid storing all the zeros of these matrices. These formats correspond, for example, to the CSR (Compressed Sparse Row) or CSC (Compressed Sparse Column) formats. These formats reduce the amount of memory required for their storage. However, in return, they require traversing tables that contain indices, that is, the row and column coordinates of non-zero elements and their values. The CSR representation consists of representing the matrix as three dense vectors. The first vector stores the location of the beginnings of the matrix's rows in the other two vectors. The second vector contains, for each successive row, the indices of the columns containing non-zero elements of the matrix.Finally, the third vector contains the undamaged elements themselves in the same order as the column indices. By traversing these three vectors, it is possible to more easily implement an "inner-product" type algorithm, although at the cost of double indirection on the elements of the vector x.

[0008] For so-called non-trivial matrices (i.e., those containing more than one non-zero element per row), data is reused when reading the vector x in the case of an "inner-product" algorithm, or when accumulating the vector y in the case of an "outer-product" algorithm. However, the hardware's ability to exploit this temporal reuse of data will depend on the time between two readings of the same element of the vector x or between two accumulations of the vector y, relative to the amount of data that the hardware can store locally.

[0009] In the case of an "inner-product" algorithm, accesses to the vector x through a processor's first-level cache are quite inefficient because they make little use of the spatial locality inherent in cache structures, and the first access to a piece of data cannot be easily predicted, causing latency spikes that reduce the overall performance of the computing system. The most powerful general-purpose processors have implemented highly sophisticated, but complex, prediction solutions whose performance is difficult to predict.

[0010] Sparse matrix-vector / dense matrix multiplication accelerators exist. Some of these accelerators use conventional caches, or cache memories, in their operation. Others use a memory access loading circuit. (a circuit called a "streamer" in English) to provide them to the processor in the order of calculations to optimize multiplications. However, the hardware cost of these solutions is very high. Summary of the invention

[0011] There is a need to propose a computer computing device optimized for matrix computation involving a sparse matrix, in particular optimizing memory management with a lower hardware cost than prior art solutions.

[0012] An embodiment overcomes all or part of the drawbacks of existing solutions and proposes a computer computing device comprising at least one main memory configured to store at least one sparse matrix in a dense vector format and to store at least one second vector or matrix, a computing unit configured to multiply the sparse matrix by the second vector or matrix, and a loading circuit comprising:

[0013] - an indexed load block comprising secondary memory configured for temporarily store values ​​of the second vector or second matrix, and a FIFO query memory configured to store queries to send values ​​stored in secondary memory to the processing unit, each of the queries having at least one location field within secondary memory of the value to be sent to the processing unit when said query is executed;

[0014] - an indexed loading engine configured to generate and store sequentially the queries in the FIFO memory of queries in an order in which the values ​​are intended to be sent to the computing unit, and to command, during the generation of each of the queries and in the absence in the secondary memory of the value intended to be sent to the computing unit during the execution of said query, the sending of said value from the main memory to the secondary memory;

[0015] and wherein the order of storage of queries is calculated and stored in main memory in the form of a microprogram comprising a data vector specifying at least, for each of the values ​​intended to be stored in secondary memory, a part of the location field of said value within secondary memory.

[0016] According to a particular embodiment, the microprogram includes a data vector specifying at least, for each of the values ​​intended to be stored in secondary memory, the location field of said value within secondary memory and a field indicating the presence or absence of said value in secondary memory.

[0017] According to a particular embodiment, the loading circuit further comprises a linear loading block including several FIFO vector memories configured to store values ​​of the dense vectors and the data vector of the firmware, and to send a first part of these values ​​to the computing unit and a second part of these values ​​to the indexed loading engine.

[0018] According to a particular embodiment, the main memory is configured to store the sparse matrix in a CSR format, and:

[0019] - a first of the vector FIFO memories is configured to store values ​​of a first of the dense vectors corresponding to row indices of non-nuisance elements of the sparse matrix, and to send these values ​​to the computing unit;

[0020] - a second vector FIFO memory is configured to store values ​​of a second of the dense vectors corresponding to the non-nuisance elements of the sparse matrix, and to send these values ​​to the computing unit;

[0021] - a third of the vector FIFO memories is configured to store values ​​of a third of the dense vectors corresponding to column indices of the non-null elements of the sparse matrix, and to send these values ​​to the indexed loading engine;

[0022] - a fourth of the vector FIFO memories is configured to store values ​​from the firmware data vector and to send these values ​​to the indexed loader.

[0023] According to a particular embodiment, the linear loading block further comprises linear loading engines configured to send to main memory requests to send the values ​​of the dense vectors and the data vector of the firmware to the FIFO vector memories.

[0024] According to a particular embodiment, the firmware data vector further specifies, for each of the values ​​intended to be stored in secondary memory, a distance field representing the number of requests to be executed between the request to send said value and a previous request to send said value.

[0025] According to a particular embodiment, the indexed load block is configured to determine a number of queries stored in the query FIFO memory, and to store in the query FIFO memory a query sent by the indexed load engine when the value of the distance field of the received query is greater than the number of queries stored in the query FIFO memory.

[0026] According to a particular embodiment, the FIFO query memory is configured to store each query with a query execution confirmation field such as:

[0027] - a query generated for a value present in secondary memory is stored in the FIFO query memory with an initial value in the query execution confirmation field indicating that the query can be executed;

[0028] - a query generated for a value not present in secondary memory is stored in FIFO memory of queries with a second value of the query execution field indicating that the query cannot be executed yet, this second value being replaced by the first value when the value is subsequently received and stored in secondary memory.

[0029] According to a particular embodiment, the firmware is calculated such that the values ​​of the location field in the data vector are determined by applying a data usage-dependent replacement policy.

[0030] According to a particular embodiment, the replacement policy implements an LRU and / or Belady type algorithm.

[0031] According to a particular embodiment, the microprogram is calculated such that, for each of the values ​​of the second vector or the second matrix intended to be sent to the computing unit and already present in secondary memory, the replacement policy is updated by considering the locations of said values ​​in secondary memory.

[0032] According to a particular embodiment, the microprogram is calculated such that, for each of the values ​​of the second vector or the second matrix intended to be sent to the computing unit and absent from secondary memory and when the secondary memory is not full, said value is stored in a free location of the secondary memory.

[0033] According to a particular embodiment, the microprogram is calculated such that, for each of the values ​​of the second vector or the second matrix intended to be sent to the computing unit and absent from secondary memory and when the secondary memory is full, a location in the secondary memory occupied by a value is chosen in accordance with the applied replacement policy and said value is stored in the chosen location in the secondary memory.

[0034] According to a particular embodiment, the indexed load block includes at least one directory configured to store, for each value intended to be stored in secondary memory, the address of said value in main memory, and in which the firmware data vector specifies, for each of the values ​​intended to be stored in secondary memory, an indication of the location of said value within secondary memory.

[0035] According to a particular embodiment, the indexed load block further comprises at least one counter configured to count the data exchanged by the indexed load block. Brief description of the drawings

[0036] These features and advantages, as well as others, will be described in detail in the following description of particular embodiments, given by way of non-limiting example, in relation to the accompanying figures, among which:

[0037] - Fig. 1 schematically represents an example of a computing device computer science according to a particular embodiment;

[0038] - Figure 2 represents an example of a logic diagram for the operation of a device computer calculation according to a particular embodiment;

[0039] - Figure 3 represents an example of a logic diagram implemented for a calculation of microprogram used in a computer computing device according to a particular embodiment;

[0040] - [Fig. 4] schematically represents an example of a computing device computer science according to a variant of the particular embodiment. Description of the implementation methods

[0041] The same elements have been designated by the same reference numerals in the different figures. In particular, the structural and / or functional elements common to the different embodiments may have the same reference numerals and may have identical structural, dimensional and material properties.

[0042] For the sake of clarity, only the steps and elements necessary for understanding the described embodiments have been shown and are detailed. In particular, various components (processing unit, main memory, secondary memory, load blocks, load motors, etc.) of the computing device are not detailed. A person skilled in the art will be able to implement these components in detail from the functional description given here.

[0043] Unless otherwise specified, when referring to two elements connected together, this means directly connected without intermediate elements other than conductors, and when referring to two elements connected (in English "coupled") together, this means that these two elements can be connected or linked through one or more other elements.

[0044] In the following description, when reference is made to absolute position qualifiers, such as the terms "front", "back", "top", "bottom", "left", "right", etc., or relative position qualifiers, such as the terms "above", "below", "superior", "inferior", etc., or to orientation qualifiers, such as the terms "horizontal", "vertical", etc., reference is made, unless otherwise specified, to the orientation of the figures in a normal position of use.

[0045] Unless otherwise specified, the expressions "approximately", "roughly", and "in the order of" mean within 10%, preferably within 5%.

[0046] Throughout the document, the term "vector" is used to refer to a row matrix or a column matrix.

[0047] An example of a computing device 100 according to a particular embodiment is described below with reference to [Fig. 1]. In the example described, the device 100 is configured to implement algorithms for multiplying a sparse matrix A with a second vector b (SpMV operation) or a second matrix B (SpMM operation).

[0048] The device 100 includes a computing unit 102 which corresponds for example to a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) or any other electronic / computer circuit adapted to the implementation of the calculations carried out by the device 100.

[0049] The device 100 also includes a main memory, or external memory, 104, for example of the RAM (Random Access Memory) type and typically a DRAM (Dynamic Random Access Memory). The main memory 104 is configured to store, in particular, at least one sparse matrix in a dense vector format.

[0050] In the example in [Fig. 1], the main memory 104 stores a sparse matrix in CSR format, that is, as data stored in a first dense vector 106 comprising the row indices of the non-nuisance elements of the sparse matrix, in a second dense vector 108 comprising the non-nuisance elements of the sparse matrix, and in a third dense vector 110 comprising the column indices of the non-nuisance elements of the sparse matrix. In CSR format, the row indices point, for each row of the matrix, to the index of the beginning of that row in the vectors of column indices and non-nuisance elements.

[0051] Alternatively, the main memory 104 may store the sparse matrix in a dense vector format other than the CSR format. The encoding format of the sparse matrix being calculated by the device 100 may, for example, be chosen according to the internal structure of the matrix and / or the characteristics of the processing unit 102.

[0052] For example, it is possible that main memory 104 stores the sparse matrix in a COO (Coordinate List) format in which each non-zero element of the sparse matrix, stored in a dense vector, is associated with its row / column coordinates in the matrix, these coordinates being stored in two other dense vectors. These coordinates can be sorted to allow a more efficient implementation of multiplying the sparse matrix by a vector or another matrix.

[0053] According to another example, it is possible for the main memory 104 to store the sparse matrix in a BCSR (Block Compressed Sparse Row) or BSR (Block Sparse Row) format, in which fixed-size dense blocks (e.g., 2 rows x 2 columns) are stored per index. This format takes advantage of the fact that matrices are often locally dense and globally sparse. It thus reduces the number of indices stored in memory at the cost of inserting zeros into the dense blocks. Heuristics can be used, for example, to choose the ideal size of these dense blocks.

[0054] According to another example, the sparse matrix can be stored in CSC format for the realization, for example, of a multiplication of the type A T ,x=y where A T is the transpose matrix of A.

[0055] Alternatively, other sparse matrix storage formats in the form of dense vectors are possible. For example, any form of deterministic indirection pointing to dense elements or subblocks of a sparse matrix can be used. In particular, the storage format of the sparse matrix in main memory 104 can be chosen to suit the way the sparse matrix is ​​traversed by the multiplication algorithm implemented in the arithmetic unit 102. For example, storing the sparse matrix in CSR format may be advantageous in the case of matrix multiplication involving an "inner-product" type algorithm, whereas storing the sparse matrix in CSC format may be advantageous in the case of matrix multiplication involving an "outer-product" type algorithm.

[0056] The main memory 104 is also configured to store at least a second vector or a second matrix intended to be multiplied with the sparse matrix. In the example of [Fig. 1], a second vector 112 is stored in the main memory 104.

[0057] The device 100 also includes a loading circuit 114, or "streamer", optimized for loading data into memory for implementing a multiplication of the sparse matrix by the second vector 112 or by the second matrix. The loading circuit 114 comprises at least:

[0058] - an indexed load block 116 comprising a secondary memory 118 configured to temporarily store values ​​of the second vector 112 or the second matrix, and a FIFO (“First In First Out”) query memory 120 configured to store requests to send values ​​stored in secondary memory 118 to the computing unit 102;

[0059] - an indexed loader 122 configured to generate and store sequentially the queries in the FIFO memory of 120 queries in an order in which the values ​​are intended to be sent to the computing unit 102, and to command, when generating each of the queries and in the absence in secondary memory 118 of the value intended to be sent to the computing unit 102 when executing the query, the sending of said value from main memory 104 to secondary memory 118.

[0060] Each of the queries intended to be stored in the query FIFO memory 120 includes at least one location field, representing the location within secondary memory 118, of the value intended to be sent to the processing unit 102 during query execution. Furthermore, in the described embodiment, the query FIFO memory 120 is configured to store each query with a query execution confirmation field such as:

[0061] - a query generated for a value present in secondary memory 118 is stored in FIFO memory of 120 queries with a first value of the query execution confirmation field indicating that the query can be executed;

[0062] - a query generated for a value not found in secondary memory 118 is stored in FIFO query memory 120 with a second query execution field value indicating that the query cannot be executed yet, this second value being replaced by the first value when the value is subsequently received and stored in secondary memory 118.

[0063] In the example of [Fig.1], the main memory 104 is also configured to store a firmware 124 comprising a data vector specifying at least, for each of the values ​​intended to be stored in the secondary memory 118, the location field of the value within the secondary memory 118 and a field indicating whether or not the value is present in the secondary memory 118. The firmware 124 includes memory command data, the function of which is detailed later, and which is decoded by the indexed loader 122.

[0064] Alternatively, the microprogram 124 could be stored within one of the dense vectors of the sparse matrix, for example the third vector 110 comprising the column indices, by reserving certain bits of this vector for the storage of this microprogram 124 (for example by using the most significant bits of the column indices of the CSR format).

[0065] In the example of [Fig. 1], the loading circuit 114 further includes a linear loading block 126 comprising several vector FIFO memories configured to store the values ​​of the dense vectors 106, 108, 110 and the firmware data vector 124 stored in the main memory 104. In the example of [Fig. 1], a first vector FIFO memory 128 is configured to receive the row index values ​​stored in the first dense vector 106, a second FIFO memory of vector 130 is configured to receive the values ​​of the non-nuisable elements of the sparse matrix stored in the second dense vector 108, a third FIFO memory of vector 132 is configured to receive the values ​​of the column indices stored in the third dense vector 110, and a fourth FIFO memory of vector 134 is configured to receive the values ​​of the firmware data vector 124.

[0066] In the example of [Fig. 1], the linear loading block 126 further includes linear loading motors 136 configured to send requests to the main memory 104 to send the values ​​of the dense vectors 106, 108, 110 and the data vector of the firmware 124 to the FIFO vector memories 128, 130, 132 and 134. In the configuration shown in [Fig. 1], these requests are sent to a first MSHR 138 (Miss Status Handling Register), or register or tracker, configured to forward these requests to the main memory 104. These linear loading motors 136 correspond, for example, to finite state machines (FSMs) programmed in software within the linear loading block 126.

[0067] Thus, the linear scheduling block 126 is configured to read and load into memory the values ​​of the dense vectors 106, 108, 110 representing the sparse matrix, as well as the microprogram 124 stored in main memory 104. Part of the information stored in the FIFO memories 128, 130, 132, and 134 is sent to the arithmetic unit 102 to execute the outer and inner loops of the multiplication of the sparse matrix by the second vector 112. In the described example, this information corresponds to the values ​​of the row indices and the non-nuisance elements stored in the first and second FIFO memories of vectors 128 and 130. Another part of this information is sent to the indexed loader 122, namely the values ​​of the column indices and the data vector of the microprogram 124 stored in the third and fourth FIFO memories 132 and 134.

[0068] The indexed load engine 122 is configured to command, during the generation of each request and in the absence in secondary memory 118 of the value intended to be sent to the processing unit 102 during the execution of said request, the sending of this value from main memory 104 (stored in the second vector 112) to secondary memory 118. In the example of [Fig. 1], this command is carried out through a second MSHR 140. The second MSHR 140 can keep track of missing elements in secondary memory 118, and therefore of accesses to main memory 104 made to obtain values ​​from the second vector 112. This second MSHR 140 can also group the requests sent to main memory 104 and thus make more efficient use of the memory bus between main memory 104 and the circuit of Loading 114. The indexed loading engine 122 can in particular be configured to calculate the addresses, within the second vector 112, at which the values ​​to be sent to secondary memory 118 are stored.

[0069] Thus, the secondary memory 118 forms a memory in which the values ​​of the second vector 112 or the second matrix are stored. Accesses to the main memory 104 are generated by the indexed loader 122 if the desired value is not present in the secondary memory 118, that is, depending on the indicator of the presence or absence of said value in the secondary memory 118 provided by the firmware 124.

[0070] Once the elements of the second vector 112 or the second matrix are present in secondary memory 118, the sequential execution of the queries stored in the query FIFO memory 120 sends the data to the processing unit 102 in the same order as the execution of these queries, so that the latter can execute the matrix multiplication algorithm between the sparse matrix and the second vector 112 or the second matrix. The send queries are therefore stored sequentially in the query FIFO memory 120 in an order in which the values ​​of the second vector 112 or the second matrix are intended to be sent to the processing unit 102. This order of query execution allows the load circuit 114 to remain synchronized with the processing unit 102 during the execution of the matrix calculation.

[0071] The data from the second vector 112 or the second matrix can be of variable precision. Thus, the number of entries in the secondary memory 118, that is to say the number of data from the second vector 112 or the second matrix that can be stored in the secondary memory 118, can depend on the chosen precision, specified by the choice of the configuration of the loading circuit 114.

[0072] In the described embodiment, the data vector of the firmware 124 further includes, for each of the values ​​intended to be stored in secondary memory 118, a distance field whose value represents the number of queries to be executed between the query sending that value and a previous query sending that value. Furthermore, in the described embodiment, the indexed load block 116 can be configured to determine a remaining number of queries stored in the query FIFO memory 120, and to store in the query FIFO memory 120 a query sent by the indexed load engine 122 when the value of the received distance field is less than the remaining number of queries stored in the query FIFO memory 120.For example, at the entry of the indexed load block 116, request and response counters, for example integrated into the second MSHR 140, can allow a request sent to main memory 104 to be blocked or allowed to pass depending on the . value of the distance field associated with the request. Such a block can be used to prevent premature replacement of data stored in secondary memory 118, and ultimately prevent erroneous data from being sent back to the processing unit 102.

[0073] In the described embodiment, the microprogram 124 can be calculated such that the values ​​of the location field in the data vector of the microprogram 124 are determined by applying a replacement policy in secondary memory 118 that is a function of data usage. The microprogram 124 can thus be used to manage the occurrence of a conflict when replacing data present in secondary memory 118 with other data from the second vector 112 or the second matrix and sent to secondary memory 118. Such a conflict can occur when the location in secondary memory 118 in which data is intended to be stored contains data not yet read by the processing unit 102.Although the replacement policy applied may minimize the occurrence of such a conflict, it may occur, for example, when a large amount of data is intended to be stored in secondary memory 118 and there are few locations in it (for example, in the case of storing large blocks of data and / or in the case of very high precision of the data stored).

[0074] An example of the operation of device 100 is described below in relation to [Fig.2] which schematically represents a logic diagram of the operation of device 100. In this logic diagram, the steps implemented are grouped by column, each corresponding to one of the elements of device 100 (the numerical reference of each of these elements is indicated in the column corresponding to that element).

[0075] During a step 202, the computing unit 102 configures the loading circuit 114 and starts the linear loading motors 136. This configuration step may consist of loading elements into the loading circuit 114, such as the base addresses of the vectors and matrices, the size of the elements of each vector and matrix and the fixed step (or "stride") separating them, the number of elements in each vector and matrix, the number of elements of the dense matrix to be loaded with each request in the case of a SpMM-type operation, the operating mode of the indexed loading block 116 (operation analogous to that of a cache memory or a FIFO memory), start and stop command signals for data loading, etc. The linear loading motors 136 receive the configuration from the computing unit 102 and start upon receiving the start command signal.The linear loading motors 136 therefore start the reading of the different dense vectors 106, 108 and 110 describing the sparse matrix as well as the corresponding microprogram 124 (steps 204, 206, 208 and 210). (respectively to the loading of vectors 106, 108, 110 and 124). For example, for the loading of each of these vectors, the linear load engines 126 group requests to send data from these vectors, for example, destined for the same cache line (typically 512 bits) or the width of the memory bus connecting the linear load block 126 to main memory 104, before sending an aggregated request to the first MSHR 138. For example, during a step 212, the first MSHR 138 can perform an arbitration between the requests received from the linear load engines 136. The first MSHR 138 can then send these requests to main memory 104 (step 214).

[0076] The main memory 104 can then send the requested values ​​to the first MSHR 138 (step 216). When the data requested in the queries are sent by the main memory 104, they are received by the first MSHR (step 218) and then sent to the different vector FIFO memories of the linear loading block 126 (steps 220, 222, 224 and 226 corresponding respectively to storage in the vector FIFO memories 128, 130, 132 and 134).

[0077] In the example described, the values ​​of the column indices and the data vector of firmware 124 are sent to the indexed loader 122, which calculates the addresses of the values ​​in the second vector 112 or the second matrix (step 228) and decodes firmware 124 (step 230). The indexed loader can then generate the queries to be sent to the indexed loader block 116 (step 232).

[0078] For each request received, the indexed load block 116 can check that there is at least one available space in the FIFO request memory 120 and blocks the processing of the request if the FIFO request memory 120 is full.

[0079] For each request received, the indexed load block 116 can then test the value of the field for the presence or absence of said value in secondary memory 118 (step 234).

[0080] If the value is already present in secondary memory 118 (occurrence of a "hit"), the indexed load block 116 can send this request into the query FIFO memory 120 with the query execution confirmation field set to a first value indicating that the query can be executed (step 236) and read the next request sent by the indexed load engine (step 238).

[0081] If the value is absent from secondary memory 118 (a "miss" occurs), the indexed load block 116 tests whether the value of the distance field of this query is null (step 240). If this value is null, the indexed load block 116 sends this query to the FIFO query memory 120 with the query execution confirmation field set to a second value indicating that the query cannot yet be executed (step 242) and sends a query to the second MSHR 140. A request to send a value from the second vector 112 or the second matrix is ​​sent (step 244). If the value of the distance field in this request is not zero, the value of this field is compared to the number of requests present in the FIFO memory of requests 120 (step 246). If the value of this field is greater than the number of requests present in the FIFO memory of requests 120, the indexed load block 116 sends this request to the FIFO memory of requests 120 with the query execution confirmation field set to the second value (step 242) and sends a request to the second MSHR 140 to send a value from the second vector 112 or the second matrix (step 244).Otherwise, this test is repeated until the number of requests present in the FIFO memory of requests 120 is less than the value of the distance field, thus blocking the request as long as the FIFO memory of requests 120 contains more requests than the value of the distance field.

[0082] Following the receipt of several requests, the second MSHR 140 can try to group one or more ongoing memory requests (steps 248, 250) and then forward them to main memory 104 (step 252) if grouping fails.

[0083] Upon receiving the request(s) sent by the second MSHR 140, the main memory 104 can send the requested values ​​to the second MSHR 140 (step 254). Upon receiving these values ​​by the second MSHR (step 256), the received value(s) can be sent to the secondary memory 118 for writing (step 258), and the request execution confirmation field can be updated in the FIFO request memory 120 by passing it to the first value (step 260). The FIFO request memory 120 can then read the next request sent by the indexed loader (step 238).

[0084] In the presence of a query stored with the confirmation field at the first value indicating that the query can be executed, at the head of the query FIFO memory 120, the secondary memory 118 transmits the stored values ​​to the computing unit 102 (step 262), the latter implementing the calculation algorithm of multiplication between the sparse matrix and the second vector 112 or the second matrix from the data received from the vector FIFO memories 128, 130, 132 and 134 and the secondary memory 118 (step 264).

[0085] As previously stated, the firmware 124 specifies at least, for each of the values ​​intended to be stored in secondary memory 118, a field for locating said value within secondary memory 118 and a field for indicating whether or not said value is present in secondary memory 118. This firmware 124 is pre-calculated before implementing the steps previously described in connection with [Fig.2].

[0086] The fact that the microprogram 124 is pre-calculated allows the implementation of a data replacement policy for data stored in secondary memory 118 (when data from the second vector 112 or the second matrix needs to be written into secondary memory 118 but it is full) which would be very expensive to implement in hardware.

[0087] Figure 3 shows an example of a logic diagram implemented for the calculation of microprogram 124. In the described example, microprogram 124 comprises a sequence of triplets (location field, presence / absence field, distance field), each associated with one of the non-nuisance elements of the sparse matrix. The location field is used to locate the data in secondary memory 118. The presence / absence field of the value in secondary memory 118 indicates whether the data is present in secondary memory 118 or whether it needs to be loaded into secondary memory 118 from main memory 104. Finally, the value of the distance field represents the number of requests to be executed between two consecutive requests to send the value associated with the request and stored in secondary memory 118, and is used to prevent overwriting data stored in secondary memory 118 that has not yet been read by the arithmetic unit 102.

[0088] This logic diagram can be implemented by the computing unit 102 or another processor (for example a host processor) or a dedicated accelerator of the device 100.

[0089] A column index of the sparse matrix is ​​first considered (step 300). In step 302, an internal representation of the secondary memory 118 contained in the program implementing this logic diagram is inspected. It is then checked whether the value of the second vector 112 or of the second matrix associated with this column index is present in the secondary memory 118 (step 304). If so, the replacement policy is updated (step 306), for example by placing this data at the end of a list of the least used data in the case of an LRU (Least Recently Used) type replacement policy, in which the least recently used row is replaced first.The data in firmware 124 (a triplet in the described example) associated with this value is then defined such that the location field points to the address in secondary memory 118 where the data is located, the presence field indicates whether the data is present in secondary memory 118, and the distance field is set to 0 (step 308). If the value is not present in secondary memory 118, it is checked whether secondary memory 118 is full (step 310). If it is not, the triplet associated with this value is defined such that the location field indicates a free address in secondary memory 118, and the presence field indicates whether the data is present in secondary memory 118. no indicates the absence of the data in secondary memory 118 and the distance field should be set to 0 (step 312).

[0090] If secondary memory 118 is full, a value stored in secondary memory 118 is chosen to be evicted (step 314). For example, in the case of applying an LRU-type replacement policy, the value at the top of the list of least used entries is chosen as the one to be evicted. Once this entry is chosen, it is checked that the replaced value is not currently in use, to ensure that the processing unit 102 has read this value. To do this, a distance between the current query and the one that last used this value is calculated (step 316), and then this distance is compared to the size of the query FIFO memory 120 (step 318). If this distance is greater than the size of the query FIFO memory 120, it is possible to consider that the value in question has already been read by the processing unit 102 when the chosen value is replaced.In this case, the triplet associated with this value is defined such that the location field indicates the address of the value to be evicted, the presence / absence field indicates the absence of the value in secondary memory 118, and the distance field is set to 0 (step 320). Otherwise, the triplet associated with this data is defined such that the location field indicates the address of the value to be evicted, the presence / absence field indicates the absence of the value in secondary memory 118, and the distance field is set to the calculated distance value (step 322).

[0091] In the example above, the replacement policy applied to define the data in the microprogram vector 124 is of the LRU type. Alternatively, this replacement policy may correspond to an optimal algorithm, or Belady's algorithm, in which the replaced value is the one that will not be used for the longest period of time in the future. Using such a replacement policy results in a better hit rate, that is, a higher frequency of cases where the values ​​being requested are already present in secondary memory 118. However, applying this replacement policy creates the risk of replacing a value that has not yet been read by the processing unit 102, which could cause a conflict and a deadlock, reducing the performance of the device 100.

[0092] According to another variant, it is possible to apply a replacement policy for values ​​stored in secondary memory 118 that combines LRU and Belady-type replacement policies. To this end, it is possible to apply the Belady-type replacement policy only to a limited number N of values ​​stored in secondary memory 118, corresponding to those least recently used. Such a variant limits the risks of conflict described above. The The value of N can be modulated according to the storage capacity of the secondary memory 118, which also depends on the size of the data to be stored.

[0093] Alternatively, other types of replacement policies can be implemented, for example of the pseudo-LRU type.

[0094] As an alternative to the examples described above, the indexed load block 116 may include a directory 142 containing, for example, for each entry in secondary memory 118, the address of that data in main memory 104. This directory 142 makes it possible to consider uses of the device 100 where only part of the replacement policy is pre-calculated, or even to temporarily implement matrix calculation without using the firmware 124, for example during the calculation of this firmware 124. Furthermore, when the indexed load block 116 includes such a directory 142, the data in firmware 124 may not include the field indicating whether or not the value of the query in question is present.Furthermore, in such a variant, the firmware 124 can provide at least partially the relevant way number of secondary memory 118 so as to consult a smaller number of entries to determine the presence or absence of the data concerned.

[0095] Figure 4 schematically represents an example of an embodiment of the device 100 comprising such a directory 142. In this variant, the device 100 comprises all the elements previously described in connection with Figure 1, as well as the directory 142 included in the indexed loading block 116. This directory 142 includes, for each entry of the secondary memory 118, its address in the secondary memory 118.

[0096] In this variant, the microprogram 124 forms a data vector specifying, for each of the values ​​intended to be stored in the secondary memory 118, an indication of the location of said value within the secondary memory 118, this indication of the location corresponding for example to a part of the way number of the cache formed by the directory 142 and the secondary memory 118. This indication of the location of the data is transmitted from the indexed loader 122 to the directory 142 which can then transmit to the FIFO query memory 120 the complete address of the data concerned.

[0097] In such a variant, the secondary memory 118 associated with directory 142 can be viewed as functioning as a set-associative cache. This variant reduces the hardware and energy cost associated with accessing directory 142.

[0098] When the firmware 124 contains, for each of the values ​​intended to be stored in secondary memory 118, only an indication of the location of said value within secondary memory 118, and the firmware 124 does not contain a field representing the presence or absence of the data in The presence or absence of the data in secondary memory 118 can be determined, for example, by partially accessing directory 142. Multiplexers can receive as input the responses from different parts of directory 142, as well as the portion of the location stored in firmware 124, to select the relevant response. The presence or absence of the data in secondary memory 118 can then be determined from the output responses of the multiplexers.

[0099] As an alternative to the example described above, it is possible that the firmware 124 may include the field indicating the presence or absence of the value of the query in question, and possibly the distance field whose value represents the number of queries to be executed between the query sending this value and a previous query sending said value.

[0100] According to one embodiment, the indexed load block 116 may also have a so-called "FIFO" operating mode in which the secondary memory 118 functions as a FIFO memory. In this case, the values ​​of the second vector 112 or the second matrix sent from the main memory 104 to the secondary memory 118 can be stored one after the other in the secondary memory 118, regardless of their address in the secondary memory 118. This mode can also be used while waiting for the firmware 124 to be calculated or when the sparse matrix has an extremely small number of non-zero values ​​per row of the matrix.

[0101] In the embodiment examples described above, it is assumed that the computing unit 102 performs a matrix calculation by implementing an "inner-product" type algorithm. Alternatively, the computing unit 102 can implement other types of algorithms (of the "outer-product" type, Gustavson, etc.) to perform these matrix calculations.

[0102] As an alternative to the embodiment examples described above, the device 100 can perform the steps described above in connection with [Fig.2] on only a part of the data of the sparse matrix, i.e. on a sub-matrix of the sparse matrix.

[0103] Device 100 can advantageously be used to perform a multiplication of a sparse matrix with a dense vector or a dense matrix. Alternatively, device 100 can be used to perform a multiplication of a sparse matrix with a vector or a matrix whose elements are not dense but are separated by a fixed gap in memory. Such a variant can be used, for example, to work on a field of a structure (a vector of complex numbers or a vector of three-dimensional vectors, for example). According to another variant, device 100 can be used to perform a multiplication of a sparse matrix with dense blocks. of a dense matrix, the elements of the dense blocks may or may not be spaced from each other with a fixed gap.

[0104] Alternatively, the indexed load block 116 may include at least one usage counter configured to count the data exchanged by the block 116. In this case, the data vector stored in the firmware 124 may not include the distance field since such counters make it possible to detect usage conflicts in secondary memory 118 and to wait for the release of a value before performing the replacement within secondary memory 118.

[0105] According to another variant, it is possible that a particular value of the distance field in the data vector of the firmware 124 serves to indicate the presence or absence of the data in secondary memory 118. This value can be replaced on the fly in the indexed load engine 122 by this value incremented by one unit.

[0106] According to one embodiment, the device 100 may further comprise an accelerator configured to work upstream or in parallel with the loading circuit 114 and execute the logic diagram previously described in connection with [Fig.3].

[0107] The different variants described above can be combined with each other.

[0108] Device 100 allows optimized management of a cache for the implementation of a multiplication of a sparse matrix by a dense vector or a dense matrix, in configurable precision.

[0109] Device 100 can pre-calculate the replacement policy in secondary memory 118 in the case of sparse matrix multiplication by a dense vector or a dense matrix. This pre-calculation can be partial: on only a part of the matrix (a submatrix) and / or on only a part of secondary memory 118 (for example, to select a subset of channels in secondary memory 118).

[0110] Device 100 can be used to form a fully associative cache without dedicated hardware for detecting conflicts on the same data sent as input to the cache. In this case, everything is pre-calculated and stored in microcode formed by firmware 124. An ideal replacement policy then becomes usable, far more efficient than what can be done in real time with hardware.

[0111] Device 100 improves the performance of multiplying a sparse matrix by a dense vector or a dense matrix by reducing the number of accesses to main memory by the processing unit, thanks to the exploitation of the temporal locality of the data used, which depends on the structure of the sparse matrix. This temporal locality arises from the fact that the numerous multiplications SpMV or SpMM implementations use the same sparse matrix data, and therefore the same memory access scheduling at each multiplication.

[0112] Device 100 allows for the pre-calculation of secondary memory management 118 to reduce hardware costs and improve performance. Device 100 allows for the pre-calculation of the behavior of secondary memory 118, that is, the calculation of the hit and the selection of a location for replacing an entry in secondary memory 118, in the form of a firmware 124.

[0113] Device 100 makes it possible to find a compromise between the hit rate and the occurrence of conflicts during data replacements in secondary memory 118 (i.e. the selection of a line of the cache still in use).

[0114] The device 100 forms an accelerator, or streamer, with extended and variable precision, in which the most critical software routines are optimized. This accelerator minimizes memory accesses, which are costly in terms of performance and energy, by integrating a cache. This accelerator is coupled to the computing core via existing data paths, thus making it compatible with multiple cores. Furthermore, this accelerator is compatible with the requirements of reading data with extended precision and is configurable to the bit level.

[0115] Device 100 can be used to form a fully associative cache at a lower cost than if such a cache were implemented purely in hardware.

[0116] Pre-calculating the indexing vector allows the use of alternative policies that would not be possible to implement in real-time hardware.

[0117] Device 100 can for example be used in the field of scientific computing or artificial intelligence, for example in algebraic solvers, eigenvalue solvers, in the inference and learning of artificial intelligence.

[0118] Various embodiments and variations have been described. A person skilled in the art will understand that certain features of these various embodiments and variations could be combined, and other variations will become apparent to a person skilled in the art.

[0119] Finally, the practical implementation of the embodiments and variants described is within the reach of a person skilled in the art, based on the functional indications given above.

Claims

1. Demands A computing device (100) comprising at least one main memory (104) configured to store at least one sparse matrix in a dense vector format (106, 108, 110) and to store at least one second vector (112) or a second matrix, a computing unit (102) configured to multiply the sparse matrix by the second vector (112) or by the second matrix, and a loading circuit (114) comprising: - an indexed load block (116) comprising a secondary memory (118) configured to temporarily store values ​​of the second vector (112) or the second matrix, and a FIFO query memory (120) configured to store queries to send values ​​stored in the secondary memory (118) to the processing unit (102), each of the queries comprising at least one location field within the secondary memory (118) of the value to be sent to the processing unit (102) during an execution of said query; - an indexed load engine (122) configured to sequentially generate and store queries in the query FIFO memory (120) in an order in which the values ​​are intended to be sent to the processing unit (102), and to command, during the generation of each query and in the absence in secondary memory (118) of the value intended to be sent to the processing unit (102) during the execution of said query, the sending of said value from main memory (104) to secondary memory (118); and in which the order of storage of the queries is calculated and stored in the main memory (104) in the form of a microprogram (124) comprising a data vector specifying at least, for each of the values ​​intended to be stored in the secondary memory (118), a part of the location field of said value within the secondary memory (118).

2. Computer computing device (100) according to claim 1, wherein the firmware (124) includes a data vector specifying at least, for each of the values ​​intended to be stored in the secondary memory (118), the location field of said value within the secondary memory (118) and a field indicating the presence or absence of said value in the secondary memory (118).

3. A computing device (100) according to any one of the preceding claims, wherein the loading circuit (114) further comprises a linear loading block (126) including several FIFO vector memories (128, 130, 132, 134) configured to store values ​​of the dense vectors and the data vector of the firmware (124), and to send a first part of these values ​​to the computing unit (102) and a second part of these values ​​to the indexed loading engine (122).

4. A computing device (100) according to claim 3, wherein the main memory (104) is configured to store the sparse matrix in a CSR format and wherein: - a first of the vector FIFO memories (128) is configured to store values ​​of a first of the dense vectors (106) corresponding to row indices of non-nuisance elements of the sparse matrix, and to send these values ​​to the computing unit (102); - a second of the vector FIFO memories (130) is configured to store values ​​of a second of the dense vectors (108) corresponding to the non-nuisance elements of the sparse matrix, and to send these values ​​to the computing unit (102);- a third of the vector FIFO memories (132) is configured to store values ​​of a third of the dense vectors (110) corresponding to column indices of the non-nuisance elements of the sparse matrix, and to send these values ​​to the indexed loader (122); - a fourth of the vector FIFO memories (134) is configured to store values ​​of the data vector; of the firmware (124) and to send these values ​​to the indexed loader (122).

5. Computer computing device (100) according to claim 3 or 4, wherein the linear load block (126) further comprises linear load motors (136) configured to send to the main memory (104) requests to send the values ​​of the dense vectors (106, 108, 110) and the data vector of the firmware (124) to the FIFO vector memories (128, 130, 132, 134).

6. A computing device (100) according to any one of the preceding claims, wherein the data vector of the firmware (124) further specifies, for each of the values ​​intended to be stored in secondary memory (118), a distance field representing the number of requests to be executed between the request to send said value and a previous request to send said value.

7. A computing device (100) according to claim 6, wherein the indexed load block (116) is configured to determine a number of queries stored in the query FIFO memory (120), and to store in the query FIFO memory (120) a query sent by the indexed load engine (122) when the value of the distance field of the received query is greater than the number of queries stored in the query FIFO memory (120).

8. A computing device (100) according to any one of the preceding claims, wherein the query FIFO memory (120) is configured to store each query with a query execution confirmation field such that: - a query generated for a value present in secondary memory (118) is stored in the query FIFO memory (120) with a first value of the query execution confirmation field indicating that the query may be executed; - a query generated for a value absent from secondary memory (118) is stored in the query FIFO memory (120) with a second value of the query execution field indicating that the query cannot not yet executed, this second value being replaced by the first value when the value is subsequently received and stored in secondary memory (118).

9. A computing device (100) according to any one of the preceding claims, wherein the firmware (124) is computed such that the values ​​of the location field in the data vector are determined by applying a data usage-dependent replacement policy.

10. Computer computing device (100) according to claim 9, wherein the replacement policy implements an LRU and / or Belady type algorithm.

11. Computer computing device (100) according to claim 9 or 10, wherein the firmware (124) is calculated such that, for each of the values ​​of the second vector (112) or of the second matrix intended to be sent to the computing unit (102) and already present in the secondary memory (118), the replacement policy is updated by considering the locations of said values ​​in the secondary memory (118).

12. A computing device (100) according to any one of claims 9 to 11, wherein the firmware (124) is calculated such that, for each of the values ​​of the second vector (112) or the second matrix intended to be sent to the computing unit (102) and absent from secondary memory (118) and when the secondary memory (118) is not full, said value is stored in a free location of the secondary memory (118).

13. A computing device (100) according to any one of claims 9 to 12, wherein the firmware (124) is calculated such that, for each of the values ​​of the second vector (112) or the second matrix intended to be sent to the computing unit (102) and absent from secondary memory (118), and when secondary memory (118) is full, a location in secondary memory (118) occupied by a value is chosen in accordance with the applied replacement policy and said value is stored in the chosen location in secondary memory (118).

14. A computing device (100) according to any one of the preceding claims, wherein the indexed load block (116) comprises at least one configured directory (142). to store, for each value intended to be stored in secondary memory (118), the address of said value in main memory (104), and in which the data vector of the firmware (124) specifies, for each of the values ​​intended to be stored in secondary memory (118), an indication of the location of said value within secondary memory (118).

15. Computer computing device (100) according to any one of the preceding claims, wherein the indexed loading block (116) further comprises at least one counter configured to count the data exchanged by the indexed loading block (116).

Images