Data read / write method and device based on multi-level storage, and multi-level storage

Abstract

The present application relates to the technical field of computers, and discloses a data read / write method and device based on multi-level storage, and multi-level storage. In the method, in response to a write-back request sent by the k-th level storage, the (k+1)-th level storage in the multi-level storage can first perform a reduction operation on data in a plurality of cache lines corresponding to a first address in the k-th level storage, and then write the result of the reduction operation into the (k+1)-th level storage. In the method, the reduction capability is offloaded from processor cores to multi-level storage, so that the reduction operation process is executed by the multi-level storage and is integrated into the data write-back process between multiple levels of storage, enabling the processor cores to be more dedicated to computation tasks, thereby improving the utilization of the computational capability of the processor cores. Furthermore, since the reduction operation process is no longer executed by the processor cores, the reduction operation process can be executed in parallel with the computation process, thereby shortening the completion time of computation tasks and improving the efficiency of completing the computation tasks.

Description

Data reading and writing method and device based on multi-level storage and multi-level storage

[0001] This application claims priority to Chinese patent application No. 202410175983.6, filed on February 7, 2024, entitled “Data reading and writing method, device and multi-level storage based on multi-level storage”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a data reading and writing method and device based on multi-level storage, and multi-level storage. Background Art

[0003] In big data processing, when the computing task is heavy, multiple processors are required to perform parallel computing to complete the computing task. For example, in the training scenario of a deep learning model, when the number of model parameters or the intermediate results generated by the calculation are too large and exceed the memory capacity of a single processor, a single processor cannot complete the training task alone, and multiple processors are required to perform parallel computing. For example, processor 1 and processor 2 jointly calculate the output of a certain layer of the model. The computing task of this layer can be abstracted as matrix multiplication X·A=Y, where matrices X and A are the input and weight of the current layer, matrix Y is the output of the current layer, and Y needs to be used as the input of the next layer after passing through the activation function. The parallel computing process of this layer is: split the matrix X from the row dimension into X1 and X2, and split the matrix A from the column dimension into A1 and A2. The processor core of processor 1 completes the calculation of X1·A1=Y1, and the processor core of processor 2 completes the calculation of X2·A2=Y2. Then, the intermediate results Y1 obtained by the processor core of processor 1 and the intermediate results Y2 obtained by the processor core of processor 2 are reduced (for example, matrix addition operation) to obtain the calculation result Y, and Y is synchronized to the memory of processor 1 and the memory of processor 2 so that the processor core of processor 1 and the processor core of processor 2 can read them to execute the computing task of the next layer of the model.

[0004] In the related art, the processor core of a processor provides computing power and reduce power. Computing power refers to the ability of the processor core to execute the computing process and obtain the intermediate results of the computing task; reduce power refers to the ability of the processor core to use the allreduce collective communication method to perform reduce operations on the intermediate results obtained by the processor cores of each processor and synchronize the results of the reduce operations to the memory of each processor. The process includes: the processor core of each processor determines the data blocks it is responsible for reducing according to the number; the processor core of each processor sends the data blocks in the memory of the processor that are responsible for reducing by the processor cores of other processors to the corresponding processor core through inter-card communication, and receives the data blocks sent by the processor cores of other processors that are responsible for reducing by the processor cores of the processor cores; each processor core performs a reduce operation on the data blocks it is responsible for reducing; each processor core writes the results of the reduce operation into the memory of the processor, and sends the results of the reduce operation obtained by each processor to other processor cores, and the other processor cores write the received results of the reduce operation into the memory of the processor, so that the memory of each processor stores the complete calculation results.

[0005] However, in the above method, the allreduce collective communication method can only be used to perform the reduce operation process after each processing core has completed the calculation process of the intermediate results, resulting in a long completion time and low completion efficiency of the computing task. Moreover, when the processor core uses its reduce capability to perform the reduce operation process, the processor core cannot use its computing capability to execute the computing task of the next layer of the model, resulting in the computing capability of the processor being idle, the utilization rate of the computing capability of the processor core being low, and a waste of computing resources.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a data reading and writing method, device, and multi-level storage based on multi-level storage, which can shorten the completion time of computing tasks, improve the completion efficiency of computing tasks, and improve the utilization rate of the computing power of processor cores, thereby reducing the waste of computing resources. The technical solution is as follows.

[0008] In a first aspect, a data reading and writing method based on multi-level storage is provided, which is applied to multi-level storage, where each level of storage in the multi-level storage includes at least one independent storage unit, and the method includes:

[0009] In multi-level storage, the k+1th level storage responds to the k+1th level write-back request for the first address sent by the kth level storage, first performs a reduce operation on the data in multiple cache lines corresponding to the first address in the kth level storage, and then writes the result of the reduce operation to the k+1th level storage, where k is an integer greater than or equal to 1.

[0010] Wherein, the k-th level storage in the multi-level storage can be any one of the storage types such as cache, main memory, phase-change memory (PCM) and solid state disk (SSD). Wherein, the cache can be a multi-level cache, for example, the cache is a three-level cache, including a first-level cache L1cache, a second-level cache L2cache and a third-level cache L3cache, which is not limited in the embodiment of the present application. The storage unit in each level of storage is also a memory device belonging to the storage type of that level of storage. For example, if the storage type of the k-th level storage in the multi-level storage is cache, then an independent storage unit in the k-th level storage is an independent cache. For another example, if the storage type of the k-th level storage in the multi-level storage is a solid state disk, then an independent storage unit in the k-th level storage is a solid state disk.

[0011] Among them, in multi-level storage, the capacity of the k-th level storage is smaller than the capacity of the k+1-th level storage, and the read and write efficiency of the k-th level storage is higher than the read and write efficiency of the k+1-th level storage. Multi-level storage uses a step-by-step write-back method to write data to improve data writing efficiency. Step-by-step write-back means: first write the data to the upper-level storage, and then write the data from the upper-level storage to the lower-level storage step by step. Multi-level storage uses a step-by-step read method to read data to improve data reading efficiency. Step-by-step read means: first read the data from the upper-level storage. If the data does not exist in the upper-level storage, read the data from the lower-level storage step by step until the data is read, and then read the data back from the storage to the upper-level storage step by step to improve the efficiency of reading the data next time.

[0012] Among them, the status of multiple cache lines corresponding to the first address in the k-th level storage is a pending reduction state. In multi-level storage, in the pending reduction state, there may be multiple cache lines corresponding to the same address. The data in the cache lines in the pending reduction state is valid, has not been reduced, and does not support reading through a read request.

[0013] The multi-level storage performs a reduce operation on the data level by level to obtain the final result of the reduce operation. The first address is the destination address of the final result of the reduce operation. For example, if the final result of the reduce operation needs to be stored in memory, the destination address of the final result of the reduce operation is the memory address, that is, the first address is the memory address.

[0014] In the above method, the reduce capability is offloaded from the processor core to the multi-level storage, and the reduce operation process is performed by the multi-level storage. The reduce operation is integrated into the data write-back process between the multi-level storage, so that the processor core can be used more to perform calculations, thereby improving the utilization rate of the computing capability of the processor core; and since the reduce operation process is no longer performed by the processor core, the reduce operation process can be parallel to the computing process performed by the processor core, thereby shortening the completion time of the computing task and improving the completion efficiency of the computing task.

[0015] Optionally, the k-th level storage sends a k+1-th level write-back request to the k+1-th level storage, including:

[0016] When k is equal to 1, the storage unit in the first-level storage sends a second-level write-back request to the storage unit in the second-level storage in response to the first-level write-back request sent by the processor core, and the first-level write-back request carries the first address; when k is greater than or equal to 2, after the storage unit in the k-th level storage completes the reduce operation indicated by the k-th level write-back request, the storage unit in the k-th level storage sends a k+1-th level write-back request to the storage unit in the k+1-th level storage.

[0017] The processor core sends the first-level write-back request to a storage unit accessible to the processor core in the first-level storage. In the case where the k+1-th level storage includes multiple independent storage units, the first storage unit in the k-th level storage sends the k+1-th level write-back request to the second storage unit in the k+1-th level storage, where the second storage unit is a storage unit accessible to the first storage unit in the k-th level storage. For example, the multi-level storage is 2-level storage, the 1st level storage is the first-level cache in the cache, including L1cache1 and L1cache2, the 2nd level storage is the second-level cache in the cache, including L2cache1 and L2cache2, processor core 1, L1cache1 and L2cache1 are located in processor 1, processor core 2, L1cache2 and L2cache2 are located in processor 2, processor core 1 can access L1cahce1, and L1cache1 can access L2cache1, then the processor core 1 sends a 1st level write back request to L1cahce1, and the L1cache1 sends a 2nd level write back request to L2cache1.

[0018] Optionally, the method further includes:

[0019] The storage unit in the k-th level storage determines the cache line corresponding to the first address from the storage units in the k-th level storage in response to a write request to the k-th level storage, and the write request carries the first address; if the state of the cache line corresponding to the first address is an invalid state or a shared state, the storage unit in the k-th level storage writes the to-be-written data corresponding to the write request into the cache line corresponding to the first address, and sets the state of the cache line corresponding to the first address to a to-be-reduced state, wherein the invalid state indicates that the data in the cache line is invalid, the shared state indicates that the data in the cache line is valid and has not yet been reduced, and the to-be-reduced state indicates that the data in the cache line is valid and has not yet been reduced; if the state of the cache line corresponding to the first address is the to-be-reduced state, the storage unit in the k-th level storage writes the to-be-written data corresponding to the write request into the cache line corresponding to the first address.

[0020] Optionally, the storage unit in the k-th level storage determines, in response to a write request to the k-th level storage, a cache line corresponding to the first address from the storage units in the k-th level storage, comprising:

[0021] When k is equal to 1, the storage unit in the first-level storage responds to the data write request sent by the processor core, determines the cache line corresponding to the first address from the storage unit in the first-level storage, and the data to be written corresponding to the data write request is the data carried by the data write request; when k is greater than or equal to 2, the storage unit in the k-th level storage responds to the k-th level write back request sent by the storage unit in the k-1-th level storage, determines the cache line corresponding to the first address from the storage unit in the k-th level storage, and the data to be written corresponding to the k-th level write back request is the result of the reduce operation indicated by the k-th level write back request.

[0022] Optionally, when k is greater than or equal to 2, after the storage unit in the k-th level storage writes the to-be-written data corresponding to the write request into the cache line corresponding to the first address, the method further includes:

[0023] The k-1th level storage sets the state of the cache line corresponding to the first address in the k-1th level storage to an invalid state.

[0024] Optionally, the method further includes:

[0025] The storage unit in the k-th level storage receives a read request for the storage unit in the k-th level storage, and the read request carries a second address; if a cache line corresponding to the second address exists in the storage unit in the k-th level storage, and the state of the cache line corresponding to the second address is a shared state, then the storage unit in the k-th level storage reads data from the cache line corresponding to the second address and returns the read data to the processor core; if a cache line corresponding to the second address exists in the storage unit in the k-th level storage, and the state of the cache line corresponding to the second address is a to-be-reduced state, then When the state of the cache line corresponding to the second address is invalid, the storage unit in the k-th level storage sends a read request to the storage unit in the k+1-th level storage based on the second address to read the data corresponding to the second address, and returns the read data to the processor core; if there is no cache line corresponding to the second address in the storage unit in the k-th level storage, the storage unit in the k-th level storage sends a read request to the storage unit in the k+1-th level storage based on the second address to read the data corresponding to the second address, and returns the read data to the processor core.

[0026] Optionally, the method further includes:

[0027] If the storage unit in the k-th level storage reads the data corresponding to the second address from the storage unit in the k+1-th level storage, the storage unit in the k-th level storage determines the target cache line from the storage unit in the k-th level storage, writes the read data into the target cache line, and sets the state of the target cache line to a shared state.

[0028] In a second aspect, a data reading and writing method based on multi-level storage is provided, which is applied to a processor core. The method includes:

[0029] The processor core calculates the intermediate result of a parallel computing task of the target computing task. After the processor core obtains the i-th data block in the first intermediate result, the processor core sends a write-back request to the multi-level storage to instruct the multi-level storage to perform a reduce operation on the i-th data block in the first intermediate result step by step, and write the final result of the reduce operation to the first address, where i is an integer greater than or equal to 1, and the i-th data block in the first intermediate result corresponds to the first address.

[0030] A data block refers to the division of data of a given size. Each group of data is called a data block. For example, a 4M-row × 4N-column floating-point matrix can be considered as data of a certain size. If it is divided into 4 rows and 4 columns, the resulting M × N 4-row × 4-column floating-point matrices can be obtained. Each 4-row × 4-column floating-point matrix can be called a data block.

[0031] The final result of the reduce operation refers to the result of the reduce operation performed by the last level of storage in the multi-level storage on the reduce operation result obtained by the previous level of storage, that is, the i-th data block in the calculation result of the target computing task.

[0032] The fact that the i-th data block in the first intermediate result corresponds to the first address means that the destination address of the final result of the reduce operation on the i-th data block in the first intermediate result is the first address. For example, if the i-th data block in the first intermediate result is X, the i-th data block in the second intermediate result is Y, the i-th data block in the third intermediate result is W, and the i-th data block in the fourth intermediate result is Z, and the final result of the reduce operation on X is X+Y+W+Z, and the destination address of X+Y+W+Z is the first address, then X, Y, Z, and W all correspond to the first address.

[0033] In the above method, in the process of executing the target computing task, the calculation process and the reduce process of the intermediate results are executed by the processor core and the multi-level storage respectively, and the processor core and the multi-level storage can execute the above two processes in parallel, so the reduce process does not have to wait for the calculation process to be completed before starting to execute, that is, the calculation process and the reduce process overlap in time, thereby shortening the overall time of the target computing task and improving the completion efficiency of the target computing task. Moreover, while the multi-level storage executes the reduce process of the previous data block, the processor core can execute the calculation process of the next data block in parallel, avoiding the idle computing power of the processor core, thereby improving the utilization rate of the computing power of the processor core.

[0034] Optionally, after calculating and obtaining the i-th data block in the first intermediate result, sending a write-back request to the multi-level storage includes:

[0035] Whenever n data blocks in the first intermediate result are calculated, a write-back request for the n data blocks is sent to the multi-level storage, where n is an integer greater than or equal to 1.

[0036] In the above method, after the processor core calculates multiple data blocks, it sends write-back requests for these multiple data blocks to the multi-level storage to instruct the reduce operation to be performed on these multiple data blocks step by step and write the final result of the reduce operation to the first address. This can reduce the number of times the processor core sends write-back requests, so that the computing power of the processor core can be used more for computing, improve the utilization rate of the computing power of the processor core, further shorten the overall time of the target computing task, and improve the completion efficiency of the target computing task.

[0037] Optionally, each level of storage in the multi-level storage includes at least one independent storage unit. After calculating and obtaining the i-th data block in the first intermediate result, the method further includes:

[0038] A data write request is sent to a storage unit in a first-level storage in a multi-level storage, where the data write request carries the first address and the data to be written, where the data to be written is the i-th data block in the first intermediate result. The data write request is used to instruct the storage unit in the first-level storage to write the data to be written into a cache line corresponding to the first address. The state of the cache line corresponding to the first address after the data to be written is written is a to-be-reduced state, where the to-be-reduced state indicates that the data in the cache line is valid and has not yet been reduced.

[0039] Optionally, the method further includes:

[0040] A read request for a second address is sent to a storage unit in the first level storage in the multi-level storage to read data corresponding to the second address from the multi-level storage, where the data corresponding to the second address is the j-th data block in the calculation result of the target computing task, where j is an integer greater than or equal to 1; the data corresponding to the second address returned by the multi-level storage is received, and a read request for a third address is sent to the multi-level storage to read data corresponding to the third address from the multi-level storage, where the data corresponding to the third address is the j+1-th data block in the calculation result; after sending the read request for the third address to the multi-level storage, the next computing task of the target computing task is executed based on the j-th data block in the calculation result.

[0041] In the above method, after the multi-level storage reads a data block in the calculation result and sends it to the processor core, the processor core executes the next calculation process based on the data block. At the same time, the multi-level storage executes the reading process of the next data block without waiting for the calculation process corresponding to the previous data block to be completed. This can shorten the overall time of the calculation task and improve the completion efficiency of the calculation task.

[0042] In a third aspect, a data reading and writing method based on multi-level storage is provided, wherein each level of storage in the multi-level storage includes at least one independent storage unit, the method comprising:

[0043] The processor core calculates an i-th data block in a first intermediate result of a first parallel computing task, where i is an integer greater than or equal to 1, and the i-th data block in the first intermediate result corresponds to a first address. The first parallel computing task is used to calculate a portion of the computing result in the target computing task.

[0044] After calculating and obtaining the i-th data block in the first intermediate result, the processor core sends a write-back request to the multi-level storage, wherein the write-back request instructs the multi-level storage to perform a reduce operation on the data in the plurality of cache lines corresponding to the first address in the multi-level storage, level by level, and write a final result of the reduce operation to the first address;

[0045] After sending the write-back request, the processor core calculates the (i+1)th data block in the first intermediate result;

[0046] While the processor core is calculating, the k-th level storage in the multi-level storage sends a k+1-th level write-back request to the k+1-th level storage in the multi-level storage, where the k+1-th level write-back request carries a first address, and k is an integer greater than or equal to 1;

[0047] The k+1th level storage responds to the k+1th level write back request, performs a reduce operation on the data in the multiple cache lines corresponding to the first address in the kth level storage, and writes the results of the reduce operation to the k+1th level storage, where different cache lines in the multiple cache lines correspond to different storage units in the kth level storage.

[0048] Optionally, the k-th level storage in the multi-level storage sends a k+1-th level write-back request to the k+1-th level storage, including:

[0049] When k is equal to 1, the storage unit in the first-level storage sends a second-level write-back request to the storage unit in the second-level storage in response to the first-level write-back request sent by the processor core, and the first-level write-back request carries the first address; when k is greater than or equal to 2, after the storage unit in the k-th level storage completes the reduce operation indicated by the k-th level write-back request, the storage unit in the k-th level storage sends a k+1-th level write-back request to the storage unit in the k+1-th level storage.

[0050] Optionally, the method further includes:

[0051] The storage unit in the k-th level storage determines the cache line corresponding to the first address from the storage units in the k-th level storage in response to a write request to the k-th level storage, and the write request carries the first address; if the state of the cache line corresponding to the first address is an invalid state or a shared state, the storage unit in the k-th level storage writes the to-be-written data corresponding to the write request into the cache line corresponding to the first address, and sets the state of the cache line corresponding to the first address to a to-be-reduced state, wherein the invalid state indicates that the data in the cache line is invalid, the shared state indicates that the data in the cache line is valid and supports reading, and the to-be-reduced state indicates that the data in the cache line is valid and has not yet been reduced; if the state of the cache line corresponding to the first address is the to-be-reduced state, the storage unit in the k-th level storage writes the to-be-written data corresponding to the write request into the cache line corresponding to the first address.

[0052] Optionally, the storage unit in the k-th level storage determines, in response to a write request to the k-th level storage, a cache line corresponding to the first address from the storage units in the k-th level storage, comprising:

[0053] When k is equal to 1, the storage unit in the first-level storage responds to the data write request sent by the processor core, determines the cache line corresponding to the first address from the storage unit in the first-level storage, and the data to be written corresponding to the data write request is the data carried by the data write request; when k is greater than or equal to 2, the storage unit in the k-th level storage responds to the k-th level write back request sent by the storage unit in the k-1-th level storage, determines the cache line corresponding to the first address from the storage unit in the k-th level storage, and the data to be written corresponding to the k-th level write back request is the result of the reduce operation indicated by the k-th level write back request.

[0054] Optionally, when k is greater than or equal to 2, after the storage unit in the k-th level storage writes the to-be-written data corresponding to the write request into the cache line corresponding to the first address, the method further includes:

[0055] The k-1th level storage sets the state of the cache line corresponding to the first address in the k-1th level storage to an invalid state.

[0056] Optionally, the method further includes:

[0057] The storage unit in the k-th level storage receives a read request for the storage unit in the k-th level storage, and the read request carries a second address; if a cache line corresponding to the second address exists in the storage unit in the k-th level storage, and the state of the cache line corresponding to the second address is a shared state, then the storage unit in the k-th level storage reads data from the cache line corresponding to the second address and returns the read data to the processor core; if a cache line corresponding to the second address exists in the storage unit in the k-th level storage, and the state of the cache line corresponding to the second address is a to-be-reduced state, then When the state of the cache line corresponding to the second address is invalid, the storage unit in the k-th level storage sends a read request to the storage unit in the k+1-th level storage based on the second address to read the data corresponding to the second address, and returns the read data to the processor core; if there is no cache line corresponding to the second address in the storage unit in the k-th level storage, the storage unit in the k-th level storage sends a read request to the storage unit in the k+1-th level storage based on the second address to read the data corresponding to the second address, and returns the read data to the processor core.

[0058] Optionally, the method further includes:

[0059] If the storage unit in the k-th level storage reads the data corresponding to the second address from the storage unit in the k+1-th level storage, the storage unit in the k-th level storage determines the target cache line from the storage unit in the k-th level storage, writes the read data into the target cache line, and sets the state of the target cache line to a shared state.

[0060] Optionally, after calculating and obtaining the i-th data block in the first intermediate result, the processor core sends a write-back request to the multi-level storage, including:

[0061] Whenever n data blocks in the first intermediate result are calculated, a write-back request for the n data blocks is sent to the multi-level storage, where n is an integer greater than or equal to 1.

[0062] Optionally, each level of storage in the multi-level storage includes at least one independent storage unit. After calculating and obtaining the i-th data block in the first intermediate result, the method further includes:

[0063] A data write request is sent to a storage unit in the first-level storage in the multi-level storage. The data write request carries the first address and the data to be written. The data to be written is the i-th data block in the first intermediate result. The data write request is used to instruct the storage unit in the first-level storage to write the data to be written into the cache line corresponding to the first address. The state of the cache line corresponding to the first address after the data to be written is written is a to-be-reduced state.

[0064] Optionally, the method further includes:

[0065] A read request for a second address is sent to a storage unit in the first level storage in the multi-level storage to read data corresponding to the second address from the multi-level storage, where the data corresponding to the second address is the j-th data block in the calculation result of the target computing task, where j is an integer greater than or equal to 1; the data corresponding to the second address returned by the multi-level storage is received, and a read request for a third address is sent to the multi-level storage to read data corresponding to the third address from the multi-level storage, where the data corresponding to the third address is the j+1-th data block in the calculation result; after sending the read request for the third address to the multi-level storage, the next computing task of the target computing task is executed based on the j-th data block in the calculation result.

[0066] In a fourth aspect, a data reading and writing device based on multi-level storage is provided, which includes at least one functional module, and the at least one functional module is used to execute the data reading and writing method based on multi-level storage provided by the aforementioned first aspect or any possible implementation method of the first aspect.

[0067] In the fifth aspect, a data reading and writing device based on multi-level storage is provided, which includes at least one functional module, and the at least one functional module is used to execute the data reading and writing method based on multi-level storage provided by the aforementioned second aspect or any possible implementation method of the second aspect.

[0068] In a sixth aspect, a multi-level storage is provided, wherein each level of storage in the multi-level storage includes at least one independent storage unit;

[0069] The k-th level storage in the multi-level storage is configured to send a k+1-th level write-back request to the k+1-th level storage in the multi-level storage, where the k+1-th level write-back request carries a first address, and k is an integer greater than or equal to 1;

[0070] The k+1th level storage is used to respond to the k+1th level write back request, perform a reduce operation on the data in the multiple cache lines corresponding to the first address in the kth level storage, and write the results of the reduce operation into the k+1th level storage, where different cache lines in the multiple cache lines correspond to different storage units in the kth level storage.

[0071] In the seventh aspect, a processor is provided, which includes multi-level storage and multiple processor cores. The multi-level storage is used to implement the steps performed by the multi-level storage in the data reading and writing method based on multi-level storage provided in the third aspect or any optional method of the third aspect. The processor core is used to implement the steps performed by the processor core in the data reading and writing method based on multi-level storage provided in the third aspect or any optional method of the third aspect.

[0072] In an eighth aspect, a computing device is provided, which includes multi-level storage and multiple processor cores, the multi-level storage is used to implement the steps performed by the multi-level storage in the data reading and writing method based on the multi-level storage provided in the third aspect or any optional method of the third aspect, and the processor core is used to implement the steps performed by the processor core in the data reading and writing method based on the multi-level storage provided in the third aspect or any optional method of the third aspect.

[0073] In the ninth aspect, a distributed cluster is provided, which includes multi-level storage and multiple processor cores. The multi-level storage is used to implement the steps performed by the multi-level storage in the data reading and writing method based on the multi-level storage provided in the third aspect or any optional method of the third aspect. The processor core is used to implement the steps performed by the processor core in the data reading and writing method based on the multi-level storage provided in the third aspect or any optional method of the third aspect.

[0074] In the tenth aspect, a storage medium is provided, which is used to store program code, and the program code can be read and executed by multi-level storage to implement the data reading and writing method based on multi-level storage provided by the first aspect or any optional method of the first aspect.

[0075] In the eleventh aspect, a storage medium is provided, which is used to store program code, and the program code can be read and executed by the processor core to implement the data reading and writing method based on multi-level storage provided by the second aspect or any optional method of the second aspect.

[0076] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] FIG1 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0078] FIG2 is a schematic diagram of a state to be reduced provided by an embodiment of the present application;

[0079] 3 is a flow chart of a data writing process in a data reading and writing method based on multi-level storage provided in an embodiment of the present application;

[0080] FIG4 is a schematic diagram of a request sending and state migration process in a storage protocol provided in an embodiment of the present application;

[0081] FIG5 is a flowchart of a data writing and reducing process in a data reading and writing method based on multi-level storage provided by an embodiment of the present application;

[0082] FIG6 is a flow chart of a data writing and reducing process in a data reading and writing method based on multi-level storage provided in an embodiment of the present application;

[0083] FIG7 is a flowchart of a data writing and reducing process in a data reading and writing method based on multi-level storage provided in an embodiment of the present application;

[0084] FIG8 is a flow chart of a data writing and reducing process in a data reading and writing method based on multi-level storage provided in an embodiment of the present application;

[0085] FIG9 is a flow chart of a data writing and reducing process in a data reading and writing method based on multi-level storage provided in an embodiment of the present application;

[0086] FIG10 is a flowchart of a data writing and reducing process in a data reading and writing method based on multi-level storage provided in an embodiment of the present application;

[0087] 11 is a flowchart of a data reading process in a data reading and writing method based on multi-level storage provided in an embodiment of the present application;

[0088] 12 is a flow chart of a data reading process in a data reading and writing method based on multi-level storage provided in an embodiment of the present application;

[0089] 13 is a flow chart of a data reading process in a data reading and writing method based on multi-level storage provided in an embodiment of the present application;

[0090] FIG14 is a flow chart of a data reading and writing method based on multi-level storage provided in an embodiment of the present application;

[0091] FIG15 is a schematic structural diagram of a data reading and writing device based on multi-level storage provided in an embodiment of the present application;

[0092] FIG16 is a schematic structural diagram of a data reading and writing device based on multi-level storage provided in an embodiment of the present application;

[0093] FIG17 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0094] FIG18 is a schematic diagram of the structure of a distributed cluster provided in an embodiment of the present application;

[0095] FIG19 is a schematic diagram of the structure of a distributed cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0096] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0097] First, the implementation environment of this application is introduced.

[0098] Figure 1 is a structural diagram of a computing device provided in an embodiment of the present application. As shown in Figure 1, the computing device includes a processor core 101 and multi-level storage 102, wherein each level of storage in the multi-level storage 102 includes at least one independent storage unit.

[0099] In which, the computing device includes one or more processors, and the multiple processor cores 101 can be distributed in one processor in the computing device, or can be distributed in multiple processors in the computing device, which is not limited in the embodiments of the present application. In some embodiments, the multi-level storage 102 is a multi-level cache in the processor in the computing device. In which, each level of storage in the multi-level storage corresponds one-to-one to each level of cache in the multi-level cache, and the number of storage units included in the k-th level storage is equal to the number of caches included in the k-th level cache, and k is an integer greater than or equal to 1. For example, the first level storage in the multi-level storage corresponds to the first level cache in the processor, and the first level cache includes multiple L1 caches, and the number of storage units included in the first level storage is equal to the number of L1 caches in the first level cache.

[0100] In other embodiments, the multi-level storage 102 includes a multi-level cache in the computing device and the memory of the computing device. For example, each processor in the computing device includes a three-level cache, then the first-level cache in each processor in the computing device corresponds to the first-level storage in the multi-level storage 102, the second-level cache in each processor in the computing device corresponds to the second-level storage in the multi-level storage 102, the third-level cache in each processor in the computing device corresponds to the third-level storage in the multi-level storage 102, and the memory of the computing device corresponds to the fourth-level storage in the multi-level storage 102. It should be noted that the above example is based on the example of the processor including the third-level cache. Those skilled in the art will know that the processor can also include a second-level cache or a first-level cache, etc., and the embodiments of the present application do not limit the level of cache in the processor. In some embodiments, the computing device also includes a solid-state disk (SSD), then the solid-state disk is the fifth-level storage in the multi-level storage 102. Those skilled in the art will know that the computing device can also include other types of storage, and other types of storage can serve as one or more levels in the multi-level storage 102, and the embodiments of the present application do not limit this.

[0101] The processor in the above implementation environment may be a general-purpose central processing unit (CPU), a graphics processing unit (GPU), a switching module processor unit (SMPU), a network processor (NP), a microprocessor, or may be one or more integrated circuits for implementing the solution of the present application, such as an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. The embodiments of the present application do not limit the type of processor.

[0102] Among them, the processor core 101 provides a computing function for performing computing tasks. Exemplarily, multiple processor cores 101 can perform computing tasks, obtain computing results, and send data write requests for the computing results to the multi-level storage 102, which stores the computing results. In some embodiments, multiple processor cores 101 can execute target computing tasks in parallel, and each processor core 101 executes a parallel computing task in the target computing task. For example, multiple processor cores 101 can execute parameter calculation tasks of the target layer in the model in parallel, and each processor core 101 executes the calculation tasks of part of the parameters of the target layer. It should be noted that the above-mentioned description of the target computing task using the parameter calculation of the target layer in the model is only exemplary. Those skilled in the art will know that multiple processor cores 101 can also execute other parallel computing tasks according to actual needs. The embodiments of this application do not limit parallel computing tasks.

[0103] Among them, the multi-level storage 102 provides storage functions and reduce functions. Exemplarily, each storage unit in each level of storage in the multi-level storage 102 includes a storage medium and an agent. The storage medium is used to store data, that is, to provide a storage function. The agent is used to perform read and write operations and reduce operations on the data in the storage medium of the previous level of storage, so as to reduce the data in the previous level of storage and store the result of the reduction in the storage medium of the current level of storage, that is, to provide a reduce function. For example, when the k-th level storage is a cache, a storage unit in the k-th level storage is a cache, and the cache includes a storage medium and an agent. In some embodiments, the agent in the cache is called a cache agent (CA); when the k+1-th level storage is a memory, a storage unit in the k-th level storage is a memory, and the memory includes a storage medium and an agent. In some embodiments, the agent of the memory is called a home agent (HA). The embodiments of the present application do not limit the name of the agent.

[0104] Each storage unit in multi-level storage 102 includes multiple cache lines, each of which includes stored data, a destination address corresponding to the stored data, and a cache line state. The amount of data corresponding to each cache line is determined based on the storage medium. The cache line state is specified by the storage protocol used by that level of storage and is used to indicate the validity and read and write rules of the data stored in the cache line. The storage protocol used by multi-level storage 102 in the embodiments of the present application and the cache line state specified by the storage protocol will be described later and are not detailed here.

[0105] It should be noted that the above FIG1 is illustrated by taking the implementation environment as a computing device as an example. In some embodiments, the implementation environment is a distributed cluster, and the distributed cluster includes multiple processor cores and multi-level storage. Among them, the distributed cluster includes multiple processors, each processor includes one or more processor cores, and the multiple processors can be distributed in one computing device or in multiple computing devices. The embodiments of the present application do not limit this. The multi-level storage includes multi-level caches in the computing devices in the distributed cluster and the memory of the computing devices. For example, each processor includes a three-level cache, and the first-level cache in the processor of each computing device in the distributed cluster corresponds to the first-level storage in the multi-level storage; the second-level cache in the processor of each computing device in the distributed cluster corresponds to the second-level storage in the multi-level storage, the third-level cache in the processor of each computing device in the distributed cluster corresponds to the third-level storage in the multi-level storage, and the memory in each computing device in the distributed cluster corresponds to the fourth-level storage in the multi-level storage. In some embodiments, the distributed cluster also includes an SSD, and the SSD is the fifth level storage in the multi-level storage. Those skilled in the art will know that the distributed cluster can also include other types of storage, and other types of storage can serve as one or more levels in the multi-level storage. The embodiments of the present application do not limit this.

[0106] In particular, when the implementation environment is a distributed cluster, the distributed cluster supports unified physical addressing. The distributed cluster can be a single machine with multiple devices, a single rack with multiple servers, a tightly coupled cluster using disaggregated memory, or a multi-rack cluster (also known as a supernode) connected via a cache coherent bus protocol. Multiple processors in a distributed cluster can access each other's memory. In some embodiments, the multiple processors meet the non-uniform memory access (NUMA) architecture. When the destination address of a read or write request issued by a processor core in any processor is not a local memory address of the processor, the processor will issue a read or write request to the memory where the destination address is located through a link, and then read or write the memory where the destination address is located. Among them, the link is an interconnection architecture that supports cache consistency, such as ultra path interconnect (UPI), asymmetric compute express link (CXL), coherent acceleration processor interface (CCIX), open coherent accelerator processor interface (OpenCAPI), generation Z (Gen-Z), etc., which is not limited to the embodiments of the present application.

[0107] An embodiment of the present application provides a data reading and writing method based on multi-level storage, in which, in the multi-level storage, the k+1-th level storage responds to the k+1-th level write-back request for the first address sent by the k-th level storage, first performs a reduce operation on the data in the multiple cache lines corresponding to the first address in the k-th level storage, and then writes the result of the reduce operation to the k+1-th level storage, wherein k is an integer greater than or equal to 1. In the above method, the reduce capability is unloaded from the processor core to the multi-level storage, the reduce operation process is executed by the multi-level storage, and the reduce operation is integrated into the data write-back process between the multi-level storage, so that the processor core can be used more to perform calculations, thereby improving the utilization rate of the computing power of the processor core; and since the reduce operation process is no longer executed by the processor core, the reduce operation process can be parallel to the computing process executed by the processor core, thereby shortening the completion time of the computing task and improving the completion efficiency of the computing task.

[0108] The above method can be applied to a scenario where multiple processor cores execute parallel computing tasks. In this scenario, each processor core calculates the intermediate result of a parallel computing task. After the processor core obtains the i-th data block in the first intermediate result (i is an integer greater than or equal to 1, and the i-th data block in the first intermediate result corresponds to the first address), the processor core sends a write-back request to the multi-level storage to instruct the multi-level storage to perform a reduce operation on the i-th data block in the first intermediate result and the i-th data block in the intermediate results calculated by other processor cores at each level (the i-th data block in the intermediate results calculated by other processor cores also corresponds to the first address), and write the final result of the reduce operation to the first address. In the above method, in the process of executing the target computing task, the calculation process and the reduce process of the intermediate results are executed by the processor core and the multi-level storage respectively, and the processor core and the multi-level storage can execute the above two processes in parallel, so the reduce process does not have to wait for the calculation process to be completed before starting to execute, that is, the calculation process and the reduce process overlap in time, thereby reducing the overall time of the target computing task, and, while the multi-level storage executes the reduce process of the previous data block, the processor core can execute the calculation process of the next data block in parallel, avoiding the idle computing power of the processor core, thereby improving the utilization rate of the computing power of the processor core.

[0109] In some embodiments, the above-mentioned parallel computing scenario is a scenario for distributed training of the model. In this scenario, multiple processor cores in the distributed cluster respectively calculate part of the parameters of the target layer in the model, and respectively obtain multiple intermediate results corresponding to the target layer. By performing reduce operations on the corresponding data blocks in the intermediate results obtained by each processor core, the calculation results of the target layer are obtained, and then based on the calculation results of the target layer, the parameters of the next layer of the target layer are calculated. In addition to the above-mentioned model distributed training scenario, the above-mentioned method can also be applied to other parallel computing scenarios, and the embodiments of the present application are not limited to this.

[0110] The above method involves data exchange between multiple levels of storage. This embodiment of the present application provides a storage protocol, which can be referred to as a parallel cache coherence protocol based on unified memory. This embodiment of the present application does not limit the name of this storage protocol. Each level of storage in the multi-level storage system exchanges data based on this storage protocol. This storage protocol is first introduced below.

[0111] First, the states of the cache lines defined by the storage protocol are introduced.

[0112] PM (Pending Reduction State): The data in this cache line is valid and has not yet been reduced. In multi-level storage, multiple cache lines corresponding to the same address can exist in the Pending Reduction state, and the data in the cache line in the Pending Reduction state cannot be read through a read request. The following example illustrates this Pending Reduction state with reference to Figure 2, which is a schematic diagram of a Pending Reduction state provided in an embodiment of the present application. As shown in Figure 2, processing core 1 writes data 11 to cache line 201 corresponding to address 0x1000 in cache 1 through PrWrNodata (data write request). After writing data 11, the state of cache line 201 is in the waiting-for-reduction state; processing core 2 writes data 22 to cache line 202 corresponding to address 0x1000 in cache 2 through PrWrNodata. After writing data 22, the state of cache line 202 is in the waiting-for-reduction state, and the state of cache line 201 is also in the waiting-for-reduction state. The data in cache lines 201 and 202 are both valid, have not been reduced, and do not support reading. That is, in the waiting-for-reduction state, there can be multiple cache lines corresponding to the same address. It should be noted that in the storage protocol used in the prior art, processing core 1 writes data 11 to cache line 201 corresponding to address 0x1000 in cache 1. When processor core 2 writes data 22 to cache line 202 corresponding to address 0x1000 in cache 2, the state of cache line 201 becomes I (invalid state), and the data 11 in cache line 201 is invalid. That is, in the storage protocol used in the prior art, there cannot be multiple cache lines corresponding to the same address, and thus the prior art cannot implement a reduce operation by the next level of storage on the data in multiple cache lines corresponding to the same address in the previous level of storage. Among them, PrWrNodata will be introduced later and will not be repeated here.

[0113] S (Shared state): The data in the cache line is valid and supports reading.

[0114] I (Invalid state): The data in this cache line is invalid.

[0115] The following describes the requests defined by this storage protocol.

[0116] PrWrNodata (data write request): Sent by the processor core to a storage unit in the first-level storage. This data write request carries the data to be written and the destination address, indicating that the data carried should be written to the destination address without reading back the original data from the destination address before writing. After receiving this data write request, the storage unit in the first-level storage writes the data carried in the data write request to the cache line corresponding to the destination address. The state of the cache line after the data is written is in the state of pending reduction.

[0117] PrWr (data write request with read): sent by the processor core to the storage unit in the first-level storage. The data write request carries the data to be written and the destination address, and is used to indicate that the carried data is written to the destination address, and before writing, the original data in the cache line corresponding to the destination address needs to be read back from the lower-level storage. After receiving the data write request with read, the storage unit in the first-level storage sends a read request for the original data to the second-level storage to obtain the original data, and writes the original data to the cache line in the storage unit corresponding to the address of the original data in the lower-level storage; then the storage unit in the first-level storage writes the data carried by the data write request with read to the cache line in the storage unit corresponding to the memory address, and the state of the cache line after the data is written is a to-be-reduced state.

[0118] BusWrNodata (cache coherence request): generated along with PrWrNodata. For example, when writing (address 1, data 1) to a storage unit in the first-level storage via PrWrNodata, the storage unit generates BusWrNodata and sends it to other storage units in the first-level storage. After receiving BusWrNodata, if the state of the cache line corresponding to address 1 in the storage unit is shared, the storage unit changes the state of the cache line to invalid.

[0119] ReducePM (k-th level write-back request, k equals 1): sent by the processor core to the storage unit in the first-level storage. The first-level write-back request carries the destination address, the number of cache lines targeted, and the reduce operation type. The storage unit in the first-level storage receives the second-level write-back request and sends it to the storage unit in the second-level storage to perform a reduce operation on the data in the multiple cache lines corresponding to the destination address in the first-level storage step by step. The reduce operation type can be INT8, FP16, and FP32 addition, etc., and the embodiment of the present application does not limit the reduce operation type. After receiving the first-level write-back request, the storage unit in the first-level storage generates a second-level write-back request based on the destination address, the number of cache lines targeted, and the reduce operation type, and sends the second-level write-back request to the storage unit in the second-level storage. After the storage unit in the second-level storage completes the reduce operation corresponding to the second-level write-back request, the storage unit in the first-level storage modifies the state of the cache line corresponding to the destination address in the storage unit in the first-level storage to an invalid state.

[0120] ReduceWb (k-th level write-back request, k greater than or equal to 2): Sent by a storage unit in the k-th level storage to a storage unit in the k-th level storage. This k-th level write-back request carries the destination address and the reduce operation type. It is used to instruct the k-th level storage to perform a reduce operation on the data in multiple cache lines corresponding to the destination address in the k-1-th level storage, and write the results of the reduce operation to the k-th level storage. After receiving this k-th level write-back request, the storage unit in the k-th level storage obtains the data to be reduced from the storage unit in the k-1-th level storage, reduces the obtained data, and writes the reduced data to the memory address.

[0121] The k-th level write-back request is illustrated below with reference to FIG3 . FIG3 is a flow chart of a data writing process in a data reading and writing method based on multi-level storage provided by an embodiment of the present application. As shown in FIG3 , FIG3 includes four processor cores (core1-core4) and multi-level storage. In the multi-level storage, the first level storage includes four storage units 3011-3014, and the cache lines corresponding to the destination address in each storage unit respectively store data X, Y, W, and Z. The second level storage includes two storage units 3021 and 3022, and the third level storage includes one storage unit 303. In response to the first-level write-back request sent by core1, the storage unit 3011 in the first-level storage sends a second-level write-back request to the storage unit 3021 in the second-level storage. The second-level write-back request carries the destination address. The storage unit 3021 in the second-level storage obtains the data X and Y in the cache line corresponding to the destination address from the storage units 3011 and 3012 in the first-level storage, performs a reduce operation on X and Y, obtains the result of the reduce operation X+Y, and stores X+Y in the storage unit 3021. The process of the storage unit 3022 in the second-level storage obtaining W+Z and storing W+Z in the storage unit 3022 is the same as that of the storage unit 302 The process of obtaining X+Y and storing X+Y in the storage unit 3021 is similar and will not be repeated here. The storage unit 3021 in the second-level storage sends a third-level write-back request to the storage unit 303 in the third-level storage, and the third-level write-back request carries the destination address. The storage unit 303 in the third-level storage obtains X+Y from the storage unit 3021 in the second-level storage and obtains W+Z from the storage unit 3022 in the second-level storage, performs a reduce operation on X+Y and W+Z, obtains the final result of the reduce operation X+Y+W+Z, and stores the final result of the reduce operation X+Y+W+Z in the storage unit 303 in the third-level storage.

[0122] ReducePull (data retrieval request): After receiving a write-back request at level k+1, a storage unit in the k+1th level of storage in one processor sends it to a storage unit in the kth level of storage in another processor to retrieve the data to be reduced from the storage units in the other processor. This data retrieval request carries a destination address and is used to retrieve the data in the cache line corresponding to the destination address. After receiving the data retrieval request, the storage unit in the kth level of storage retrieves the data from the cache line corresponding to the destination address and sends the retrieved data to the storage unit in the k+1th level of storage. After sending the data, the storage unit in the kth level of storage changes the state of the cache line corresponding to the destination address to invalid.

[0123] PrRd (Read Request): Sent by the processor core to a storage unit in the first level of storage, or from a storage unit in the k-th level of storage to a storage unit in the k-th level of storage. This read request carries a destination address and is used to read the data corresponding to the destination address. After the storage unit in the k-th level storage receives the read request, if there is a cache line corresponding to the destination address in the storage unit in the k-th level storage and the state of the cache line is a shared state, the storage unit in the k-th level storage reads the data from the cache line and returns the read data to the processor core; if there is a cache line corresponding to the destination address in the storage unit in the k-th level storage and the state of the cache line is a pending reduction state, the storage unit in the k-th level storage waits for the state of the cache line to be invalid, and then sends a read request to the storage unit in the k+1-th level storage based on the destination address to read the data corresponding to the destination address, and returns the read data to the processor core; if there is no cache line corresponding to the destination address in the storage unit in the k-th level storage, then the storage unit in the k-th level storage sends a read request to the storage unit in the k+1-th level storage based on the destination address to read the data corresponding to the destination address, and returns the read data to the processor core.

[0124] It should be noted that, in order to simplify the description of the embodiments of the present application and highlight the implementation process of the data reading and writing method based on multi-level storage provided by the embodiments of the present application, the embodiments of the present application will collectively refer to the multi-level cache as cache, with cache as the first level storage in the multi-level storage, and memory as the second level storage in the multi-level storage. The data interaction between the processor core, cache and memory is used as an example to introduce the method, and will not be described later. The data interaction process between multi-level caches or the data interaction process between memory and other storage (such as a solid-state drive) is similar to the data interaction process between cache and memory, and will not be repeated in the embodiments of the present application.

[0125] The following uses Figure 4 to illustrate the sending of requests and the state migration process of the corresponding cache lines in the above-mentioned storage protocol. Figure 4 is a schematic diagram of the request sending and state migration process in a storage protocol provided by an embodiment of the present application. As shown in Figure 4, the upper figure in Figure 4 includes GPU1 and GPU2, GPU1 includes core1, first agent 1 (HA1), cache1, second agent (CA1) and video memory 1, GPU2 includes core2, first agent 2 (HA2), cache2, second agent 2 (CA2) and video memory 2; the lower figure in Figure 4 includes PM state, S state and I state. It should be understood that in Figure 4, CA1 and cache1 constitute a storage unit in the first-level storage, CA2 and cache2 constitute a storage unit in the first-level storage, HA1 and video memory 1 constitute a storage unit in the second-level storage, and HA2 and video memory 2 constitute a storage unit in the second-level storage. Among them, the storage function is provided by the cache and video memory, and the data reduce function is provided by CA and HA.

[0126] Data write process: core1 sends a PrWrNodata request to CA1, which carries data 1 and memory address A. CA1 writes data 1 carried in the PrWrNodata request to cache line 1 in cache1, which corresponds to memory address A. If the state of cache line 1 before writing data 1 is I or S, CA1 writes data 1 to cache line 1 and changes the state of cache line 1 to PM. If the state of cache line 1 before writing data 1 is PM, CA1 first writes back the original data in cache line 1 level by level, changing the state of cache line 1 to I, then writes data 1 to cache line 1 and changes the state of cache line 1 from I to PM. CA1 sends a BusWrNodata request to CA2 in GPU2. If the state of cache line 2 is S, CA2 changes the state of cache line 2 from S to I. Cache line 2 is the cache line in cache 2 in GPU2 that corresponds to memory address A.

[0127] The process of reducing data step by step: core1 sends a ReducePM request for memory address A to CA1, and CA1 obtains data 1 from cache line 1 based on memory address A; CA1 sends a ReduceWb request to HA1, and the ReduceWb request carries data 1 and memory address A; HA1 writes data 1 carried in the ReduceWb request to video memory 1 based on memory address A, and sends a ReducePull request to CA2 in GPU2, and the ReducePull carries memory address A (taking memory address A corresponding to GPU1's memory as an example); when data 2 has been written to cache line 2 and the state of cache line 2 is PM, CA2 sends data 2 in cache line 2 to HA1, and CA2 changes the state of cache line 2 from PM to I; after HA1 receives data 2, HA1 reads data 1 from video memory 1 based on memory address A and reduces data 1 and data 2. Operation, write the result of the reduce operation to memory address A in video memory 1, HA1 returns the completion response of the ReduceWb request to CA1, and after receiving the completion response, CA1 changes the state of cache line 1 from PM to I.

[0128] Data reading process: core1 sends a PrRd request to CA1, which carries memory address A; CA1 sends a read request for memory address A to HA1 (taking the memory address A corresponding to GPU1's memory as an example); HA1 sends the data in memory address A (the result of the reduce operation) to CA1, and CA1 writes the data to the cache line corresponding to memory address A in cache1, changes the state of the cache line to S, and sends the data to core1.

[0129] It should be noted that the example of the distributed cluster including GPU1 and GPU2 in FIG4 is merely exemplary, and the embodiment of the present application does not limit the type and number of processors in the distributed cluster.

[0130] The above introduces a storage protocol provided by an embodiment of the present application. Below, taking the training process of the target layer in the model as an example, the specific process of a data reading and writing method based on multi-level storage provided by an embodiment of the present application is introduced.

[0131] First, the data writing and reducing process is introduced. Figure 5 is a flowchart of the data writing and reducing process in a data reading and writing method based on multi-level storage provided by an embodiment of the present application. As shown in Figure 5, the method is applied to the first processor in a distributed cluster, and the distributed cluster includes multiple processors. The multiple processors can access each other's memory. The first processor includes a processor core, a first agent, a second agent, a memory and a cache. The first agent in the first processor is used to receive requests for memory and perform read and write operations on the memory of the first processor. The second agent in the first processor is used to receive requests for cache and perform read and write operations on the cache of the first processor. The second agent and cache in the first processor constitute a storage unit in the first level storage in the multi-level storage, and the first agent and memory in the first processor constitute a storage unit in the second level storage in the multi-level storage. The method includes the following steps 501 to 506.

[0132] 501. The processor core calculates the i-th data block in the first intermediate result of the first parallel computing task, where i is an integer greater than or equal to 1. The i-th data block in the first intermediate computing result corresponds to a first address. The first parallel computing task is used to implement the calculation of part of the computing results in the target computing task.

[0133] Among them, the first parallel computing task can be abstracted as the multiplication of the first input sub-matrix and the first weight sub-matrix of the target layer in the model, wherein the first input sub-matrix is ​​obtained based on row cutting of the input matrix of the target layer, and the first weight sub-matrix is ​​obtained based on column cutting of the input matrix of the target layer, that is, the target layer is tensor parallelized using the row cutting method. For example, the input matrix of the target layer in the model is X, the scale of X is 1024×2048, the weight matrix of the target layer is A, the scale of A is 2048×1024, the calculation result of the target layer is Y, Y=X·A, the scale of Y is 1024×1024; perform row cutting on the input matrix X to obtain the first sub-matrix X1 and the second sub-matrix X2, the scale of X1 and X2 is 1024×1024, perform column cutting on the weight matrix A to obtain the first sub-weight matrix A1 and the second sub-weight matrix A2. The scales of the weight matrices A2, A1, and A2 are 1024×1024, the first parallel computing task of the target layer is X1·A1=Y1, and the second parallel computing task is X2·A2=Y2, wherein the first parallel computing task is executed by the first processor, and the second parallel computing task is executed by the second processor, Y1 is the first intermediate calculation result, Y2 is the second intermediate calculation result, the scales of Y1 and Y2 are 1024×1024 (the same as the scale of Y), and Y=Y1+Y2. It should be noted that the above example of the first parallel computing task is only exemplary and does not limit the number of processors in the distributed cluster, the scale of the input matrix and weight matrix of the target layer, the cutting results of the input matrix and weight matrix, etc., and the cutting process can be completed by the first processor or by other processors. The embodiment of the present application does not limit the cutting process.

[0134] In which, when calculating the data blocks in the first intermediate result, the processor core calculates each data block in the first intermediate result in sequence. For example, after the processor core calculates the i-th data block in the first intermediate result, it calculates the i+1-th data block in the first intermediate result until all the data blocks in the first intermediate result are obtained.

[0135] 502. After calculating the i-th data block in the first intermediate result, the processor core sends a data write request and a first-level write back request to the second agent. After sending the first-level write back request, the processor core calculates the i+1-th data block in the first intermediate result. The data write request indicates that the i-th data block in the first intermediate result is written into the cache of the first processor. The first-level write back request indicates that a second-level write back request is sent to the first agent. The second-level write back request is used to instruct the first agent of the first processor to perform a reduce operation on the data in the multiple cache lines corresponding to the first address in the cache, and write the result of the reduce operation to the first address in the memory of the first processor.

[0136] The data write request carries the i-th data block and memory address in the first intermediate result. In some embodiments, the amount of data in each data block in the first intermediate result is equal to an integer multiple of the capacity of a cache line in the cache of the first processor. Continuing with the example in step 501 above, the first intermediate result is represented by a first intermediate result matrix, each element in the first intermediate calculation result is a 2-byte half-precision floating-point number, the first intermediate calculation result is divided into 8×8 blocks, and the first intermediate calculation result is divided into 16384 blocks (Z1, Z2, Z3, ... Z16384), each block is used as a data block in the first intermediate result, then the amount of data of each data block is 128 bytes, the capacity of a cache line in the cache is also 128 bytes, and the amount of data of each data block is exactly equal to the capacity of a cache line in the cache of the first processor. In the above embodiment, by making the data volume of the data block in the first intermediate result equal to an integer multiple of the cache line capacity in the cache, when the data block is written into the cache, each data block can just fill an integer number of cache lines, thereby avoiding the problem of cache space waste caused by the misalignment of the data volume of the data block and the capacity of the cache line.

[0137] The data write request is, for example, the PrWrNodata request or the PrWr request, and the first-level write-back request is, for example, the ReducePM request. The PrWrNodata request, the PrWr request, and the ReducePM request are not described in detail here.

[0138] In some embodiments, the first address range in the memory of the first processor is used as the storage space for the calculation results of the target layer, and the memory address carried by the data write request corresponds to the first address range. The embodiment of the present application does not limit the first address range.

[0139] It should be noted that step 502 is an implementation method of sending a write-back request for n data blocks to the multi-level storage every time n data blocks in the first intermediate result are calculated. In some embodiments, this process is also implemented based on other methods, and the embodiments of the present application do not limit this.

[0140] 503. While the processor core is calculating, the second agent responds to the data write request in parallel, writes the i-th data block in the first intermediate result to the first cache line in the cache of the first processor, and sets the state of the first cache line to a pending reduction state, where the pending reduction state indicates that the data in the first cache line is valid and has not yet been reduced.

[0141] The second agent responds to the data write request, determines a first cache line in the cache of the first processor based on the first address carried in the data write request, and writes the i-th data block in the first intermediate calculation result into the first cache line.

[0142] In some embodiments, there is a cache line corresponding to the first address in the cache of the first processor, and the cache line is the first cache line. The second agent writes the i-th data block in the first intermediate result carried by the data write request into the first cache line.

[0143] In other embodiments, there is no cache line corresponding to the first address in the cache of the first processor, and the cache of the first processor is not full, then the second agent determines any cache line from the cache lines with a status of I in the cache, and the determined cache line is the first cache line. The second agent writes the i-th data block in the first intermediate result into the first cache line, and after writing, changes the status of the first cache line from I to PM.

[0144] In some further embodiments, there is no cache line corresponding to the first address in the cache of the first processor, and the cache of the first processor is full, then the second agent determines the first cache line from the cache; if the state of the first cache line is I or S, then the second agent directly writes the i-th data block in the first intermediate result into the first cache line, and after writing, modifies the state of the first cache line to PM; if the state of the first cache line is PM, then the second agent waits until the state of the first cache line is I, and then writes the i-th data block in the first intermediate result into the first cache line, and after writing, modifies the state of the first cache line from I to PM. Among them, the method for the second agent to determine the first cache line from the cache includes: least recently used algorithm (LRU), first in first out algorithm (FIFO) and random replacement algorithm (RAND, rand), etc., among which the least recently used algorithm is used to determine the cache line that has been least accessed in the recent period as the first cache line; the first in first out algorithm is used to determine the cache line with the longest existence time as the first cache line; the random replacement algorithm is used to determine the cache line determined by a random number as the first cache line. It should be noted that the above three replacement algorithms are only exemplary and can be set according to actual needs. The embodiments of this application do not limit this.

[0145] It should be noted that the above steps 502 and 503 are an implementation method of sending a write-back request to the multi-level storage after calculating the i-th data block in the first intermediate result. The write-back request instructs the multi-level storage to perform a reduce operation on the data in the multiple cache lines corresponding to the first address in the multi-level storage step by step, and write the final result of the reduce operation to the first address. In some embodiments, this process is also implemented based on other methods, which is not limited in the embodiments of the present application.

[0146] 504. The second agent sends a second-level write-back request to the first agent in the first processor in response to the first-level write-back request.

[0147] The first-level write-back request carries the i-th data block, memory address, and reduce operation type in the first intermediate result. The reduce operation type can be INT8, FP16, and FP32 addition, etc. The embodiment of the present application does not limit the reduce operation type. The second-level write-back request carries the first address and the reduce operation type. In response to the first-level write-back request, the second agent sends a second-level write-back request to the first agent in the first processor based on the first address and the reduce operation type.

[0148] The second-level write-back request is, for example, the ReduceWb request mentioned above, and the ReduceWb request will not be described in detail here.

[0149] 505. The first agent in the first processor responds to the second-level write-back request, obtains the i-th data block in the first intermediate result from the cache line corresponding to the first address in the cache of the first processor, obtains the i-th data block in the second intermediate result of the second parallel computing task from the cache line corresponding to the first address in the cache of the second processor, performs a reduce operation on the i-th data block in the first intermediate result and the i-th data block in the second intermediate result to obtain a result of the reduce operation, and writes the result of the reduce operation to the first address in the memory of the first processor. The second parallel computing task is used to implement the calculation of another part of the parameters of the target layer.

[0150] Among them, the first agent in the first processor is pre-configured with information about other processors in the distributed cluster, so that the first agent can send data acquisition requests to other processors. The i-th data block in the first intermediate result and the i-th data block in the second intermediate result both correspond to the first address in the first processor. The first agent in the first processor responds to the second-level write-back request and obtains data blocks from the cache lines corresponding to the first address in the cache of other processors in the distributed cluster. The obtained data blocks are also the i-th data blocks in the intermediate results of other parallel computing tasks in the target computing task. Then, the obtained data blocks are reduced with the i-th data block in the first intermediate result to obtain the result of the reduce operation. The result of the reduce operation is also the i-th data block in the computing result of the target layer. The result of the reduce operation is written to the memory of the first processor. The process includes the following steps 5051 to 5053.

[0151] 5051. A first agent in a first processor sends a data acquisition request for the first address to a second agent in a second processor. The data acquisition request is used to acquire a data block from a cache line corresponding to the first address in a cache of the second processor.

[0152] The data acquisition request carries the first address. The data acquisition request is, for example, the ReducePull request, which will not be described in detail here.

[0153] The second agent in the second processor responds to the data acquisition request and determines, based on the first address, a cache line corresponding to the first address from the cache; if the state of the cache line is a pending reduction state, the second agent in the second processor obtains data from the cache line to obtain a data block corresponding to the first address.

[0154] 5052. The first agent in the first processor receives the data block in the cache line corresponding to the first address returned by the second agent of the second processor to obtain the i-th data block in the first intermediate result.

[0155] 5053. The first agent in the first processor performs a reduce operation on the i-th data block in the second intermediate result and the i-th data block in the first intermediate result to obtain a result of the reduce operation, and writes the result of the reduce operation to the first address in the first processor.

[0156] The first agent in the first processor performs the reduce operation specified in the second-level writeback request on the i-th data block in the second intermediate result and the i-th data block in the first intermediate calculation result. For example, if the reduction operation type carried in the second-level writeback request is 32-bit single-precision floating-point number (FP32) addition, the first agent in the first processor performs the FP32 addition operation on the i-th data block in the second intermediate result and the i-th data block in the first intermediate result to obtain the result of the reduce operation, which is the i-th data block in the calculation result of the target layer.

[0157] In some embodiments, the first agent in the first processor responds to the second-level write-back request, first writes the i-th data block in the first intermediate result carried by the second-level write-back request to the first memory address based on the first address carried by the second-level write-back request, and then sends a data acquisition request for the first address to the second agent of the second processor to obtain the i-th data block in the second intermediate result from the cache of the second processor; after the first agent in the first processor obtains the i-th data block in the second intermediate result, it reads the i-th data block in the first intermediate result from the first address in the memory of the first processor, performs a reduce operation on the i-th data block in the first intermediate result and the i-th data block in the second intermediate result to obtain the result of the reduce operation; the first agent of the first processor writes the result of the reduce operation to the first address in the memory of the first processor.

[0158] It should be noted that the above step 505 is described using the example of a distributed cluster including two processors (a first processor and a second processor). In some embodiments, the distributed cluster includes three or more processors. In response to the second-level write-back request, the first agent in the first processor obtains the data in the cache line corresponding to the first address from the caches of the other processors except the first processor, obtains the i-th data block of the intermediate results of the other parallel computing tasks of the target computing task, and performs a reduce operation on the multiple obtained data blocks and the i-th data block in the first intermediate result to obtain the result of the reduce operation. In some embodiments, the first agent in the first processor performs a reduce operation each time it obtains a data block corresponding to the first address from the caches of the other processors, until the data in the multiple cache lines corresponding to the first address are completed. For example, a distributed cluster includes four processors, namely processor 1, processor 2, processor 3 and processor 4. In response to the second-level write-back request, processor 1 obtains data in multiple cache lines corresponding to the first address from the caches of processors 2, 3 and 4 respectively; after processor 1 obtains the i-th data block in the second intermediate result from the cache of processor 2, it reduces the i-th data block in the second intermediate result with the i-th data block in the first intermediate result to obtain a first intermediate reduced result; after processor 1 obtains the i-th data block in the third intermediate result from the cache of processor 3, it reduces the i-th data block in the third intermediate result with the first intermediate reduced result to obtain a second intermediate reduced result; after processor 1 obtains the i-th data block in the fourth intermediate result from the cache of processor 4, it reduces the i-th data block in the fourth intermediate result with the second intermediate reduced result to obtain the result of the reduce operation, which is the i-th data block in the calculation result of the target layer.

[0159] It should be noted that the above steps 504 and 505 are an implementation method in which, when k is equal to 1, the storage unit in the first-level storage responds to the first-level write-back request sent by the processor core and sends a second-level write-back request to the storage unit in the second-level storage; when k is greater than or equal to 2, after the storage unit in the k-th level storage completes the reduce operation indicated by the k-th level write-back request, the storage unit in the k-th level storage sends a k+1-th level write-back request to the storage unit in the k+1-th level storage. In some embodiments, this process is also implemented based on other methods, which is not limited to the embodiments of the present application.

[0160] 506. The second agent sets the state of the first cache line to an invalid state, where the invalid state indicates that the data in the first cache line is invalid.

[0161] It should be noted that the above step 506 is an optional step. In some embodiments, step 506 is not performed.

[0162] It should be noted that step 506 is an implementation method in which the k-1 level storage sets the state of the cache line corresponding to the first address in the k-1 level storage to an invalid state. In some embodiments, this process is also implemented based on other methods, and the embodiments of the present application do not limit this.

[0163] The process shown in steps 501 to 506 above is illustrated below with reference to Figure 6. Figure 6 is a flow chart of the data writing and reduce process in a data reading and writing method based on multi-level storage provided in an embodiment of the present application. As shown in Figure 6, the processor core sequentially calculates cl1, cl2, cl3, ... cln (cli corresponds to the i-th data block in the first intermediate result). After the processor core calculates cl1, it communicates with the first and second agents. Specifically, the processor core sends a data write request for cl1 to write cl1 to a cache line in the cache and changes the state of the cache line to PM (the grid fill in FIG6 indicates that the data block has not yet been reduced). After the processor core sends the data write request, it sends a first-level write-back request for cl1 to reduce cl1 step by step. While the first and second agents perform the reduce operation on cl1, the processor core calculates cl2. After cl1 is reduced, the state of the cache line corresponding to cl1 is I (black fill in FIG6 indicates that the reduce operation has been performed), indicating that the data in the cache line has been reduced, the result of the reduce operation has been written to the memory, and the data in the cache line is invalid. The same applies to cl2, cl3, ..., cln, and cl1, and will not be described in detail.

[0164] In the above method, during model training, the calculation process and the reduction process of the intermediate result of the target layer in the model are respectively executed by the processor core and the first agent in the processor. The processor core and the first agent can execute the above two processes in parallel, so the reduction process does not have to wait for the calculation process to complete before starting. In other words, the calculation process and the reduction process overlap in time, thereby reducing the overall time required for model training. In addition, while the first agent executes the reduction process for the previous data block, the processor core can execute the calculation process for the next data block in parallel, avoiding idle computing power of the processor core, thereby improving the utilization of the processor's computing power. In addition, by ensuring that the data size of the data block in the first intermediate result is equal to an integer multiple of the capacity of the cache line in the cache, when the data block is written to the cache, each data block can exactly fill an integer number of cache lines, thereby avoiding the problem of cache space waste caused by the misalignment of the data size of the data block and the cache line capacity. In addition, cross-device data reduction is completed by the hardware in the processor (the first agent and the second agent), and the reduce process does not require software configuration and initiation, which can reduce software stack overhead and reduce data synchronization time.

[0165] It should be noted that the above steps 501 to 506 are described using the example of the processor core of the first processor sending a data write request and a write-back request for each data block calculated to the second agent in the first processor. In other embodiments, the processor core sends a data write request for each data block calculated to the second agent, and sends write-back requests for the target number of data blocks to the second agent whenever the number of calculated data blocks reaches a target number. This process is described below with reference to FIG7 .

[0166] Figure 7 is a flowchart of the data writing and reducing process in a data reading and writing method based on multi-level storage provided by an embodiment of the present application. As shown in Figure 7, the method is applied to a first processor in a distributed cluster, and the distributed cluster includes multiple processors. The multiple processors can access each other's memory. The first processor includes a processor core, a first agent, a second agent, a memory and a cache. The first agent in the first processor is used to receive requests for memory and perform read and write operations on the memory of the first processor. The second agent in the first processor is used to receive requests for cache and perform read and write operations on the cache of the first processor. The second agent and cache in the first processor constitute a storage unit in the first level storage in the multi-level storage, and the first agent and memory in the first processor constitute a storage unit in the second level storage in the multi-level storage. The method includes the following steps 701 to 707.

[0167] 701. The processor core calculates an i-th data block in a first intermediate result of a first parallel computing task, where i is an integer greater than or equal to 1, and the i-th data block in the first intermediate result corresponds to a first address.

[0168] The step 701 is similar to the above step 501 and will not be described in detail.

[0169] 702. After calculating the i-th data block in the first intermediate result, the processor core sends a data write request to the second agent. After sending the data write request, the processor core calculates the i+1-th data block in the first intermediate result.

[0170] Among them, the process of the processor core sending a data write request to the second agent is the same as the process of the processor core sending a data write request to the second agent in the above step 502. The difference is that in step 702, after the processor core sends the data write request, it calculates the i+1th data block in the first intermediate result. The similarities are not repeated here.

[0171] 703. While the processor core is calculating, the second agent responds to the data write request in parallel, writes the i-th data block in the first intermediate result to the first cache line in the cache of the first processor, and sets the state of the first cache line to a to-be-reduced state.

[0172] The step 703 is similar to the above step 503 and will not be described in detail.

[0173] Continuing with the example in step 502 above, and illustrating steps 701 to 703 above in conjunction with Figures 8 and 9, Figure 8 is a flow chart of the data writing and reduce process in a data reading and writing method based on multi-level storage provided by an embodiment of the present application, and Figure 9 is a flow chart of the data writing and reduce process in a data reading and writing method based on multi-level storage provided by an embodiment of the present application. As shown in step 0 in Figure 8, in the initial state, the state of each cache line in the cache of the first processor is 1; as shown in Figure 9, the processor core first calculates Z1, and after calculating Z1, the processor core sends a data write request (PrWrNodata) to CA1, and the data write request carries Z1 and memory address 0x0. In response to the data write request, CA1 writes Z1 to a cache line in the cache and sets the state of the cache line to PM (corresponding to step 1 in Figure 8); when CA1 writes data, the processor core continues to calculate Z2.

[0174] 704. Every time the processor core calculates and obtains the target number of data blocks in the first intermediate result, it sends a first-level write-back request for the target number of data blocks to the second agent. After sending the first-level write-back request, the processor core calculates the data blocks that are not obtained in the first intermediate result.

[0175] The first-level write-back request carries the first address, the number of cache lines targeted (i.e., the target number), and the reduce operation type. The target number can be set according to actual needs, and the embodiment of the application does not limit the target number.

[0176] Continuing to use Figures 8 and 9 as an example to illustrate step 704. As shown in Figure 8, after calculating Z2, the processor core sends a data write request (PrWrNodata) to CA1. The data write request carries Z2 and the memory address 0x80. In response to the data write request, CA1 writes Z2 to a cache line in the cache and sets the state of the cache line to PM (corresponding to step 2 in Figure 8). After the processor core calculates two data blocks (Z1 and Z2), it sends a first-level write-back request (ReducePM) for Z1 and Z2 to CA1. The first-level write-back request carries Z1, Z2, the starting address 0x0 corresponding to the write-back request, the number of cache lines 2 (len=2) targeted, and the type of reduce operation (corresponding to step 3 (1) in Figure 8); after sending the first-level write-back request, the processor core continues to calculate Z3.

[0177] 705. While the processor core is calculating, the second agent responds to the first-level write-back request in parallel and sends second-level write-back requests for the target number of data blocks to the first agent in the first processor, respectively, wherein the first agent of the first processor performs a reduce operation on the data in the multiple cache lines corresponding to the first address in the cache for the second-level write-back request for the i-th data block in the first intermediate result, and writes the result of the reduce operation to the first address in the memory of the first processor.

[0178] The second agent sending the second-level write-back request for the target number of data blocks to the first agent in the first processor respectively means that the second agent sends multiple second-level write-back requests to the first agent in the first processor for the multiple cache lines where the target number of data blocks are located. For example, the first-level write-back request is for the i-th data block to the i+x-th data block (x is a positive integer) in the first intermediate result, and these x data blocks are written to x cache lines in the cache of the first processor. Then, the second agent sends the second-level write-back request for these x cache lines to the first agent in the first processor respectively, so as to perform reduce operations on the i-th data block, the i+1-th data block, ... the i+x-th data block in the first intermediate calculation result, and write the results of the reduce operations to the memory of the first processor respectively.

[0179] 8 and 9 are used as examples to further illustrate step 705. As shown in FIG9 , in response to the first-level write-back requests (ReducePM) for Z1 and Z2, CA1 sends second-level write-back requests (ReduceWb) for the cache lines where Z1 and Z2 are located to HA, respectively. The second-level write-back request for the cache line where Z1 is located carries Z1 and memory address 0x0 (corresponding to step 3(2) in FIG8 ), and the second-level write-back request for the cache line where Z2 is located carries Z2 and memory address 0x80.

[0180] It should be noted that step 705 is an implementation method of sending a write-back request for n data blocks to the multi-level storage every time n data blocks in the first intermediate result are calculated. In some embodiments, this process is also implemented based on other methods, and the embodiments of the present application do not limit this.

[0181] 706. In response to the second-level write-back request for the cache line where the i-th data block in the first intermediate result is located, the first agent obtains the i-th data block in the first intermediate result from the cache line corresponding to the first address in the cache of the first processor, obtains the i-th data block in the second intermediate result of the second parallel computing task from the cache line corresponding to the first address in the cache of the second processor, performs a reduce operation on the i-th data block in the first intermediate result and the i-th data block in the second intermediate result, obtains a result of the reduce operation, and writes the result of the reduce operation to the first address in the memory of the first processor.

[0182] Step 706 is similar to step 507 and will not be described in detail.

[0183] 8 and 9 are used as examples to illustrate step 706. As shown in FIG9 , HA1 responds to the second-level write-back request (ReduceWb) for Z1 and Z2 and performs a reduce operation on Z1 and Z2 respectively. As shown in step 3 (3) in FIG8 , HA1 responds to the second-level write-back request for Z1 and first writes Z1 to memory address 0x0; HA1 obtains the data block corresponding to 0x0 from the cache of the second processor, HA1 reads Z1 from memory address 0x0 (corresponding to sub-step i in step 3 (3) in FIG8 ), performs a reduce operation on Z1 and the data block obtained from the cache of the second processor (corresponding to sub-step ii in step 3 (3) in FIG8 ), and writes the result of the reduce operation to memory address 0x0 (corresponding to sub-step iii in step 3 (3) in FIG8 ). The process of HA1 performing a reduce operation on Z2 is the same as the process of performing a reduce operation on Z1, and will not be repeated. As shown in Figure 9, while HA1 performs the reduce operation on Z1 and Z2, CA1 writes Z3 and Z4 calculated by the processor core. The subsequent process is the same as the processing of Z1 and Z2, which will not be repeated here.

[0184] 707 . The second agent sets the state of the first cache line to an invalid state, where the invalid state indicates that the data in the first cache line is invalid.

[0185] 8 and 9 are used as examples to illustrate step 706. As shown in step 4 in FIG8, after the results of the reduce operations corresponding to Z1 and Z2 are written, the states of the cache lines corresponding to Z1 and Z2 in the CA1 cache are set to 1.

[0186] In the above method, during the model training process, the calculation process and the reduce process of the intermediate calculation results of the target layer in the model are executed by the processor core and the first agent in the processor respectively, and the processor core and the first agent can execute the above two processes in parallel, so the reduce process does not have to wait for the calculation process to be completed before it starts to be executed, that is, the calculation process and the reduce process overlap in time, thereby reducing the overall time of model training, and while the first agent executes the reduce process for the previous data block, the processor core can execute the calculation process for the next data block in parallel, avoiding the idle computing power of the processor core, thereby improving the utilization rate of the processor's computing power; in addition, by making the data volume of the data block in the first intermediate result equal to an integer multiple of the capacity of the cache line in the cache, When data blocks are written into the cache, each data block can just fill an integer number of cache lines, thereby avoiding the problem of cache space waste caused by the misalignment of the data size of the data block and the capacity of the cache line; in addition, cross-device data reduction is completed through the hardware in the processor (the first agent and the second agent), and the reduce process does not require software configuration and initiation, which can reduce software stack overhead and reduce data synchronization time; further, after the processor core calculates multiple data blocks, it sends a write-back request for these multiple data blocks to the second agent to instruct the reduce operation to be performed on these multiple data blocks and write the results of the reduce operation to the memory, which can reduce the number of times the processor core sends requests, so that the computing power of the processor core can be used more for calculation, improve the utilization of the processor's computing power, and further reduce the overall time of model training.

[0187] It should be noted that the embodiments shown in Figures 5 and 7 above are described using the example of a first processor obtaining the target layer's calculation results and storing each data block in the calculation results in the first processor's memory. In some embodiments, the first processor obtains a portion of the target layer's calculation results, and the second processor obtains another portion of the target layer's calculation results, with each processor storing the obtained data blocks in its respective memory. For example, the first processor obtains data blocks 1 to N-1 of the target layer's calculation results (i.e., in the embodiments corresponding to Figures 5 and 7 above, the value of i is less than N), and the second processor obtains data blocks N and subsequent data blocks in the target layer's calculation results. When the first processor calculates the mth data block (m is an integer greater than or equal to N) in the first intermediate result, the first processor first writes the data block to the first processor's cache. When the second processor sends a data acquisition request to the first processor, the first processor sends the data block to the second processor, which then reduces the data block to obtain the mth data block in the target layer's calculation results. This process is described below.

[0188] Figure 10 is a flowchart of the data writing and reducing process in a data reading and writing method based on multi-level storage provided by an embodiment of the present application. As shown in Figure 10, the method is applied to a distributed cluster, which includes multiple processors. The multiple processors can access each other's memory. The first processor includes a processor core, a first agent, a second agent, a memory and a cache. The first agent in the first processor is used to receive requests for memory and perform read and write operations on the memory of the first processor. The second agent in the first processor is used to receive requests for cache and perform read and write operations on the cache of the first processor. The second agent and cache in the first processor constitute a storage unit in the first level storage in the multi-level storage, and the first agent and memory in the first processor constitute a storage unit in the second level storage in the multi-level storage. The method includes the following steps 1001 to 1006.

[0189] 1001. The processor core calculates the mth data block in the first intermediate result of the first parallel computing task, where m is an integer greater than or equal to N.

[0190] In some embodiments, the first address range in the memory of the first processor is used as the storage space for some data blocks in the calculation result of the target layer, and the second address range in the memory of the second processor is used as the storage space for another part of the data blocks in the calculation result of the target layer. For any data block in the first intermediate result calculated by the processor core, if the memory address corresponding to the data block corresponds to the first address range, the first processor performs a reduce operation on the data block and stores the result of the reduce operation; if the memory address corresponding to the data block corresponds to the second address range, the second processor performs a reduce operation on the data block and stores the result of the reduce operation. Continuing with the example in step 502 above, the memory capacity of the first processor and the second processor are both 4 gigabytes (GB), the memory address range of the first processor is 0x0 to 0xFFFFFFFF, and the memory address range of the second processor is 0x100000000 to 0x1FFFFFFFF, and the memory addresses of the two processors are continuous. The calculation result of the target layer has a size of 1024×1024, and each data is a 2-byte half-precision floating-point number. Therefore, the data volume corresponding to the calculation result of the target layer is 2 megabytes (MB). The first processor and the second processor respectively store 1MB of data in the calculation result of the target layer. The first address range (0x0 to 0xFFFFF, with a capacity of 1MB) in the first processor is used as the storage space for some data blocks in the calculation result of the target layer, and the second address range (0x100000000 to 0x1000FFFFF, with a capacity of 1MB) in the second processor is used as the storage space for another part of the data blocks in the calculation result of the target layer.

[0191] Among them, the i-th data block (i is an integer greater than 0 and less than N) in the first intermediate result corresponds to the first address range in the first processor, and the m-th data block (m is greater than N) in the first intermediate calculation result corresponds to the second address range in the second processor.

[0192] The process of the processor core calculating the mth data block in the first intermediate calculation result is the same as the process of calculating the ith data block in the first intermediate result in step 501 above, and will not be repeated here.

[0193] 1002. After calculating the mth data block in the first intermediate result, the processor core sends a data write request to the second agent. After sending the data write request, the processor core calculates the m+1th data block in the first intermediate result. The data write request indicates that the mth data block in the first intermediate result is written into the cache of the first processor.

[0194] 1003. While the processor core is calculating, the second agent responds to the data write request in parallel, writes the mth data block in the first intermediate result into the first cache line in the cache of the first processor, and sets the state of the first cache line to a to-be-reduced state.

[0195] The above steps 1002 and 1003 are similar to the above steps 702 and 703 and are not described in detail.

[0196] 1004. In response to the data acquisition request of the first agent in the second processor for the mth data block in the first intermediate result, the second agent sends the mth data block in the first intermediate result to the first agent in the second processor.

[0197] In response to the data acquisition request, the second agent retrieves data from the cache line corresponding to the first address in the cache of the first processor based on the first address carried in the data acquisition request. The data is the mth data block in the first intermediate result. The second agent sends the retrieved data to the first agent in the second processor. In response to the data acquisition request, the second agent determines the cache line corresponding to the first address in the cache of the first processor based on the first address. If the cache line is in a pending reduction state, the second agent retrieves data from the cache line to obtain the mth data block in the first intermediate result.

[0198] 1005. The first agent in the second processor performs a reduce operation on the mth data block in the first intermediate result and the mth data block in the second intermediate result to obtain a result of the reduce operation, and writes the result of the reduce operation to a first address in the memory of the second processor.

[0199] 1006. The second agent sets the state of the first cache line to an invalid state.

[0200] Step 1005 and step 1006 are similar to the above-mentioned steps 505 and 506 and are not described in detail.

[0201] The above embodiments shown in Figures 5, 7 and 10 introduce the data writing and reduce processes in a data reading and writing method based on multi-level storage provided by an embodiment of the present application. The following introduces the data reading process in a data reading and writing method based on multi-level storage provided by an embodiment of the present application.

[0202] Figure 11 is a flowchart of a data reading process in a data reading and writing method based on multi-level storage provided by an embodiment of the present application. As shown in Figure 11, the method is applied to a first processor in a distributed cluster, and the distributed cluster includes multiple processors. The multiple processors can access each other's memory. The first processor includes a processor core, a first agent, a second agent, a memory and a cache. The first agent in the first processor is used to receive requests for memory and perform read and write operations on the memory of the first processor. The second agent in the first processor is used to receive requests for cache and perform read and write operations on the cache of the first processor. The second agent and cache in the first processor constitute a storage unit in the first level storage in the multi-level storage, and the first agent and memory in the first processor constitute a storage unit in the second level storage in the multi-level storage. The method includes the following steps 1101 to 1105.

[0203] 1101. After the processor core calculates the last data block in the first intermediate result, the processor core sends a read request for the second address to the second agent. The second address is the memory address corresponding to the j-th data block in the calculation result of the target layer, and j is an integer greater than 0.

[0204] 1102. In response to the read request, the second agent sends a read request for the second address to the destination first agent to obtain the jth data block in the calculation result of the target layer. The destination first agent is the first agent in the processor corresponding to the second address among multiple processors.

[0205] Wherein, when the second address corresponds to the memory of the first processor, the second agent responds to the read request and sends a read request for the second address to the first agent in the first processor to obtain the j-th data block in the calculation result of the target layer; when the second address corresponds to the memory of the second processor, the second agent responds to the read request and sends a read request for the second address to the first agent in the second processor to obtain the j-th data block in the calculation result of the target layer.

[0206] In some embodiments, when the second address corresponds to a memory of the first processor and there is no cache line corresponding to the second address in the cache of the first processor, the second agent sends a read request for the second address to the first agent in the first processor, and receives a data block at the second address returned by the first agent in the first processor. In response to the read request, the first agent in the first processor reads the data block at the second address from the memory of the first processor and returns the data block to the second agent.

[0207] In other embodiments, when the second address corresponds to the memory of the first processor, and a cache line corresponding to the second address exists in the cache of the first processor and the cache line is in a pending-reduce state, the second agent waits for the cache line to be in an invalid state, sends a read request for the second address to the first agent in the first processor, and receives the data block at the second address returned by the first agent in the first processor. The pending-reduce state of the cache line indicates that the data in the cache line has not yet been reduced and cannot be read via a data read request (PrRd); the invalid state of the cache line indicates that the data in the cache line has been reduced and written to the memory of the first processor.

[0208] In some embodiments, after the second agent sends a read request for the second address to the destination first agent to obtain the j-th data block in the calculation result of the target layer, the second agent writes the j-th data block in the calculation result of the target layer into the second cache line in the cache of the first processor, and sets the state of the second cache line to a shared state, which indicates that the data in the second cache line is valid and supports reading through the data read request. In the above embodiment, after the second agent reads the data block from the second address, it writes the data block into the cache line in the cache and sets the state of the cache line to a shared state, so that the data block can be directly obtained from the cache the next time it is read, without having to read the memory through the first agent, thereby speeding up the reading efficiency of the data block.

[0209] It should be noted that the above step 1102 is explained by taking the j-th data block in the calculation result of the target layer as an example that the j-th data block in the calculation result of the target layer does not exist in the cache of the first processor. In some embodiments, the second address corresponds to the memory of the first processor, and the cache line corresponding to the second address exists in the memory of the first processor and the state of the cache line is a shared state (indicating that the j-th data block in the calculation result of the target layer exists in the cache of the first processor). Then, the second agent directly reads the j-th data block in the calculation result of the target layer from the cache line without sending a read request for the first address to the destination first agent.

[0210] 1103. The second agent sends the j-th data block in the calculation result of the target layer to the processor core.

[0211] 1104. The processor core sends a read request for the third address to the second agent. The third address is the memory address corresponding to the j+1th data block in the calculation result of the target layer. After the processor core sends the read request for the third address, it executes the parallel computing task of the next layer of the target layer based on the jth data block in the calculation result of the target layer.

[0212] 1105. While the processor core is calculating, the second agent reads the j+1th data block in the calculation result of the target layer from the memory of multiple processors in parallel.

[0213] 8 , and in conjunction with FIG12 and FIG13 , illustrate the process shown in steps 1101 to 1105. FIG12 is a flow chart illustrating a data reading process in a multi-level storage-based data reading and writing method provided in an embodiment of the present application. As shown in FIG12 , the second agent reads cl1, and the processor core performs calculations based on cl1. While the processor core performs calculations based on cl1, the second agent reads cl2. The reading process from cl2 to cln is similar to the reading process of cl1 and will not be further described. FIG13 is a flow chart of a data reading process in a data reading and writing method based on multi-level storage provided by an embodiment of the present application. As shown in step 1 in FIG13 , the processor core sends a data read request (PrRd) to CA1, and the data read request carries the second address 0x0; CA1 responds to the read request and determines that the second address 0x0 carried by the read request corresponds to the memory of the first processor, and there is a cache line corresponding to the second address 0x0 in the cache of the processor, and the state of the cache line is PM; as shown in step 2 (1) in FIG13 , CA1 saves the data read request until the cache line corresponding to the second address 0x0 is received. The reduce operation of the data in the cache line is completed; as shown in step 2 (2) in Figure 13, when the reduce operation of the data in the cache line corresponding to the second address 0x0 is completed, CA1 changes the state of the cache line corresponding to the second address 0x0 to I; as shown in step 3 in Figure 13, CA1 sends a data read request to HA1 in the first processor to obtain the data in the second address 0x0, and CA1 receives the data returned by HA1; as shown in step 4 in Figure 13, after CA1 reads back the data in 0x0, it writes the data to the cache line corresponding to 0x0 in the cache and changes the state of the cache line to S.

[0214] In the above method, during the model training process, the calculation process and the reduce process of the intermediate calculation results of the target layer in the model are executed by the processor core and the first agent in the processor respectively, and the processor core and the first agent can execute the above two processes in parallel, so the reduce process does not have to wait for the calculation process to be completed before starting to execute, that is, the calculation process and the reduce process overlap in time, thereby reducing the overall time of model training. Moreover, while the first agent executes the reduce process for the previous data block, the processor core can execute the calculation process for the next data block in parallel, avoiding the idle computing power of the processor core, thereby improving the utilization rate of the processor's computing power; further, after the processor core of the first processor completes the calculation of the last data block in the first intermediate calculation result, the processor core can perform a read operation on the calculation result of the target layer to start executing the parallel computing task in the next layer of the target layer without waiting for the calculation of the entire calculation result of the target layer to be completed, which can further reduce the overall time of model training and improve the utilization rate of the processor's computing power. As shown in Figure 14, Figure 14 is a flow chart of a data reading and writing method based on multi-level storage provided by an embodiment of the present application. In Figure 14, for the parallel computing task of the Lth layer in the model, the processor i outputs the intermediate calculation result Yi cache line by cache line. While the processor i outputs the intermediate calculation result Yi, Yi is synchronized cache line by cache line, and finally the calculation result Y of the Lth layer is obtained. After the intermediate calculation result Yi is calculated, while determining the calculation result Y, the determined data blocks in the calculation result Y can be read cache line by cache line, thereby executing the parallel computing task of the L+1 layer.

[0215] Figure 15 is a structural diagram of a data reading and writing device based on multi-level storage provided in an embodiment of the present application. It is applied to multi-level storage, and each level of storage includes at least one independent storage unit. The device includes a sending module 1501 and a reduction module 1502.

[0216] The sending module 1501 is configured to send a k+1th level write-back request to a k+1th level storage in the multi-level storage through the kth level storage in the multi-level storage, where the k+1th level write-back request carries a first address, and k is an integer greater than or equal to 1;

[0217] The reduction module 1502 is used to respond to the k+1th level write back request, perform a reduce operation on the data in the multiple cache lines corresponding to the first address in the kth level storage through the k+1th level storage, and write the result of the reduce operation into the k+1th level storage, where different cache lines in the multiple cache lines correspond to different storage units in the kth level storage.

[0218] Optionally, the sending module 1501 is configured to:

[0219] When k is equal to 1, in response to the first-level write-back request sent by the processor core, sending a second-level write-back request to the storage unit in the second-level storage through the storage unit in the first-level storage, where the first-level write-back request carries the first address;

[0220] When k is greater than or equal to 2, after the storage unit in the k-th level storage completes the reduce operation indicated by the k-th level write back request, the k+1-th level write back request is sent to the storage unit in the k+1-th level storage through the storage unit in the k-th level storage.

[0221] Optionally, the device further comprises:

[0222] a determining module configured to determine, in response to a write request to the k-th level storage, from storage cells in the k-th level storage, a cache line corresponding to the first address, the write request carrying the first address;

[0223] Write modules for:

[0224] If the state of the cache line corresponding to the first address is an invalid state or a shared state, writing the to-be-written data corresponding to the write request into the cache line corresponding to the first address through a storage unit in the k-th level storage, and setting the state of the cache line corresponding to the first address to a to-be-reduced state, wherein the invalid state indicates that the data in the cache line is invalid, the shared state indicates that the data in the cache line is valid and supports reading, and the to-be-reduced state indicates that the data in the cache line is valid and has not yet been reduced;

[0225] If the state of the cache line corresponding to the first address is the to-be-reduced state, the to-be-written data corresponding to the write request is written into the cache line corresponding to the first address through the storage unit in the k-th level storage.

[0226] Optionally, the determining module is configured to:

[0227] When k is equal to 1, in response to a data write request sent by the processor core, determining, through a storage unit in the first-level storage, a cache line corresponding to the first address from the storage unit in the first-level storage, and the data to be written corresponding to the data write request is the data carried by the data write request;

[0228] When k is greater than or equal to 2, in response to the k-th level write back request sent by the storage unit in the k-1-th level storage, the cache line corresponding to the first address is determined from the storage unit in the k-th level storage through the storage unit in the k-th level storage, and the data to be written corresponding to the k-th level write back request is the result of the reduce operation indicated by the k-th level write back request.

[0229] Optionally, when k is greater than or equal to 2, the apparatus further comprises:

[0230] The setting module is configured to set the state of the cache line corresponding to the first address in the k-1th level storage to an invalid state through the k-1th level storage.

[0231] Optionally, the device further comprises:

[0232] Read module for:

[0233] receiving, through the storage unit in the k-th level storage, a read request for a storage unit in the k-th level storage, the read request carrying a second address;

[0234] If a cache line corresponding to the second address exists in the storage unit in the k-th level storage, and the state of the cache line corresponding to the second address is a shared state, reading data from the cache line corresponding to the second address through the storage unit in the k-th level storage, and returning the read data to the processor core;

[0235] If a cache line corresponding to the second address exists in the storage unit in the k-th level storage, and the state of the cache line corresponding to the second address is in a pending reduction state, then when the state of the cache line corresponding to the second address becomes an invalid state through the storage unit in the k-th level storage, a read request is sent to the storage unit in the k+1-th level storage based on the second address to read the data corresponding to the second address, and the read data is returned to the processor core;

[0236] If there is no cache line corresponding to the second address in the storage unit in the k-th level storage, a read request is sent to the storage unit in the k+1-th level storage based on the second address to read the data corresponding to the second address, and the read data is returned to the processor core.

[0237] Optionally, the setting module is further configured to:

[0238] If the storage unit in the k-th level storage reads the data corresponding to the second address from the storage unit in the k+1-th level storage, the storage unit in the k-th level storage determines the target cache line from the storage unit in the k-th level storage, writes the read data into the target cache line, and sets the state of the target cache line to a shared state.

[0239] It should be noted that, in other embodiments, the steps that the above modules are responsible for implementing can be specified as needed, and the above modules can respectively implement different steps in the above-mentioned data reading and writing method based on multi-level storage to realize all the functions of the above-mentioned device. That is, the data reading and writing device based on multi-level storage provided in the above-mentioned embodiment is only illustrated by the division of the above-mentioned functional modules when implementing the data reading and writing method based on multi-level storage. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device provided in the above-mentioned embodiment and the corresponding method embodiment belong to the same concept, and its specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0240] FIG16 is a schematic structural diagram of a data reading and writing device based on multi-level storage provided in an embodiment of the present application, which is applied to a processor core. The device includes a computing module 1601 and a sending module 1602 .

[0241] The computing module 1602 is configured to compute an i-th data block in a first intermediate result of a first parallel computing task, where i is an integer greater than or equal to 1, and the i-th data block in the first intermediate result corresponds to a first address. The first parallel computing task is configured to compute a portion of the computing results in the target computing task.

[0242] The sending module 1602 is configured to, after calculating the i-th data block in the first intermediate result, send a write-back request to the multi-level storage, wherein the write-back request instructs the multi-level storage to perform a reduce operation on the data in the plurality of cache lines corresponding to the first address in the multi-level storage, level by level, and write a final result of the reduce operation to the first address;

[0243] The calculation module 1602 is further configured to calculate the (i+1)th data block in the first intermediate result after sending the write-back request.

[0244] Optionally, the sending module 1602 is configured to:

[0245] Whenever n data blocks in the first intermediate result are calculated, a write-back request for the n data blocks is sent to the multi-level storage, where n is an integer greater than or equal to 1.

[0246] Optionally, each level of storage in the multi-level storage includes at least one independent storage unit, and the sending module 1602 is further configured to:

[0247] A data write request is sent to a storage unit in the first-level storage in the multi-level storage, where the data write request carries the first address and the data to be written, where the data to be written is the i-th data block in the first intermediate result. The data write request is used to instruct the storage unit in the first-level storage to write the data to be written into the cache line corresponding to the first address, where the state of the cache line corresponding to the first address after the data to be written is written is a to-be-reduced state, where the to-be-reduced state indicates that the data in the cache line is valid and has not yet been reduced.

[0248] Optionally, the sending module 1602 is further configured to:

[0249] Sending a read request for a second address to a storage unit in a first-level storage in the multi-level storage, so as to read data corresponding to the second address from the multi-level storage, where the data corresponding to the second address is a j-th data block in a calculation result of the target computing task, where j is an integer greater than or equal to 1;

[0250] Receive the data corresponding to the second address returned by the multi-level storage, and send a read request for a third address to the multi-level storage to read the data corresponding to the third address from the multi-level storage, where the data corresponding to the third address is the j+1th data block in the calculation result;

[0251] The calculation module 1601 is further configured to execute the next calculation task of the target calculation task based on the j-th data block in the calculation result after sending a read request for the third address to the multi-level storage.

[0252] It should be noted that, in other embodiments, the steps that the above modules are responsible for implementing can be specified as needed, and the above modules can respectively implement different steps in the above-mentioned data reading and writing method based on multi-level storage to realize all the functions of the above-mentioned device. That is, the data reading and writing device based on multi-level storage provided in the above-mentioned embodiment is only illustrated by the division of the above-mentioned functional modules when implementing the data reading and writing method based on multi-level storage. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device provided in the above-mentioned embodiment and the corresponding method embodiment belong to the same concept, and its specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0253] Among them, the sending module 1501, the reduction module 1502, the calculation module 1601, and the sending module 1602 can all be implemented by software or hardware. For example, the implementation of the calculation module 1601 is described below using the calculation module 1601 as an example. Similarly, the implementation of the sending module 1501, the reduction module 1502, and the sending module 1602 can refer to the implementation of the calculation module 1601.

[0254] As an example of a software functional unit, the computing module 1601 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the computing module 1601 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0255] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0256] As an example of a hardware functional unit, computing module 1601 may include at least one computing device, such as a server. Alternatively, computing module 1601 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0257] The multiple computing devices included in computing module 1601 can be distributed in the same region or in different regions. The multiple computing devices included in computing module 1601 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in computing module 1601 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0258] The present application also provides a computing device 1700. FIG17 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. As shown in FIG17 , the computing device 1700 includes a bus 1701, a processor 1702, a memory 1703, and a communication interface 1704. The processor 1702, the memory 1703, and the communication interface 1704 communicate with each other via the bus 1701. The computing device 1700 can be a computing device or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 1700.

[0259] Bus 1701 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG17 shows only one line, but this does not imply that there is only one bus or only one type of bus. Bus 1701 may include a path for transmitting information between various components of computing device 1700 (e.g., memory 1703, processor 1702, and communication interface 1704).

[0260] The processor 1702 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0261] Memory 1703 may include volatile memory, such as random access memory (RAM). Memory 1703 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0262] Memory 1703 stores executable program code, which processor 1702 executes to implement the functions of sending module 1501 and reduction module 1502, respectively, thereby implementing the multi-level storage-based data reading and writing method. That is, memory 1703 stores instructions for executing the multi-level storage-based data reading and writing method.

[0263] Alternatively, the memory 1703 stores executable program code, and the processor 1702 executes the executable program code to implement the functions of the aforementioned calculation module 1601 and sending module 1602, thereby implementing the data reading and writing method based on multi-level storage. In other words, the memory 1703 stores instructions for executing the data reading and writing method based on multi-level storage.

[0264] The communication interface 1704 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1700 and other devices or a communication network.

[0265] Embodiments of the present application also provide a distributed cluster. Figure 18 is a schematic diagram of a distributed cluster provided by an embodiment of the present application. As shown in Figure 18, the distributed cluster includes at least one computing device 1700. The memory 1703 of one or more computing devices 1700 in the distributed cluster may store the same instructions for executing the multi-level storage-based data reading and writing method.

[0266] In some possible implementations, the memory 1703 of one or more computing devices 1700 in the distributed cluster may also store partial instructions for executing the data reading and writing method based on multi-level storage. In other words, the combination of one or more computing devices 1700 can jointly execute the instructions for executing the data reading and writing method based on multi-level storage.

[0267] It should be noted that the memories 1703 in different computing devices 1700 in the distributed cluster can store different instructions, each used to execute a portion of the functions of the content generation apparatus. In other words, the instructions stored in the memories 1703 in different computing devices 1700 can implement the functions of one or more of the aforementioned sending module 1501, reduction module 1502, calculation module 1601, and sending module 1602.

[0268] It should be understood that the functions of the computing device 1700 shown in FIG. 18 may also be performed by multiple computing devices 1700 .

[0269] In some possible implementations, one or more computing devices in a distributed cluster can be connected via a network. The network can be a wide area network (WAN), a local area network (LAN), or the like. FIG. 19 illustrates one possible implementation. FIG. 19 is a schematic diagram of a possible implementation of a distributed cluster provided by an embodiment of the present application. As shown in FIG. 19 , two computing devices 1700 are connected via a network. Specifically, the network is connected via a communication interface in each computing device.

[0270] The present application also provides another distributed cluster. The connection relationship between the computing devices in this distributed cluster can be similar to the connection method shown in Figures 18 or 19 . However, the memory 1703 in one or more computing devices 1700 in this distributed cluster can store the same instructions for executing the multi-level storage-based data reading and writing method.

[0271] In some possible implementations, the memory 1703 of one or more computing devices 1700 in the distributed cluster may also store partial instructions for executing the data reading and writing method based on multi-level storage. In other words, the combination of one or more computing devices 1700 can jointly execute the instructions for executing the data reading and writing method based on multi-level storage.

[0272] It should be noted that the memories 1703 in different computing devices 1700 in the distributed cluster can store different instructions, each used to execute a portion of the functions of the content generation apparatus. In other words, the instructions stored in the memories 1703 in different computing devices 1700 can implement the functions of one or more of the aforementioned sending module 1501, reduction module 1502, calculation module 1601, and sending module 1602.

[0273] An embodiment of the present application provides a computer-readable storage medium, which can be any available medium that a computing device can store or a data storage device such as a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes a program code, and when the program code is executed by a processor core, the processor core executes the steps performed by the processor core in the data reading and writing method based on multi-level storage provided in the above method embodiment, or, when the program code is executed by a multi-level storage, the multi-level storage executes the steps performed by the multi-level storage in the data reading and writing method based on multi-level storage provided in the above method embodiment.

[0274] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the data involved in this application were obtained with full authorization.

[0275] Those skilled in the art will appreciate that the various method steps and units described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0276] Those skilled in the art will clearly understand that, for the sake of convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0277] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.

[0278] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0279] In addition, each unit in each embodiment of the present application may be integrated into a processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The above-mentioned integrated units may be implemented in the form of hardware or software units.

[0280] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computing device (which can be a personal computer, a server, or a computing device, etc.) to execute all or part of the steps of the method in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk or an optical disk.

[0281] In this application, the terms "first," "second," and the like are used to distinguish between identical or similar items having substantially the same function or effect. It should be understood that "first," "second," and "nth" do not have a logical or temporal dependency, nor do they limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first," "second," and the like to describe various elements, these elements should not be limited by these terms. These terms are simply used to distinguish one element from another.

[0282] In this application, the term "at least one" means one or more, and the term "plurality" means two or more. In this document, the terms "system" and "network" are often used interchangeably.

[0283] It should also be understood that the term "if" may be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting." Similarly, the phrase "if it is determined that..." or "if [stated condition or event] is detected" may be interpreted to mean "upon determining that..." or "in response to determining that..." or "upon detecting [stated condition or event]" or "in response to detecting [stated condition or event]," depending on the context.

[0284] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0285] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer program instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0286] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired or wireless method. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a digital video disc (DVD), or a semiconductor medium (e.g., a solid-state drive)), etc.

[0287] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0288] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data reading and writing method based on multi-level storage, characterized in that: Applied to multi-level storage, each level of storage includes at least one independent storage unit, the method includes: The k-th level storage in the multi-level storage sends a k+1-th level write-back request to the k+1-th level storage in the multi-level storage, where the k+1-th level write-back request carries a first address, and k is an integer greater than or equal to 1; The k+1th level storage responds to the k+1th level write back request, performs a reduce operation on the data in the multiple cache lines corresponding to the first address in the kth level storage, and writes the results of the reduce operation into the k+1th level storage, where different cache lines in the multiple cache lines correspond to different storage units in the kth level storage.

2. The method according to claim 1, characterized in that The k-th level storage sends a k+1-th level write-back request to the k+1-th level storage, comprising: When k is equal to 1, the storage unit in the first-level storage sends a second-level write-back request to the storage unit in the second-level storage in response to the first-level write-back request sent by the processor core, wherein the first-level write-back request carries the first address; When k is greater than or equal to 2, after the storage unit in the k-th level storage completes the reduce operation indicated by the k-th level write back request, the storage unit in the k-th level storage sends a k+1-th level write back request to the storage unit in the k+1-th level storage.

3. The method according to claim 1 or 2, characterized in that The method further comprises: The storage unit in the k-th level storage determines, in response to a write request to the k-th level storage, a cache line corresponding to the first address from the storage units in the k-th level storage, the write request carrying the first address; If the state of the cache line corresponding to the first address is an invalid state or a shared state, the storage unit in the k-th level storage writes the to-be-written data corresponding to the write request into the cache line corresponding to the first address, and sets the state of the cache line corresponding to the first address to a to-be-reduced state, where the invalid state indicates that the data in the cache line is invalid, the shared state indicates that the data in the cache line is valid and supports reading, and the to-be-reduced state indicates that the data in the cache line is valid and has not yet been reduced; If the state of the cache line corresponding to the first address is the to-be-reduced state, the storage unit in the k-th level storage writes the to-be-written data corresponding to the write request into the cache line corresponding to the first address.

4. The method according to claim 3, characterized in that The storage unit in the k-th level storage determines, in response to a write request to the k-th level storage, a cache line corresponding to the first address from the storage units in the k-th level storage, comprising: When k is equal to 1, the storage unit in the first-level storage determines, in response to the data write request sent by the processor core, a cache line corresponding to the first address from the storage unit in the first-level storage, and the data to be written corresponding to the data write request is the data carried by the data write request; When k is greater than or equal to 2, the storage unit in the k-th level storage responds to the k-th level write back request sent by the storage unit in the k-1-th level storage, determines the cache line corresponding to the first address from the storage unit in the k-th level storage, and the data to be written corresponding to the k-th level write back request is the result of the reduce operation indicated by the k-th level write back request.

5. The method according to claim 3 or 4, characterized in that When k is greater than or equal to 2, after the storage unit in the k-th level storage writes the to-be-written data corresponding to the write request into the cache line corresponding to the first address, the method further includes: The k-1th level storage sets the state of the cache line corresponding to the first address in the k-1th level storage to an invalid state.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: The storage unit in the k-th level storage receives a read request for the storage unit in the k-th level storage, wherein the read request carries a second address; If a cache line corresponding to the second address exists in the storage unit in the k-th level storage, and the state of the cache line corresponding to the second address is a shared state, the storage unit in the k-th level storage reads data from the cache line corresponding to the second address and returns the read data to the processor core; If a cache line corresponding to the second address exists in the storage unit in the k-th level storage, and the state of the cache line corresponding to the second address is in a pending reduction state, the storage unit in the k-th level storage is waiting for the state of the cache line corresponding to the second address to be reduced. When the memory is in an invalid state, a read request is sent to the storage unit in the k+1-th level storage based on the second address to read the data corresponding to the second address, and the read data is returned to the processor core; If there is no cache line corresponding to the second address in the storage unit in the k-th level storage, the storage unit in the k-th level storage sends a read request to the storage unit in the k+1-th level storage based on the second address to read the data corresponding to the second address and returns the read data to the processor core.

7. The method according to claim 6, characterized in that The method further comprises: If the storage unit in the k-th level storage reads the data corresponding to the second address from the storage unit in the k+1-th level storage, the storage unit in the k-th level storage determines the target cache line from the storage unit in the k-th level storage, writes the read data to the target cache line, and sets the state of the target cache line to a shared state.

8. A data reading and writing method based on multi-level storage, characterized in that: Applied to a processor core, the method includes: Calculate an i-th data block in a first intermediate result of a first parallel computing task, where i is an integer greater than or equal to 1, the i-th data block in the first intermediate result corresponds to a first address, and the first parallel computing task is used to calculate a portion of the computing result in the target computing task; After calculating the i-th data block in the first intermediate result, sending a write-back request to the multi-level storage, wherein the write-back request instructs the multi-level storage to perform a reduce operation on the data in the plurality of cache lines corresponding to the first address in the multi-level storage, level by level, and write a final result of the reduce operation to the first address; After sending the write-back request, the (i+1)th data block in the first intermediate result is calculated.

9. The method according to claim 8, characterized in that After calculating and obtaining the i-th data block in the first intermediate result, sending a write-back request to the multi-level storage includes: Whenever n data blocks in the first intermediate result are calculated, a write-back request for the n data blocks is sent to the multi-level storage, where n is an integer greater than or equal to 1.

10. The method according to claim 8 or 9, characterized in that Each level of storage in the multi-level storage includes at least one independent storage unit. After calculating and obtaining the i-th data block in the first intermediate result, the method further includes: A data write request is sent to a storage unit in the first-level storage in the multi-level storage, where the data write request carries the first address and the data to be written, where the data to be written is the i-th data block in the first intermediate result, and the data write request is used to instruct the storage unit in the first-level storage to write the data to be written into the cache line corresponding to the first address. The state of the cache line corresponding to the first address after the data to be written is written is a to-be-reduced state, and the to-be-reduced state indicates that the data in the cache line is valid and has not yet been reduced.

11. The method according to any one of claims 8 to 10, characterized in that The method further comprises: Sending a read request for a second address to a storage unit in a first-level storage in the multi-level storage, so as to read data corresponding to the second address from the multi-level storage, where the data corresponding to the second address is a j-th data block in a calculation result of the target computing task, where j is an integer greater than or equal to 1; receiving data corresponding to the second address returned by the multi-level storage, and sending a read request for a third address to the multi-level storage to read the data corresponding to the third address from the multi-level storage, where the data corresponding to the third address is the j+1th data block in the calculation result; After sending a read request for the third address to the multi-level storage, the next computing task of the target computing task is executed based on the j-th data block in the computing result.

12. A data reading and writing method based on multi-level storage, characterized in that: Each level of storage in the multi-level storage includes at least one independent storage unit, and the method includes: The processor core calculates an i-th data block in a first intermediate result of a first parallel computing task, where i is an integer greater than or equal to 1, the i-th data block in the first intermediate result corresponds to a first address, and the first parallel computing task is used to implement calculation of a portion of the computing result in the target computing task; After calculating and obtaining the i-th data block in the first intermediate result, the processor core sends a write-back request to the multi-level storage, wherein the write-back request instructs the multi-level storage to perform a reduce operation on the data in the plurality of cache lines corresponding to the first address in the multi-level storage, level by level, and write a final result of the reduce operation to the first address; After sending the write-back request, the processor core calculates the (i+1)th data block in the first intermediate result; While the processor core is calculating, the k-th level storage in the multi-level storage sends a k+1-th level write-back request to the k+1-th level storage in the multi-level storage, where the k+1-th level write-back request carries a first address, and k is an integer greater than or equal to 1; The k+1-th level storage performs a reduce operation on the data in the multiple cache lines corresponding to the first address in the k-th level storage in response to the k+1-th level write back request, and writes the result of the reduce operation to the k+1-th level storage, different cache lines in the multiple cache lines correspond to different storage units in the k-th level storage, and the status of the multiple cache lines is a pending reduction status, and the pending reduction status indicates that the data in the cache lines is valid and has not yet been reduced.

13. A data reading and writing device based on multi-level storage, characterized in that: Applied to multi-level storage, each level of storage includes at least one independent storage unit, the device includes: a sending module, configured to send a k+1th level write-back request to a k+1th level storage in the multi-level storage through the kth level storage in the multi-level storage, wherein the k+1th level write-back request carries a first address, and k is an integer greater than or equal to 1; A reduction module is configured to, in response to the k+1th level write-back request, perform a reduce operation on data in a plurality of cache lines corresponding to the first address in the kth level storage through the k+1th level storage, and write a result of the reduce operation into the k+1th level storage, wherein different cache lines among the plurality of cache lines correspond to different storage units in the kth level storage, and the states of the plurality of cache lines are in a pending reduction state, and the pending reduction state indicates that the data in the cache lines is valid and has not yet been reduced.

14. A data reading and writing device based on multi-level storage, characterized in that: Applied to a processor core, the device comprises: a computing module, configured to compute an i-th data block in a first intermediate result of a first parallel computing task, where i is an integer greater than or equal to 1, the i-th data block in the first intermediate result corresponds to a first address, and the first parallel computing task is configured to compute a partial computing result in a target computing task; a sending module, configured to, after calculating the i-th data block in the first intermediate result, send a write-back request to the multi-level storage, wherein the write-back request instructs the multi-level storage to perform a reduce operation on the data in the plurality of cache lines corresponding to the first address in the multi-level storage, level by level, and write a final result of the reduce operation to the first address; The calculation module is further configured to calculate the (i+1)th data block in the first intermediate result after sending the write-back request.

15. A multi-level storage, characterized in that: Each level of storage in the multi-level storage includes at least one independent storage unit; The k-th level storage in the multi-level storage is configured to send a k+1-th level write-back request to the k+1-th level storage in the multi-level storage, wherein the k+1-th level write-back request carries a first address, and k is an integer greater than or equal to 1; The k+1th level storage is used to respond to the k+1th level write back request, perform a reduce operation on the data in the multiple cache lines corresponding to the first address in the kth level storage, and write the results of the reduce operation into the k+1th level storage, different cache lines in the multiple cache lines correspond to different storage units in the kth level storage, and the status of the multiple cache lines is a pending reduction state, and the pending reduction state indicates that the data in the cache lines is valid and has not yet been reduced.

16. A processor, characterized in that: The computing device includes multi-level storage and multiple processor cores, the multi-level storage is used to implement the steps performed by the multi-level storage in the data reading and writing method based on multi-level storage as described in claim 12 above, and the processor core is used to implement the steps performed by the processor core in the data reading and writing method based on multi-level storage as described in claim 12 above.

17. A computing device, characterized in that The computing device includes multi-level storage and multiple processor cores, the multi-level storage is used to implement the steps performed by the multi-level storage in the data reading and writing method based on multi-level storage as described in claim 12 above, and the processor core is used to implement the steps performed by the processor core in the data reading and writing method based on multi-level storage as described in claim 12 above.

18. A distributed cluster, characterized in that: The distributed cluster includes multi-level storage and multiple processor cores, the multi-level storage is used to implement the steps performed by the multi-level storage in the data reading and writing method based on multi-level storage as described in claim 12 above, and the processor core is used to implement the steps performed by the processor core in the data reading and writing method based on multi-level storage as described in claim 12 above.

19. A storage medium, characterized in that: The storage medium is used to store program code, and the program code can be read and executed by multi-level storage to implement the data reading and writing method based on multi-level storage as described in any one of claims 1 to 7 above.

20. A storage medium, characterized in that The storage medium is used to store program code, and the program code can be read and executed by the processor core to implement the data reading and writing method based on multi-level storage as described in any one of claims 8 to 11 above.

Images