Data processing method, device, storage medium, and program product

Abstract

Embodiments of the present disclosure provide a data processing method, a device, a storage medium, and a program product. In the embodiments of the present disclosure, for a computing task transmitted across networks, address computation and data computation are offloaded to a network transmission module. On one hand, the network transmission module performs computation on the basis of meta-information of a first data subset in a memory of a first computing node to generate memory distribution information of the first data subset, and reads data of the first data subset from the memory on the basis of the memory distribution information; and on the other hand, the network transmission module receives data of a second data subset in a memory of a second computing node, performs data computation on the data of the first data subset and the data of the second data subset, and writes the computed result data into the memory of the first computing node. Processor resources on a computing node are freed up from a computing task, reducing the consumption of the computing resources on the computing node; and in addition, by offloading the computing task, the frequency of memory access is reduced, thereby improving computing efficiency.

Description

Data processing methods, equipment, storage media and software products Technical Field

[0001] This disclosure relates to the field of network communication technology, and in particular to a data processing method, device, storage medium, and program product. Background Technology

[0002] With the development of deep learning technology, tensor computing has been widely used in applications such as artificial intelligence (AI) and high-performance computing. Tensors are an extension of the array concept to a higher dimension. They can be understood as multidimensional arrays of three or more dimensions, and are gradually becoming a new data structure standard.

[0003] In actual model training or inference, in order to improve performance and efficiency, different tensor computation tasks are assigned to different computing nodes for execution. In this process, a computing node may need Tensor Tile (localized sub-blocks in tensor data) on another computing node to participate in the computation with Tensor Tile in its own tensor data, which involves tensor computation process transmitted across the network.

[0004] Existing tensor computation processes involving cross-network transmission are primarily completed by applications at both ends. These processes include calculating the memory address list of Tensor Tiles, reading memory from both Tensor Tiles, transmitting the remote Tensor Tile over the network, and performing computations on both Tensor Tiles. The entire tensor computation process is not only computationally resource-intensive but also computationally inefficient. Summary of the Invention

[0005] This disclosure provides a data processing method, apparatus, storage medium, and program product to reduce the computational resource overhead of computing nodes performing data computations and the frequency of accessing node memory, thereby improving computational efficiency.

[0006] This disclosure provides a data processing method applicable to a first network transmission module on a first computing node, wherein a first application runs on the first computing node. The method includes: receiving a computing task issued by the first application, the computing task including metadata of a first subset of data participating in the computing task from a first dataset, the data in the first subset being distributed and stored in the memory of the first computing node; generating memory distribution information of the first subset in the memory of the first computing node based on the metadata of the first subset; reading data from the memory of the first subset based on the memory distribution information of the first subset; receiving data from a second subset sent by a second computing node, the second subset being a subset of data participating in the computing task from a second dataset; performing calculations on the data in the first subset and the data in the second subset, and writing the calculated data into the memory of the first computing node.

[0007] This disclosure also provides a data processing method applied to a second network transmission module on a second computing node, wherein a second application runs on the second computing node. The method includes: receiving a data transmission task issued by the second application, the data transmission task including metadata of a second data subset in the second data set; storing the data in the second data subset in a distributed manner in the memory of the second computing node; generating memory distribution information of the second data subset based on the metadata of the second data subset; reading the data in the second data subset from the memory of the second computing node based on the memory distribution information of the second data subset; and sending the data in the second data subset to a first computing node.

[0008] This disclosure also provides a computing node, including: a processing unit and a network transmission module; wherein, the processing unit is configured to run a first application, and the network transmission module is configured to execute steps in various methods provided in this disclosure for a first network transmission module applicable to a first computing node; and / or, the processing unit is configured to run a second application, and the network transmission module is configured to execute steps in various methods provided in this disclosure for a second network transmission module applicable to a second computing node.

[0009] This disclosure also provides a data transmission system, including: a first computing node and a second computing node, the first computing node including a first network transmission module and a first processing unit, and the second computing node including a second network transmission module and a second processing unit; the first processing unit is configured to run a first application, the first network transmission module is configured to execute steps in various methods provided in this disclosure applicable to the first network transmission module on the first computing node; the second processing unit is configured to run a second application, and the second network transmission module is configured to execute steps in various methods provided in this disclosure applicable to the second network transmission module on the second computing node.

[0010] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the processor to perform the steps in the methods described above.

[0011] This disclosure also provides a computer program product, which includes a computer program / instructions that, when executed by a processor, enable the processor to perform the steps described in the method embodiments above.

[0012] In this embodiment of the disclosure, for computing tasks transmitted across a network, the calculation of data addresses and data calculations are delegated to the network transmission module. The network transmission module calculates and generates memory distribution information for the first data subset based on the metadata of the first data subset in the memory of the first computing node, and reads the data of the first data subset from memory based on this memory distribution information. Simultaneously, it receives data from the second data subset in the memory of the second computing node. Furthermore, it performs data calculations on the data of the first and second data subsets and writes the results of the data calculations into the memory of the first computing node. This frees up the processor resources on the computing node from address and data calculations, reducing the consumption of computing resources on the computing node. In addition, based on the delegation of computing tasks, the frequency of memory access is reduced, improving computing efficiency. Attached Figure Description

[0013] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the present disclosure and are used to explain the disclosure, but do not constitute an undue limitation of the disclosure. In the drawings:

[0014] Figures 1a and 1b are schematic diagrams of the architecture and process of tensor computation performed by processors that rely on computing nodes;

[0015] Figure 2a is a schematic diagram of a data processing method based on the internal structure of a computing node, provided by an exemplary embodiment of the present disclosure.

[0016] Figure 2b is a schematic diagram of the memory access process for tensor computation within a computing node provided in an exemplary embodiment of this disclosure;

[0017] Figure 3a is a schematic diagram of a data processing method based on the internal structure of a computing node, provided by another exemplary embodiment of this disclosure;

[0018] Figure 3b is a schematic diagram of the data packet format of two transmission methods used by the network transmission module provided in another exemplary embodiment of this disclosure;

[0019] Figure 3c is a schematic diagram of the memory access process for tensor computation within a computing node provided in another exemplary embodiment of this disclosure;

[0020] Figure 4a is a schematic diagram of a data processing method provided in yet another exemplary embodiment of this disclosure;

[0021] Figure 4b is a schematic diagram of another data processing method provided in yet another exemplary embodiment of this disclosure;

[0022] Figure 5 is a schematic diagram of the structure of a computing node provided in another exemplary embodiment of this disclosure. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0024] In the cases involving user information in the embodiments of this disclosure, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. In addition, the various models involved in this disclosure (including but not limited to language models or large models) comply with relevant laws and standards.

[0025] Tensor data, as multidimensional arrays, and memory, as a linear address space, are stored linearly in memory according to the order of their multiple dimensions; that is, tensor data, as a multidimensional array, is mapped to a contiguous one-dimensional memory space for storage. The storage order of tensor data can be row-major or column-major, but is not limited to either. Row-major storage means that the elements in the tensor data are stored sequentially according to their rows; column-major storage means that the elements in the tensor data are stored sequentially according to their columns.

[0026] For example, consider a three-dimensional tensor data set with a shape of (M, N, K), where M is the number of rows, N is the number of columns, and K is the number of data points at each position (the intersection of rows and columns), which could be an array. The storage methods for this three-dimensional tensor data set are described below, in both row-major and column-major order.

[0027] If the 3D tensor data is stored in row-major order, the first K data points of the first row and first column are stored first, followed by the K data points of the first row and second column, and so on, until the data corresponding to each column of the first row is stored. Then the storage of the second row can begin. The data storage method for the second row can refer to the storage method for the first row. The storage of the 3D tensor data is considered complete when all rows of the 3D tensor data have been stored.

[0028] If the 3D tensor data is stored in column-major order, the first K data points of the first row of the first column are stored first, followed by the K data points of the second row of the first column, and so on, until the data corresponding to each row of the first column is stored. Then the storage of the data in the second column can begin. The storage method for the second column can refer to the storage method for the first column. The storage of the 3D tensor data is considered complete when all columns of the data have been stored.

[0029] Tensor computation is typically performed at the tensor tile level to improve performance, especially when dealing with high-dimensional tensor data. By dividing tensor data into smaller, local tensor tiles as needed, multiple processing units can perform parallel computations on different tensor tiles, thus improving performance. Although tensor data is linearly stored in memory, the data within a particular tensor tile is scattered across different memory regions. This is because the storage order affects how the tensor data is arranged in memory. For example, row-major storage results in contiguous data within a row, but not between different columns. If a tensor tile involves multiple rows, the data corresponding to those rows will be scattered across different memory locations. Similarly, column-major storage results in contiguous data within a column, but not between different rows. If a tensor tile involves multiple columns, the data corresponding to those columns will be scattered across different memory locations.

[0030] In memory, the data of a Tensor Tile may be divided into multiple data segments. For a given data segment, the data within that segment is contiguous, while different data segments may be separate. For example, if the tensor data is a three-dimensional array, it is equivalent to a large cube. A small cube is cut out from the large cube, and the small cube is an example of a Tensor Tile. The storage location of its corresponding data in memory can be regarded as a set of separate contiguous memory spaces. Here, "contiguous" in "separate contiguous memory space" means contiguous within a segment; "separate" means that different data segments are separated, that is, scattered on different physical addresses.

[0031] When transferring Tensor Tiles over a network, Direct Memory Access (DMA) technology can be used. The application calculates a Scatter-Gather-List (SGL) of the Tensor Tiles and provides it to the DMA. The DMA reads data fragments from the Tensor Tiles one by one according to the memory addresses in the SGL, encapsulates them into data packets, and transmits them over the network to the other end. The SGL is a data structure used to describe the scattered data fragments. The SGL includes multiple sequentially arranged descriptors, forming a list of memory addresses. Each descriptor points to the starting address and size of the data fragment to be transferred. Therefore, the memory address of the corresponding data fragment can be determined based on the descriptor, allowing the data fragment to be retrieved from the memory space corresponding to that address, thus enabling the network transfer of data fragments. The calculation of SGL is a loop operation related to the dimension of tensor data, which consumes corresponding computing resources. For example, for 3-dimensional tensor data of 1024*1024*1024, if the memory space corresponding to the Tensor Tile contains 1024*1024=1M scattered contiguous spaces, an address calculation must be performed once for each data segment in a contiguous space. Therefore, to achieve data access to the Tensor Tile, 1M address calculations are required.

[0032] Based on the above, referring to Figures 1a and 1b, the tensor computation process involving cross-network transmission includes: Referring to S1 in Figure 1a, the local application and the remote application respectively calculate the memory address list of the Tensor Tile participating in the tensor computation; then, referring to S2 in Figure 1a, the local application and the remote application respectively send computation tasks to the network interface card (NIC) device, the computation tasks including their respective memory address lists or the starting address of the memory address lists; referring to S3 in Figure 1a, the remote NIC device obtains the remote Tensor Tile from the memory of the remote application based on the corresponding memory address list; referring to S4 in Figure 1a, the local NIC device requests the remote Tensor Tile through the network; referring to step 5 in Figure 1b, the local NIC device writes the remote Tensor Tile into its local memory; referring to S6-S9 in Figure 1b, when the data preparation is complete, the local application, based on the calculated memory address list, obtains its own Tensor Tile from its memory, and then obtains the remote Tensor Tile from its memory, performing computation on the local Tensor Tile. The Tile performs computation operations with the remote Tensor Tile and writes the computation results back to the local memory.

[0033] In the tensor computation process shown in Figures 1a-1b, tensor computation and memory address list computation consume computational resources and involve multiple memory accesses. These multiple memory accesses include the local network interface card writing the remote Tensor Tile to local memory; retrieving the local Tensor Tile from local memory; reading the remote Tensor Tile from local memory; and writing the computation result back to local memory. Therefore, at least four memory accesses are required. This not only wastes computational resources but also leads to low computational efficiency due to frequent memory accesses.

[0034] In the above content, taking tensor data as a data structure and its calculation process as an example, we will provide an exemplary illustration of the problems of wasted computing resources in data calculation involving cross-network transmission, and the problem of low computing efficiency caused by frequent memory access during data calculation.

[0035] In this embodiment of the disclosure, data with similar data structures to tensor data are collectively referred to as datasets. Data computation is the computation performed on the data in the dataset, and data computation transmitted across networks may involve computation on data in datasets stored on different endpoints.

[0036] Furthermore, computations between datasets are performed at the data subset level, allowing multiple processing units to perform parallel computations on different data subsets, thereby improving computational performance. Here, a data subset, as a subset of the dataset, has its data distributed and stored in the memory of the computing nodes. This distributed data can be organized into multiple data segments; within a given data segment, the segments are contiguous, while different data segments may be separate.

[0037] Furthermore, based on the tensor calculation process shown in Figures 1a-1b, it can be seen that the data in the dataset faces the problem of wasted computing resources and low computing efficiency due to frequent memory access during the data calculation process transmitted across networks.

[0038] Based on this, in this embodiment of the disclosure, for computing tasks transmitted across the network, the calculation of data addresses and data calculations are delegated to the network transmission module. The network transmission module, on the one hand, calculates and generates memory distribution information for the first data subset based on the metadata of the first data subset in the memory of the first computing node, and reads the data of the first data subset from memory based on this memory distribution information; on the other hand, it receives data of the second data subset in the memory of the second computing node; then, it performs data calculations on the data of the first and second data subsets, and writes the result of the data calculations into the memory of the first computing node, thereby freeing up the processor resources on the computing node from data calculations and address calculations, thus reducing the consumption of computing resources on the computing node; furthermore, based on the delegation of computing tasks, the frequency of memory access is reduced, improving computing efficiency.

[0039] Optionally, in the tensor computation process shown in Figures 1a-1b, if the processing unit of the computing node includes multiple processor cores and these cores share memory, frequent access to this shared memory by the processor core responsible for performing the tensor computation task will interfere with access to the shared memory by other processor cores, resulting in a decrease in the overall utilization and performance of the processing unit. Therefore, similar problems of low processing unit utilization and performance degradation due to shared memory also exist during the computation of data in the dataset. Accordingly, in the optional embodiments of this disclosure, by offloading the computation task to the network transmission module, the frequency of memory access on the computing node is reduced, thereby reducing interference with access to shared memory by other processor cores and improving the overall utilization and performance of the processing unit.

[0040] Optionally, during the tensor computation process as shown in Figures 1a-1b, when transmitting tensor data across the network, the data is transmitted one data segment at a time, with each segment encapsulated into a data packet. Each data packet includes not only the data segment but also protocol fields such as source and destination addresses. The network bandwidth consumed by these protocol fields can be termed network coding overhead. If each data segment is small, the effective data load of each data packet will be small, meaning the actual data to be transmitted is small. Since network coding is required for each data packet, the network coding overhead is relatively large, resulting in significant network bandwidth overhead. Therefore, similar issues of high network bandwidth overhead can arise during the computation of data within a dataset. Accordingly, in the optional embodiments of this disclosure, based on the offloading of memory distribution information to the network transmission module, the network transmission module can obtain all data segments included in the data subset. It can then package multiple data segments corresponding to the data subset into a single data packet and send it to the computing node at the other end. This significantly reduces the new overhead of network coding for data packets and increases the effective data load of a single data packet.

[0041] The technical solutions provided by the embodiments of this disclosure are described in detail below with reference to the accompanying drawings.

[0042] It should be noted in advance that, in the various embodiments of this disclosure, tensor data and tensor computation transmitted across networks are used as examples, but this does not constitute a limitation on the embodiments of this disclosure. That is to say, the data processing method provided by the embodiments of this disclosure can be applied to, but is not limited to, tensor data as a data structure, and the data of the applicable data structure is collectively referred to as a dataset. Data computation refers to the computation of data in the dataset, and the computation is performed at the granularity of a data subset. The data subset can be implemented as the Tensor Tile mentioned above.

[0043] Referring to Figures 2a-2b, the tensor computation process transmitted across the network involves: a first computation node 20 and a second computation node 21 participating in the tensor computation task; wherein, the first computation node 20 is the local end and the second computation node 21 is the remote end. In this embodiment, it is not limited whether the first computation node 20 is the local end or the remote end. This is just an example to facilitate the explanation of related content in subsequent embodiments. Similarly, it is not limited whether the second computation node 21 is the remote end or the local end. In this embodiment, the second computation node is used as the remote end as an example.

[0044] In this embodiment, the first computing node 20 and the second computing node 21 refer to nodes that include certain computing resources. These computing resources are various hardware resources with computing capabilities provided by the computing nodes, such as, but not limited to, processing units like CPUs (Central Processing Units), DPUs (Data Processing Units), NPUs (Neural Network Processing Units), and GPUs. Furthermore, the specific implementation of the first and second computing nodes is not limited; for example, they can be terminal devices such as desktop computers, laptops, smartphones, or IoT (Internet of Things) devices, or server-side devices such as conventional servers, cloud servers, or server arrays.

[0045] In this embodiment, the first computing node 20 includes a first network transmission module 24 and runs a first application. The first application broadly refers to various applications that rely on machine learning tasks, such as shopping applications, video applications, live streaming applications, game applications, language processing applications, etc. The training and inference processes of machine learning tasks involve a large number of tensor computation tasks. Tensor computation tasks refer to tasks that perform various operations and calculations on tensor data, including but not limited to basic arithmetic operations on tensor data, such as addition, subtraction, multiplication, and division; as well as matrix operations, tensor reduction, tensor convolution, and tensor shrinkage, etc.

[0046] Further, as shown in S201 of Figure 2a, the first application sends the tensor computation task to the first network transmission module 24. In this embodiment, the method of sending the tensor computation task is not limited. Optionally, the first application can directly communicate with the first network transmission module to send the tensor computation task directly to the first network transmission module; or, the first application can write the tensor computation task into the memory 22 of the first computing node, for example, into a queue pair in the memory 22 of the first computing node; then, as shown in S202 of Figure 2a, the first network transmission module retrieves the tensor computation task from the memory 22 of the first computing node. For example, if the tensor computation task is written into the queue corresponding to the queue pair, the first network transmission module can periodically poll the queue to read the tensor computation task from it; or, the first application can send an interrupt signal to the first network transmission module after writing the tensor computation task into the queue pair to trigger the first network transmission module to read the tensor computation task from the corresponding queue.

[0047] In this embodiment, for any tensor computation task, the tensor data participating in the tensor computation task includes a first sub-tensor located in memory 22 of the first computing node and a second sub-tensor located in memory 23 of the second computing node; the first sub-tensor and the second sub-tensor can be implemented as the Tensor Tile mentioned above; wherein, the first sub-tensor is obtained by splitting the first tensor as needed, and the size of the splitting for different tensor computation tasks can be the same or different, depending on the needs of the tensor computation task; similarly, the second sub-tensor is obtained by splitting the second tensor as needed, and its size is also not limited.

[0048] As mentioned earlier, the data in a Tensor Tile is distributed across different memory regions. In this embodiment, when the first sub-tensor can be implemented as a Tensor Tile, the data in the first sub-tensor is distributed and stored in the memory 22 of the first computing node. To index the data in the first sub-tensor, in this embodiment, the tensor computation task includes the metadata of the first sub-tensor. Furthermore, as shown in S203 of FIG2a, the first network transmission module 24 can generate memory distribution information of the first sub-tensor in the memory 22 of the first computing node based on the metadata of the first sub-tensor. This memory distribution information describes the distribution of data in the first sub-tensor in the memory 22 of the first computing node. Optionally, this memory distribution information can be implemented as a memory address list, which includes at least one memory address. Each memory address corresponds to a segment of continuously stored data, i.e., a data fragment. As shown in FIG2a, the first sub-tensor in the first tensor includes data fragment a1, data fragment a2, and data fragment an, where n≥2 and n is a natural number. It should be understood that FIG2a uses the example of the first sub-tensor including multiple data fragments; the first sub-tensor can also include one data fragment, and this is not a limitation.

[0049] In the embodiments of this disclosure, for any data segment, the data segment is continuous within the segment; different memory addresses may correspond to different data segments, and different data segments may be separated. The reasons for the separation of different data segments can be referred to in subsequent embodiments.

[0050] Furthermore, in this embodiment, as shown in S204 of FIG2a, the first network transmission module 24 reads data in the first sub-tensor from the memory 22 of the first computing node based on the memory distribution information of the first sub-tensor; and as shown in S205 of FIG2a, receives data in the second sub-tensor sent by the second computing node 21; wherein, in this embodiment, the order in which the first network transmission module 24 reads the first sub-tensor and receives the second sub-tensor is not limited.

[0051] After the first network transmission module 24 completes the reading of the first sub-tensor and the receiving of the second sub-tensor, the process of performing tensor calculation by the first network transmission module is further described with reference to FIG2b. As shown in FIG2b, S204 and S205 in this process correspond to the reading of the data of the first sub-tensor and the receiving of the data of the second sub-tensor, respectively. Furthermore, as shown in S206 in FIG2b, the first network transmission module 24 performs tensor calculation on the data in the first sub-tensor and the data in the second sub-tensor. As shown in S207 in FIG2b, the first network transmission module 24 writes the result data of the tensor calculation into the memory 22 of the first computing node. The result data of the tensor calculation can be written into a first storage location. The first storage location can be determined based on the metadata of the first sub-tensor. The specific method of determining the first storage location may vary depending on the first application, and this embodiment does not limit it. In one optional embodiment, the storage location of the data fragment in the first sub-tensor in the memory 22 of the first computing node is used as the first storage location, so that the result data of the tensor calculation can be written to the first storage location to achieve the purpose of the result data overwriting the old data fragment; of course, in addition to this implementation, the first storage location can also be other storage locations that will not overwrite the old data fragment.

[0052] In this embodiment of the disclosure, for computing tasks transmitted across a network, the calculation of data addresses and data calculations are delegated to the network transmission module. The network transmission module calculates and generates memory distribution information for the first data subset based on the metadata of the first data subset in the memory of the first computing node, and reads the data of the first data subset from memory based on this memory distribution information. Simultaneously, it receives data from the second data subset in the memory of the second computing node. Furthermore, it performs data calculations on the data of the first and second data subsets and writes the results of the data calculations into the memory of the first computing node. This frees up the processor resources on the computing node from data calculations and address calculations, thereby reducing the consumption of computing resources on the computing node. In addition, based on the delegation of computing tasks, the frequency of memory access is reduced, improving computing efficiency.

[0053] In this embodiment, tensor data and tensor computation transmitted across networks are still used as examples, but this does not constitute a limitation on the embodiments of this disclosure. That is to say, the data processing method provided by the embodiments of this disclosure can be applied to, but is not limited to, tensor data as a data structure, and the data of the applicable data structure is collectively referred to as a dataset. Data computation refers to the computation of data in the dataset, and the computation is performed at the granularity of a data subset. The data subset can be implemented as the Tensor Tile mentioned above.

[0054] Referring to Figures 3a-3b, another cross-network transmission tensor computation process provided in this embodiment involves: a first computing node 20 and a second computing node 21; wherein, the first computing node 20 is the local end, and the second computing node 21 is the remote end, and the first computing node 20 and the second computing node 21 each include their respective network transmission modules; and, a first application runs on the first computing node, and a second application runs on the second computing node; in this embodiment, the computation process participating in the tensor computation task includes the first application and the second application; the first application issues a tensor computation task to the first network transmission module 24, which involves reading the first sub-tensor of the first tensor participating in the tensor computation task at the local end, and receiving data of the second sub-tensor of the second tensor at the second computing node 21; the second application is responsible for instructing the second network transmission module 25 to send the data of the second sub-tensor participating in the tensor computation task to the first computing node 20; then, after completing the reading of the first sub-tensor and the receiving of the second sub-tensor, tensor computation is performed, and the result data is written to the local memory. The cross-network transmission tensor computation process involved in this embodiment will be described in detail below.

[0055] As shown in Figure 3a, in this embodiment, the first network transmission module receives a tensor computation task issued by the first application. The tensor computation task includes the metadata of the first sub-tensors participating in the tensor computation task within the first tensor. The data of the first sub-tensors is distributed and stored in the memory of the first computing node. Optionally, the metadata of the first sub-tensors includes the metadata of the first tensor and the position information of the first sub-tensors within the first tensor. Based on this information, the memory distribution information of the first sub-tensors can be quickly generated. Specifically, the process by which the first network transmission module generates the memory distribution information of the first sub-tensors based on the metadata of the first sub-tensors includes: determining the global memory region occupied by the first tensor in the memory of the first computing node based on the metadata of the first tensor. The global memory region refers to the memory region occupied by each sub-tensor included in the first tensor within the memory of the first computing node; then, the memory distribution information of the first sub-tensors can be generated based on the position information of the first sub-tensors within the first tensor and the global memory region; wherein, the data in the first sub-tensors is distributed and stored in the memory of the first computing node, and this memory distribution information is used to describe the distribution of the data in the first sub-tensors within the memory of the first computing node.

[0056] In an optional embodiment, the metadata of the first tensor includes at least: the number of dimensions of the first tensor, the size of each dimension, the data type, and data padding strategy information. The number of dimensions of the first tensor can be one-dimensional or multi-dimensional, without limitation. The size of any dimension is the number of elements included in that dimension. The data type refers to the type of data stored, such as single-precision floating-point, integer, double-precision floating-point, etc. Different data types generally correspond to different byte counts. The data padding strategy information determines how to handle the tensor's boundaries. This information is used to add extra values ​​(which can be 0) to the boundaries of the tensor data, thereby changing the size of the tensor data to suit different computational needs. Based on the above metadata of the first tensor, the global memory area occupied by the first tensor in the memory of the first computing node can be determined simply and quickly.

[0057] Specifically, in the process of determining the global memory region occupied by the first tensor in the memory of the first computing node based on the metadata of the first tensor, the memory size required by the first tensor is calculated based on the number of dimensions, the size of each dimension, the data type, and the data filling strategy information of the first tensor; then, based on the memory size and the starting address of the memory allocated to the first tensor, the global memory region occupied by the first tensor in the memory of the first computing node is determined; wherein, the data in the first tensor is stored continuously in the global memory region according to the specified dimension order.

[0058] In this embodiment, at least one sub-tensor, comprising the first tensor, is stored in the global memory region. Each sub-tensor has a corresponding storage location in the global memory region, and its storage location in the global memory region can be determined based on its location information. In an optional embodiment, the location information of the first sub-tensor within the first tensor includes at least: the offset and size of the first tensor in each dimension; wherein the offset in each dimension refers to the number of elements offset relative to the starting address; and the size in each dimension refers to the number of elements in that dimension. Based on this information, the memory distribution information of the first sub-tensor can be generated simply and quickly.

[0059] Based on the above, in the process of generating the memory distribution information of the first sub-tensor, firstly, according to the offset and size of the first sub-tensor in each dimension and the order of that dimension, the global memory region is traversed to obtain at least one data segment corresponding to the first sub-tensor; in this embodiment, any data segment refers to data stored in a continuous memory space.

[0060] In this embodiment, the order of traversing the multiple dimensions of the tensor data during the traversal of the global memory region corresponding to the first tensor is not limited. Preferredly, the global memory region can be traversed in conjunction with the storage order of the multiple dimensions of the tensor data. For example, it can be considered whether the data is stored in row-major or column-major order. If it is stored in row-major order, it means that elements within the same row in memory are consecutive, so traversal can be performed row-wise first. If it is stored in column-major order, it means that elements within the same column in memory are consecutive, so traversal can be performed column-wise first. This fully utilizes the advantage of accessing consecutive data in memory, improving the hit rate of the data fragments of the first sub-tensor.

[0061] As shown in Figure 3a, by traversing the global memory region corresponding to the first tensor, at least one data segment corresponding to the first sub-tensor is obtained, such as data segment a1, data segment a2, and data segment an, etc.; where n≥2, and n is a natural number; Figure 3a shows an example where the first sub-tensor includes multiple data segments, but this does not constitute a limitation on this embodiment, and the first sub-tensor may also include one data segment. Similarly, in subsequent embodiments, the third and fourth sub-tensors also include one or more data segments, which will not be elaborated further in subsequent embodiments.

[0062] Next, the first network transmission module determines at least one local memory region occupied by the first sub-tensor based on the storage location of at least one data segment in the global memory region. The at least one local memory region is related to the dimension of the first tensor. If the first tensor has two or more dimensions, the number of local memory regions is multiple; if the first tensor has only one dimension, the number of local memory regions is one. In the case of multiple local memory regions, these regions can be distributed across different storage locations within the global memory region.

[0063] Furthermore, the first network transmission module can generate an address linked list as the memory distribution information of the first sub-tensor based on the starting address, size, and sequential relationship between at least one local memory region. For example, for each local memory region, the starting address and size of each contiguous local memory region are used as an SGL Descriptor; an SGL Descriptor plus a pointer to the next SGL Descriptor forms an SGL Segment; SGL Segments form an address linked list; further, this address linked list is used as the memory distribution information of the first sub-tensor, so that the first network transmission module can read data from the memory of the first computing node based on the memory distribution information of the first sub-tensor. It should be understood that the address linked list is an example of memory distribution information and does not constitute a limitation on this embodiment.

[0064] In this embodiment, as shown in FIG3a, the tensor computation task also involves receiving a second sub-tensor from a remote second computing node to perform tensor computation using the second sub-tensor. Optionally, before receiving the data in the second sub-tensor sent by the second computing node, the first network transmission module may receive a prefetch request (not shown in FIG3a) issued by the first application. This prefetch request is used to instruct the second sub-tensor to be retrieved from the second computing node. The first network transmission module sends the prefetch request to the second computing node to trigger the second computing node to return the second sub-tensor after listening to the ready state of the second sub-tensor as ready. The monitoring of whether the ready state of the second sub-tensor is ready means monitoring whether the data of the second sub-tensor has been written into the memory of the second computing node so as to read the data of the second sub-tensor from the memory of the second computing node. Further optionally, if the ready state of the second sub-tensor is detected as ready, the second computing node may return a response message to the first network transmission module in response to the prefetch request; upon receiving the response message returned by the second computing node, the first network transmission module waits to receive the data in the second sub-tensor sent from the second computing node.

[0065] Optionally, the second network transmission module can send a subscription request for the second sub-tensor to the second application. This subscription request is used to request to listen to the ready state of the second sub-tensor, that is, to trigger listening to whether the data of the second sub-tensor has been written into the memory of the second computing node. When the second network transmission module receives a request response from the second application for the subscription request, that is, the data of the second sub-tensor is readable, it can read the data in the second sub-tensor from the memory of the second computing node based on the memory distribution information of the second sub-tensor. This request response is sent by the second application after listening to the ready state of the second sub-tensor as ready. Then, as shown in S205 of Figure 3a, the second network transmission module sends the data read from the second sub-tensor to the first computing node.

[0066] In the above optional embodiments, the problem of transmitting data fragments in a subtensor is transformed into the problem of monitoring the state of the subtensor in memory. The application (or the processing unit of the computing node) and the network transmission module can transmit the monitoring results through request and response. Therefore, the processing unit and the network transmission module no longer need to perform complex interactions according to the traditional basic semantics such as receive / send, which can effectively reduce the transmission latency of tensor data.

[0067] Accordingly, the second application can instruct the second network transmission module to send the data in the second sub-tensor. As shown in S2051 of Figure 3a, the second application issues a data sending task to the second network transmission module. This data sending task includes the metadata of the second sub-tensor in the second tensor. The metadata of the second sub-tensor includes the metadata of the second tensor and the position information of the second sub-tensor in the second tensor. Further, as shown in S2053 of Figure 3a, the second network transmission module generates the memory distribution information of the second sub-tensor based on the metadata of the second sub-tensor. The method by which the second application issues the data sending task is not limited and can be referenced to the method of issuing tensor computation tasks, which will not be elaborated here.

[0068] In the process of generating the memory distribution information of the second sub-tensor, the second network transmission module determines the global memory region occupied by the second tensor in the memory of the second computing node based on the metadata of the second tensor. The global memory region refers to the memory region occupied by each sub-tensor included in the second tensor in the memory of the second computing node. Then, based on the position information of the second sub-tensor in the second tensor and the global memory region, the memory distribution information of the second sub-tensor is generated. The data in the second sub-tensor is distributed and stored in the memory of the second computing node. This memory distribution information is used to describe the distribution of the data in the second sub-tensor in the memory of the second computing node.

[0069] In one optional embodiment, the metadata of the second tensor includes at least: the number of dimensions of the second tensor, the size of each dimension, the data type, and the data filling strategy information; wherein, the number of dimensions of the second tensor can be one-dimensional or multi-dimensional, and there is no limitation thereto. Regarding the number of dimensions, the size of each dimension, the data type, and the data filling strategy information of the second tensor, the relevant content regarding the number of dimensions, the size of each dimension, the data type, and the data filling strategy information of the first tensor can be referred to, and will not be repeated here.

[0070] Furthermore, in determining the global memory region occupied by the second tensor in the memory of the second computing node based on the metadata of the second tensor, the memory size required by the second tensor is calculated based on the number of dimensions, the size of each dimension, the data type, and the data filling strategy information of the second tensor. Then, based on the memory size and the starting address of the memory allocated to the second tensor, the global memory region occupied by the second tensor in the memory of the second computing node is determined. Among them, the data in the second tensor is stored continuously in the global memory region according to the specified dimension order.

[0071] The second tensor includes at least one sub-tensor stored in the global memory region. Each sub-tensor has a corresponding storage location in the global memory region, and its storage location in the global memory region can be determined based on its location information. In an optional embodiment, the location information of the second sub-tensor in the second tensor includes at least: the offset and size of the second tensor in each dimension; wherein, the offset in each dimension refers to the number of elements offset relative to the starting address; and the size in each dimension refers to the number of elements in that dimension. Furthermore, in the process of generating the memory distribution information of the second sub-tensor, the global memory region is traversed according to the offset and size of the second sub-tensor in each dimension and the order of that dimension to obtain multiple data fragments corresponding to the second sub-tensor. The traversal order of the multiple dimensions included in the tensor data is not limited during the traversal process; for details, please refer to the relevant content of the global memory region corresponding to the first tensor, which will not be elaborated here.

[0072] The second subtensor obtained through traversal has multiple data fragments that are distributed and stored in the memory of the second computing node. Each data fragment refers to data stored in a contiguous memory space. The second network transmission module determines at least one local memory region occupied by the second subtensor based on the storage location of the multiple data fragments in the global memory region. The at least one local memory region is related to the dimension of the second tensor. If the second tensor has two or more dimensions, the number of local memory regions is multiple. If the second tensor has only one dimension, the number of local memory regions is one. When there are multiple local memory regions, they can be distributed in different storage locations in the global memory region. Furthermore, the second network transmission module can generate an address linked list based on the starting address, size, and sequential relationship between the at least one local memory region as the memory distribution information of the second subtensor. For example, for each local memory region, the starting address and size of each local contiguous memory region are used as an SGL Descriptor; an SGL Descriptor plus a pointer to the next SGL Descriptor forms an SGL Segment; SGL Segments form an address linked list; further, this address linked list is used as the memory distribution information of the second sub-tensor, so that the second network transmission module can read the data in the second sub-tensor from the memory of the second computing node according to the memory distribution information of the second sub-tensor.

[0073] As shown in Figure 3a, the second sub-tensor in the second tensor includes data fragment b1, data fragment b2, and data fragment bm; where m ≥ 2 and m is a natural number; multiple data fragments are stored in the global memory region corresponding to the second tensor, and different data fragments are stored in different local memory regions; as shown in S2054 in Figure 3a, the second network transmission module reads the data in the second sub-tensor from the memory of the second computing node based on the memory distribution information of the second sub-tensor.

[0074] In this embodiment, the method of transmitting data segments between the first network transmission module and the second network transmission module is not limited. For example, one method is to package multiple data segments into a single data packet and send it to the other end, which saves network bandwidth resources; another method is to divide the data into multiple data packets and send multiple data segments to the other end sequentially, which facilitates timely and flexible transmission of data segments. Therefore, combining these two data segment transmission methods, an implementation method for transmitting data segments between the second network transmission module and the first network transmission module is given. Optionally, the second network transmission module packages multiple data segments of the second sub-tensor into a single data packet, i.e., the first data packet; and sends the first data packet to the first network transmission module; the first network transmission module receives the first data packet sent by the second computing node, as shown in Figure 3b. The first data packet includes multiple first payload fields, such as first payload field c1-first payload field ck, where k>1 and k is a natural number; the first payload field refers to the field that carries the data of the second sub-tensor; wherein, one first payload field carries one data segment; from the first data packet... Multiple data fragments are parsed from multiple first payload fields in the second sub-tensor to obtain the data in the second sub-tensor; or, the second network transmission module packages multiple data fragments into multiple data packets, i.e., multiple second data packets, and sends the multiple second data packets to the first network transmission module in batches; the first network transmission module receives multiple second data packets sent from the second computing node, as shown in Figure 3b. The second data packet includes a second payload field, such as second payload field d1, second payload field d2, and second payload field dj, where j≥2 and j is a natural number; the second payload field carries a data fragment; multiple data fragments are parsed from the second payload fields in the multiple second data packets to obtain the data in the second sub-tensor. As shown in Figure 3c, after the first network transmission module completes the reading of the first sub-tensor (as shown in S204 of Figure 3c) and the receiving of the second sub-tensor (as shown in S205 of Figure 3c), the tensor calculation task can be executed using the first network transmission module on the first computing node. The first computing node includes a first network transmission module 24, the memory 22 of the first computing node, and a first processing unit 26. The first application runs on the first processing unit 26 and can send tensor calculation tasks to the first network transmission module.Optionally, as shown in Figure 3b, the first data packet or the second data packet used to transmit the second sub-tensor further includes a source address field and a destination address field; wherein, the source address field carries the metadata of the second sub-tensor, and the destination address field carries the metadata of the first sub-tensor; that is, the first network transmission module can also receive the metadata of the second sub-tensor sent by the second computing node; furthermore, when the first network transmission module performs tensor calculation on the data in the first sub-tensor and the data in the second sub-tensor, and writes the result data of the tensor calculation into the memory of the first computing node, it includes: determining the data correspondence between the first sub-tensor and the second sub-tensor based on the metadata of the first sub-tensor and the second sub-tensor, wherein the data correspondence refers to the correspondence between the data segments in the first sub-tensor and the corresponding data segments in the second sub-tensor; performing tensor calculation on the data segments in the first sub-tensor and the corresponding data segments in the second sub-tensor based on the data correspondence to obtain the result data of the tensor calculation; determining the first storage location of the result data of the tensor calculation in the memory of the first computing node based on the metadata of the first sub-tensor; and writing the result data of the tensor calculation into the memory of the first computing node based on the first storage location.

[0075] It should be understood that the example of the first storage location being the storage location of the result data does not constitute a limitation on this embodiment.

[0076] In this embodiment, the implementation structure of the network transmission module is not limited. Any device structure capable of performing the functions of memory distribution information calculation, tensor calculation, and data transmission described in the above embodiments is within the protection scope of this embodiment. In an optional embodiment, as shown in FIG3c, one implementation structure of the first network transmission module includes a newly added TTCA (Tensor Tile Communication Accelerator) module. The TTCA module may include a control path 27 and a tensor calculation module 28. The control path 27 may be configured to generate memory distribution information, configure calculation parameters, and control DMA data, etc.; the tensor calculation module 28 is configured to perform tensor calculations.

[0077] It should be noted that, in this embodiment of the disclosure, in addition to tensor computation involving cross-network transmission, there may also be a cross-network tensor sending and receiving process. This sending and receiving process does not involve tensor computation; that is, one computing node may need to transmit local tensor data to another computing node. The sending and receiving process of tensor data will be described below with reference to Figures 4a-4b.

[0078] It should be understood that, in this embodiment, tensor data cross-network transmission is still used as an example, but this does not constitute a limitation on the embodiments of this disclosure. That is to say, the data sending and receiving process in this embodiment can be, but is not limited to, tensor data as a data structure, and the data of applicable data structures are collectively referred to as datasets, and the data subset can be implemented as the Tensor Tile mentioned above.

[0079] As shown in Figure 4a, in this embodiment, as shown in S401a of Figure 4a, the first application sends a data receiving task to the memory of the first computing node; then, as shown in S402a of Figure 4a, the first network transmission module obtains the data receiving task sent by the first application. The method by which the first network transmission module obtains the data receiving task is not limited, and can be implemented with reference to the method by which the first network transmission module obtains the tensor computation task.

[0080] The data receiving task is used to instruct the acquisition of the third sub-tensor from the second tensor from the second computing node. The third sub-tensor can be implemented as a Tensor Tile as mentioned above, as shown in Figure 4a. The third sub-tensor corresponds to at least one data segment, such as data segment q1, data segment q2, and data segment qn, etc. Wherein, n≥2, and n is a natural number. Figure 4a takes the first sub-tensor including multiple data segments as an example, but it does not constitute a limitation on this embodiment. The third sub-tensor can also include one data segment.

[0081] Furthermore, in this embodiment, the first network transmission module performs a data receiving task to receive the data of the third sub-tensor. As shown in S403a of FIG4a, when the first network transmission module receives the data receiving task, it generates the memory distribution information of the third sub-tensor based on the metadata of the third sub-tensor. This memory distribution information can be used to determine the second storage location of the third sub-tensor in the memory of the first computing node. Further, as shown in S404a of FIG4a, the first network transmission module receives the data in the third sub-tensor sent by the second computing node. As shown in S405a of FIG4a, the first network transmission module determines the second storage location of the data in the third sub-tensor in the memory of the first computing node based on the memory distribution information of the third sub-tensor. Then, as shown in S406a of FIG4a, the data in the third sub-tensor is stored in the memory of the first computing node according to the second storage location.

[0082] In this embodiment, the determination of the second storage location of the third sub-tensor based on the metadata of the third sub-tensor, and the order in which the data of the third sub-tensor is received, are not limited.

[0083] As shown in Figure 4b, in this embodiment, as shown in S401b of Figure 4b, the first application sends a data sending task to the memory of the first computing node; then, as shown in S402b of Figure 4b, the first network transmission module obtains the data sending task sent by the first application. The method by which the first network transmission module obtains the data receiving task is not limited, and can be implemented with reference to the method by which the first network transmission module obtains the tensor calculation task.

[0084] In this embodiment, the data transmission task includes the metadata of the fourth sub-tensor in the first tensor; the data in the fourth sub-tensor is distributed and stored in the memory of the first computing node. The fourth sub-tensor can be implemented as a Tensor Tile mentioned above, as shown in Figure 4b. The fourth sub-tensor corresponds to at least one data segment, such as data segment w1, data segment w2, and data segment wn, etc.; where n≥2, and n is a natural number; Figure 4b shows an example where the fourth sub-tensor includes multiple data segments, but this does not constitute a limitation on this embodiment. The fourth sub-tensor can also include one data segment.

[0085] Furthermore, in this embodiment, as shown in S403b of FIG4b, memory distribution information of the fourth sub-tensor is generated based on the metadata of the fourth sub-tensor; then, as shown in S404b of FIG4b, data in the fourth sub-tensor is read from the memory of the first computing node based on the memory distribution information of the fourth sub-tensor; as shown in S405b of FIG4b, the data in the fourth sub-tensor read from the memory of the first computing node is written into the first network transmission module; further, as shown in S406b of FIG4b, the data in the fourth sub-tensor is sent, for example, it can be sent to the second computing node, but is not limited thereto.

[0086] Optionally, the data in the fourth subtensor includes multiple data fragments distributed and stored in the memory of the first computing node, with different data fragments stored in different memory regions; sending the data in the fourth subtensor to the second computing node includes: adding the multiple data fragments in the fourth subtensor to multiple payload fields of the first data packet, and sending the third data packet with the multiple data fragments added to it to the second computing node; wherein, the third data packet is obtained by adding the multiple data fragments to be transmitted to multiple payload fields of the first data packet, and the third data packet has the same structure as the first data packet, which can be implemented with reference to the structure of the first data packet, except that the third data packet carries multiple data fragments.

[0087] Optionally, the memory of the first computing node also includes a queue pair, comprising a first queue and a second queue. The first queue is used to carry tensor computation tasks or data sending tasks issued by the first application, and the second queue is used to carry data receiving tasks issued by the first application. In an optional embodiment, the first computing node further includes a first processing unit, which creates the queue pair in memory according to the queue creation command issued by the first application.

[0088] In this embodiment, the method by which the first network transmission module receives the tensor computation task or data transmission task issued by the first application is not limited; for example, periodic polling and / or interrupt methods can be used. In an optional embodiment, the first queue can be periodically polled to read the tensor computation task or data transmission task written by the first application from the first queue; and / or, an interrupt signal triggered after the first application writes the tensor computation task or data transmission task to the first queue can be received, and the tensor computation task or data transmission task can be read from the first queue according to the interrupt signal.

[0089] In one optional embodiment, receiving a data receiving task issued by a first application includes: periodically polling a second queue to read the data receiving task written by the first application from the second queue; and / or receiving an interrupt signal triggered after the first application writes a data receiving task to the second queue, and reading the data receiving task from the second queue according to the interrupt signal.

[0090] Further optionally, the queue pair may also include a third queue adapted to the queue pair. The third queue is used to write the completion messages of the tensor computation task, data sending task, or data receiving task issued by the first application. The third queue may be notified by periodic polling or interruption, without limitation.

[0091] In this disclosure, the implementation of the network transmission module is not limited, but two exemplary implementation methods are provided. One method is to implement the network transmission module as a dedicated network transmission module, such as a network interface card (NIC) device specifically designed for performing various tensor calculations and transmissions provided in the embodiments of this disclosure. This NIC device includes a processor, which may be referred to as the TTCA mentioned above. The application writes tensor calculation tasks, receive tasks, or send tasks to a queue pair, and the TTCA executes the steps of the various data processing methods provided in the foregoing embodiments of this disclosure for processing. Another approach is to reuse the existing RDMA network card on the computing node in the network transmission module. This can be considered an improvement on the RDMA (Remote Direct Memory Access) transmission protocol. The network transmission module includes the original processor and the newly added TTCA module. The application can reuse the RDMA programming interface for development. In this case, the original RDMA queue pairs can be reused, and corresponding command types can be added for tensor computation tasks to distinguish them from traditional RDMA commands. The application sends tensor computation tasks, receive tasks, or send tasks to the TTCA module through the new RDMA commands, and the TTCA executes the steps of the various data processing methods provided in the foregoing embodiments of this disclosure for processing. Of course, the application can also send other tasks to the processor through traditional RDMA commands for the processor to perform network transmission processing on the other tasks.

[0092] The detailed implementation methods and beneficial effects of each step in this embodiment have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0093] The execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 401a to 403a can be device A; or the execution subject of steps 401a and 402a can be device A, and the execution subject of step 403a can be device B; and so on.

[0094] Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 401a, 402a, etc., are used to distinguish different operations, and the numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. The descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0095] Figure 5 shows a schematic diagram of the structure of a computer node 50 provided in an embodiment of this disclosure. The computing node 50 includes a processing unit 51 and a network transmission module 52. The processing unit 51 is configured to run a first application, and the network transmission module 52 is configured to: receive a computing task issued by the first application, the computing task including metadata of a first subset of data participating in the computing task from a first dataset, the data in the first subset being distributed and stored in the memory of the first computing node; generate memory distribution information of the first subset in the memory of the first computing node based on the metadata of the first subset; read data from the memory of the first subset based on the memory distribution information of the first subset; receive data from a second subset sent by a second computing node, the second subset being a subset of data participating in the computing task from a second dataset; perform calculations on the data in the first subset and the data in the second subset, and write the calculation result data into the memory of the first computing node.

[0096] In an optional embodiment, the metadata of the first data subset includes the metadata of the first dataset and the location information of the first data subset in the first dataset;

[0097] When the network transmission module 52 generates the memory distribution information of the first data subset based on the metadata of the first data subset, it is specifically configured to: generate the memory distribution information of the first data subset based on the metadata of the first data subset, including: determining the global memory region occupied by the first dataset in the memory of the first computing node based on the metadata of the first dataset; and generating the memory distribution information of the first data subset based on the location information of the first data subset in the first dataset and the global memory region.

[0098] In an optional embodiment, the metadata of the first dataset includes at least: the number of dimensions of the first dataset, the size of each dimension, the data type, and the data filling strategy information; when the network transmission module 52 determines the global memory area occupied by the first dataset in the memory of the first computing node based on the metadata of the first dataset, it is specifically configured to: calculate the memory size required by the first dataset based on the number of dimensions, the size of each dimension, the data type, and the data filling strategy information of the first dataset; determine the global memory area based on the memory size and the starting address of the memory allocated to the first dataset; wherein, the data in the first dataset is stored continuously in the global memory area according to the specified dimensional order.

[0099] In an optional embodiment, the location information of the first data subset in the first dataset includes at least: the offset and size of the first data subset in each dimension; when the network transmission module 52 generates the memory distribution information of the first data subset based on the location information of the first data subset in the first dataset and the global memory region, it is specifically configured to: traverse the global memory region based on the offset and depth information of the first data subset in each dimension and the specified dimensional order to obtain at least one data segment included in the first data subset; determine at least one local memory region occupied by the first data subset based on the storage location of the at least one data segment in the global memory region; generate an address linked list based on the starting address, space size, and order relationship between the at least one local memory region, and use the address linked list as the memory distribution information of the first data subset.

[0100] In one optional embodiment, the data in the second data subset includes multiple data segments distributed and stored in the memory of the second computing node, with different data segments stored in different memory regions. When the network transmission module 52 receives data from the second data subset sent by the second computing node, it is specifically configured to: receive a first data packet sent by the second computing node, the first data packet including multiple first payload fields, each first payload field carrying a data segment; parse the multiple data segments from the multiple first payload fields in the first data packet to obtain the data in the second data subset; or, receive multiple second data packets sent by the second computing node, the second data packets including second payload fields, each second payload field carrying a data segment; parse the multiple data segments from the second payload fields in the multiple second data packets to obtain the data in the first data subset.

[0101] In an optional embodiment, the network transmission module 52 is further configured to: receive metadata of the second data subset sent by the second computing node; then, when the network transmission module 52 performs calculations on the data in the first data subset and the data in the second data subset, and writes the calculated result data into the memory of the first computing node, it is specifically configured to: determine the data correspondence between the first data subset and the second data subset for tensor calculation based on the metadata of the first data subset and the metadata of the second data subset; perform calculations on the data segments in the first data subset and the corresponding data segments in the second data subset based on the data correspondence to obtain the calculated result data; determine the first storage location of the calculated result data in the memory of the first computing node based on the metadata of the first data subset; and write the calculated result data into the memory of the first computing node based on the first storage location.

[0102] In an optional embodiment, before receiving data from the second data subset sent by the second computing node, the network transmission module 52 is further configured to: receive a prefetch request issued by the first application, the prefetch request being used to indicate that the second data subset is obtained from the second computing node; and send the prefetch request to the second computing node to trigger the second computing node to return the second data subset after listening to the ready state of the second data subset as ready.

[0103] In an optional embodiment, the network transmission module 52 is further configured to: receive a response message returned by the second computing node in response to the prefetch request, the response message being used to instruct the first network transmission module to wait to receive data from the second data subset sent by the second computing node.

[0104] In an optional embodiment, the network transmission module 52 is further configured to: receive data from the third data subset sent by the second computing node according to a data receiving task issued by the first application; the data receiving task is used to instruct the acquisition of the third data subset from the second dataset from the second computing node; determine a second storage location of the data in the third data subset in the memory of the first computing node according to the metadata of the third data subset; and store the data in the third data subset in the memory of the first computing node according to the second storage location.

[0105] In an optional embodiment, the network transmission module 52 is further configured to: receive a data transmission task issued by a first application, the data transmission task including metadata of a fourth data subset in the first data set, the data in the fourth data subset being distributed and stored in the memory of the first computing node; generate memory distribution information of the fourth data subset based on the metadata of the fourth data subset; read the data in the fourth data subset from the memory of the first computing node based on the memory distribution information of the fourth data subset; and send the data in the fourth data subset to the second computing node.

[0106] In an optional embodiment, the data in the fourth data subset includes multiple data segments distributed and stored in the memory of the first computing node, with different data segments stored in different memory regions; when the network transmission module 52 sends the data in the fourth data subset to the second computing node, it is specifically configured to: add the multiple data segments in the fourth data subset to multiple payload fields of the first data packet, and send the third data packet with the multiple data segments added to it to the second computing node.

[0107] In an optional embodiment, the memory of the first computing node further includes a queue pair, which includes a first queue and a second queue. The first queue is used to carry tensor computation tasks or data sending tasks issued by the first application, and the second queue is used to carry data receiving tasks issued by the first application. When the network transmission module 52 receives a computation task or data sending task issued by the first application, it is specifically configured to: obtain the computation task or data sending task from the first queue in a polling and / or interrupt manner. And when the network transmission module 52 receives a data receiving task issued by the first application, it is specifically configured to: obtain the data receiving task from the second queue in a polling and / or interrupt manner.

[0108] In an optional embodiment, when the network transmission module 52 receives a computing task or data transmission task issued by the first application, it is specifically configured to: periodically poll the first queue to read the computing task or data transmission task written by the first application from the first queue; and / or, receive an interrupt signal triggered after the first application writes a computing task or data transmission task to the first queue, and read the computing task or data transmission task from the first queue according to the interrupt signal.

[0109] In an optional embodiment, when the network transmission module 52 receives a data receiving task issued by the first application, it is specifically configured to: periodically poll the second queue to read the data receiving task written by the first application from the second queue; and / or receive an interrupt signal triggered after the first application writes a data receiving task to the first queue, and read the data receiving task from the first queue according to the interrupt signal.

[0110] The processing unit may also be configured to run a second application, and the network transmission module 52 may be configured to: receive a data transmission task issued by the second application, the data transmission task including metadata of a second data subset in the second data set; the data in the second data subset is distributed and stored in the memory of the second computing node; generate memory distribution information of the second data subset based on the metadata of the second data subset; read the data in the second data subset from the memory of the second computing node based on the memory distribution information of the second data subset; and send the data in the second data subset to the first computing node.

[0111] In an optional embodiment, when the network transmission module 52 reads data from the memory of the second data subset based on the memory distribution information of the second data subset, it is specifically configured to: send a subscription request for the second data subset to the second application, the subscription request being used to request monitoring the ready state of the second data subset; upon receiving a request response from the second application for the subscription request, read the data from the memory of the second data subset based on the memory distribution information of the second data subset; wherein the request response is issued by the second application after monitoring that the ready state of the second data subset is ready.

[0112] In an optional embodiment, before sending a subscription request for the second data subset to the second application, the network transmission module 52 is further configured to: receive a prefetch request sent by the first computing node, the prefetch request being used to trigger the second network transmission module to return to the second data subset after listening to the ready state of the second data subset as ready.

[0113] Furthermore, as shown in Figure 5, the computing node also includes other components such as a display 53, a power supply component 54, and an audio component 55. Figure 5 schematically shows only some components and does not imply that the computing node only includes the components shown in Figure 5. Additionally, the components within the dashed boxes in Figure 5 are optional, not mandatory, and their specific inclusion depends on the product form of the computing node. The computing node in this embodiment can be implemented as a terminal device such as a desktop computer, laptop computer, smartphone, or IoT device, or as a server-side device such as a conventional server, cloud server, or server array. If the computing node in this embodiment is implemented as a terminal device such as a desktop computer, laptop computer, or smartphone, it may include the components within the dashed boxes in Figure 5; if the computing node in this embodiment is implemented as a server-side device such as a conventional server, cloud server, or server array, it may not include the components within the dashed boxes in Figure 5.

[0114] This disclosure also provides a data transmission system, including: a first computing node and a second computing node, the first computing node including a first network transmission module and a first processing unit, the second computing node including a second network transmission module and a second processing unit; the first processing unit is configured to run a first application, the first network transmission module is configured to execute the steps in the above method embodiments; the second processing unit is configured to run a second application, and the second network transmission module is configured to execute the steps in the above method embodiments.

[0115] Accordingly, this disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above method embodiments.

[0116] Accordingly, this disclosure also provides a computer program product, which includes a computer program / instructions that, when executed by a processor, enable the processor to implement the steps in the above method embodiments.

[0117] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0118] The aforementioned communication components are configured to facilitate wired or wireless communication between the device containing the communication components and other devices. The device containing the communication components can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication components receive broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication components also include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth (BT), and other technologies.

[0119] The aforementioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors can sense the boundaries of touch or swipe actions, and also detect the duration and pressure associated with the touch or swipe operation.

[0120] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0121] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker configured to output audio signals.

[0122] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0123] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0126] In a typical configuration, a computing device includes one or more processors (Central Processing Units, CPUs), input / output interfaces, network interfaces, and memory.

[0127] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0128] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be configured to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0129] The terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0130] The above are merely embodiments of this disclosure and are not intended to limit the scope of this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of the claims of this disclosure.

Claims

1. A data processing method, applicable to a first network transmission module on a first computing node, wherein a first application runs on the first computing node, the method comprising: The system receives a computing task issued by a first application. The computing task includes metadata of a first subset of data in a first dataset that participates in the computing task. The data in the first subset of data is stored in the memory of the first computing node. Based on the metadata of the first data subset, generate memory distribution information of the first data subset in the memory of the first computing node; Based on the memory distribution information of the first data subset, data from the first data subset is read from the memory of the first computing node; Receive data from a second data subset sent by a second computing node, wherein the second data subset is a subset of data in a second dataset that participates in the computing task; The data in the first data subset and the data in the second data subset are calculated, and the calculated data is written into the memory of the first computing node.

2. The method according to claim 1, wherein, The metadata of the first data subset includes the metadata of the first dataset and the location information of the first data subset in the first dataset; Based on the metadata of the first data subset, memory distribution information of the first data subset is generated, including: Based on the metadata of the first dataset, determine the global memory region occupied by the first dataset in the memory of the first computing node; Based on the location information of the first data subset in the first dataset and the global memory region, the memory distribution information of the first data subset is generated.

3. The method according to claim 2, wherein, The metadata of the first dataset includes at least: the number of dimensions of the first dataset, the size of each dimension, the data type, and the data filling strategy information; Based on the metadata of the first dataset, determine the global memory region occupied by the first dataset in the memory of the first computing node, including: Calculate the memory size required for the first dataset based on the number of dimensions, the size of each dimension, the data type, and the data filling strategy information. Based on the memory size and the starting address of the memory allocated for the first dataset, determine the global memory region occupied by the first tensor in the memory of the first computing node; The data in the first dataset are stored sequentially in the global memory area according to a specified dimension.

4. The method according to claim 2, wherein, The location information of the first data subset in the first dataset includes at least: the offset and size of the first data subset in each dimension; Based on the location information of the first data subset within the first dataset and the global memory region, memory distribution information of the first data subset is generated, including: Based on the offset and depth information of the first data subset in each dimension and the specified dimensional order, the global memory region is traversed to obtain at least one data segment included in the first data subset. Based on the storage location of the at least one data segment in the global memory region, determine at least one local memory region occupied by the first data subset; An address list is generated based on the starting address, size, and order relationship between the at least one local memory region, and the address list is used as the memory distribution information of the first data subset.

5. The method according to any one of claims 1-4, wherein, The data in the second data subset includes multiple data segments distributed and stored in the memory of the second computing node, with different data segments stored in different memory regions; Receive data from the second data subset sent by the second computing node, including: The system receives a first data packet sent by the second computing node. The first data packet includes multiple first payload fields, and each first payload field carries a data segment. The system then parses the multiple data segments from the multiple first payload fields in the first data packet to obtain the data in the second data subset. or Receive multiple second data packets sent by the second computing node, the second data packets including a second payload field, the second payload field carrying a data segment; parse the multiple data segments from the second payload field of the multiple second data packets to obtain data in the first data subset.

6. The method according to any one of claims 1-4, wherein, Also includes: Receive metadata about the second data subset sent by the second computing node; The calculation is performed on the data in the first data subset and the data in the second data subset, and the calculation result is written into the memory of the first computing node, including: Based on the metadata of the first data subset and the metadata of the second data subset, determine the data correspondence between the first data subset and the second data subset for tensor computation; Based on the data correspondence, the data segments in the first data subset and the corresponding data segments in the second data subset are calculated to obtain the calculated result data; Based on the metadata of the first data subset, determine the first storage location of the computation result data in the memory of the first computing node; The calculation result data is written into the memory of the first computing node according to the first storage location.

7. The method according to claim 1, wherein, Before receiving data from the second subset of data sent by the second computing node, the process also includes: Receive a prefetch request from the first application, the prefetch request being used to instruct the second data subset to be retrieved from the second computing node; The prefetch request is sent to the second computing node to trigger the second computing node to return the second data subset after it detects that the second data subset is ready.

8. The method according to claim 7, wherein, Also includes: The first network transmission module receives a response message from the second computing node in response to the prefetch request. The response message is used to instruct the first network transmission module to wait to receive data from the second data subset sent by the second computing node.

9. The method according to any one of claims 1-8, wherein, Also includes: Receive tasks based on data sent by the first application; Receive data from the third data subset sent by the second computing node; The data receiving task is used to instruct the acquisition of a third data subset from the second dataset from the second computing node; Based on the metadata of the third data subset, determine the second storage location of the data in the third data subset in the memory of the first computing node; According to the second storage location, the data in the third data subset is stored in the memory of the first computing node.

10. The method according to any one of claims 1-8, wherein, Also includes: Receive a data sending task issued by a first application. The data sending task includes metadata of a fourth data subset in the first dataset. The data in the fourth data subset is distributed and stored in the memory of the first computing node. Generate memory distribution information of the fourth data subset based on the metadata of the fourth data subset. Based on the memory distribution information of the fourth data subset, data in the fourth data subset is read from the memory of the first computing node; The data in the fourth data subset is sent to the second computing node.

11. The method according to claim 10, wherein, The data in the fourth data subset includes multiple data segments distributed and stored in the memory of the first computing node, with different data segments stored in different memory regions; Sending data from the fourth data subset to the second computing node includes: Multiple data fragments from the fourth data subset are added to multiple payload fields of the first data packet, and a third data packet with the multiple data fragments added is sent to the second computing node.

12. The method according to claim 11, wherein, The memory of the first computing node also includes a queue pair, which includes a first queue and a second queue. The first queue is used to carry tensor computing tasks or data sending tasks issued by the first application, and the second queue is used to carry data receiving tasks issued by the first application. Receiving computing tasks or data transmission tasks issued by the first application includes: retrieving computing tasks or data transmission tasks from the first queue in a polling and / or interrupt manner; Receiving data reception tasks issued by the first application includes: retrieving data reception tasks from the second queue in a polling and / or interrupt manner.

13. The method according to claim 12, wherein the queue pair further includes a third queue adapted to the queue pair, the third queue being configured to write completion messages of the tensor computation task, the data sending task, or the data receiving task issued by the first application.

14. A data processing method applied to a second network transmission module on a second computing node, wherein a second application runs on the second computing node, the method comprising: Receive a data sending task issued by the second application, wherein the data sending task includes metadata of a second subset of data in the second dataset; The data in the second data subset is distributed and stored in the memory of the second computing node. Based on the metadata of the second data subset, the memory distribution information of the second data subset is generated. Based on the memory distribution information of the second data subset, data from the second data subset is read from the memory of the second computing node; The data in the second data subset is sent to the first computing node.

15. The method according to claim 14, wherein, Based on the memory distribution information of the second data subset, data from the second data subset is read from the memory of the second computing node, including: Send a subscription request for the second data subset to the second application, the subscription request being used to request listening to the ready state of the second data subset; Upon receiving a response to the subscription request from the second application, data from the second data subset is read from the memory of the second computing node based on the memory distribution information of the second data subset. The request response is issued by the second application after it detects that the second data subset is ready.

16. The method according to claim 15, wherein, Before sending a subscription request for the second data subset to the second application, the method further includes: The system receives a prefetch request sent by the first computing node. The prefetch request is used to trigger the second network transmission module to return the second data subset after it detects that the second data subset is ready.

17. A computing node, comprising: Processing unit and network transmission module; Wherein, the processing unit is configured to run a first application, and the network transmission module is configured to execute the steps of the method according to any one of claims 1-13; and / or, the processing unit is configured to run a second application, and the network transmission module is configured to execute the steps of the method according to any one of claims 14-16.

18. A data transmission system, comprising: A first computing node and a second computing node, the first computing node including a first network transmission module and a first processing unit, the second computing node including a second network transmission module and a second processing unit; the first processing unit is configured to run a first application, the first network transmission module is configured to execute the steps of the method according to any one of claims 1-13; the second processing unit is configured to run a second application, the second network transmission module is configured to execute the steps of the method according to any one of claims 14-16.

19. A computer storage medium storing a computer program / instructions that, when executed by a processor, cause the processor to perform the steps of the method according to any one of claims 1-13 or 14-16.

20. A computer program product comprising a computer program / instructions that, when executed by a processor, cause the processor to perform the steps of the method of any one of claims 1-13 or 14-16.

Images