Data processing method and device, storage medium and program product

Abstract

The embodiment of the invention provides a data processing method and device, a storage medium and a program product. In the embodiment of the invention, address calculation and data calculation are issued to a network transmission module for a calculation task of cross-network transmission; on one hand, the network transmission module calculates and generates memory distribution information of a first data subset according to meta-information of the first data subset in a memory of a first calculation node, and reads data of the first data subset from the memory according to the memory distribution information; on the other hand, receiving data of a second data subset in a memory in a second computing node; furthermore, data calculation is carried out on the data of the first data subset and the data of the second data subset, calculation result data is written into a memory of the first calculation node, processor resources on the calculation nodes are liberated from calculation tasks, and consumption of the calculation resources on the calculation nodes is reduced; in addition, based on the distribution of the calculation task, the access frequency of the memory is reduced, and the calculation efficiency is improved.

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202411524323.0 (22) Application Date 2024.10.29 (71) Applicant Cloud Intelligent Asset Holdings (Singapore) Pte Ltd Address 51 Bras Basa Road, Singapore, Laizaanda Tower #03-06 (72) Inventors Fang Xing, Fang Zhiyong, Yang Hang (74) Patent Agency Beijing Taihe Jiusi Intellectual Property Agency Co., Ltd. 11610 Patent Attorney Zhang Ai, Liu Ge (51) Int.Cl. G06F 9 / 50 (2006.01) (54) Invention Title Data Processing Method, Device, Storage Medium and Program Product (57) Abstract This application provides a data processing method, device, storage medium and program product. In this embodiment, for computational tasks transmitted across a network, address calculation and data calculation are delegated to the network transmission module. The network transmission module calculates and generates memory distribution information for the first data subset based on the metadata of the first data subset in the memory of the first computing node, and reads the data of the first data subset from memory based on this memory distribution information. Simultaneously, it receives data from the second data subset in the memory of the second computing node. Furthermore, it performs data calculations on the data of the first and second data subsets and writes the calculated results into the memory of the first computing node. This frees up processor resources on the computing node from computational tasks, reducing the consumption of computational resources on the computing node. Additionally, by delegating computational tasks, the frequency of memory access is reduced, improving computational efficiency. Claims 4 pages, Description 19 pages, Drawings 7 pages, CN 121957834 A 2026.05.01 CN 1 21 95 78 34 A 1. A data processing method, characterized in that it is applicable to a first network transmission module on a first computing node, wherein a first application runs on the first computing node, the method comprising: receiving a computing task issued by the first application, the computing task including metadata of a first data subset participating in the computing task in a first dataset, wherein the data in the first data subset is dispersedly stored in the memory of the first computing node; generating memory distribution information of the first data subset in the memory of the first computing node based on the metadata of the first data subset; reading data in the first data subset from the memory of the first computing node based on the memory distribution information of the first data subset; receiving data in a second data subset sent by a second computing node, the second data subset being a data subset participating in the computing task in a second dataset; performing calculations on the data in the first data subset and the data in the second data subset, and converting the calculated results into a data set.The data is written into the memory of the first computing node. 2. The method according to claim 1, wherein the metadata of the first data subset includes the metadata of the first dataset and the location information of the first data subset within the first dataset; generating memory distribution information of the first data subset based on the metadata of the first data subset includes: determining the global memory region occupied by the first dataset in the memory of the first computing node based on the metadata of the first dataset; generating memory distribution information of the first data subset based on the location information of the first data subset within the first dataset and the global memory region. 3. The method according to claim 2, wherein the metadata of the first dataset includes at least: the number of dimensions of the first dataset, the size of each dimension, the data type, and the data filling strategy information; determining the global memory region occupied by the first dataset in the memory of the first computing node based on the metadata of the first dataset includes: calculating the memory size required by the first dataset based on the number of dimensions, the size of each dimension, the data type, and the data filling strategy information; determining the global memory region occupied by the first tensor in the memory of the first computing node based on the memory size and the starting address of the memory allocated to the first dataset; wherein the data in the first dataset is stored continuously in the global memory region according to the specified dimensional order. 4. The method according to claim 2, wherein the location information of the first data subset in the first dataset includes at least: the offset and size of the first data subset in each dimension; generating memory distribution information of the first data subset based on the location information of the first data subset in the first dataset and the global memory region includes: traversing the global memory region based on the offset and depth information of the first data subset in each dimension and the specified dimensional order to obtain at least one data segment included in the first data subset; determining at least one local memory region occupied by the first data subset based on the storage location of the at least one data segment in the global memory region; generating an address linked list based on the starting address, space size, and order relationship between the fields of the at least one local memory region, and using the address linked list as the memory distribution information of the first data subset. 5. The method according to any one of claims 1-4, characterized in that the data in the second data subset comprises multiple data segments distributed and stored in the memory of the second computing node, with different data segments stored in different memory regions; receiving the data in the second data subset sent by the second computing node includes:The system receives a first data packet sent by the second computing node, the first data packet including multiple first payload fields, each first payload field carrying a data segment; parses the multiple data segments from the multiple first payload fields in the first data packet to obtain data in the second data subset; or receives multiple second data packets sent by the second computing node, the second data packet including a second payload field, each second payload field carrying a data segment; parses the multiple data segments from the second payload fields in the multiple second data packets to obtain data in the first data subset. 6. The method according to any one of claims 1-4, characterized in that it further comprises: receiving metadata of the second data subset sent by the second computing node; performing calculations on the data in the first data subset and the data in the second data subset, and writing the calculation result data into the memory of the first computing node, comprising: determining the data correspondence between the first data subset and the second data subset for tensor calculation based on the metadata of the first data subset and the metadata of the second data subset; performing calculations on the data segments in the first data subset and the corresponding data segments in the second data subset based on the data correspondence to obtain the calculation result data; determining a first storage location of the calculation result data in the memory of the first computing node based on the metadata of the first data subset; and writing the calculation result data into the memory of the first computing node based on the first storage location. 7. The method according to claim 1, characterized in that, before receiving data in the second data subset sent by the second computing node, it further comprises: receiving a prefetch request issued by the first application, the prefetch request being used to indicate obtaining the second data subset from the second computing node; sending the prefetch request to the second computing node to trigger the second computing node to return the second data subset after listening to the ready state of the second data subset as ready. 8. The method according to claim 7, characterized in that, it further comprises: receiving a response message returned by the second computing node in response to the prefetch request, the response message being used to indicate that the first network transmission module waits to receive data in the second data subset sent by the second computing node. 9. The method according to any one of claims 1-8, characterized in that, it further comprises: receiving data in the third data subset sent by the second computing node according to a data receiving task issued by the first application; the data receiving task being used to indicate obtaining the third data subset in the second data subset from the second computing node; determining a second storage location of the data in the third data subset in the memory of the first computing node according to the metadata of the third data subset; Claims 2 / 4 pages 3 CN 121957834A. Storing the data in the third data subset into the memory of the first computing node according to the second storage location. 10. The method according to any one of claims 1-8, further comprising: receiving a data sending task issued by a first application, the data sending task including metadata of a fourth data subset in the first data set, the data in the fourth data subset being distributed and stored in the memory of the first computing node; generating memory distribution information of the fourth data subset based on the metadata of the fourth data subset; reading the data in the fourth data subset from the memory of the first computing node based on the memory distribution information of the fourth data subset; and sending the data in the fourth data subset to the second computing node. 11. The method according to claim 10, wherein the data in the fourth data subset includes multiple data fragments distributed and stored in the memory of the first computing node, with different data fragments stored in different memory regions; sending the data in the fourth data subset to the second computing node includes: adding the multiple data fragments in the fourth data subset to multiple payload fields of a first data packet, and sending a third data packet with the multiple data fragments added to it to the second computing node. 12. The method according to claim 11, wherein the memory of the first computing node further includes a queue pair, the queue pair including a first queue and a second queue, the first queue being used to carry tensor computation tasks or data transmission tasks issued by the first application, and the second queue being used to carry data reception tasks issued by the first application; receiving computation tasks or data transmission tasks issued by the first application includes: obtaining computation tasks or data transmission tasks from the first queue in a polling and / or interrupt manner; receiving data reception tasks issued by the first application includes: obtaining data reception tasks from the second queue in a polling and / or interrupt manner. 13. A data processing method, characterized in that a second network transmission module is applied to a second computing node, the second computing node running a second application, the method comprising: receiving a data transmission task issued by the second application, the data transmission task including metadata of a second data subset in a second dataset; the data in the second data subset being dispersedly stored in the memory of the second computing node, generating memory distribution information of the second data subset based on the metadata of the second data subset; reading data from the memory of the second data subset based on the memory distribution information of the second data subset; and sending the data in the second data subset to a first computing node. 14. The method according to claim 13, characterized in that reading data from the memory of the second data subset based on the memory distribution information of the second data subset includes:Sending a subscription request for the second data subset to the second application, the subscription request being used to request monitoring the ready state of the second data subset; upon receiving a request response from the second application for the subscription request, reading data from the memory of the second data subset based on the memory distribution information of the second data subset; wherein the request response is issued by the second application after monitoring the ready state of the second data subset as ready. 15. The method according to claim 14, characterized in that, before sending the subscription request for the second data subset to the second application, it further includes: receiving a prefetch request sent by the first computing node, the prefetch request being used to trigger the second network transmission module to return the second data subset after monitoring the ready state of the second data subset as ready. 16. A computing node, characterized in that it comprises: a processing unit and a network transmission module; wherein the processing unit is configured to run a first application, and the network transmission module is configured to execute the steps of the method according to any one of claims 1-12; and / or, the processing unit is configured to run a second application, and the network transmission module is configured to execute the steps of the method according to any one of claims 13-15. 17. A data transmission system, characterized in that it comprises: a first computing node and a second computing node, the first computing node comprising a first network transmission module and a first processing unit, the second computing node comprising a second network transmission module and a second processing unit; the first processing unit is configured to run a first application, and the first network transmission module is configured to execute the steps of the method according to any one of claims 1-12; the second processing unit is configured to run a second application, and the second network transmission module is configured to execute the steps of the method according to any one of claims 13-15. 18. A computer storage medium, characterized in that it stores a computer program / instructions, which, when executed by a processor, cause the processor to implement the steps of the method according to any one of claims 1-12 or claims 13-15. 19. A computer program product, characterized in that it comprises a computer program / instructions, which, when executed by a processor, cause the processor to perform the steps of the method according to any one of claims 1-12 or 13-15. Claims 4 / 4 Page 5 CN 121957834 A Data Processing Method, Apparatus, Storage Medium and Program Product Technical Field

[0001] This application relates to the field of network communication technology, and more particularly to a data processing method, apparatus, storage medium and program product. Background Art

[0002] With the development of deep learning technology, tensor computation has been widely used in applications such as Artificial Intelligence (AI) and high-performance computing. Tensors are an extension of the array concept to a higher dimension, and can be understood as multidimensional arrays of three dimensions or more, and are gradually becoming a new data structure standard.

[0003] In actual model training or inference, in order to improve performance and efficiency, different tensor computation tasks are assigned to different computing nodes for execution. In this process, one computing node may need the Tensor Tile (localized sub-block in tensor data) on another computing node to participate in the computation with the Tensor Tile in the local tensor data, which involves the tensor computation process of cross-network transmission.

[0004] The existing tensor computation process involving cross-network transmission is mainly completed by the two applications, involving the computation process of the memory address list of Tensor Tile, the memory reading process of the Tensor Tile at both ends, the network transmission process of the remote Tensor Tile, and the computation process of the Tensor Tile at both ends. The entire tensor computation process not only has a large computational resource overhead, but also has low computational efficiency.

[0005] This application provides a data processing method, apparatus, storage medium, and program product to reduce the computational resource overhead of computing nodes performing data computation and the frequency of accessing node memory, thereby improving computational efficiency.

[0006] An embodiment of this application provides a data processing method applicable to a first network transmission module on a first computing node, wherein a first application runs on the first computing node. The method includes: receiving a computation task issued by the first application, wherein the computation task includes metadata of a first subset of data participating in the computation task in a first dataset, wherein the data in the first subset of data is dispersedly stored in the memory of the first computing node; generating memory distribution information of the first subset of data in the memory of the first computing node based on the metadata of the first subset of data; reading data in the first subset of data from the memory of the first computing node based on the memory distribution information of the first subset of data; receiving data in a second subset of data sent by a second computing node, wherein the second subset of data is a subset of data participating in the computation task in a second dataset; performing computation on the data in the first subset of data and the data in the second subset of data, and writing the computation result data into the memory of the first computing node.

[0007] This application embodiment also provides a data processing method applied to a second network transmission module on a second computing node, wherein a second application runs on the second computing node, the method comprising: receiving data sent by the second application.A data transmission task is performed, the data transmission task including metadata of a second subset of a second dataset; the data in the second subset is distributed and stored in the memory of the second computing node, and memory distribution information of the second subset is generated based on the metadata of the second subset; based on the memory distribution information of the second subset, the data in the second subset is read from the memory of the second computing node; the data in the second subset is transmitted to the first computing node.

[0008] This application embodiment also provides a computing node, including: a processing unit and a network transmission module; wherein, the processing unit is used to run a first application, the network transmission module is used to execute the steps in the various methods of the first network transmission module provided in this application embodiment applicable to the first computing node; and / or, the processing unit is used to run a second application, the network transmission module is used to execute the steps in the various methods of the second network transmission module provided in this application embodiment applicable to the second computing node.

[0009] This application embodiment also provides a data transmission system, including: a first computing node and a second computing node, the first computing node including a first network transmission module and a first processing unit, the second computing node including a second network transmission module and a second processing unit; the first processing unit is used to run a first application, the first network transmission module is used to execute the steps in the various methods provided in this application embodiment applicable to the first network transmission module on the first computing node; the second processing unit is used to run a second application, the second network transmission module is used to execute the steps in the various methods provided in this application embodiment applicable to the second network transmission module on the second computing node.

[0010] This application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the above methods.

[0011] This application embodiment also provides a computer program product, which includes a computer program / instructions, which, when executed by a processor, causes the processor to implement the steps in the above method embodiments.

[0012] In this embodiment of the application, for computing tasks involving cross-network transmission, the calculation of data addresses and data calculations are delegated to the network transmission module. The network transmission module, on the one hand, calculates and generates memory distribution information for the first data subset based on the metadata of the first data subset in the memory of the first computing node, and reads the data of the first data subset from memory based on this memory distribution information; on the other hand, it receives data of the second data subset in the memory of the second computing node; furthermore, it performs data calculations on the data of the first data subset and the data of the second data subset, and writes the result of the data calculations into the first...In the memory of the computing node, the processor resources on the computing node are freed from address calculation and data calculation, reducing the consumption of computing resources on the computing node; in addition, based on the decentralization of computing tasks, the frequency of memory access is also reduced, improving computing efficiency.

[0013] The accompanying drawings, which are included to provide a further understanding of this application and constitute a part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an improper limitation of this application. In the accompanying drawings:

[0014] Figures 1a and 1b are schematic diagrams of the architecture and process of tensor computation performed by a processor dependent on a computing node;

[0015] Figure 2a is a schematic diagram of a data processing method based on the internal structure of a computing node provided in an exemplary embodiment of this application;

[0016] Figure 2b is a schematic diagram of the memory access process for tensor computation performed inside a computing node provided in an exemplary embodiment of this application;

[0017] Figure 3a is a schematic diagram of a data processing method based on the internal structure of a computing node provided in another exemplary embodiment of this application;

[0018] Figure 3b is a schematic diagram of the data packet format of two transmission methods used by a network transmission module provided in another exemplary embodiment of this application;

[0019] Figure 3c is a schematic diagram of the memory access process for tensor computation performed inside a computing node provided in another exemplary embodiment of this application (page 2 / 19, CN 121957834 A);

[0020] Figure 4a is a schematic diagram of a data processing method provided in yet another exemplary embodiment of this application;

[0021] Figure 4b is a schematic diagram of another data processing method provided in yet another exemplary embodiment of this application;

[0022] Figure 5 is a schematic diagram of the structure of a computing node provided in another exemplary embodiment of this application. Detailed Description

[0023] To make the purpose, technical solution and advantages of this application clearer, the technical solution of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] It should be noted that, in the case where the embodiments of this application involve user information, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with the relevant laws, regulations and standards of relevant countries and regions, andA corresponding operation entry point is provided for users to choose to authorize or deny. In addition, the various models involved in this application (including but not limited to language models or large models) comply with relevant laws and standards.

[0025] Tensor data is a multidimensional array, and memory is a linear address space. Tensor data is linearly stored in memory according to the order between multiple dimensions; that is, tensor data, as a multidimensional array, will be mapped to a one-dimensional continuous memory space for storage. The storage order of tensor data can be stored in row-major or column-major order, but is not limited to this. Among them, storing in row-major order means that the elements in the tensor data will be stored in the order of rows; storing in column-major order means that the elements in the tensor data will be stored in the order of columns.

[0026] For example, for three-dimensional tensor data, assuming that the shape of the three-dimensional tensor data is (M, N, K), where M is the number of rows, N is the number of columns, and K is the number of data included at each position, i.e., the intersection of rows and columns, for example, it can be an array. The storage methods of the three-dimensional tensor data in row-major and column-major order are described below respectively.

[0027] If the three-dimensional tensor data is stored in row-major order, the first row and first column of K data are stored first, followed by the first row and second column of K data. This continues until the data corresponding to each column of the first row is stored. Then, the storage of the second row of data can begin. The data storage method for the second row can refer to the storage method for the first row. The storage of the three-dimensional tensor data is complete when all rows of the data are stored.

[0028] If the three-dimensional tensor data is stored in column-major order, the first column and first row of K data are stored first, followed by the first column and second row of K data. This continues until the data corresponding to each row of the first column is stored. Then, the storage of the second column of data can begin. The data storage method for the second column can refer to the storage method for the first column. The storage of the three-dimensional tensor data is complete when all columns of the data are stored.

[0029] Tensor computation is typically performed at the tensor tile level to optimize tensor computation performance, especially when dealing with high-dimensional tensor data. By dividing the tensor data into smaller, local tensor tiles as needed, multiple processing units can be used to perform parallel computation on different tensor tiles, thereby improving tensor computation performance. Although tensor data is linearly stored in memory, the data in a particular tensor tile will be scattered across different memory regions. This is because the storage order affects how the tensor data is arranged in memory. For example, if stored in row-major order, the data within a row is continuous, but the data between different columns is not continuous. If the tensor tile involves...If there are multiple rows, the data corresponding to these rows will be scattered in different locations in memory; for example, if the data is stored in column-major order, the data within a column is continuous, but the data between different rows is not continuous. If a Tensor Tile involves multiple columns, the data corresponding to these columns will be scattered in different locations in memory.

[0030] The data of a Tensor Tile may be divided into multiple data segments in memory. For a certain data segment, the data within the segment is continuous, but the data segments may be separated. For example, if the tensor data is a three-dimensional array, it is equivalent to a large cube. If a small cube is cut out from the large cube, the small cube is an example of a Tensor Tile. The storage location of its corresponding data in memory can be regarded as a set of separate continuous memory spaces. In the "separate continuous memory space", the continuity refers to the continuity within the segment; the separation refers to the separation between different data segments, that is, they are scattered on different physical addresses.

[0031] When transmitting Tensor Tile data over a network, DMA (Direct Memory Access) technology can be used. The application calculates the Scatter-Gather-List (SGL) of the Tensor Tile and provides it to the DMA. The DMA reads data fragments from the Tensor Tile one by one according to the memory addresses in the SGL and encapsulates them into data packets for transmission over the network to the other end. Here, the SGL is a data structure used to describe the scattered data fragments. The SGL includes multiple sequentially arranged descriptors, forming a list of memory addresses. Each descriptor points to the starting address and size of the data fragment to be transmitted. Therefore, the memory address of the corresponding data fragment can be determined according to the descriptor, so that the data fragment in the Tensor Tile can be retrieved from the memory space corresponding to that memory address, thereby realizing the network transmission of data fragments. Among them, calculating SGL is a loop operation related to the dimension of tensor data, which will consume corresponding computing resources. For example, for 3-dimensional tensor data of 1024*1024*1024, if the memory space corresponding to the Tensor Tile contains 1024*1024=1M scattered continuous spaces, each time a data segment in a continuous space is accessed, an address calculation must be performed. Therefore, to achieve data access to the Tensor Tile, 1M address calculations need to be performed.

[0032] Based on the above, referring to Figures 1a and 1b, the tensor calculation process involving cross-network transmission includes: referring to S1 in Figure 1a, the local application and the remote application respectively calculate the memory address list of the Tensor Tile participating in the tensor calculation; then referring toIn step S2 of Figure 1a, the local application and the remote application respectively send computing tasks to the network interface card (NIC) device. The computing tasks include their respective memory address lists or the starting address of the memory address lists. Referring to step S3 of Figure 1a, the remote NIC device retrieves the remote Tensor Tile from the memory of the remote application based on the corresponding memory address list. Referring to step S4 of Figure 1a, the local NIC device requests the remote Tensor Tile through the network. Referring to step 5 of Figure 1b, the local NIC device writes the remote Tensor Tile into its local memory. Referring to steps S6-S9 of Figure 1b, when the data preparation is complete, the local application retrieves its own Tensor Tile from its memory based on the calculated memory address list, then retrieves the remote Tensor Tile from its memory, performs computing operations on the local Tensor Tile and the remote Tensor Tile, and writes the computing results back to its local memory.

[0033] In the tensor computation process shown in Figures 1a-1b, tensor computation and memory address list computation consume computational resources; and multiple memory accesses are involved, including the local network card device writing the remote Tensor Tile to its local memory; retrieving the local Tensor Tile from its local memory; reading the remote Tensor Tile from its local memory; and writing the computation result back to its local memory. Therefore, at least four memory accesses are required, which not only wastes computational resources but also leads to low computational efficiency due to frequent memory accesses.

[0034] In the above content, taking tensor data as a data structure and its computation process as an example, the problem of wasted computational resources faced by data computation involving cross-network transmission and the problem of low computational efficiency caused by frequent memory accesses during data computation are illustrated by example.

[0035] In the embodiments of this application, data with data structures similar to tensor data are collectively referred to as datasets. Data computation means performing computations on the data in the dataset, and cross-network transmission data computation may involve performing computations on data in datasets stored on different ends. Instruction manual 4 / 19 pages 9 CN 121957834 A

[0036] Further, the computation between datasets is performed at the data subset granularity, thereby enabling parallel computation of different data subsets using multiple processing units, improving computational performance. Here, a data subset is a subset of the dataset, and the data within the data subset is distributed and stored in the memory of the computing nodes. The distributed data can be organized into multiple data segments; for a given data segment, the data within the segment is continuous, while different data segments may be separate.

[0037] Further, combined with the tensor computation process shown in Figures 1a-1b, it can be seen that the data in the dataset is transmitted across networks...During data computation, there are problems such as wasted computing resources and low computational efficiency caused by frequent memory access.

[0038] Based on this, in this embodiment of the application, for computing tasks transmitted across the network, the calculation of data address and data computation are delegated to the network transmission module. On the one hand, the network transmission module calculates and generates memory distribution information of the first data subset based on the metadata of the first data subset in the memory of the first computing node, and reads the data of the first data subset from the memory based on the memory distribution information; on the other hand, it receives the data of the second data subset in the memory of the second computing node; then, it performs data computation on the data of the first data subset and the data of the second data subset, and writes the result data of the data computation into the memory of the first computing node, thereby freeing the processor resources on the computing node from data computation and address computation, thereby reducing the consumption of computing resources on the computing node; in addition, based on the delegation of computing tasks, the frequency of memory access is also reduced, and the computational efficiency is improved.

[0039] Further optionally, in the tensor calculation process shown in Figures 1a-1b, if the processing unit of the computing node includes multiple processor cores and these cores share memory, if the processor core responsible for performing the tensor calculation task frequently accesses this shared memory, it will interfere with other processor cores accessing the shared memory, thereby reducing the overall utilization and performance of the processing unit. It can be seen that similar problems of low utilization and performance degradation of the processing unit due to shared memory also exist in the calculation process of data in the dataset. Accordingly, in the optional embodiment of this application, since the calculation task is offloaded to the network transmission module for execution, the frequency of memory access on the computing node is reduced, thereby reducing interference with other processor cores accessing the shared memory and improving the overall utilization and performance of the processing unit.

[0040] Further optionally, in the tensor calculation process shown in Figures 1a-1b, when transmitting tensor data across the network, the data is transmitted one data segment at a time, with data segments in continuous memory regions as the granularity. Each data segment is encapsulated into a data packet. Each data packet includes not only the data segment but also protocol fields such as source address and destination address. The network bandwidth consumed by these protocol fields can be called network coding overhead. If each data segment is small, this will result in a small effective data load for each data packet, meaning the actual data to be transmitted is small. Since network coding is required for each data packet, the network coding overhead is relatively large, leading to a large network bandwidth overhead problem. Therefore, similar large network bandwidth overhead problems also exist in the data calculation process within the dataset. Accordingly, in the optional embodiment of this application, based on the offloading of memory distribution information to the network transmission module, the network...The transmission module can acquire all data fragments included in the data subset, and then package multiple data fragments corresponding to the data subset into a data packet and send it to the computing node at the other end. This can significantly reduce the new overhead of network encoding of the data packet and improve the effective data load of a single data packet.

[0041] The technical solutions provided by the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0042] It should be noted in advance that in the embodiments of this application, tensor data and tensor computation transmitted across the network are used as examples, but this does not constitute a limitation on the embodiments of this application. That is to say, the data processing method provided by the embodiments of this application can be applied to, but is not limited to, tensor data as a data structure, and the data of the applicable data structure is collectively referred to as a dataset. Data computation refers to the computation of data in the dataset, and the computation is performed at the granularity of the data subset. The data subset can be implemented as the Tensor Tile described above.

[0043] Referring to Figures 2a-2b, the tensor computation process transmitted across the network involves: a first computing node 20 and a second computing node 21 participating in the tensor computation task; wherein, the first computing node 20 is the local end and the second computing node 21 is the remote end. It should be noted that in this embodiment, there is no limitation on whether the first computing node 20 is the local end or the remote end. This is only an example to facilitate the explanation of related content in subsequent embodiments. Similarly, there is no limitation on whether the second computing node 21 is the remote end or the local end. In this embodiment, the second computing node is used as the remote end as an example.

[0044] In the embodiments of this application, the first computing node 20 and the second computing node 21 refer to nodes that include certain computing resources. The computing resources are various hardware resources with computing capabilities provided by the computing nodes, such as including but not limited to: CPU (Central Processing Unit), DPU (Data Processing Unit), NPU (Neural Network Processing Unit), and GPU and other processing units. Furthermore, the specific implementation of the first computing node and the second computing node is not limited. For example, they can be terminal devices such as desktop computers, laptops, smartphones, or IoT (Internet of Things) devices, or server devices such as conventional servers, cloud servers, or server arrays.

[0045] In this embodiment, the first computing node 20 includes a first network transmission module 24 and runs a first application. The first application generally refers to various applications that rely on machine learning tasks, such as shopping applications, video applications, live streaming applications, etc.Applications include games, language processing applications, etc. Among them, the training and inference process of machine learning tasks involves a large number of tensor computation tasks. Tensor computation tasks refer to tasks that perform various operations and calculations on tensor data, including but not limited to basic arithmetic operations on tensor data, such as addition, subtraction, multiplication and division; as well as matrix operations, tensor reduction, tensor convolution and tensor shrinkage, etc.

[0046] Further, as shown in S201 of FIG2a, the first application sends the tensor computation task to the first network transmission module 24. In this embodiment, the method of sending the tensor computation task is not limited. Optionally, the first application can communicate directly with the first network transmission module and send the tensor computation task directly to the first network transmission module; or, the first application can write the tensor computation task into the memory 22 of the first computing node, for example, it can write it into the queue pair in the memory 22 of the first computing node; then, as shown in S202 of FIG2a, the first network transmission module obtains the tensor computation task from the memory 22 of the first computing node. For example, when a tensor computation task is written into the corresponding queue of a queue pair, the first network transmission module can periodically poll the queue to read the tensor computation task from the queue; or, the first application can send an interrupt signal to the first network transmission module after writing the tensor computation task into the corresponding queue of the queue pair to trigger the first network transmission module to read the tensor computation task from the corresponding queue.

[0047] In this embodiment, for any tensor computation task, the tensor data participating in the tensor computation task includes a first sub-tensor located in the memory 22 of the first computing node and a second sub-tensor located in the memory 23 of the second computing node; the first sub-tensor and the second sub-tensor can be implemented as the Tensor Tile mentioned above; wherein, the first sub-tensor is obtained by splitting the first tensor as needed, and the size of the splitting for different tensor computation tasks can be the same or different, depending on the needs of the tensor computation task; similarly, the second sub-tensor is obtained by splitting the second tensor as needed, and its size is also not limited.

[0048] As mentioned above, the data in a Tensor Tile is distributed across different memory regions. In this embodiment, when the first sub-tensor can be implemented as a Tensor Tile, the data in the first sub-tensor is distributed and stored in the memory 22 of the first computing node. In order to index the data in the first sub-tensor, in this embodiment, the tensor computation task includes the metadata of the first sub-tensor; furthermore, as shown in S203 of FIG2a, the first network transmission module 24 can generate the memory distribution information of the first sub-tensor in the memory 22 of the first computing node based on the metadata of the first sub-tensor; this memory distribution specification 6 / 19 pages 11 CN 121957834 AThe distribution information is used to describe the distribution of data in the first sub-tensor in the memory 22 of the first computing node. Optionally, the memory distribution information can be implemented as a memory address list, which includes at least one memory address. Each memory address corresponds to a segment of continuously stored data, i.e., a data fragment. As shown in Figure 2a, the first sub-tensor in the first tensor includes data fragment a1, data fragment a2, and data fragment an, where n≥2 and n is a natural number. It should be understood that Figure 2a uses the example of the first sub-tensor including multiple data fragments. The first sub-tensor may also include only one data fragment, and there is no limitation on this.

[0049] It should be noted that in the embodiments of this application, for any data fragment, the data fragment is continuous within the segment; different memory addresses may correspond to different data fragments, and different data fragments may be separated. The reasons for the separation of different data fragments can be referred to in the following embodiments.

[0050] Further, in this embodiment, as shown in S204 of FIG2a, the first network transmission module 24 reads data in the first sub-tensor from the memory 22 of the first computing node based on the memory distribution information of the first sub-tensor; and, as shown in S205 of FIG2a, receives data in the second sub-tensor sent by the second computing node 21; wherein, in this embodiment, the order in which the first network transmission module 24 reads the first sub-tensor and receives the second sub-tensor is not limited.

[0051] After the first network transmission module 24 completes the reading of the first sub-tensor and the receiving of the second sub-tensor, the process of the first network transmission module performing tensor calculation will be further described with reference to FIG2b. As shown in Figure 2b, S204 and S205 in this process correspond to reading the data of the first sub-tensor and receiving the data of the second sub-tensor, respectively. Furthermore, as shown in S206 of Figure 2b, the first network transmission module 24 performs tensor calculations on the data in the first and second sub-tensors. As shown in S207 of Figure 2b, the first network transmission module 24 writes the result data of the tensor calculation into the memory 22 of the first computing node. The result data of the tensor calculation can be written to a first storage location, which can be determined based on the metadata of the first sub-tensor. The specific method for determining the first storage location may vary depending on the first application, and this embodiment does not limit this. In an optional embodiment, the storage location of the data fragment in the first sub-tensor in the memory 22 of the first computing node is used as the first storage location. The result data of the tensor calculation can then be written to the first storage location to achieve the purpose of overwriting the old data fragment. Of course, besides this implementation, the first storage location can also be any other storage location that will not overwrite the old data fragment.

[0052] In this embodiment of the application, for computing tasks involving cross-network transmission, the calculation of data address and data calculation are combined.The computation is decentralized to the network transmission module. On one hand, the network transmission module calculates and generates memory distribution information for the first data subset based on the metadata of the first data subset in the memory of the first computing node, and reads the data of the first data subset from memory based on this memory distribution information. On the other hand, it receives data from the second data subset in the memory of the second computing node. Then, it performs data computation on the data of the first and second data subsets, and writes the computational results into the memory of the first computing node. This frees up the processor resources on the computing node from data computation and address computation, thereby reducing the consumption of computing resources on the computing node. Furthermore, based on the decentralization of computational tasks, the frequency of memory access is reduced, improving computational efficiency.

[0053] In this embodiment, tensor data and tensor computation transmitted across networks are still used as examples, but this does not constitute a limitation on the embodiments of this application. That is to say, the data processing method provided by the embodiments of this application can be applied to, but is not limited to, tensor data as a data structure, and the data of the applicable data structure is collectively referred to as a dataset. Data computation refers to the computation of data in the dataset, and the computation is performed at the granularity of a data subset. The data subset can be implemented as the Tensor Tile mentioned above.

[0054] Referring to Figures 3a-3b, another cross-network transmission tensor computation process provided in this embodiment involves: a first computing node 20 and a second computing node 21; wherein, the first computing node 20 serves as the local end, and the second computing node 21 serves as the remote end, the first computing node 20 and the second computing node 21 each include their respective network transmission modules; and, a first application runs on the first computing node, and a second application runs on the second computing node; in this embodiment, the computation process participating in the tensor computation task includes the first application and the second application; the first application sends a tensor computation task to the first network transmission module 24, which involves reading the first sub-tensor of the first tensor participating in the tensor computation task on the local end. And involves receiving data of the second sub-tensor of the second tensor of the second computing node 21; the second application is responsible for instructing the second network transmission module 25 to send the data of the second sub-tensor participating in the tensor calculation task to the first computing node 20; then, after completing the reading of the first sub-tensor and the reception of the second sub-tensor, the tensor calculation is performed, and the result data is written to the local memory. The tensor calculation process of cross-network transmission involved in this embodiment will be described in detail below.

[0055] As shown in FIG3a, in this embodiment, the first network transmission module receives the tensor calculation task issued by the first application. The tensor calculation task includes the meta-information of the first sub-tensor participating in the tensor calculation task in the first tensor.The data of the tensor is distributed and stored in the memory of the first computing node. Optionally, the metadata of the first sub-tensor includes the metadata of the first tensor and the position information of the first sub-tensor in the first tensor. Based on this information, the memory distribution information of the first sub-tensor can be quickly generated. Specifically, the process by which the first network transmission module generates the memory distribution information of the first sub-tensor based on the metadata of the first sub-tensor includes: determining the global memory region occupied by the first tensor in the memory of the first computing node based on the metadata of the first tensor. The global memory region refers to the memory region occupied by each sub-tensor included in the first tensor in the memory of the first computing node; then, the memory distribution information of the first sub-tensor can be generated based on the position information of the first sub-tensor in the first tensor and the global memory region; wherein, the data in the first sub-tensor is distributed and stored in the memory of the first computing node, and this memory distribution information is used to describe the distribution of the data in the first sub-tensor in the memory of the first computing node.

[0056] In an optional embodiment, the metadata of the first tensor includes at least: the number of dimensions of the first tensor, the size of each dimension, the data type, and data padding strategy information; wherein, the number of dimensions of the first tensor can be one-dimensional or multi-dimensional, without limitation; the size of any dimension is the number of elements included in the tensor data in that dimension; the data type refers to the type of data stored, such as single-precision floating-point, integer, double-precision floating-point, etc.; the number of bytes corresponding to different data types is generally different; the data padding strategy information determines how to handle the boundaries of the tensor, and the data padding strategy information is used to add extra values ​​(which can be 0) to the boundaries of the tensor data, thereby changing the size of the tensor data to suit different computational needs. Based on the above metadata of the first tensor, the global memory area occupied by the first tensor in the memory of the first computing node can be determined simply and quickly.

[0057] Specifically, in the process of determining the global memory region occupied by the first tensor in the memory of the first computing node based on the metadata of the first tensor, the memory size required by the first tensor is calculated based on the number of dimensions, the size of each dimension, the data type, and the data filling strategy information of the first tensor; then, the global memory region occupied by the first tensor in the memory of the first computing node is determined based on the memory size and the memory starting address allocated to the first tensor; wherein, the data in the first tensor is stored continuously in the global memory region according to the specified dimensional order.

[0058] Wherein, at least one sub-tensor included in the first tensor is stored in the global memory region, and each sub-tensor has a corresponding storage location in the global memory region. The storage location of the sub-tensor in the global memory region can be determined based on the location information corresponding to each sub-tensor. In an optional embodiment, the location information of the first sub-tensor in the first tensor includes at least: the offset and size of the first tensor in each dimension; wherein, the offset of each dimension refers to the offset relative to the starting address.The number of elements offset from the address; the size of each dimension refers to the number of elements in that dimension. Based on this information, the memory distribution information of the first sub-tensor can be generated simply and quickly.

[0059] Based on the above, in the process of generating the memory distribution information of the first sub-tensor, firstly, according to the offset and size of the first sub-tensor in each dimension on page 8 / 19 of the specification 13 CN 121957834 A and the order of that dimension, the global memory region is traversed to obtain at least one data segment corresponding to the first sub-tensor; in this embodiment, any data segment refers to data stored in a continuous memory space.

[0060] In this embodiment, in the process of traversing the global memory region corresponding to the first tensor, the order of traversing the multiple dimensions included in the tensor data is not limited. Firstly, the global memory region can be traversed by combining the storage order of multiple dimensions of tensor data. For example, it can be referenced whether the storage order is row-first or column-first. If it is stored in row-first order, it means that the elements in the same row in memory are consecutive, so the traversal can be performed first along the row; if it is stored in column-first order, it means that the elements in the same column in memory are consecutive, so the traversal can be performed first along the column. This can make full use of the advantage of memory access to continuous data and improve the hit rate of data segments of the first sub-tensor.

[0061] As shown in Figure 3a, by traversing the global memory region corresponding to the first tensor, at least one data segment corresponding to the first sub-tensor is obtained, such as data segment a1, data segment a2 and data segment an, etc.; where n≥2, and n is a natural number; Figure 3a takes the first sub-tensor including multiple data segments as an example, but it does not constitute a limitation on this embodiment. The first sub-tensor can also include one data segment. In subsequent embodiments, the third and fourth sub-tensors are similar, including one or more data segments, which will not be repeated in subsequent embodiments.

[0062] Next, the first network transmission module determines at least one local memory region occupied by the first sub-tensor based on the storage location of at least one data segment in the global memory region. The at least one local memory region is related to the dimension of the first tensor. If the first tensor has two or more dimensions, the number of local memory regions is multiple; if the first tensor has only one dimension, the number of local memory regions is one. In the case of multiple local memory regions, these regions can be distributed across different storage locations in the global memory region.

[0063] Furthermore, the first network transmission module can generate an address linked list based on the starting address, size, and sequential relationship between the at least one local memory region, serving as the memory distribution information of the first sub-tensor. For example, for each local memory region, the starting address and size of each contiguous local memory region are used as an SGL.Descriptor; SGL Descriptor + pointer to the next SGL Descriptor form an SGL Segment; SGL Segments form an address linked list; further, this address linked list is used as the memory distribution information of the first sub-tensor, so that the first network transmission module can read the data in the first sub-tensor from the memory of the first computing node according to the memory distribution information of the first sub-tensor. It should be understood that the address linked list is only an example of memory distribution information and does not constitute a limitation on this embodiment.

[0064] In this embodiment, as shown in FIG3a, the tensor computation task also involves receiving a second sub-tensor from a remote second computing node to perform tensor computation using the second sub-tensor. Optionally, before receiving data from the second sub-tensor sent by the second computing node, the first network transmission module may receive a prefetch request (not shown in Figure 3a) issued by the first application. This prefetch request indicates that the second sub-tensor should be retrieved from the second computing node. The first network transmission module sends the prefetch request to the second computing node to trigger the second computing node to return the second sub-tensor after detecting that its ready state is ready. Monitoring the ready state of the second sub-tensor means monitoring whether the data of the second sub-tensor has been written into the memory of the second computing node, so as to read the data of the second sub-tensor from the memory of the second computing node. Further optionally, if the ready state of the second sub-tensor is detected as ready, the second computing node may return a response message to the first network transmission module in response to the prefetch request. Upon receiving the response message returned by the second computing node, the first network transmission module waits to receive data from the second sub-tensor sent by the second computing node.

[0065] Optionally, the second network transmission module can send a subscription request for the second sub-tensor to the second application. This subscription request is used to request to listen to the ready state of the second sub-tensor, that is, whether the data that triggers listening to the second sub-tensor has been written into the memory of the second computing node. When the second network transmission module receives a request response from the second application for the subscription request, that is, the data of the second sub-tensor is readable, it can read the data in the second sub-tensor from the memory of the second computing node based on the memory distribution information of the second sub-tensor. This request response is sent by the second application after listening to the ready state of the second sub-tensor as ready. Then, as shown in S205 in FIG3a, the second network transmission module sends the data read from the second sub-tensor to the first computing node.

[0066] In the above optional embodiment, the problem of transmitting data fragments in the sub-tensor is transformed into transmitting data in the sub-tensor in memory.Regarding the state monitoring problem, the application (or the processing unit of the computing node) and the network transmission module can transmit the monitoring results through request and response. Therefore, the processing unit and the network transmission module no longer need to perform complex interactions according to the traditional basic semantics such as receive / send, which can effectively reduce the transmission latency of tensor data.

[0067] Accordingly, the second application can instruct the second network transmission module to send the data in the second sub-tensor. As shown in S2051 of FIG3a, the second application issues a data sending task to the second network transmission module. The data sending task includes the metadata of the second sub-tensor in the second tensor. The metadata of the second sub-tensor includes the metadata of the second tensor and the position information of the second sub-tensor in the second tensor. Further, as shown in S2053 of FIG3a, the second network transmission module generates the memory distribution information of the second sub-tensor according to the metadata of the second sub-tensor. The method of the second application issuing the data sending task is not limited. It can be referred to the method of issuing tensor calculation tasks, which will not be elaborated here.

[0068] In the process of generating the memory distribution information of the second sub-tensor, the second network transmission module determines the global memory region occupied by the second tensor in the memory of the second computing node based on the metadata of the second tensor. The global memory region refers to the memory region occupied by each sub-tensor included in the second tensor in the memory of the second computing node. Then, based on the position information of the second sub-tensor in the second tensor and the global memory region, the memory distribution information of the second sub-tensor is generated. The data in the second sub-tensor is distributed and stored in the memory of the second computing node. The memory distribution information is used to describe the distribution of the data in the second sub-tensor in the memory of the second computing node.

[0069] In an optional embodiment, the metadata of the second tensor includes at least: the number of dimensions of the second tensor, the size of each dimension, the data type, and the data filling strategy information. The number of dimensions of the second tensor can be one-dimensional or multi-dimensional, and there is no limitation on this. Regarding the number of dimensions, the size of each dimension, the data type, and the data filling strategy information of the second tensor, the relevant content of the number of dimensions, the size of each dimension, the data type, and the data filling strategy information of the first tensor can be referred to, and will not be repeated here.

[0070] Furthermore, in determining the global memory region occupied by the second tensor in the memory of the second computing node based on the metadata of the second tensor, the memory size required by the second tensor is calculated based on the number of dimensions, the size of each dimension, the data type, and the data filling strategy information of the second tensor; furthermore, the global memory region occupied by the second tensor in the memory of the second computing node is determined based on the memory size and the starting address of the memory allocated to the second tensor; wherein, the data in the second tensor is stored continuously in the global memory region according to the specified dimensional order.

[0071] The second tensor includes at least one sub-tensor stored in the global memory region. Each sub-tensor has a corresponding storage location in the global memory region, and its storage location in the global memory region can be determined based on its location information. In an optional embodiment, the location information of the second sub-tensor in the second tensor includes at least: the offset and size of the second tensor in each dimension; wherein, the offset in each dimension refers to the number of elements offset relative to the starting address; and the size in each dimension refers to the number of elements in that dimension. Furthermore, in the process of generating the memory distribution information of the second sub-tensor, the global memory region is traversed according to the offset and size of the second sub-tensor in each dimension and the order of that dimension to obtain multiple data fragments corresponding to the second sub-tensor. The traversal order of the multiple dimensions included in the tensor data is not limited during the traversal process; for details, please refer to the relevant content of the memory region on pages 10 / 19 of the global specification corresponding to the first tensor (CN 121957834 A), which will not be elaborated here.

[0072] The multiple data fragments of the second sub-tensor obtained by traversal are scattered and stored in the memory of the second computing node. Any data fragment refers to data stored in a contiguous memory space. The second network transmission module determines at least one local memory region occupied by the second sub-tensor based on the storage location of the multiple data fragments in the global memory region. The at least one local memory region is related to the dimension of the second tensor. If the dimension of the second tensor is two or more, the number of local memory regions is multiple. If the dimension of the second tensor is one, the number of local memory regions is one. In the case of multiple local memory regions, the multiple local memory regions can be different storage locations scattered in the global memory region. Furthermore, the second network transmission module can generate an address linked list based on the starting address, space size, and sequential relationship between at least one local memory region as the memory distribution information of the second sub-tensor. For example, for each local memory region, the starting address and size of each local contiguous memory region are used as an SGL Descriptor; an SGL Descriptor plus a pointer to the next SGL Descriptor forms an SGL Segment; SGL Segments form an address linked list; further, this address linked list is used as the memory distribution information of the second sub-tensor, so that the second network transmission module can read the data in the second sub-tensor from the memory of the second computing node according to the memory distribution information of the second sub-tensor.

[0073] As shown in Figure 3a, the second sub-tensor in the second tensor includes data segment b1, data segment b2 and data segment bm; where m≥2 and m is a natural number; multiple data segments are stored in the global memory region corresponding to the second tensor, notThe same data fragments are stored in different local memory regions; as shown in S2054 of Figure 3a, the second network transmission module reads the data in the second sub-tensor from the memory of the second computing node based on the memory distribution information of the second sub-tensor.

[0074] In the embodiments of this application, the method of transmitting data fragments between the first network transmission module and the second network transmission module is not limited. For example, one method is to package multiple data fragments into a data packet and send it to the other end together. This method can save network bandwidth resources; another method is to divide the data into multiple data packets and send multiple data fragments to the other end in sequence. This method is conducive to the timely and flexible transmission of data fragments. Therefore, combining these two data fragment transmission methods, the implementation method of transmitting data fragments between the second network transmission module and the first network transmission module is given. Optionally, the second network transmission module packages multiple data fragments of the second sub-tensor into a data packet, i.e., the first data packet; sends the first data packet to the first network transmission module; the first network transmission module receives the first data packet sent by the second computing node, as shown in Figure 3b. The first data packet includes multiple first payload fields, such as first payload field c1 - first payload field ck, where k > 1 and k is a natural number; the first payload field refers to the field that carries the data of the second sub-tensor; one first payload field carries one data segment; multiple data segments are parsed from the multiple first payload fields in the first data packet to obtain the data in the second sub-tensor; or, the second network transmission module packages multiple data segments into multiple data packets, i.e., multiple second data packets, and sends the multiple second data packets to the first network transmission module in batches; the first network transmission module receives multiple second data packets sent from the second computing node, as shown in Figure 3b. The second data packet includes second payload fields, such as second payload field d1, second payload field d2, and second payload field dj, where j ≥ 2 and j is a natural number; the second payload field carries one data segment; multiple data segments are parsed from the second payload fields in the multiple second data packets to obtain the data in the second sub-tensor. As shown in Figure 3c, after the first network transmission module completes the reading of the first sub-tensor (as shown in S204 of Figure 3c) and the receiving of the second sub-tensor (as shown in S205 of Figure 3c), the tensor calculation task can be executed using the first network transmission module on the first computing node. The first computing node includes a first network transmission module 24, the memory 22 of the first computing node, and a first processing unit 26. The first application runs on the first processing unit 26 and can send tensor calculation tasks to the first network transmission module. Optionally, as shown in Figure 3b, the first data packet or second data packet used to transmit the second sub-tensor further includes a source address field and a destination address field; wherein, the source address field carries the specification page 11 / 19 16 CN 121957834 AThe destination address field carries the metadata of the first sub-tensor; that is, the first network transmission module can also receive the metadata of the second sub-tensor sent by the second computing node; furthermore, when the first network transmission module performs tensor calculations on the data in the first sub-tensor and the data in the second sub-tensor, and writes the result data of the tensor calculation into the memory of the first computing node, it includes: determining the data correspondence between the first sub-tensor and the second sub-tensor based on the metadata of the first sub-tensor and the second sub-tensor, the data correspondence referring to the correspondence between the data segments in the first sub-tensor and the corresponding data segments in the second sub-tensor; performing tensor calculations on the data segments in the first sub-tensor and the corresponding data segments in the second sub-tensor based on the data correspondence to obtain the result data of the tensor calculation; determining the first storage location of the result data of the tensor calculation in the memory of the first computing node based on the metadata of the first sub-tensor; and writing the result data of the tensor calculation into the memory of the first computing node based on the first storage location.

[0075] It should be understood that the first storage location is only an example of the storage location of the result data and does not constitute a limitation on this embodiment.

[0076] In this embodiment, the implementation structure of the network transmission module is not limited. Any device structure capable of performing the functions of memory distribution information calculation, tensor calculation, and data transmission described in the above embodiments is within the protection scope of this embodiment. In an optional embodiment, as shown in FIG3c, one implementation structure of the first network transmission module includes a newly added TTCA (Tensor Tile Communication Accelerator) module. The TTCA module may include a control path 27 and a tensor calculation module 28. The control path 27 can be used to generate memory distribution information, configure calculation parameters, and control DMA data, etc.; the tensor calculation module 28 is used to perform tensor calculation.

[0077] It should be noted that in this embodiment, in addition to tensor calculation involving cross-network transmission, cross-network tensor transmission and reception processes may also be involved. These transmission and reception processes do not involve tensor calculation, that is, a computing node may need to transmit local tensor data to another computing node. The tensor data transmission and reception process will be described below with reference to FIG4a-FIG.

[0078] It should be understood that, in this embodiment, cross-network transmission of tensor data is still used as an example, but this does not constitute a limitation on the embodiments of this application. That is to say, the data sending and receiving process in this embodiment can be, but is not limited to, tensor data as a data structure, and the data of applicable data structures are collectively referred to as datasets, while data subsets can be implemented as Tensor Tile as described above.

[0079] As shown in Figure 4a, in this embodiment, as shown in S401a of Figure 4a, the first application sends a data receiving task to the memory of the first computing node; then, as shown in S402a of Figure 4a, the first network transmission module obtains the data receiving task sent by the first application. The method by which the first network transmission module obtains the data receiving task is not limited, and can be implemented with reference to the method by which the first network transmission module obtains the tensor calculation task.

[0080] The data receiving task is used to instruct the third sub-tensor in the second tensor to be obtained from the second computing node; the third sub-tensor can be implemented as the Tensor Tile mentioned above, as shown in Figure 4a, the third sub-tensor corresponds to at least one data segment, such as data segment q1, data segment q2 and data segment qn, etc.; where n≥2, and n is a natural number; Figure 4a takes the first sub-tensor including multiple data segments as an example, but this does not constitute a limitation on this embodiment, and the third sub-tensor can also include one data segment.

[0081] Further, in this embodiment, the first network transmission module performs a data receiving task to receive the data of the third sub-tensor. As shown in S403a of FIG4a, when the first network transmission module receives the data receiving task, it generates the memory distribution information of the third sub-tensor according to the metadata of the third sub-tensor. The memory distribution information can be used to determine the second storage location of the third sub-tensor in the memory of the first computing node. Further, as shown in S404a of FIG4a, the first network transmission module receives the data in the third sub-tensor sent by the second computing node. As shown in S405a of FIG4a, the first network transmission module determines the second storage location of the data in the third sub-tensor in the memory of the first computing node according to the memory distribution information of the third sub-tensor. Then, as shown in S406a of FIG4a, the data in the third sub-tensor is stored in the memory of the first computing node according to the second storage location.

[0082] In this embodiment, the second storage location of the third sub-tensor is determined based on the metadata of the third sub-tensor; and the order in which the data of the third sub-tensor is received is not limited.

[0083] As shown in FIG4b, in this embodiment, as shown in S401b of FIG4b, the first application sends a data sending task to the memory of the first computing node; furthermore, as shown in S402b of FIG4b, the first network transmission module obtains the data sending task sent by the first application. The method by which the first network transmission module obtains the data receiving task is not limited, and can be implemented with reference to the method by which the first network transmission module obtains the tensor calculation task.

[0084] In this embodiment, the data sending task includes the metadata of the fourth sub-tensor in the first tensor; the data in the fourth sub-tensor is stored in the memory of the first computing node, and the fourth sub-tensor can be implemented as the Tensor mentioned above.Tile, as shown in Figure 4b, is at least one data segment corresponding to the fourth sub-tensor, such as data segment w1, data segment w2, and data segment wn; where n≥2 and n is a natural number; Figure 4b shows an example where the fourth sub-tensor includes multiple data segments, but this does not constitute a limitation on this embodiment, and the fourth sub-tensor may also include one data segment.

[0085] Further, in this embodiment, as shown in S403b of Figure 4b, the memory distribution information of the fourth sub-tensor is generated according to the meta-information of the fourth sub-tensor; then, as shown in S404b of Figure 4b, the data in the fourth sub-tensor is read from the memory of the first computing node based on the memory distribution information of the fourth sub-tensor; as shown in S405b of Figure 4b, the data in the fourth sub-tensor read from the memory of the first computing node is written into the first network transmission module; further, as shown in S406b of Figure 4b, the data in the fourth sub-tensor is sent, for example, it can be sent to the second computing node, but is not limited thereto.

[0086] Optionally, the data in the fourth sub-tensor includes multiple data fragments stored separately in the memory of the first computing node, with different data fragments stored in different memory regions; sending the data in the fourth sub-tensor to the second computing node includes: adding the multiple data fragments in the fourth sub-tensor to multiple payload fields of the first data packet, and sending the third data packet with the added multiple data fragments to the second computing node; wherein, the third data packet is obtained by adding the multiple data fragments to be transmitted to multiple payload fields of the first data packet, and the third data packet has the same structure as the first data packet, and can be implemented with reference to the structure of the first data packet, except that the third data packet carries multiple data fragments.

[0087] Optionally, the memory of the first computing node also includes a queue pair, which includes a first queue and a second queue. The first queue is used to carry tensor calculation tasks or data sending tasks issued by the first application, and the second queue is used to carry data receiving tasks issued by the first application. In an optional embodiment, the first computing node further includes: a first processing unit, which creates a queue pair in memory according to a queue creation command issued by the first application.

[0088] In this embodiment, the method by which the first network transmission module receives the tensor calculation task or data transmission task issued by the first application is not limited. For example, periodic polling and / or interruption methods can be used. In an optional embodiment, the first queue can be periodically polled to read the tensor calculation task or data transmission task written by the first application from the first queue; and / or, an interrupt signal triggered after the first application writes the tensor calculation task or data transmission task to the first queue can be received, and the tensor calculation task or data transmission task can be read from the first queue according to the interrupt signal.

[0089] In an optional embodiment, receiving the data reception task issued by the first application includes: periodically polling the second queue.The queue is configured to read data receiving tasks written by the first application from the second queue; and / or to receive an interrupt signal triggered after the first application writes data receiving tasks to the second queue, and to read data receiving tasks from the second queue according to the interrupt signal.

[0090] Further optionally, the queue pair also includes a third queue adapted to the queue pair, the third queue being used to write completion messages of tensor calculation tasks, data sending tasks or data receiving tasks issued by the first application (pages 13 / 19 of the specification, CN 121957834 A), and the third queue can be notified by periodic polling or interruption, without limitation.

[0091] In the embodiments of this application, the implementation method of the network transmission module is not limited, and two implementation methods of the network transmission module are provided exemplarily. One approach is to implement the network transmission module as a dedicated network transmission module, such as a network card device specifically designed for performing various tensor calculations and transmissions provided in the embodiments of this application. This network card device includes a processor, which may be referred to as the TTCA mentioned above. The application performs processing by writing tensor calculation tasks, receiving tasks, or sending tasks to a queue pair, and the TTCA executes the steps in the various data processing methods provided in the foregoing embodiments of this application. Another approach is to reuse the existing RDMA network card on the computing node in the network transmission module. This can be considered an improvement on the RDMA (Remote Direct Memory Access) transmission protocol. The network transmission module includes the original processor and the newly added TTCA module. The application can reuse the RDMA programming interface for development. In this case, the original RDMA queue pair can be reused, and corresponding command types can be added for tensor computation tasks to distinguish them from traditional RDMA commands. The application sends tensor computation tasks, receiving tasks, or sending tasks to the TTCA module through the new RDMA commands, and the TTCA executes the steps of the various data processing methods provided in the foregoing embodiments of this application for processing. Of course, the application can also send other tasks to the processor through traditional RDMA commands for the processor to perform network transmission processing on the other tasks.

[0092] The detailed implementation methods and beneficial effects of each step in the method of this embodiment have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0093] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the executing entity for steps 401a to 403a can be device A; or, for example, the executing entity for steps 401a and 402a can be device A, and the executing entity for step 403a can be device B; and so on.

[0094] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 401a, 402a, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0095] As shown in FIG5, this is a schematic diagram of the structure of a computer node 50 provided in an embodiment of the present application. The computing node 50 includes: a processing unit 51 and a network transmission module 52; wherein, the processing unit 51 is used to run a first application, and the network transmission module 52 is used to: receive a computing task issued by the first application, the computing task including metadata of a first data subset participating in the computing task in a first dataset, the data in the first data subset being dispersedly stored in the memory of the first computing node; generate memory distribution information of the first data subset in the memory of the first computing node based on the metadata of the first data subset; read data in the first data subset from the memory of the first computing node based on the memory distribution information of the first data subset; receive data in a second data subset sent by a second computing node, the second data subset being a data subset participating in the computing task in a second dataset; perform calculations on the data in the first data subset and the data in the second data subset, and write the calculation result data into the memory of the first computing node.

[0096] In an optional embodiment, the metadata of the first data subset includes the metadata of the first dataset and the location information of the first data subset in the first dataset; Specification 14 / 19 pages 19 CN 121957834 A

[0097] When the network transmission module 52 generates the memory distribution information of the first data subset based on the metadata of the first data subset, it is specifically used for: generating the memory distribution information of the first data subset based on the metadata of the first data subset, including: determining the global memory area occupied by the first dataset in the memory of the first computing node based on the metadata of the first dataset; generating the memory distribution information of the first data subset based on the location information of the first data subset in the first dataset and the global memory area.

[0098] In an optional embodiment, the metadata of the first dataset includes at least: the dimension of the first dataset.The network transmission module 52, when determining the global memory area occupied by the first dataset in the memory of the first computing node based on the metadata of the first dataset, specifically performs the following: calculates the memory size required by the first dataset based on the number of dimensions, the size of each dimension, the data type, and the data filling strategy information; determines the global memory area based on the memory size and the starting address of the memory allocated to the first dataset; wherein, the data in the first dataset is stored continuously in the global memory area according to the specified dimensional order.

[0099] In an optional embodiment, the location information of the first data subset in the first dataset includes at least: the offset and size of the first data subset in each dimension; when the network transmission module 52 generates the memory distribution information of the first data subset based on the location information of the first data subset in the first dataset and the global memory region, it is specifically used to: traverse the global memory region based on the offset and depth information of the first data subset in each dimension and the specified dimensional order to obtain at least one data segment included in the first data subset; determine at least one local memory region occupied by the first data subset based on the storage location of the at least one data segment in the global memory region; generate an address linked list based on the starting address, space size and order relationship between the at least one local memory region, and use the address linked list as the memory distribution information of the first data subset.

[0100] In an optional embodiment, the data in the second data subset includes multiple data segments distributed and stored in the memory of the second computing node, with different data segments stored in different memory regions; when the network transmission module 52 receives the data in the second data subset sent by the second computing node, it is specifically used to: receive a first data packet sent by the second computing node, the first data packet including multiple first payload fields, one first payload field carrying one data segment; parse the multiple data segments from the multiple first payload fields in the first data packet to obtain the data in the second data subset; or, receive multiple second data packets sent by the second computing node, the second data packet including a second payload field, the second payload field carrying one data segment; parse the multiple data segments from the second payload fields in the multiple second data packets to obtain the data in the first data subset.

[0101] In an optional embodiment, the network transmission module 52 is further used to: receive the metadata of the second data subset sent by the second computing node; then the network transmission module 52, when processing the data in the first data subset and the second data subset, parses the multiple data segments from the second payload fields in the second data packets to obtain the data in the first data subset.When performing calculations on data in a data subset and writing the calculation results into the memory of the first computing node, the process specifically involves: determining the data correspondence between the first and second data subsets for tensor calculations based on the metadata of the first and second data subsets; performing calculations on the data segments in the first and second data subsets based on the data correspondence to obtain the calculation results; determining the first storage location of the calculation results in the memory of the first computing node based on the metadata of the first data subset; and writing the calculation results into the memory of the first computing node based on the first storage location.

[0102] In an optional embodiment, before receiving data from the second data subset sent by the second computing node, the network transmission module 52 is further configured to: receive a prefetch request issued by the first application, the prefetch request indicating that the second data subset be obtained from the second computing node; and send the prefetch request to the second computing node to trigger the second computing node to return the second data subset after detecting that the ready state of the second data subset is ready.

[0103] In an optional embodiment, the network transmission module 52 is further configured to: receive a response message returned by the second computing node in response to the prefetch request, the response message indicating that the first network transmission module waits to receive data from the second data subset sent by the second computing node.

[0104] In an optional embodiment, the network transmission module 52 is further configured to: receive data in the third data subset sent by the second computing node according to a data receiving task issued by the first application; the data receiving task is used to instruct the acquisition of the third data subset in the second dataset from the second computing node; determine the second storage location of the data in the third data subset in the memory of the first computing node according to the metadata of the third data subset; and store the data in the third data subset in the memory of the first computing node according to the second storage location.

[0105] In an optional embodiment, the network transmission module 52 is further configured to: receive a data sending task issued by the first application, the data sending task including metadata of the fourth data subset in the first dataset, the data in the fourth data subset being distributed and stored in the memory of the first computing node; generate memory distribution information of the fourth data subset according to the metadata of the fourth data subset; read the data in the fourth data subset from the memory of the first computing node based on the memory distribution information of the fourth data subset; and send the data in the fourth data subset to the second computing node.

[0106] In an optional embodiment, the data in the fourth data subset includes multiple data segments distributed and stored in the memory of the first computing node, with different data segments stored in different memory regions; when the network transmission module 52 sends the data in the fourth data subset to the second computing node, it is specifically used to: add the multiple data segments in the fourth data subset to multiple payload fields of the first data packet, and send the third data packet with the multiple data segments added to it to the second computing node.

[0107] In an optional embodiment, the memory of the first computing node also includes a queue pair, the queue pair including a first queue and a second queue, the first queue being used to carry tensor computation tasks or data sending tasks issued by the first application, and the second queue being used to carry data receiving tasks issued by the first application; then when the network transmission module 52 receives computation tasks or data sending tasks issued by the first application, it is specifically used to: obtain computation tasks or data sending tasks from the first queue in a polling and / or interrupt manner; and when the network transmission module 52 receives data receiving tasks issued by the first application, it is specifically used to: obtain data receiving tasks from the second queue in a polling and / or interrupt manner.

[0108] In an optional embodiment, when the network transmission module 52 receives a computing task or data sending task issued by the first application, it is specifically configured to: periodically poll the first queue to read the computing task or data sending task written by the first application from the first queue; and / or, receive an interrupt signal triggered after the first application writes a computing task or data sending task to the first queue, and read the computing task or data sending task from the first queue according to the interrupt signal.

[0109] In an optional embodiment, when the network transmission module 52 receives a data receiving task issued by the first application, it is specifically configured to: periodically poll the second queue to read the data receiving task written by the first application from the second queue; and / or, receive an interrupt signal triggered after the first application writes a data receiving task to the first queue, and read the data receiving task from the first queue according to the interrupt signal. Instruction manual, pages 16 / 19, 21 CN 121957834 A

[0110] The processing unit can also be used to run a second application. The network transmission module 52 is used to: receive a data transmission task issued by the second application, the data transmission task including metadata of a second data subset in the second data set; the data in the second data subset is distributed and stored in the memory of the second computing node; based on the metadata of the second data subset, memory distribution information of the second data subset is generated; based on the memory distribution information of the second data subset, the data in the second data subset is read from the memory of the second computing node; and the first...Data from the second data subset is sent to the first computing node.

[0111] In an optional embodiment, when the network transmission module 52 reads data from the memory of the second computing node based on the memory distribution information of the second data subset, it is specifically used to: send a subscription request for the second data subset to the second application, the subscription request being used to request to listen to the ready state of the second data subset; upon receiving a request response from the second application for the subscription request, read data from the memory of the second data subset based on the memory distribution information of the second data subset; wherein, the request response is sent by the second application after listening to the ready state of the second data subset as ready.

[0112] In an optional embodiment, before sending the subscription request for the second data subset to the second application, the network transmission module 52 is further used to: receive a prefetch request sent by the first computing node, the prefetch request being used to trigger the second network transmission module to return to the second data subset after listening to the ready state of the second data subset as ready.

[0113] Further, as shown in FIG5, the computing node also includes other components such as a display 53, a power supply component 54, and an audio component 55. FIG5 only schematically shows some components and does not mean that the computing node only includes the components shown in FIG5. In addition, the components within the dashed boxes in FIG5 are optional components, not mandatory components, and can be determined according to the product form of the computing node. The computing node of this embodiment can be implemented as a terminal device such as a desktop computer, a laptop computer, a smartphone, or an IoT device, or as a server device such as a conventional server, a cloud server, or a server array. If the computing node of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, or a smartphone, it may include the components within the dashed boxes in FIG5; if the computing node of this embodiment is implemented as a server device such as a conventional server, a cloud server, or a server array, it may not include the components within the dashed boxes in FIG5.

[0114] This application embodiment also provides a data transmission system, including: a first computing node and a second computing node, the first computing node including a first network transmission module and a first processing unit, the second computing node including a second network transmission module and a second processing unit; the first processing unit is used to run a first application, the first network transmission module is used to execute the steps in the above method embodiment; the second processing unit is used to run a second application, the second network transmission module is used to execute the steps in the above method embodiment.

[0115] Correspondingly, this application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the above method embodiment.

[0116] Accordingly, this application also provides a computer program product, which includes a computer program / instructions that, when executed by a processor, cause the processor to implement the steps in the above method embodiments.

[0117] The above memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), Magnetic Storage, Flash Memory, Disk, or Optical Disk.

[0118] The above communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, and other mobile communication networks, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0119] The above-mentioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen may be implemented as a touch screen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. Touch sensors can not only sense the boundaries of touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operation.

[0120] The power supply component described above provides power to various components of the device in which the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.

[0121] The audio component described above may be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device in which the audio component is located is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in a memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0122] Those skilled in the art will understand that embodiments of this application may be provided as methods, systems, or computer program products. Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application may take the form of a computer program product implemented on one or more computer-readable storage media (including, but not limited to, disk storage, compact disc read-only memory, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0124] These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby enabling the computer or other programmable data processing apparatus to perform a series of operational steps to produce a computer-implemented process.Instructions that execute on other programmable devices provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0126] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interfaces, network interfaces, and memory.

[0127] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0128] Computer-readable media includes both permanent and non-persistent, removable and non-removable media that can be used to store information by any method or technology. Information may be computer-readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0129] It should also be noted that the terms “comprising,” “including,” or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0130] The above are merely embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.19 / 19 Page 24 CN 121957834 A Figure 1a Figure 1b Appendix 1 / 7 Page 25 CN 121957834 A Figure 2a Appendix 2 / 7 Page 26 CN 121957834 A Figure 2b Appendix 3 / 7 Page 27 CN 121957834 A Figure 3a Figure 3b Appendix 4 / 7 Page 28 CN 121957834 A Figure 3c Appendix 5 / 7 Page 29 CN 121957834 A Figure 4a Appendix 6 / 7 Page 30 CN 121957834 A Figure 4b Figure 5 Appendix 7 / 7 Page 31 CN 121957834 A Abstract The embodiment of the invention provides a data processing method and device, a storage medium and a program product. In the embodiment of the invention, address calculation and data calculation are issued to a network transmission module for a calculation task of cross-network transmission; on one hand, the network transmission module calculates and generates memory distribution information of a first data subset according to meta-information of the first data subset in a memory of a first calculation node, and reads data of the first data subset from the memory according to the memory distribution information; on the other hand, receiving data of a second data subset in amemory in a second computing node; furthermore, data calculation is carried out on the data of the first data subset and the data of the second data subset, calculation result data is written into a memory of the first calculation node, processor resources on the calculation nodes are liberated from calculation tasks, and consumption of the calculation resources on the calculation nodes is reduced; in addition, based on the distribution of the calculation task, the access frequency of the memory is reduced, and the calculation efficiency is improved.

Claims

1. A data processing method, characterized in that, The method, applicable to a first network transmission module on a first computing node, wherein a first application is running on the first computing node, comprises: The system receives a computing task issued by a first application. The computing task includes metadata of a first subset of data in a first dataset that participates in the computing task. The data in the first subset of data is stored in the memory of the first computing node. Based on the metadata of the first data subset, generate memory distribution information of the first data subset in the memory of the first computing node; Based on the memory distribution information of the first data subset, data from the first data subset is read from the memory of the first computing node; Receive data from a second data subset sent by a second computing node, wherein the second data subset is a subset of data in a second dataset that participates in the computing task; The data in the first data subset and the data in the second data subset are calculated, and the calculated data is written into the memory of the first computing node.

2. The method according to claim 1, characterized in that, The metadata of the first data subset includes the metadata of the first dataset and the location information of the first data subset in the first dataset; Based on the metadata of the first data subset, memory distribution information of the first data subset is generated, including: Based on the metadata of the first dataset, determine the global memory region occupied by the first dataset in the memory of the first computing node; Based on the location information of the first data subset in the first dataset and the global memory region, the memory distribution information of the first data subset is generated.

3. The method according to claim 2, characterized in that, The metadata of the first dataset includes at least: the number of dimensions of the first dataset, the size of each dimension, the data type, and the data filling strategy information; Based on the metadata of the first dataset, determine the global memory region occupied by the first dataset in the memory of the first computing node, including: Calculate the memory size required for the first dataset based on the number of dimensions, the size of each dimension, the data type, and the data filling strategy information. Based on the memory size and the starting address of the memory allocated for the first dataset, determine the global memory region occupied by the first tensor in the memory of the first computing node; The data in the first dataset are stored sequentially in the global memory area according to a specified dimension.

4. The method according to claim 2, characterized in that, The location information of the first data subset in the first dataset includes at least: the offset and size of the first data subset in each dimension; Based on the location information of the first data subset within the first dataset and the global memory region, memory distribution information of the first data subset is generated, including: Based on the offset and depth information of the first data subset in each dimension and the specified dimensional order, the global memory region is traversed to obtain at least one data segment included in the first data subset. Based on the storage location of the at least one data segment in the global memory region, determine at least one local memory region occupied by the first data subset; An address list is generated based on the starting address, size, and order relationship between the at least one local memory region, and the address list is used as the memory distribution information of the first data subset.

5. The method according to any one of claims 1-4, characterized in that, The data in the second data subset includes multiple data segments distributed and stored in the memory of the second computing node, with different data segments stored in different memory regions; Receive data from the second data subset sent by the second computing node, including: The system receives a first data packet sent by the second computing node. The first data packet includes multiple first payload fields, and each first payload field carries a data segment. The system then parses the multiple data segments from the multiple first payload fields in the first data packet to obtain the data in the second data subset. or Receive multiple second data packets sent by the second computing node, the second data packets including a second payload field, the second payload field carrying a data segment; parse the multiple data segments from the second payload field of the multiple second data packets to obtain data in the first data subset.

6. The method according to any one of claims 1-4, characterized in that, Also includes: Receive metadata about the second data subset sent by the second computing node; The calculation is performed on the data in the first data subset and the data in the second data subset, and the calculation result is written into the memory of the first computing node, including: Based on the metadata of the first data subset and the metadata of the second data subset, determine the data correspondence between the first data subset and the second data subset for tensor computation; Based on the data correspondence, the data segments in the first data subset and the corresponding data segments in the second data subset are calculated to obtain the calculated result data; Based on the metadata of the first data subset, determine the first storage location of the computation result data in the memory of the first computing node; The calculation result data is written into the memory of the first computing node according to the first storage location.

7. The method according to claim 1, characterized in that, Before receiving data from the second subset of data sent by the second computing node, the process also includes: Receive a prefetch request from the first application, the prefetch request being used to instruct the second data subset to be retrieved from the second computing node; The prefetch request is sent to the second computing node to trigger the second computing node to return the second data subset after it detects that the second data subset is ready.

8. The method according to claim 7, characterized in that, Also includes: The first network transmission module receives a response message from the second computing node in response to the prefetch request. The response message is used to instruct the first network transmission module to wait to receive data from the second data subset sent by the second computing node.

9. The method according to any one of claims 1-8, characterized in that, Also includes: Receive tasks based on data sent by the first application. Receive data from the third data subset sent by the second computing node; The data receiving task is used to instruct the acquisition of a third data subset from the second dataset from the second computing node; Based on the metadata of the third data subset, determine the second storage location of the data in the third data subset in the memory of the first computing node; According to the second storage location, the data in the third data subset is stored in the memory of the first computing node.

10. The method according to any one of claims 1-8, characterized in that, Also includes: Receive a data sending task issued by a first application. The data sending task includes metadata of a fourth data subset in the first dataset. The data in the fourth data subset is distributed and stored in the memory of the first computing node. Generate memory distribution information of the fourth data subset based on the metadata of the fourth data subset. Based on the memory distribution information of the fourth data subset, data in the fourth data subset is read from the memory of the first computing node; The data in the fourth data subset is sent to the second computing node.

11. The method according to claim 10, characterized in that, The data in the fourth data subset includes multiple data segments distributed and stored in the memory of the first computing node, with different data segments stored in different memory regions; Sending data from the fourth data subset to the second computing node includes: Multiple data fragments from the fourth data subset are added to multiple payload fields of the first data packet, and a third data packet with the multiple data fragments added is sent to the second computing node.

12. The method according to claim 11, characterized in that, The memory of the first computing node also includes a queue pair, which includes a first queue and a second queue. The first queue is used to carry tensor computing tasks or data sending tasks issued by the first application, and the second queue is used to carry data receiving tasks issued by the first application. Receiving computing tasks or data transmission tasks issued by the first application includes: retrieving computing tasks or data transmission tasks from the first queue in a polling and / or interrupt manner; Receiving data reception tasks issued by the first application includes: retrieving data reception tasks from the second queue in a polling and / or interrupt manner.

13. A data processing method, characterized in that, A second network transmission module applied to a second computing node, wherein a second application runs on the second computing node, the method comprising: Receive a data sending task issued by the second application, wherein the data sending task includes metadata of a second subset of data in the second dataset; The data in the second data subset is distributed and stored in the memory of the second computing node. Based on the metadata of the second data subset, the memory distribution information of the second data subset is generated. Based on the memory distribution information of the second data subset, data from the second data subset is read from the memory of the second computing node; The data in the second data subset is sent to the first computing node.

14. The method according to claim 13, characterized in that, Based on the memory distribution information of the second data subset, data from the second data subset is read from the memory of the second computing node, including: Send a subscription request for the second data subset to the second application, the subscription request being used to request listening to the ready state of the second data subset; Upon receiving a response to the subscription request from the second application, data from the second data subset is read from the memory of the second computing node based on the memory distribution information of the second data subset. The request response is issued by the second application after it detects that the second data subset is ready.

15. The method according to claim 14, characterized in that, Before sending a subscription request for the second data subset to the second application, the method further includes: The system receives a prefetch request sent by the first computing node. The prefetch request is used to trigger the second network transmission module to return the second data subset after it detects that the second data subset is ready.

16. A computing node, characterized in that, include: Processing unit and network transmission module; Wherein, the processing unit is used to run a first application, the network transmission module is used to execute the steps in the method of any one of claims 1-12; and / or, the processing unit is used to run a second application, the network transmission module is used to execute the steps in the method of any one of claims 13-15.

17. A data transmission system, characterized in that, include: A first computing node and a second computing node, the first computing node including a first network transmission module and a first processing unit, the second computing node including a second network transmission module and a second processing unit; the first processing unit is used to run a first application, the first network transmission module is used to execute the steps of the method according to any one of claims 1-12; the second processing unit is used to run a second application, the second network transmission module is used to execute the steps of the method according to any one of claims 13-15.

18. A computer storage medium, characterized in that, The device stores a computer program / instructions that, when executed by a processor, cause the processor to perform the steps of the method according to any one of claims 1-12 or 13-15.

19. A computer program product, characterized in that, Includes a computer program / instruction that, when executed by a processor, causes the processor to perform the steps of the method according to any one of claims 1-12 or 13-15.

Images