Data processing method, computing system, device, storage medium and program product

Direct access to the memory of adjacent computing nodes through the chip interconnection bus, solving the problem of multiple data replication in data access between computing nodes, and achieving efficient data transmission and improvement in computing task efficiency.

WO2024239667A1PCT designated stage expired Publication Date: 2024-11-28HUAWEI TECH CO LTD

Patent Information

Application Number
PCT/CN2024/071099
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-22
Filing Date
2024-01-08
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

Data access between computing nodes is repeated due to network transmission, resulting in low transmission speed and high latency, which affects the efficiency of computing tasks.

Method used

Direct access to the memory of adjacent computing nodes through chip interconnection buses (such as PCIe bus, CXL bus, QPI bus), avoid network transmission, and use the shared virtual address mechanism to achieve data transmission.

Benefits of technology

It reduces data transmission delay, improves data access efficiency between computing nodes, and improves the operation efficiency of computing tasks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024071099_28112024_PF_FP_ABST
    Figure CN2024071099_28112024_PF_FP_ABST
Patent Text Reader

Abstract

The present application relates to the technical field of information, and particularly relates to a data processing method, a computing system, a device, a storage medium and a program product. The method is applied to a computing system, which comprises a plurality of computing nodes, wherein processors of different computing nodes in the computing system are connected to each other by means of a chip interconnection bus. One computing node in the computing system may use a shared virtual address to access data in another computing node by means of the chip interconnection bus, such that repeated data replication is not required, so as to facilitate a reduction in the latency for data transmission between the computing nodes and an improvement in the data transmission efficiency between the computing nodes, thereby improving the operation efficiency of a computation task.
Need to check novelty before this filing date? Find Prior Art

Description

Data processing method, computing system, device, storage medium and program product

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 22, 2023, with application number 202310581596.8 and application name “Data processing method, computing system, device, storage medium and program product”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of information technology, and in particular to a data processing method, a computing system, a device, a storage medium, and a program product. Background Art

[0003] As the complexity of user computing tasks increases, multiple computing nodes often need to collaboratively process the same computing task. For example, different computing nodes can each handle one or more subtasks of the task. During the collaborative processing of multiple computing nodes, data access between computing nodes is often required. For example, one computing node can retrieve data from the memory of another computing node, or one computing node can store data in the memory of another computing node.

[0004] Currently, data access between computing nodes is typically achieved through a network, such as through protocols like Remote Direct Memory Access (RDMA) and Transmission Control Protocol / Internet Protocol (TCP / IP). However, during network data access between computing nodes, data may be copied multiple times, such as between the memory, network interface card, and cache of a computing node, or between the memories corresponding to different computing units within a computing node. This results in low transmission speeds and high latency for data access between computing nodes over the network, impacting the efficiency of data access between computing nodes.

[0005] Summary of the Invention

[0006] In view of this, the present application provides a data processing method, a computing system, a device, a storage medium, and a program product.

[0007] In a first aspect, the present application provides a data processing method, which is applied to a computing system, wherein the computing system includes a first computing node and a second computing node, wherein the first computing node includes a first processor, the second computing node includes a second processor and a first memory, and the first processor and the second processor are connected through a chip interconnect bus; and the method includes: the first processor detects a cross-node access request, and the cross-node access request is used to access the first memory; the first processor accesses the first memory through the chip interconnect bus.

[0008] That is to say, the first computing node accesses the first memory in the second computing node through the chip interconnection bus without accessing through the network, which can avoid multiple data copies and is conducive to improving data access efficiency between computing nodes in the computing system.

[0009] In a possible implementation of the first aspect above, the cross-node access request includes a first virtual address, and the first processor accesses the first memory through the chip interconnect bus, including: the first processor determines the first physical address corresponding to the first virtual address; the first processor accesses the first physical storage space corresponding to the first physical address in the first memory through the chip interconnect bus.

[0010] In a possible implementation of the first aspect above, the first processor runs a first process, and the first virtual address is the virtual address of the first process; and determining the first physical address corresponding to the first virtual address includes: the first processor queries the first physical address in a first page table through the first virtual address, wherein the first page table is the page table of the first process, and the first page table includes a first mapping relationship between the first virtual address and the first physical address.

[0011] That is, the first computing node and the second computing node implement data transmission via the chip interconnect bus by sharing a virtual address.

[0012] In a possible implementation of the first aspect, the second processor runs a second process, and the first physical address is a physical address corresponding to a second virtual address of the second process.

[0013] In a possible implementation of the first aspect above, the above-mentioned first mapping relationship is determined in the following manner: the second processor sends the second mapping relationship between the second virtual address and the first physical address to the first processor; the first processor adds the first mapping relationship between the first virtual address and the first physical address to the first page table based on the second mapping relationship.

[0014] In a possible implementation of the first aspect above, the first computing node runs a first virtual memory sharing engine, and the second computing node runs a second virtual memory sharing engine; and the second processor sends a second mapping relationship between the second virtual address and the first physical address to the first processor, including: after the second processor adds the second mapping relationship to the first queue, it uses the second virtual memory sharing engine to synchronize the first queue to the first virtual memory sharing engine; wherein the first queue is a shared virtual memory queue corresponding to the first computing node and the second computing node, and the first queue is stored in both the first virtual memory sharing engine and the second virtual memory sharing engine.

[0015] In a possible implementation of the first aspect, the first virtual address and the second virtual address are the same.

[0016] In a possible implementation of the first aspect, the first virtual address and the second virtual address are different.

[0017] That is, the first virtual address and the second virtual address may be different, which can avoid the situation where the mapping relationship between the second virtual address and the first physical address cannot be established when the second virtual address of the first process is already in use.

[0018] In a possible implementation of the first aspect above, the above-mentioned adding the first mapping relationship between the first virtual address and the first physical address to the first page table includes: the first processor obtains the second mapping relationship from the first queue of the second virtual memory sharing engine; the first processor determines the first mapping relationship based on the second mapping relationship, and adds the first mapping relationship to the first page table.

[0019] In a possible implementation of the first aspect, the first computing node and the second computing node are computing nodes in a first associated group.

[0020] In a possible implementation of the first aspect above, a node topology management unit is running in the above-mentioned second computing node, and the first association group is determined in the following manner: the second computing node uses the node topology management unit to establish a topological relationship between computing nodes in the computing system; the second computing node establishes a first association group based on the topological relationship, and the first association group includes the device identification of each device node in the first computing node and the second computing node.

[0021] In a possible implementation of the first aspect, the first memory is a memory corresponding to the second processor.

[0022] In a possible implementation of the first aspect, the second computing node further includes a third processor, and the first memory is a memory corresponding to the third processor.

[0023] In a possible implementation of the first aspect above, the first processor or the second processor includes any one of a central processing unit CPU, a graphics processing unit GPU, a field programmable gate array FPGA, an application-specific integrated circuit ASIC, a neural network processing unit NPU, and an artificial intelligence chip.

[0024] In a possible implementation of the first aspect, the chip interconnect bus includes at least one of the following buses: a high-speed serial computer expansion bus standard PCIe bus, a computing express interconnect CXL bus, and a quick path interconnect QPI bus.

[0025] In a possible implementation of the first aspect, at least one of the first computing node and the second computing node is a heterogeneous computing node.

[0026] In a second aspect, the present application provides a computing system, wherein the computing system includes a first computing node and a second computing node, wherein the first computing node includes a first processor, the second computing node includes a second processor and a first memory, and the first processor and the second processor are connected through a chip interconnect bus; the first processor is used to access the first memory through the chip interconnect bus when a cross-node access request to access the first memory is detected.

[0027] That is to say, the first computing node in the computing system accesses the first memory in the second computing node through the chip interconnection bus without accessing through the network, which can avoid multiple data copies and is conducive to improving the data access efficiency between computing nodes in the computing system.

[0028] In a third aspect, the present application provides a device comprising: a memory for storing instructions; and at least one processor for instructing the instructions to enable the device to implement the above-mentioned first aspect and any possible implementation method provided by the above-mentioned first aspect.

[0029] In a fourth aspect, the present application provides a readable storage medium, which stores instructions. When the instructions are executed by a device, the device implements the above-mentioned first aspect and any possible implementation method provided by the above-mentioned first aspect.

[0030] In a fifth aspect, the present application provides a computer program product, which, when running on a device, enables the device to implement the method provided in the first aspect and any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] FIG1A shows a schematic diagram of a computing node CP1 transmitting data to a computing node CP2 via the TCP / IP protocol according to some embodiments of the present application;

[0032] FIG1B shows a schematic diagram of a computing node CP3 transmitting data to a computing node CP4 via an RDMA protocol;

[0033] FIG2A shows a schematic structural diagram of a host according to some embodiments of the present application;

[0034] FIG2B shows a schematic structural diagram of a general computing node according to some embodiments of the present application;

[0035] FIG2C shows a schematic diagram of the structure of a heterogeneous computing node according to some embodiments of the present application;

[0036] FIG2D shows a schematic diagram of the structure of another heterogeneous computing node according to some embodiments of the present application;

[0037] FIG3 shows a schematic structural diagram of a computing system 0 according to some embodiments of the present application;

[0038] FIG4 shows a schematic diagram of a data transmission path for data access between a heterogeneous computing node P-1 and a heterogeneous computing node P-2 according to some embodiments of the present application;

[0039] FIG5 shows a schematic diagram of a data transmission path for data access between a heterogeneous computing node P-3 and a heterogeneous computing node P-4 according to some embodiments of the present application;

[0040] FIG6 shows a schematic diagram of a scenario of data access between a general computing node CP1 and a general computing node CP2 according to some embodiments of the present application;

[0041] FIG7 shows a schematic diagram of a scenario of data access between a heterogeneous computing node P1 and a heterogeneous computing node P2 according to some embodiments of the present application;

[0042] FIG8 is a schematic diagram showing a scenario of data access between a heterogeneous computing node P3 and a heterogeneous computing node P4 according to some embodiments of the present application;

[0043] FIG9A is a schematic diagram showing a process pr2 accessing data in a memory 13 of a process pr1 using a shared virtual address via a chip interconnect bus according to some embodiments of the present application;

[0044] FIG9B is a schematic diagram showing a process pr2 accessing data in the memory 16 of the process pr1 using a shared virtual address via a chip interconnect bus according to some embodiments of the present application;

[0045] FIG9C is a schematic diagram showing a process pr4 accessing data in the memory 33 of the process pr3 using a shared virtual address via a chip interconnect bus according to some embodiments of the present application;

[0046] FIG9D is a schematic diagram showing a process pr4 accessing data in the memory 36 of the process pr3 using a shared virtual address via a chip interconnect bus according to some embodiments of the present application;

[0047] FIG10 shows a schematic diagram of a software architecture running in a host and an AI chip according to some embodiments of the present application;

[0048] FIG11A is a schematic diagram showing a software architecture of the computing system 2 shown in FIG7 when accessing data through an SVM method according to some embodiments of the present application;

[0049] FIG11B shows a schematic diagram of a software architecture of the computing system 3 shown in FIG8 when accessing data through an SVM method according to some embodiments of the present application;

[0050] FIG12A shows a schematic diagram of an interactive process for device node registration according to some embodiments of the present application;

[0051] FIG12B shows a schematic diagram of an interaction process for deregistration of a device node D31 according to some embodiments of the present application;

[0052] FIG13 shows a schematic diagram of an interactive process for creating an association group according to some embodiments of the present application;

[0053] FIG14 shows a schematic diagram of an interactive process for deleting an association group according to some embodiments of the present application;

[0054] FIG15 is a schematic diagram showing an interactive process for initializing a shared virtual address of a process according to some embodiments of the present application;

[0055] FIG16 shows a schematic diagram of dividing the virtual address space of a process according to some embodiments of the present application;

[0056] FIG17 is a schematic diagram showing an interactive flow of a data processing method according to some embodiments of the present application;

[0057] FIG18 is a schematic diagram showing a process pr2 accessing data of process pr1 in memory 13 through a shared virtual address according to some embodiments of the present application;

[0058] FIG19 is a schematic diagram showing a process pr2 accessing data of process pr1 in memory 16 through a shared virtual address according to some embodiments of the present application;

[0059] FIG20 is a schematic diagram showing a process pr4 accessing data in a memory 33 of a process pr3 through a shared virtual address according to some embodiments of the present application;

[0060] FIG21 is a schematic diagram showing a process pr4 accessing data in a memory 36 of a process pr3 through a shared virtual address according to some embodiments of the present application;

[0061] FIG22 is a schematic diagram showing an interactive flow of a data processing method according to some embodiments of the present application;

[0062] FIG23 is a schematic diagram showing another method of dividing the virtual address space of a process according to some embodiments of the present application;

[0063] FIG24 shows another schematic diagram of a process pr2 accessing data in the memory 13 of the process pr1 through a shared virtual address according to some embodiments of the present application;

[0064] FIG25 is a schematic diagram showing another example of a process pr2 accessing data in the memory 16 of a process pr1 through a shared virtual address according to some embodiments of the present application;

[0065] FIG26 is a schematic diagram showing another example of a process pr4 accessing data in a memory 33 of a process pr3 through a shared virtual address according to some embodiments of the present application;

[0066] FIG27 is a schematic diagram showing another example of a process pr4 accessing data in a memory 36 of a process pr3 through a shared virtual address according to some embodiments of the present application;

[0067] FIG28 shows a schematic structural diagram of a server 100 according to some embodiments of the present application. DETAILED DESCRIPTION

[0068] Illustrative embodiments of the present application include, but are not limited to, data processing methods, computing systems, devices, storage media, and program products.

[0069] As mentioned above, data access between compute nodes in a computing system is typically achieved through a network. However, this network access process involves multiple data replications, such as between the compute node's memory, network interface card, and cache. This results in low transmission speeds and high latency, impacting the efficiency of data access between compute nodes.

[0070] For example, FIG1A shows a schematic diagram of a computing node CP1 transmitting data to a computing node CP2 through a TCP / IP protocol according to some embodiments of the present application.

[0071] As shown in Figure 1A, according to the requirements of the TCP / IP protocol, at the data sending end (computing node CP1), the application layer (e.g., the application sending the data) first copies the data to the network programming interface (sockets) protocol layer. The sockets protocol layer then copies the data to the network card (NIC) via the transport layer and the network card driver. The NIC then sends the data over the network. At the data receiving end (computing node CP2), after receiving the data over the network, the NIC first copies the data to the sockets protocol layer via the NIC driver and the transport layer. The sockets protocol layer then copies the data to the application layer (e.g., the application receiving the data). In other words, the data in computing node CP1 is transmitted to the application of computing node CP2 via the TCP / IP protocol, and the data undergoes four replications.

[0072] For another example, FIG1B shows a schematic diagram of a computing node CP3 transmitting data to a computing node CP4 through an RDMA protocol.

[0073] As shown in Figure 1B , according to the RDMA protocol, at the data sending end (compute node CP3), the application layer (e.g., the application sending the data) first copies the data from the network card driver to the RDMA network card, which then sends the data over the network. At the data receiving end (compute node CP4), after receiving the data over the network, the RDMA network card also needs to copy the data to the application layer (e.g., the application receiving the data) via the network card driver. In other words, the data in compute node CP3 is transferred to the application in compute node CP4 via the RDMA protocol, requiring two copies of the data.

[0074] That is to say, when data is transmitted between computing nodes through the network, whether the TCP / IP protocol or the RDMA protocol is used, the data will be copied multiple times, resulting in low transmission speed and high latency for data access between computing nodes through the network, affecting the efficiency of data access between computing nodes.

[0075] It can be understood that computing nodes may include general computing nodes (also called homogeneous computing nodes) and heterogeneous computing nodes.

[0076] A general computing node can be physically implemented by a host, where the host generally refers to at least one general-purpose processor (such as a central processing unit (CPU)) and a collection of memories connected to the at least one general-purpose processor.

[0077] For example, referring to Figure 2A , a host H may include a CPU and a memory, and the CPU and the memory in the host H are connected via a bus. Referring to Figure 2B , a general computing node CP may be implemented by the host H.

[0078] It should be understood that in other embodiments, the host may also include other associated circuits or devices, such as a network card.

[0079] Heterogeneous computing nodes can be physically implemented through a host and heterogeneous computing resources that can be called by the host.

[0080] It should be understood that heterogeneous computing resources may include heterogeneous processors and memories connected to or disposed within heterogeneous processors. Among them, heterogeneous processors may include one or more of a graphics processing unit (GPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a neural processing unit (NPU), and an artificial intelligence (AI) chip.

[0081] For example, referring to FIG. 2C , the heterogeneous computing node HP can be implemented by the aforementioned host H and an AI chip AIC that can be called by the host H.

[0082] For another example, referring to FIG2D , the heterogeneous computing node HP′ can be implemented by the aforementioned host H and the acceleration unit CUA and memory space MA in the AI ​​chip AIC that can be called by the host H. The heterogeneous computing node HP″ can be implemented by the aforementioned host H and the acceleration unit CUB and memory space MB in the AI ​​chip AIC that can be called by the host H.

[0083] The following is an introduction in conjunction with a specific computing system.

[0084] For example, FIG3 shows a schematic structural diagram of a computing system 0 according to some embodiments of the present application.

[0085] As shown in Figure 3, the computing system 0 includes N heterogeneous computing nodes (heterogeneous computing node P-1, heterogeneous computing node P-2, ..., heterogeneous computing node PN, N>1), and the computing node Pk is implemented by a heterogeneous server Sk (1≤k≤N). The AI ​​application server and each heterogeneous server are connected to the network through a network switch, and data transmission is realized based on network protocols such as TCP / IP and RDMA protocol. Among them, the CPU and AI chip in each heterogeneous server are connected through a bus, for example, through a high-speed serial computer expansion bus standard (Peripheral Component Interconnect Express, PCIe) bus, a compute express link (CXL) bus, a quick path interconnect (QPI) bus, etc.

[0086] It should be understood that a heterogeneous server refers to a server that includes both general-purpose processors and heterogeneous processors.

[0087] AI application servers can split computing tasks (such as AI inference computing tasks or AI training computing tasks) into multiple sub-computing tasks and assign the sub-computing tasks to the AI ​​chips in each heterogeneous computing node for execution. When running sub-computing tasks, heterogeneous computing nodes usually need to obtain data from other heterogeneous computing nodes, such as obtaining intermediate inference results or intermediate training parameters generated by other heterogeneous computing nodes. Because the heterogeneous servers that implement each heterogeneous computing node communicate data over the network, data replication occurs when data is transmitted between heterogeneous computing nodes, resulting in high latency, affecting the communication efficiency between heterogeneous computing nodes, and thus affecting the operating efficiency of computing tasks.

[0088] For example, FIG4 shows a schematic diagram of a data transmission path for data access between a heterogeneous computing node P-1 and a heterogeneous computing node P-2 in a computing system 0 according to some embodiments of the present application.

[0089] As shown in Figure 4, heterogeneous compute node P-1 is implemented using AI chip A-1 and host H1, which includes a CPU and memory connected to it. Host H1 (as a root complex (RC)) is connected to AI chip A-1 (as an endpoint (ED)) via a PCIe link. Heterogeneous compute node P-2 is implemented using AI chip A-2 and host H2, which includes a CPU and memory connected to it. Host H2 (as a RC) is connected to AI chip A-2 (as an ED) via a PCIe link.

[0090] When the accelerator in AI chip A-2 reads data from AI chip A-1, the data must first be transferred from the accelerator in AI chip A-1 to its memory. Then, AI chip A-1's memory transfers the data to host-H1's memory via the PCIe link. Host-H1 then transfers the data to host-H2's memory via the network. Next, the data is transferred from host-H2's memory to AI chip A-2's memory via the PCIe link. Finally, the accelerator in AI chip A-2 reads the data from AI chip A-2's memory. However, during the network transfer from host-H1 to host-H2's memory, whether using TCP / IP or RDMA, multiple data copies occur, affecting data transmission efficiency.

[0091] For another example, FIG5 shows a schematic diagram of a data transmission path for data access between a heterogeneous computing node P-3 and a heterogeneous computing node P-4 according to some embodiments of the present application.

[0092] As shown in Figure 4, heterogeneous computing node P-3 is implemented using host-H3, the first memory space in AI chip A-3, and the first acceleration unit. Heterogeneous computing node P-4 is implemented using host-H4, the second memory space in AI chip A-3, and the second acceleration unit. Host-H3, host-H4, and AI chip A-3 are connected via a PCIe switch.

[0093] When the second acceleration unit in heterogeneous computing node P-4 reads data from the first memory space, the data must first be transferred from the first memory space to the memory of host-H3, and then transferred from host-H3 to the memory of host-H4 via the network. The data is then transferred from host-H4's memory to the second memory space. Finally, the second acceleration unit reads the data from the second memory space. During the data transfer from host-H3 to host-H4 over the network, whether using TCP / IP or RDMA, multiple data copies occur, reducing data transmission efficiency.

[0094] That is to say, when data is transmitted between heterogeneous computing nodes P-1 and P-2 through the network, or when data is transmitted between heterogeneous computing nodes P-3 and P-4 through the network, the data will be copied multiple times, which affects the efficiency of data transmission and reduces the operating efficiency of the computing task.

[0095] To improve the efficiency of data transmission between computing nodes, embodiments of the present application provide a data processing method for use in a computing system comprising multiple computing nodes. In this computing system, data transmission between different computing nodes is not performed over a network, but rather over a chip interconnect bus, such as a PCIe bus, a CXL bus, or a QPI bus. Because data transmission over the chip interconnect bus eliminates the need for data replication, this method reduces data transmission latency between computing nodes, improves the efficiency of data transmission between computing nodes, and thereby improves the operational efficiency of computing tasks.

[0096] Specifically, the computing units (including general-purpose processors and / or heterogeneous processors) physically used to implement the computing system can be connected through a chip interconnect bus (such as a PCIe bus, a CXL bus, a QPI bus, etc.). When a computing node wants to access the data of another computing node, it can directly access the other computing node through the chip interconnect bus. For example, it can read data from the memory of another computing node or write data to the memory of another computing node.

[0097] It should be understood that if the computing nodes in a computing system are deployed on the same server, the computing units in the server can be physically connected via a chip interconnect bus. If the computing nodes in a computing system are deployed on different servers, the different servers can also be physically connected via a chip interconnect bus.

[0098] It should be understood that the technical solution provided in the embodiments of the present application can be applied to data access between general-purpose computing nodes, data access between heterogeneous computing nodes, and data access between general-purpose computing nodes and heterogeneous computing nodes, without limitation here.

[0099] It should be understood that computing nodes in a computing system can be physically deployed in the same server or in different servers, which is not limited here.

[0100] It should be understood that in each embodiment of the present application, the heterogeneous processor used to implement the heterogeneous computing node can be any heterogeneous processor, including but not limited to GPU, FPGA, ASIC, NPU, AI chip, etc. For ease of description, the following description uses the AI ​​chip as an example of the heterogeneous processor used to implement the heterogeneous computing node.

[0101] For example, for a computing system including only general-purpose computing nodes, hosts in the general-purpose computing nodes may be connected via a chip interconnect bus, so that data may be transmitted between the general-purpose computing nodes via the chip interconnect bus.

[0102] For example, FIG6 shows a schematic diagram of a scenario of data access between a general computing node CP5 and a general computing node CP6 according to some embodiments of the present application.

[0103] As shown in Figure 6, computing system 1 includes general-purpose computing nodes CP5 and CP6. General-purpose computing node CP5 is implemented by host-CH1, and general-purpose computing node CP6 is implemented by host-CH2. Host-CH1 includes CPU C1 and memory M1, while host-CH6 includes CPU C2 and memory M2. Host-CH1 and host-CH2 are connected via a chip interconnect bus.

[0104] When CPU C2 reads data in memory M1, it can directly read data from memory M1 through the chip interconnection bus without transmitting it through the network.

[0105] It should be understood that in other embodiments, the computing system 1 may further include more computing nodes, and the computing nodes in the computing system 1 may be connected via a chip interconnect bus.

[0106] For another example, for a computing system that only includes heterogeneous computing nodes, the heterogeneous computing nodes, as well as the host used to implement each heterogeneous computing node and the corresponding AI chip, can be connected through a chip interconnection bus, and data can be transmitted through the chip interconnection bus.

[0107] For example, FIG7 shows a schematic diagram of a scenario of data access between a heterogeneous computing node P1 and a heterogeneous computing node P2 according to some embodiments of the present application.

[0108] As shown in FIG7 , computing system 2 includes heterogeneous computing nodes P1 and P2 .

[0109] The heterogeneous computing node P1 is implemented by a host 11, a chip interconnect bus switch E1, and an AI chip 14. The host 11 includes a CPU 12 and a memory 13, and the AI ​​chip 14 includes an accelerator 15 and a memory 16.

[0110] The heterogeneous computing node P2 is implemented by a host 21, a chip interconnect bus switch E2, and an AI chip 24. The host 21 includes a CPU 22 and a memory 23, and the AI ​​chip 24 includes an accelerator 25 and a memory 26.

[0111] The host 11 and the AI ​​chip 14 are connected to the chip interconnect bus interactive machine E1 through the chip interconnect bus. The host 21 and the AI ​​chip 24 are connected to the chip interconnect bus interactive machine E2 through the chip interconnect bus. The chip interconnect bus switch E1 and the chip interconnect bus switch E2 are connected through the chip interconnect bus.

[0112] In computing system 2, any processor in a heterogeneous computing node (CPU 12, accelerator 15, CPU 22, or accelerator 25) can access the memory of other processors via the chip interconnect bus. For example, when accelerator 25 accesses data in memory 16, it can use the chip interconnect bus to directly access the data in memory 16 through chip interconnect bus switch E1 and chip interconnect bus switch E2. For example, it can read data from memory 16 or write data to memory 16.

[0113] It should be understood that in other embodiments, the computing system 2 may include more computing nodes (general-purpose heterogeneous computing nodes and / or heterogeneous computing nodes). The computing nodes in the computing system 2, and the hosts in the heterogeneous computing nodes and the corresponding AI chips, can all be connected via a chip interconnect bus, and data can be transmitted via the chip interconnect bus.

[0114] It should be understood that the host 11, chip interconnect bus switch E1, AI chip 14, host 21, chip interconnect bus switch E2 and AI chip 24 can be deployed in the same server or in different servers, which is not limited here.

[0115] For another example, FIG8 shows a schematic diagram of a scenario of data access between a heterogeneous computing node P3 and a heterogeneous computing node P4 according to some embodiments of the present application.

[0116] As shown in Figure 8, computing system 3 includes heterogeneous computing nodes P3 and P4. Computing system 3 is implemented using hosts 31 and 41, an AI chip 34, and a chip interconnect bus switch E3. Host 31 includes a CPU 32 and memory 33, host 41 includes a CPU 42 and memory 43, and AI chip 34 includes an accelerator 35 and memory 36. Hosts 31, 41, and AI chip 34 are each connected to chip interconnect bus switch E3 via a chip interconnect bus.

[0117] Heterogeneous computing node P3 is implemented by host 31, acceleration unit 351, and memory space 361 in memory 36. Heterogeneous computing node P4 is implemented by host 41, acceleration unit 352, and memory space 362 in memory 36. In other words, heterogeneous computing node P3 and heterogeneous computing node P4 share AI chip 34.

[0118] Any computing unit in heterogeneous computing node P3 or heterogeneous computing node P4 can access data in other computing units through the chip interconnect bus. For example, when acceleration unit 352 reads data from memory space 361, it can read the data directly from memory space 361 through the chip interconnect bus without transmitting it over the network.

[0119] It should be understood that the host 31, host 41, and AI chip 34 can be deployed in the same server, or they can be deployed in different servers respectively, or they can be partially set in the same server, which is not limited here.

[0120] It should be understood that in other embodiments, the computing system 3 may also include more computing nodes (general computing nodes or heterogeneous computing nodes), and the computing nodes in the computing system 3 can be connected through a chip interconnection bus, which is not limited here.

[0121] It should be understood that the chip interconnect bus switch can implement data and address exchange according to the chip interconnect bus protocol. In some embodiments, the chip interconnect bus switch can include a PCIe switch, a CXL switch, etc.

[0122] It should be understood that in some embodiments, data access can be achieved between heterogeneous computing nodes, between the host of a heterogeneous computing node and the corresponding AI chip through shared virtual memory (SVM). In other words, the host or AI chip can store the mapping relationship between the shared virtual address (SVA) of other computing nodes and the physical address of the memory. When the host or AI chip of a computing node wants to access the memory space in other computing nodes, it can query the corresponding physical address through the shared virtual address of other computing nodes, and then access the corresponding memory space through the chip interconnect bus based on the physical address.

[0123] For example, referring to FIG9A , corresponding to the computing system 2 shown in FIG7 , assume that process pr1 of application program AP1 running on host 11 of heterogeneous computing node P1 wants to share the memory space corresponding to physical address PA0 in memory 13 with process pr2 of application program AP2 running on host 21. Process pr1 can store the mapping relationship (VA_S→PA0) between physical address PA0 and the shared virtual address VA_S corresponding to physical address PA0 in process pr1's page table, for example, in memory management unit (MMU) 121 corresponding to CPU 12 and system memory management unit (SMMU) 151 corresponding to accelerator 15. Heterogeneous computing node P1 then sends this mapping relationship to process pr2 running on heterogeneous computing node P2. Process pr2 can store the received mapping relationship (VA_S→PA0) in its page table, for example, in memory MMU 221 corresponding to CPU 22 and SMMU 251 corresponding to accelerator 25. If process pr2 wants to access the memory space corresponding to physical address PA0 in memory 13, it can query the physical address PA0 from MMU 221 or SMMU 251 through the shared virtual address VA_S, and then use the chip interconnect bus to directly access the memory space corresponding to physical address PA0 in memory 13 through physical address PA0.

[0124] For another example, referring to FIG9B , corresponding to computing system 2 shown in FIG7 , assume that process pr1 of application program AP1 running on host 11 of heterogeneous computing node P1 wants to share the memory space corresponding to physical address PA01 in memory 16 with process pr2 of application program AP2 running on host 21. Process pr1 may store a mapping relationship (VA_S1→PA01) between physical address PA01 and the shared virtual address VA_S1 corresponding to physical address PA01 in its page table, for example, in SMMU 151. Heterogeneous computing node P1 then sends this mapping relationship to process pr2 running on computing node P2. Process pr2 may store the received mapping relationship (VA_S1→PA01) in its page table, for example, in SMMU 251 corresponding to accelerator 25. If process pr2 wants to access the memory space corresponding to physical address PA01 in memory 16, it can query the physical address PA01 from SMMU 251 through the shared virtual address VA_S1, and then use the chip interconnect bus to directly access the memory space corresponding to physical address PA01 in memory 16 through physical address PA01.

[0125] For another example, referring to FIG9C , corresponding to computing system 3 shown in FIG8 , assume that process pr3 of application program AP3 running on host 31 of heterogeneous computing node P3 shares the memory space corresponding to physical address PA02 in memory 33 with process pr4 of application program AP4 running on host 41. Process pr3 may store the mapping relationship (VA_S2→PA02) between physical address PA02 and the shared virtual address VA_S2 corresponding to physical address PA02 in process pr3's page table, for example, in MMU 321 corresponding to CPU 32 and SMMU 3511 corresponding to acceleration unit 351. Heterogeneous computing node P3 then sends this mapping relationship to process pr4 running on heterogeneous computing node P4. Process pr4 may store the received mapping relationship (VA_S2→PA02) in process pr4's page table, for example, in MMU 421 corresponding to CPU 42 and SMMU 3521 corresponding to acceleration unit 352. If process pr4 wants to access the memory space corresponding to physical address PA02 in memory 33, it can query the physical address PA02 from MMU 421 or SMMU 3521 through the shared virtual address VA_S2, and then use the chip interconnect bus to directly access the memory space corresponding to physical address PA02 in memory 33 through physical address PA02.

[0126] For example, referring to FIG9D , corresponding to computing system 3 shown in FIG8 , assume that process pr3 of application program AP3 running on host 31 of heterogeneous computing node P3 wants to share the memory space corresponding to physical address PA03 in memory 36 with process pr4 of application program AP4 running on host 41. Process pr3 may store the mapping relationship (VA_S3→PA03) between physical address PA03 and the shared virtual address VA_S3 corresponding to physical address PA03 in its page table, for example, in MMU 321 corresponding to CPU 32 and SMMU 3511 corresponding to acceleration unit 351. Heterogeneous computing node P3 may then send this mapping relationship to process pr4 running on heterogeneous computing node P4. Process pr4 may store the received mapping relationship (VA_S3→PA03) in its page table, for example, in SMMU 3521 corresponding to acceleration unit 352. If process pr4 wants to access the memory space corresponding to physical address PA03 in memory 36, it can query the physical address PA03 from SMMU 3521 through the shared virtual address VA_S3, and then use the chip interconnect bus to directly access the memory space corresponding to physical address PA03 in memory 36 through physical address PA03.

[0127] The following describes a technical solution for implementing data access between heterogeneous computing nodes using a chip interconnect bus through an SVM approach, in conjunction with the aforementioned computing systems 2 and 3.

[0128] First, the software architecture of the embodiment of the present application is introduced.

[0129] Figure 10 shows a schematic diagram of the software architecture running in a host and an AI chip according to some embodiments of the present application.

[0130] As shown in Figure 10, the software architecture includes shared virtual memory management and shared virtual memory management agent, where the shared virtual memory management runs in the host and the shared virtual memory management agent runs in the AI ​​chip.

[0131] The shared virtual memory management may include an association node client, a page table management unit, and a virtual memory sharing unit.

[0132] The association node client is used to implement device node topology management functions, such as device node registration, device node deregistration, and device node topology information synchronization. A device node refers to a host or an AI chip.

[0133] In some embodiments, the association node client can be used to maintain an association node information table. The association node information table is used to record association domains in the computing system. An association domain is used to indicate a group of device nodes that can access data via the SVM method.

[0134] It should be understood that the association domain information of an association domain may include the association domain name, host device node, and associated device nodes. The host device node refers to the device node that created the association domain, and the associated device nodes refer to other device nodes in the association domain except the host device node.

[0135] It should be understood that for one association domain, there is only one host device node, but there can be one or more association device nodes.

[0136] It should be understood that the specific process of the associated node client implementing device node registration and deregistration will be introduced in the embodiments of Figures 12A and 12B and will not be repeated here.

[0137] The virtual memory sharing unit is used to apply for, destroy, and maintain shared virtual memory queues between processes. For example, it sends requests to the virtual memory sharing engine on the AI ​​chip to apply for, destroy, or maintain shared virtual memory queues. The specific methods by which the virtual memory sharing unit applies for, destroys, and maintains shared virtual memory queues are described in the embodiments of Figures 17 through 27 below and are not detailed here.

[0138] The page table management unit is responsible for creating, maintaining, and synchronizing the shared virtual address management table. The shared virtual address management table is used to record the usage of shared virtual addresses corresponding to each process on the host in the computing system, such as the usage of virtual addresses of each process that are only used by processes running on the computing node (hereinafter referred to as intra-computing node shared virtual addresses) and the usage of virtual addresses of each process that can be used by processes running on other computing nodes (hereinafter referred to as inter-computing node shared virtual addresses).

[0139] The shared virtual memory management agent can include a node topology management unit, a page table management unit, and a virtual memory sharing engine. Each AI chip can run a shared virtual memory management agent.

[0140] The node topology management unit is used to maintain the topological relationships of each device node in the computing system, such as maintaining a system node information status table. The system node information status table may include status information of all device nodes in the computing system (such as device node identification, connection relationships between device nodes, etc.), as well as the contents of the aforementioned associated node information table.

[0141] It should be understood that the specific process of the node topology management unit implementing the creation and maintenance of the associated node information table will be introduced in the embodiments of Figures 13 and 14 below and will not be described in detail here.

[0142] The page table management unit is responsible for creating, maintaining, and synchronizing the shared virtual address management table. The shared virtual address management table is used to record the usage of shared virtual addresses corresponding to each process on the host in the computing system, such as the usage of shared virtual addresses within the computing node and shared virtual addresses between computing nodes of each process.

[0143] The virtual memory sharing engine manages, maintains, and synchronizes shared virtual memory queues. These queues store information about virtual memory shared within the computing system using the SVM. Each piece of virtual memory sharing information in the queue corresponds to a share and may include information such as the queue name, the virtual memory address between computing nodes, the physical address corresponding to the virtual memory address between computing nodes, and the size of the virtual memory address between computing nodes.

[0144] For example, the virtual memory sharing engine can be used to receive and respond to the application, destruction and maintenance requests of the shared virtual memory queue sent by the virtual memory sharing unit. The details will be introduced in the embodiments of Figures 17 to 27 below and will not be repeated here.

[0145] It should be understood that the software architecture shown in FIG10 is only an example. In other embodiments, the software architecture may include more or fewer modules, or some modules may be merged or split, which is not limited here.

[0146] It should be understood that in a computing system, each host can run the aforementioned shared virtual memory management, and each AI chip can run the aforementioned shared virtual memory management agent.

[0147] For example, FIG11A shows a schematic diagram of a software architecture of the computing system 2 shown in FIG7 when accessing data through an SVM manner according to some embodiments of the present application.

[0148] As shown in Figure 11A, the shared virtual memory management D11-1 and application AP1 can be run in the host 11 (device identifier is D11), the shared virtual memory management D21-1 and application AP2 can be run in the host 21 (device identifier is D21), the AI ​​chip 14 (device identifier is D14) can run the shared virtual memory management agent D14-1, and the AI ​​chip 24 (device identifier is D24) can run the shared virtual memory management agent D24-1.

[0149] The shared virtual memory management D11 - 1 may include an associated node client D11 - 2 , a page table management unit D11 - 3 and a virtual memory sharing unit D11 - 4 .

[0150] The shared virtual memory management D21 - 1 may include an associated node client D21 - 2 , a page table management unit D21 - 3 , and a virtual memory sharing unit D21 - 4 .

[0151] The shared virtual memory management agent D14 - 1 may include a node topology management unit D14 - 2 , a page table management unit D14 - 3 , and a virtual memory sharing engine D14 - 4 .

[0152] The shared virtual memory management agent D24 - 1 may include a node topology management unit D24 - 2 , a page table management unit D24 - 3 , and a virtual memory sharing engine D24 - 4 .

[0153] It should be understood that the functions of each module can refer to the embodiment shown in FIG10 and will not be described in detail here.

[0154] For another example, FIG11B shows a schematic diagram of a software architecture when the computing system 3 shown in FIG8 accesses data through an SVM manner according to some embodiments of the present application.

[0155] As shown in Figure 11B, the shared virtual memory management D31-1 and application AP3 can run in the host 31 (device identifier is D31), the shared virtual memory management D41-1 and application AP4 can run in the host 41 (device identifier is D41), and the shared virtual memory management agent D34-1 runs in the AI ​​chip 34 (device identifier is D34).

[0156] The shared virtual memory management D31 - 1 may include an associated node client D31 - 2 , a page table management unit D31 - 3 and a virtual memory sharing unit D31 - 4 .

[0157] The shared virtual memory management D41 - 1 may include an associated node client D41 - 2 , a page table management unit D41 - 3 , and a virtual memory sharing unit D41 - 4 .

[0158] The shared virtual memory management agent D34 - 1 may include a node topology management unit D34 - 2 , a page table management unit D34 - 3 , and a virtual memory sharing engine D34 - 4 .

[0159] It should be understood that the functions of each module can refer to the embodiment shown in FIG10 and will not be described in detail here.

[0160] The following describes the technical solution of the present application in conjunction with the software architecture shown in FIG. 10 , FIG. 11A , and FIG. 11B and the situations shown in FIG. 7 to FIG. 9D .

[0161] Registering and unregistering device nodes

[0162] When each computing node of the computing system is started, for example, when the shared virtual memory management in the computing node is initialized, a device standby node registration request can be initiated to the corresponding shared virtual memory management agent to facilitate establishment of topology information of the device nodes in the computing system.

[0163] The following describes the technical solution for registering and deregistering device nodes in the computing system 3 in conjunction with the software architecture shown in FIG. 11B .

[0164] It should be understood that in the software architecture shown in FIG11B , shared virtual memory management D31-1 and shared virtual memory management D41-1 can send a device node registration request to shared virtual memory management agent D34-1 during initialization. Upon receiving a device node registration request from a device node, shared virtual memory management agent D34-1 can add the device node information of the device node to the system node information status table to establish a topology map of the device nodes in computing system 3.

[0165] Specifically, FIG12A shows a schematic diagram of an interactive process for device node registration according to some embodiments of the present application. As shown in FIG12A , the interactive process includes the following steps:

[0166] S1201: The associated node client D31-2 sends a device node registration request message to the node topology management unit D34-2.

[0167] When the shared virtual memory management unit D31-1 is initialized, the associated node client D31-2 sends a device node registration request message to the node topology management unit D34-2. The device node registration request message may include the device identifier (D31) of the host (host 31) where the associated node client D31-2 is located.

[0168] S1202: The node topology management unit D34-2 processes the device node registration request and adds the device node information to the system node information status table.

[0169] When receiving the device node registration request, the node topology management unit D34 - 2 adds the device node information (eg, device identification) to the system node information status table.

[0170] For example, after receiving the device identification of the associated node client D31 - 2 , the node topology management unit D34 - 2 may add the device identification “ D31 ” to the system node information status table.

[0171] It should be understood that each node topology management unit will add the device identifier of its own device to the system node information status table when it initializes. For example, node topology management unit D34-2 may add the device identifier "D34" of AI chip 34 to the system node information status table when it initializes.

[0172] S1203: The node topology management unit D34-2 synchronizes the system node information status table to other device node topology management units.

[0173] After the node topology management unit D34-2 adds the device node information to the system node information status table, if the device node (AI chip 34) where the node topology management unit D34-2 is located is also connected to other AI chips through the chip interconnection bus, the node topology management unit D34-2 can synchronize the system node information status table to the node topology management units of other AI chips.

[0174] It should be understood that after all device nodes in the computing system are registered, the system node information status table stored by each node topology management unit includes the device identifiers of all device nodes in the computing system. For example, for the aforementioned computing system 3, after host 31, host 41, and AI chip 34 are all registered, the device identifiers in the system node information status table stored by node topology management unit D34-2 may include: D31, D41, and D34.

[0175] It should be understood that when a device node in the computing system 3 goes offline, it may send a device node deregistration request to the shared virtual memory management agent D34-1. Upon receiving the device node deregistration request from a device node, the shared virtual memory management agent D34-1 may delete the device node information of the device node from the system node information status table to update the topology map of the device nodes in the computing system 3.

[0176] Specifically, FIG12B shows a schematic diagram of an interaction process for deregistering a device node D31 according to some embodiments of the present application. As shown in FIG12B , the interaction process includes the following steps:

[0177] S1204: The associated node client D31-2 sends a device node deregistration request message to the node topology management unit D34-2.

[0178] When the host D31 is offline, it can send a device node deregistration request message to the node topology management unit D34-2 through the associated node client D31-2. The device node deregistration request message can include the device identifier "D31" of the host 31.

[0179] S1205: The node topology management unit D34-2 processes the device node deregistration request and deletes the device node information from the system node information status table.

[0180] When receiving the device node deregistration request, the node topology management unit D34 - 2 deletes the device node information (eg, device identification) from the system node information status table.

[0181] For example, after receiving the device identification of the associated node client D31 - 2 , the node topology management unit D34 - 2 may delete the device identification “ D31 ” from the system node information status table.

[0182] It should be understood that when the device node where each node topology management unit is located goes offline, it can delete the device identifier of its own device node from the system node information status table. For example, when AI chip 34 goes offline, node topology management unit D34-2 can delete the device identifier "D34" of AI chip 34 from the system node information status table.

[0183] S1206: The node topology management unit D34-2 synchronizes the system node information status table to other node topology management units.

[0184] After node topology management unit D34-2 modifies the system node information status table (for example, after deleting device identifier D31), if the device node (AI chip 34) where node topology management unit D34-2 resides is still connected to other AI chips via the chip interconnect bus, node topology management unit D34-2 can synchronize the modified system node information status table with the node topology management units of other AI chips. This allows the system node information status table recorded by each AI chip to be updated immediately.

[0185] Affiliate group creation and maintenance

[0186] It should be understood that after the device nodes in the computing system are registered, the applications running in each host (such as applications for AI training or applications for AI reasoning) can establish an association group including multiple device nodes through the system management program of the computing system to facilitate communication between heterogeneous computing nodes. The system management program can create and delete association groups, add and delete device nodes in the association group, etc. through the node topology management unit according to the needs of the application. For example, device nodes running subtasks of the same computing task can be established in an association group so that each device node can obtain data from the memory of other device nodes or write data to the memory of other device nodes through the SVM.

[0187] It should be understood that in some embodiments, the shared virtual address of the computing nodes in an affinity group can only be used by the computing nodes in the affinity group, and cannot be used by computing nodes outside the affinity group.

[0188] Next, the process of establishing an association group for application programs in the computing system will be described by continuing to use the aforementioned computing system 3 as an example.

[0189] Specifically, FIG13 shows a schematic diagram of an interactive process for creating an association group according to some embodiments of the present application. As shown in FIG13 , the interactive process includes the following steps:

[0190] S1301: The system management program calls the association node client D31-2's create association group request interface.

[0191] When the system management program running in the host 31 receives an instruction from an application program (eg, application program AP3) that requires inter-node data communication, it can call the association node client D31-2's create association group request interface provided by the shared virtual memory manager D31-1.

[0192] S1302: The association node client D31-2 sends a create association group request message to the node topology management unit D34-2.

[0193] When the association node client D31-2 detects that the association group creation request interface is called, it can send a create association group request message to the node topology management unit D34-2 based on the input data when the association group creation request interface is called. The input data or request message can include the name of the association group and the device identifiers of the device nodes included in the association group.

[0194] For example, for computing system 3, the input data and request message may include the name of the association group "GP1", and the device identifiers of the device nodes included in the association group GP1: "D31", "D41", and "D34".

[0195] S1303: The node topology management unit D34-2 processes the request to create an association group and creates an association group in the system node information status table.

[0196] When receiving the create association group request message, the node topology management unit D34-2 can create an association group in the system node information status table according to the content of the request message, for example, add the name of the association group and the device identifiers of the device nodes included in the association group to the system node information status table.

[0197] Exemplarily, corresponding to the aforementioned computing system 3, when the host 31 and the host 41 use the AI ​​chip 34 to perform AI reasoning or AI training, the established association group can be the association group GP1 (that is, the name of the association group is GP1), and the device identifiers of the device nodes included in the association group GP1 are: "D31", "D41" and "D34".

[0198] S1304: The node topology management unit D34-2 synchronizes the association group information of the established association group to other device nodes.

[0199] After the node topology management unit D34-2 establishes a new association group, it can synchronize the association group information of the established association group to other device nodes, such as other AI chips in the computing system, the device nodes included in the association group, etc.

[0200] For example, for the aforementioned computing system 3 , the node topology management unit D34 - 2 may synchronize the information of the association group GP1 to the association node client D41 - 2 in the host 41 .

[0201] It should be understood that the node topology management unit D34-2 can synchronize only the association group information of the newly added association group to other device nodes, or can synchronize the association group information of all association groups (such as the association node information table) to other device nodes, which is not limited here.

[0202] S1305: The node topology management unit D34-2 obtains the synchronization results returned by other device nodes, where the synchronization results include the association group information sent by other device nodes.

[0203] The node topology management unit D34 - 2 may obtain synchronization results returned by other device nodes, where the synchronization results include association group information sent by other device nodes.

[0204] For example, if an application in another device node creates, modifies, or deletes association group information of one or more association groups, the association group information of these association groups may be sent to the node topology management unit D34 - 2 .

[0205] S1306: The node topology management unit D34-2 sends a create association group response message to the association node client D31-2. The response message includes the latest association group information.

[0206] After receiving the synchronization results returned by other device nodes, the node topology management unit D34-2 can combine the received synchronization results with the association group information stored in the AI ​​chip 34 to generate the latest association group information (such as an updated association node information table), and send the latest association group information to the association node client D31-2 via an application message.

[0207] For example, when the synchronization result includes an association group that is not included in the association group information in the AI ​​chip 34 , the node topology management unit D 34 - 2 may add the association group information of the association group to the AI ​​chip 34 .

[0208] For another example, when the synchronization result indicates that a certain association group needs to be deleted, the node topology management unit D34 - 2 may delete the association group information of the association group from the stored association group information.

[0209] For another example, if the synchronization result indicates that the association group information of a certain association group has changed, such as the included device nodes have increased or decreased, the node topology management unit D34-2 can update the changed information to the association group information of the association group.

[0210] S1307: The associated node client D31-2 processes the response message and updates the associated node information table.

[0211] After receiving the association group creation response message, the association node client D31 - 2 may update the association node information table stored in the host 31 according to the content of the association group creation response message.

[0212] S1308: The association node client D31-2 sends the association group information of the created association group to the system management program.

[0213] After updating the association node information table, the association node client D31 - 2 may send the association group information of the created association group to the system management program.

[0214] It should be understood that after receiving the association group information of the created association group, the system management program may send the association group information to the corresponding application (eg, application AP3) so that the application can access inter-node data based on the association group information.

[0215] It should be understood that when an application in a host no longer needs to perform data interaction between heterogeneous computing nodes, the corresponding association group can be deleted.

[0216] For example, FIG14 shows a schematic diagram of an interactive process for deleting an association group according to some embodiments of the present application. As shown in FIG14 , the interactive process includes the following steps:

[0217] S1401: The system management program calls the association group deletion request interface of the association node client D31-2.

[0218] When the system management program running in the host 31 receives an instruction from an application program (e.g., application program AP3) that needs to exchange data between nodes and wants to delete a certain association group, it can call the association node client D31-2's association group deletion request interface provided by the shared virtual memory manager D31-1.

[0219] S1402: The association node client D31-2 sends a delete association group request message to the node topology management unit D34-2.

[0220] When the association node client D31-2 detects that the association group deletion request interface is called, it can send a deletion association group request message to the node topology management unit D34-2 based on the association group information of the association group to be deleted input when the deletion association group request interface is called. The deletion association group request message includes the name of the association group to be deleted.

[0221] For example, for the computing system 3, the above-mentioned deletion association group request message may include the name of the association group "GP1".

[0222] S1403: The node topology management unit D34-2 processes the request to delete the association group and finds the association group to be deleted in the system node information status table.

[0223] S1404: The node topology management unit D34-2 synchronizes the association group deletion request to other device nodes.

[0224] After finding the association group to be deleted, the node topology management unit D34 - 2 synchronizes the association group deletion request to other device nodes, such as other device nodes in the association group to be deleted, according to the device identifier recorded in the association group.

[0225] For example, when the node topology management unit D34 - 2 finds that the device identifiers in the association group GP1 include “D31 ”, “D41 ” and “D34 ”, it may synchronize a request to delete the association group GP1 to the association node client D41 - 2 in the host 41 .

[0226] It should be understood that after receiving the association group deletion request, other device nodes may delete the corresponding association group from the association node information table or the system node information status table.

[0227] S1405: The node topology management unit D34-2 obtains the synchronization results returned by other device nodes.

[0228] The node topology management unit D34 - 2 obtains the synchronization results returned by other device nodes, which may include information about addition, deletion, and modification of the associated group by other device nodes.

[0229] S1406: The node topology management unit D34-2 maintains the system node information status table according to the acquired synchronization result.

[0230] After receiving the synchronization results returned by other device nodes, the node topology management unit D34-2 can maintain the system node information status table based on the synchronization results to obtain the latest associated node information. For example, the node topology management unit D34-2 can first delete the associated group found in step S1403 and then maintain the system node information status table based on the synchronization results.

[0231] For example, when the synchronization result includes an association group that is not in the system node information status table of the AI ​​chip 34 , the node topology management unit D34 - 2 may add the association group information of the association group to the system node information status table of the AI ​​chip 34 .

[0232] For another example, when the synchronization result indicates that a certain association group needs to be deleted, the node topology management unit D34 - 2 may delete the association group information of the association group from the stored system node information status table.

[0233] For another example, if the synchronization result indicates that the association group information of a certain association group has changed, such as the included device nodes have increased or decreased, the node topology management unit D34-2 can update the changed information to the association group information of the association group in the system node information status table.

[0234] S1407: The node topology management unit D34-2 sends a delete association group response message to the associated node client D31-2.

[0235] After maintaining the system node information status table, the node topology management unit D34 - 2 may send a delete association group response message to the associated node client D31 - 2 . The response message may include the latest association group information.

[0236] S1408: The associated node client D31-2 processes the response message and updates the associated node information table.

[0237] After receiving the delete association group response message, the association node client D31-2 can update the association node information table stored in the host 31 according to the content of the delete association group response message, such as deleting the association group information of a certain association group, modifying the association group information of a certain association group, or adding a new association group.

[0238] S1409: The association node client D31-2 sends a response message for deleting the association group to the system management program.

[0239] After updating the associated node information table, the associated node client D31 - 2 may send a response message to the system management program to delete the associated group.

[0240] It should be understood that after receiving the response information of deleting the association group, the system management program may send the response information to the corresponding application (eg, application AP3) so that the application no longer accesses inter-node data through the association group.

[0241] It should be understood that in some embodiments, if the association group information of a certain association group needs to be maintained, such as adding a device node or reducing a device node, the system management application in the host can call the corresponding interface of the association node client to maintain the association group information.

[0242] For example, if you want to delete the host 41 from the aforementioned association group GP1, the system management application running in the host 41 can call the node deletion request interface of the association node client D41-2. Then, the association node client D41-2 sends a node deletion request to the node topology management unit D34-2. The node deletion request can include the device identifier of the device node to be deleted (the device identifier D41 of the host 41) and the association group name GP1. Secondly, the node topology management unit D34-2 searches for the association group GP1 in the system node information status table based on the association group name (GP1) in the node deletion request, and deletes the device identifier D41 from the association group GP1. Thirdly, the node topology management unit D34-2 can also synchronize the node deletion request to other device nodes so that other device nodes can also find the association group GP1 in their respective system node information status tables and delete the device identifier D41 from the association group GP1.

[0243] For another example, if you want to add host 42 to the aforementioned association group GP1, the system management application running in host 42 can call the add node request interface of the corresponding association node client. Then, the association node client running in host 42 sends an add node request to the node topology management unit D34-2. The add node request can include the device identifier of the device node to be added (the device identifier D42 of host 42) and the association group name GP1. Secondly, the node topology management unit D34-2 searches for the association group GP1 in the system node information status table based on the association group name (GP1) in the add node request, and adds the device identifier D42 to the association group GP1. Thirdly, the node topology management unit D34-2 can also synchronize the add node request to other device nodes so that other device nodes can also search for the association group GP1 in their respective system node information status tables and add the device identifier D42 to the association group GP1.

[0244] It should be understood that after the association group is created, the application programs running in each computing node can access data between the computing nodes through the SVM.

[0245] Shared virtual address initialization

[0246] If an application in a computing node wants to access data between nodes, it can first initialize the shared virtual memory function through shared virtual memory management to divide the virtual address space of the application process into the process-private virtual address space, the shared virtual address space within the computing node, and the shared virtual address space between computing nodes.

[0247] The private virtual address space of a process is the virtual address space that can be accessed by the process but not by other processes.

[0248] The shared virtual address space within a compute node refers to the virtual address space that is accessible to other processes on the compute node where the process is located but is inaccessible to processes on other compute nodes.

[0249] The shared virtual address space between computing nodes refers to the virtual address space that can be accessed by other processes on the computing node where the process is located and processes on other computing nodes.

[0250] The following describes the technical solution for initializing the shared virtual address in conjunction with the computing system 3 shown in FIG8 and the software architecture shown in FIG11B .

[0251] For example, FIG15 shows a schematic diagram of an interactive process for initializing a shared virtual address of a process according to some embodiments of the present application. As shown in FIG15 , the process includes the following steps:

[0252] S1501: Application program AP3 calls a shared virtual address space initialization request interface.

[0253] When the application program AP3 is started or needs to communicate data with applications in other computing nodes, it can call the shared virtual address space initialization request interface.

[0254] S1502: The page table management unit D31-3 sends a request for obtaining device node topology information to the page table management unit D34-3.

[0255] In response to the application program AP3 calling the shared virtual address space initialization request interface, the page table management unit D31 - 3 sends a request for obtaining device node topology information to the page table management unit D34 - 3 .

[0256] S1503: The page table management unit D34-3 obtains the device node topology information from the system node information status table, and divides the virtual address space of the process of the application program AP3 according to the device node topology information.

[0257] The page table management unit D34 - 3 obtains the device node topology information from the system node information status table in response to the device node topology information acquisition request, and divides the virtual address space of the process of the application program AP3 according to the device node topology information.

[0258] Exemplarily, the page table management unit D34 - 3 may divide the virtual address space of the process of the application program AP3 into a process-private virtual address space, an intra-computing-node shared virtual address space, and an inter-computing-node shared virtual address space.

[0259] For example, referring to FIG16 , if the operating system running on the host 31 is 64-bit, the virtual address space corresponding to the application program AP3 may include 128TB. The page table management unit D34-3 may divide the 128TB virtual address space into a process-private virtual address space, a shared virtual address space within a computing node, and a shared virtual address space between computing nodes. The shared virtual address space between computing nodes may be divided based on the number of device nodes in the device node topology information. For example, the shared virtual address space between computing nodes may be divided into N virtual address spaces of equal or different sizes, where N is the number of device nodes in the computing system.

[0260] S1504: The page table management unit D34-3 sends the device node topology information to the page table management unit D31-3.

[0261] After acquiring the device node topology information from the system node information status table, the page table management unit D34 - 3 may send the device node topology information to the page table management unit D31 - 3 .

[0262] It should be understood that in some embodiments, step S1503 and step S1504 can be combined into the same step, or the execution order can be swapped, which is not limited here.

[0263] S1505: The page table management unit D31-3 divides the virtual address space of the process of the application program AP3 according to the device node topology information.

[0264] After receiving the device node topology information, the page table management unit D31 - 3 may divide the virtual address space of the process of the application program AP3 according to the device node topology information.

[0265] It should be understood that page table management unit D31-3 and page table management unit D34-3 use the same method to divide the virtual address space of each process. Because page table management unit D31-3 and page table management unit D34-3 use the same device node topology information when performing the division, page table management unit D31-3 and page table management unit D34-3 also divide the virtual address space for the same process into the same manner.

[0266] It should be understood that after the virtual memory address of the application process running in the computing node is initialized, the application process can create a shared virtual memory queue so that processes in other computing nodes can establish a mapping relationship between the virtual memory address and the physical address, and access the physical address in the computing node through the shared virtual address.

[0267] Data access via SVM

[0268] The following describes a specific solution for data access between computing nodes through SVM, in combination with computing system 2 shown in Figure 7, computing system 3 shown in Figure 8, scenarios shown in Figures 9A to 9D, and software architecture shown in Figures 10 to 11B.

[0269] Figure 17 shows a schematic diagram of an interactive process of a data processing method according to some embodiments of the present application. As shown in Figure 17, the interactive process includes the following steps:

[0270] S1701: The host process in the host computing node sends a shared virtual memory request to the page table management unit.

[0271] When the host process of the host computing node needs to communicate data with other computing nodes, it may send a shared virtual memory request to the page table management unit in the host computing node. The request may include the size of the memory to be shared.

[0272] It should be understood that the host process refers to a process that initiates virtual memory sharing, and the computing node that runs the host process can be called a host computing node.

[0273] It should be understood that if the shared virtual memory requested by the host process is the memory in the host of the host computing node, the host process may apply for the shared virtual memory from the page table management unit in the host.

[0274] For example, for the situations shown in Figures 7, 9A and 11A, if the host process is process pr1 of application AP1, and the memory that process pr1 wants to share is the memory in the host 11, then process pr1 can apply for shared virtual memory from the page table management unit D11-3.

[0275] For another example, corresponding to the situations shown in Figures 8, 9C and 11B, if the host process is process pr3 of application AP3, and the memory that process pr3 wants to share is the memory in the host 31, then process pr3 can apply for shared virtual memory to the page table management unit D31-3.

[0276] It should be understood that if the shared virtual memory requested by the host process is the memory in the AI ​​chip of the host computing node, the main process can apply for the shared virtual memory from the page table management unit in the AI ​​chip.

[0277] For example, for the situations shown in Figures 7, 9B and 11A, if the host process is process pr1 of application AP1, and the memory that process pr1 wants to share is the memory in the AI ​​chip 14, then process pr1 can apply for shared virtual memory from the page table management unit D14-3.

[0278] For example, corresponding to the situations shown in Figures 8, 9D and 11B, if the host process is process pr3 of application AP3, and the memory that process pr3 wants to share is the memory in the AI ​​chip 34, then process pr3 can apply for shared virtual memory from the page table management unit D34-3.

[0279] It should be understood that in some embodiments, when the host process applies for a shared virtual address between computing nodes, it may also apply for a shared virtual address within the computing node at the same time, so that other processes running on the host computing node can communicate data with the host process through the shared virtual address within the computing node.

[0280] S1702: The page table management unit in the host computing node allocates a shared virtual address between computing nodes, and adds a mapping relationship between the allocated shared virtual address between computing nodes and the corresponding physical address to the page table of the host process.

[0281] After receiving a shared virtual memory request from the host process, the page table management unit in the host compute node allocates a virtual address segment from the host process's shared virtual address space as the shared virtual address segment based on the shared virtual address size specified in the request. It also allocates a physical address in the host compute node's memory that corresponds to the shared virtual address size. The mapping between the allocated shared virtual address segment and the physical address is then inserted into the page table of the processor corresponding to the physical address.

[0282] For example, referring to FIG18 , corresponding to the scenarios shown in FIG7 , FIG9A , and FIG11A , the host computing node may be heterogeneous computing node P1, and the host process may be process pr1 of application program AP1. If the virtual memory to be shared by process pr1 is memory in host computer 11, page table management unit D11-3 may allocate a virtual address VA_S shared between computing nodes, and the physical address corresponding to the virtual address VA_S shared between computing nodes is physical address PA0 in memory 13. The mapping relationship (VA_S→PA0) between the virtual address VA_S shared between computing nodes and the physical address PA0 is then inserted into the page table of process pr1, and the page table is stored in MMU 121 of the processor (CPU 12) corresponding to memory 13.

[0283] For another example, referring to FIG19 , corresponding to the scenarios shown in FIG7 , FIG9B , and FIG11A , the host computing node may be heterogeneous computing node P1, and the host process may be process pr1 of application program AP1. If the virtual memory to be shared by process pr1 is the memory in AI chip 14 , page table management unit D14-3 may allocate a virtual address VA_S1 shared between computing nodes, and the physical address corresponding to the virtual address VA_S1 shared between computing nodes is physical address PA01 in memory 16 . The mapping relationship (VA_S1→PA01) between the virtual address VA_S1 shared between computing nodes and the physical address PA01 is then inserted into the page table of process pr1, and the page table is stored in SMMU 151 of accelerator 15 .

[0284] For another example, referring to FIG20 , corresponding to the scenarios shown in FIG8 , FIG9C , and FIG11B , the host computing node may be computing node P3, and the host process may be process pr3 of application program AP3. If the virtual memory to be shared by process pr3 is memory in host computer 31, page table management unit D31-3 may allocate a virtual address VA_S2 shared between computing nodes. The physical address corresponding to the virtual address VA_S2 shared between computing nodes is physical address PA02 in memory 33. The mapping relationship (VA_S2→PA02) between the virtual address VA_2S shared between computing nodes and the physical address PA02 is then inserted into the page table of process pr3, and the page table is stored in MMU 321 of the processor (CPU 32) corresponding to memory 33.

[0285] For example, referring to FIG21 , corresponding to the scenarios shown in FIG8 , FIG9D , and FIG11B , the host computing node may be computing node P3, and the host process may be process pr3 of application program AP3. If the virtual memory to be shared by process pr3 is the memory in AI chip 34, page table management unit D34-3 may allocate a virtual address VA_S3 shared between computing nodes, and the physical address corresponding to the virtual address VA_S3 shared between computing nodes is physical address PA03 in memory 36. The mapping relationship (VA_S3→PA03) between the virtual address VA_S3 shared between computing nodes and the physical address PA03 is then inserted into the page table of process pr3, and the page table is stored in SMMU 3511 of acceleration unit 351.

[0286] It should be understood that in some embodiments, if the host process also applies for a shared virtual address within the computing node, the page table management unit can also allocate a shared virtual address within the computing node for the host process, and add the mapping relationship between the allocated shared virtual address within the computing node and the corresponding physical address to the page table of the host process.

[0287] For example, referring to Figures 18 and 19, if the host process is process pr1, the page table management unit D14-3 may allocate the intra-compute node shared virtual address VA_1 to process pr1 from the intra-compute node shared virtual address space of process pr1, and the physical address corresponding to VA_1 is the physical address PA1 in memory 16. If the host process is process pr2 of application program AP2, the page table management unit D34-3 may allocate the intra-compute node shared virtual address VA_2 to process pr2 from the intra-compute node shared virtual address space of process pr2, and the physical address corresponding to VA_2 is the physical address PA2 in memory 26.

[0288] For another example, referring to Figures 20 and 21 , if the host process is process pr3, the page table management unit D34-3 may allocate the intra-compute node shared virtual address VA_3 to process pr3 from the intra-compute node shared virtual address space of process pr3, and the physical address corresponding to VA_3 is the physical address PA3 in memory 36. If the host process is process pr4, the page table management unit D24-3 may allocate the intra-compute node shared virtual address VA_4 to process pr4 from the intra-compute node shared virtual address space of process pr4, and the physical address corresponding to VA_4 is the physical address PA4 in memory 36.

[0289] It should be understood that since the shared virtual addresses VA1, VA2, VA3, and VA4 within the computing node can only be used by the processes of the computing node, VA2 can be different from or the same as VA1, and VA4 can be different from or the same as VA3.

[0290] S1703: The page table management unit in the host computing node sends the allocation result to the host process.

[0291] After allocating the shared virtual address between computing nodes, the page table management unit in the host computing node can send the allocation result to the host process. The allocation result may include the shared virtual address between computing nodes, the physical address corresponding to the shared virtual address between computing nodes, the size of the shared virtual address between computing nodes, etc.

[0292] For another example, corresponding to the situation shown in Figure 18, the page table management unit D11-3 can send the virtual address VA_S shared between computing nodes, the size of the virtual address VA_S shared between computing nodes (for example, 10KB), and the physical address PA0 corresponding to VA_S to process pr1.

[0293] For example, corresponding to the situation shown in Figure 19, the page table management unit D14-3 can send the virtual address VA_S1 shared between computing nodes, the size of VA_S1 (for example, 10KB), and the physical address PA01 corresponding to the virtual address VA_S1 shared between computing nodes to process pr1.

[0294] For another example, corresponding to the situation shown in Figure 20, the page table management unit D31-3 can send the virtual address VA_S2 shared between computing nodes, the size of VA_S2 (for example, 10KB), and the physical address PA02 corresponding to the virtual address VA_S2 shared between computing nodes to process pr3.

[0295] For example, corresponding to the situation shown in Figure 21, the page table management unit D34-3 can send the virtual address VA_S3 shared between computing nodes, the size of VA_S3 (for example, 10KB), and the physical address PA03 corresponding to the virtual address VA_S3 shared between computing nodes to process pr3.

[0296] S1704: The host process sends a request to the host virtual memory sharing engine to establish a shared virtual memory queue.

[0297] After receiving the allocation result, if the host process wants to communicate data with other computing nodes, it can send a request to the host virtual memory sharing engine to establish a shared virtual memory queue. The request may include the sharing name, the host computing node identifier (such as the device identifier of the host and AI chip in the host computing node), and the destination computing node identifier (such as the device identifier of the host and AI chip in the host computing node).

[0298] It should be understood that in some embodiments, the host process can send a request to establish a shared virtual memory queue to the virtual sharing engine in the AI ​​chip through the virtual memory sharing unit in the host.

[0299] For example, for the situations shown in Figures 18 and 19, process pr1 can send a request to establish a shared virtual memory queue to the virtual memory sharing engine D14-4 through the virtual memory sharing unit D11-4. The request includes the shared virtual memory queue Q1, the host computing node identifier (the device identifier D11 of the host 11 and the device identifier D14 of the AI ​​chip 14), and the destination computing node identifier (the device identifier D21 of the host 21 and the device identifier D24 of the AI ​​chip 24).

[0300] For another example, in the scenarios shown in Figures 20 and 21, process pr1 can send a request to establish a shared virtual memory queue to virtual memory sharing engine D34-4 via virtual memory sharing unit D31-4. The request includes the shared name Q2, the host computing node identifier (device identifier D31 of host 31 and device identifier D34 of AI chip 34), and the destination computing node identifier (device identifier D41 of host 41 and device identifier D34 of AI chip 34).

[0301] It should be understood that the destination computing node may be one or more.

[0302] It should be understood that if a shared virtual memory queue has been established between the host process and the destination process (ie, the process in the destination computing node that communicates data with the host process), steps S1704 to S1706 may be skipped.

[0303] S1705: The host virtual memory sharing engine establishes a shared virtual memory queue and synchronizes it to the destination virtual memory sharing engine.

[0304] After receiving the request to establish virtual memory sharing, the host virtual memory sharing engine can respond to the request, establish a corresponding shared virtual memory queue, and synchronize it to each destination virtual memory sharing engine (the virtual memory sharing engine on the destination computing node).

[0305] For example, corresponding to the scenarios shown in Figures 18 and 19, virtual memory sharing engine D14-4 may establish a shared virtual memory queue Q1, which includes device nodes identified as D31, D41, and D34. Virtual memory sharing engine D14-4 then synchronizes shared virtual memory queue Q1 to virtual memory sharing engine D24-4.

[0306] For another example, corresponding to the scenarios shown in Figures 20 and 21, a shared virtual memory queue Q2 can be established by virtual memory sharing engine D34-4. The shared virtual memory queue Q2 includes device nodes identified as D31, D41, and D34. Since the host virtual memory sharing engine and the destination virtual memory sharing engine are the same, synchronization is not required.

[0307] It should be understood that in some embodiments, the host virtual memory sharing engine may also forward the virtual memory sharing establishment request to the destination virtual memory sharing engine instead of sending the established shared virtual memory queue, which is not limited here.

[0308] It should be understood that the process of the host virtual memory sharing engine synchronizing the shared virtual memory queue to the destination virtual memory sharing engine can be achieved through data transmission through the chip interconnection bus, or through other forms, such as data transmission through the network. Here, there is no limitation on the data transmission method in the process of the host virtual memory sharing engine synchronizing the shared virtual memory queue to the destination virtual memory sharing engine.

[0309] S1706: The destination virtual memory sharing engine establishes a shared virtual memory queue.

[0310] After receiving the shared virtual memory queue or the request to establish a shared virtual memory queue sent by the host virtual memory sharing engine, the destination virtual memory sharing engine may establish the same shared virtual memory queue as that in the host virtual memory sharing engine.

[0311] For example, corresponding to the situations shown in Figures 18 and 19, when the virtual memory sharing engine D24-4 receives the shared virtual memory queue Q1, it can establish the same shared virtual memory queue Q1, and the device nodes corresponding to the shared virtual memory queue Q1 are identified as D31, D41 and D34.

[0312] S1707: The host process sends a virtual memory sharing information enqueue request to the host virtual memory sharing engine.

[0313] When the host process needs to communicate with other computing nodes in the established shared virtual memory queue, it can send a virtual memory sharing information enqueue request to the host virtual memory sharing engine. The request may include the shared virtual address between computing nodes, the size of the shared virtual address between computing nodes, the physical address corresponding to the shared virtual address between computing nodes, and the shared virtual memory queue name, etc.

[0314] It should be understood that in some embodiments, the host process can send a request to establish a shared virtual memory queue to the virtual sharing engine in the AI ​​chip through the virtual memory sharing unit in the host.

[0315] For example, corresponding to the situation shown in Figure 18, process pr1 can send a virtual memory sharing information enqueue request to the virtual memory sharing engine D14-4 through the virtual memory sharing unit D11-4. The request may include the shared virtual memory queue name Q1, the shared virtual address VA_S between computing nodes, the shared virtual address size 10KB between computing nodes, and the physical address PA0 corresponding to the shared virtual address VA_S between computing nodes.

[0316] For another example, corresponding to the situation shown in Figure 19, process pr1 can send a virtual memory sharing information enqueue request to the virtual memory sharing engine D14-4 through the virtual memory sharing unit D11-4. The request may include the shared virtual memory queue name Q1, the shared virtual address VA_S1 between computing nodes, the shared virtual address size 10KB between computing nodes, and the physical address PA01 corresponding to the shared virtual address VA_S1 between computing nodes.

[0317] For another example, corresponding to the situation shown in Figure 20, process pr3 can send a virtual memory sharing information enqueue request to the virtual memory sharing engine D34-4 through the virtual memory sharing unit D31-4. The request may include the shared virtual memory queue name Q2, the shared virtual address VA_S2 between computing nodes, the shared virtual address size 10KB between computing nodes, and the physical address PA02 corresponding to the shared virtual address VA_S2 between computing nodes.

[0318] For example, corresponding to the situation shown in Figure 21, process pr3 can send a virtual memory sharing information enqueue request to the virtual memory sharing engine D34-4 through the virtual memory sharing unit D31-4. The request may include the shared virtual memory queue name Q2, the shared virtual address VA_S3 between computing nodes, the shared virtual address size 10KB between computing nodes, and the physical address PA03 corresponding to the shared virtual address VA_S3 between computing nodes.

[0319] It should be understood that in some embodiments, the destination process may also add virtual memory sharing information to the shared virtual memory queue by sending a virtual memory sharing information enqueue request to the destination virtual memory sharing engine, so that the host process or other destination processes can access the memory in the computing node where the destination process is located.

[0320] S1708: The host virtual memory sharing engine adds the virtual memory sharing information to the shared virtual memory queue and synchronizes it to the destination virtual memory sharing engine.

[0321] The host virtual memory sharing engine responds to the virtual memory information enqueue request, adds the virtual memory sharing information in the virtual memory sharing information enqueue request, such as the shared virtual address between computing nodes, the shared virtual address size between computing nodes and the physical address corresponding to the shared virtual address between computing nodes, etc., to the corresponding shared virtual memory queue, and synchronizes it to the destination virtual memory sharing engine in the queue.

[0322] It should be understood that after receiving the virtual memory sharing information, the destination virtual memory sharing engine may add the virtual memory sharing information to the shared virtual memory queue of the destination computing node.

[0323] For example, in the scenario shown in FIG18 , virtual memory sharing engine D14-4 can add virtual memory sharing information (the shared virtual address VA_S between computing nodes, the physical address PA0 corresponding to the shared virtual address VA_S between computing nodes, and the size of the shared virtual address VA_S between computing nodes being 10 KB) to shared virtual memory queue Q1. Virtual memory sharing engine D14-4 can then synchronize the virtual memory sharing information with virtual memory sharing engine D24-4.

[0324] For another example, in the scenario shown in FIG19 , virtual memory sharing engine D14-4 can add virtual memory sharing information (the inter-computing node shared virtual address VA_S1, the physical address PA01 corresponding to the inter-computing node shared virtual address VA_S1, and the size of the inter-computing node shared virtual address VA_S1 being 10 KB) to shared virtual memory queue Q1. Virtual memory sharing engine D14-4 can then synchronize the virtual memory sharing information with virtual memory sharing engine D24-4.

[0325] For another example, for the situation shown in Figure 20, the virtual memory sharing engine D34-4 can add the shared virtual address VA_S2 between computing nodes, the physical address PA02 corresponding to the shared virtual address VA_S2 between computing nodes, and the size of 10KB of the shared virtual address VA_S2 between computing nodes to the shared virtual memory queue Q2.

[0326] For example, for the situation shown in Figure 21, the virtual memory sharing engine D34-4 can add the shared virtual address VA_S3 between computing nodes, the physical address PA03 corresponding to the shared virtual address VA_S3 between computing nodes, and the size of 10KB of the shared virtual address VA_S3 between computing nodes to the shared virtual memory queue Q2.

[0327] It should be understood that in the scenarios shown in FIG. 20 and FIG. 21 , since both the host virtual memory sharing engine and the destination host virtual memory sharing engine are virtual memory sharing engines D34 - 4 , there is no need to synchronize the information of the shared virtual memory queue Q2 to the destination virtual memory sharing engine.

[0328] It should be understood that the process of the host virtual memory sharing engine synchronizing the virtual memory sharing information to the destination virtual memory sharing engine can be achieved through data transmission through the chip interconnection bus, or through other forms, such as data transmission through the network. Here, there is no limitation on the data transmission method in the process of the host virtual memory sharing engine synchronizing the virtual memory sharing information to the destination virtual memory sharing engine.

[0329] S1709: The host virtual memory sharing engine adds the mapping relationship between the shared virtual address between computing nodes and the corresponding physical address to the page table of the host process in the host computing node.

[0330] The host virtual memory sharing engine adds the mapping between the shared virtual address and the corresponding physical address to the host process's page table in the host compute node. If the physical address corresponding to the shared virtual address between compute nodes is a physical address in the host's memory, the host virtual memory sharing engine can add the page table to the host's MMU and the AI ​​chip's SMMU. If the physical address corresponding to the shared virtual address between compute nodes is a physical address in the AI ​​chip's memory, the host virtual memory sharing engine can add the page table to the AI ​​chip's SMMU.

[0331] For example, corresponding to the situation shown in Figure 18, the virtual memory sharing engine D14-4 can add the mapping relationship "VA_S→PA0" between the shared virtual address VA_S and the physical address PA0 between the computing nodes to the page table of the process pr1, for example, stored in the SMMU 151 corresponding to the accelerator 15 and the MMU 121 corresponding to the CPU 12.

[0332] For another example, corresponding to the situation shown in FIG19 , the virtual memory sharing engine D14 - 4 may add the mapping relationship “VA_S1→PA01” between the shared virtual address VA_S1 and the physical address PA01 between the computing nodes to the page table of the process pr1, for example, stored in the SMMU 151 corresponding to the accelerator 15 .

[0333] For another example, corresponding to the situation shown in Figure 20, the virtual memory sharing engine D34-4 can add the mapping relationship "VA_S2→PA02" between the shared virtual address VA_S2 and the physical address PA02 between the computing nodes to the page table of the process pr3, for example, stored in the SMMU 3511 corresponding to the acceleration unit 351 and the MMU 321 corresponding to the CPU 32.

[0334] For example, corresponding to the situation shown in Figure 21, the virtual memory sharing engine D34-4 can add the mapping relationship "VA_S3→PA03" between the shared virtual address VA_S3 and the physical address PA03 between the computing nodes to the page table of the process pr3, for example, stored in the SMMU 3511 corresponding to the acceleration unit 351.

[0335] It should be understood that in some embodiments, if the physical address corresponding to the shared virtual address between computing nodes is less than or equal to 4KB, the host virtual memory sharing engine may establish a page table entry to store the mapping relationship between the shared virtual address between computing nodes and the corresponding physical address. If the physical address corresponding to the shared virtual address between computing nodes is greater than 4KB, the host virtual memory sharing engine may establish a page table entry for storing the mapping relationship between the shared virtual address between computing nodes and the corresponding physical address based on a granularity of 4KB per physical page. In other words, the mapping relationship between the shared virtual address between computing nodes and the corresponding physical address can be stored via one or more page table entries.

[0336] It can be understood that in some embodiments, the execution order of step S1708 and step S1709 can be exchanged or combined into one step, which is not limited here.

[0337] S1710: The destination process sends a virtual memory sharing information dequeue request to the destination virtual memory sharing engine.

[0338] The destination process sends a virtual memory sharing information dequeue request to the destination virtual memory sharing engine. The request may include the name of the shared virtual memory queue.

[0339] It should be understood that the destination process can send a virtual memory sharing information dequeue request to the destination virtual memory sharing engine when it detects a change in the information in the shared virtual memory queue, or receives a message from the host process requiring data communication, or wants to obtain data from the host process, or when other methods agreed upon with the host process are met.

[0340] It should be understood that, in some embodiments, the destination process may send a virtual memory sharing information dequeue request to the destination virtual memory sharing engine via a virtual memory sharing unit running in the destination computing node.

[0341] For example, for the situations shown in Figures 18 and 19, the process pr2 of the application AP2 can send a virtual memory sharing information dequeue request to the virtual memory sharing engine D24-4 through the virtual memory sharing unit D21-4, and the request may include the name Q1 of the shared virtual memory queue.

[0342] For another example, for the situations shown in FIG20 and FIG21, the process pr4 of the application program AP4 may send a virtual memory sharing information dequeue request to the virtual memory sharing engine D34-4 through the virtual memory sharing unit D41-4, and the request may include the name Q2 of the shared virtual memory queue.

[0343] S1711: The destination virtual memory sharing engine adds the mapping relationship between the shared virtual address between computing nodes and the corresponding physical address to the page table of the destination process in the destination computing node.

[0344] The destination virtual memory sharing engine responds to the virtual memory sharing information dequeue request, searches for the corresponding shared virtual memory information according to the shared virtual memory queue name in the request, and adds the mapping relationship between the shared virtual address and the corresponding physical address between the computing nodes to the page table of the destination process in the destination computing node based on the queried shared virtual memory information.

[0345] For example, corresponding to the scenario shown in FIG18 , the virtual memory sharing engine D24-4 can query the shared virtual address VA_S between computing nodes and the corresponding physical address PA0 from the shared virtual memory queue according to the shared virtual memory queue name Q1. The mapping relationship between the shared virtual address VA_S between computing nodes and the physical address PA0 is then added to the page table of process pr2. This page table can be stored in the SMMU 251 corresponding to the accelerator 25 and the MMU 221 corresponding to the CPU 22.

[0346] For another example, corresponding to the scenario shown in FIG19 , the virtual memory sharing engine D24-4 can query the shared virtual address VA_S1 between computing nodes and the corresponding physical address PA01 from the shared virtual memory queue according to the shared virtual memory queue name Q1. The mapping relationship between the shared virtual address VA_S1 between computing nodes and the physical address PA01 is then added to the page table of process pr2. This page table can be stored in the SMMU 251 corresponding to the accelerator 25.

[0347] For another example, corresponding to the scenario shown in FIG20 , the virtual memory sharing engine D34-4 can query the shared virtual address VA_S2 between computing nodes and the corresponding physical address PA02 from the shared virtual memory queue according to the shared virtual memory queue name Q2. The mapping relationship between the shared virtual address VA_S2 between computing nodes and the physical address PA02 is then added to the page table of process pr4. This page table can be stored in the SMMU 3521 corresponding to the acceleration unit 352 and the MMU 421 corresponding to the CPU 42.

[0348] For example, corresponding to the scenario shown in FIG21 , the virtual memory sharing engine D34-4 can query the shared virtual address VA_S3 between computing nodes and the corresponding physical address PA03 from the shared virtual memory queue according to the shared virtual memory queue name Q2. The mapping relationship between the shared virtual address VA_S3 between computing nodes and the physical address PA03 is then added to the page table of process pr4. This page table can be stored in the SMMU 3521 corresponding to the acceleration unit 352.

[0349] It should be understood that in some embodiments, if the physical address corresponding to the shared virtual address between computing nodes is less than or equal to 4KB, the destination virtual memory sharing engine may establish a page table entry to store the mapping relationship between the shared virtual address between computing nodes and the corresponding physical address. If the physical address corresponding to the shared virtual address between computing nodes is greater than 4KB, the destination virtual memory sharing engine may establish a page table entry for storing the mapping relationship between the shared virtual address between computing nodes and the corresponding physical address based on a granularity of 4KB per physical page. In other words, the mapping relationship between the shared virtual address between computing nodes and the corresponding physical address can be stored via one or more page table entries.

[0350] S1712: The destination virtual memory sharing engine sends the shared virtual address between computing nodes and the corresponding address size to the destination process.

[0351] After adding the mapping relationship between the shared virtual address between computing nodes and the corresponding physical address to the page table of the destination process, the destination virtual memory sharing engine may send the shared virtual address between computing nodes and the corresponding address size to the destination process.

[0352] It should be understood that in some embodiments, the destination virtual memory sharing engine may send the shared virtual address and the corresponding address size between computing nodes to the destination process via the virtual memory sharing unit in the destination computing node.

[0353] For example, for the situation in FIG18 , the virtual memory sharing engine D24 - 4 may send the shared virtual address VA_S between computing nodes and the corresponding address size 10KB to the process pr2 through the virtual memory sharing unit D21 - 4 .

[0354] For another example, in the scenario of FIG19 , the virtual memory sharing engine D24 - 4 may send the shared virtual address VA_S1 between computing nodes and the corresponding address size 10KB to the process pr2 through the virtual memory sharing unit D21 - 4 .

[0355] For another example, in the situation of FIG20 , the virtual memory sharing engine D34 - 4 may send the shared virtual address VA_S2 between computing nodes and the corresponding address size 10KB to the process pr4 through the virtual memory sharing unit D41 - 4 .

[0356] For example, in the situation of FIG. 21 , the virtual memory sharing engine D34 - 4 may send the inter-computing node shared virtual address VA_S3 and the corresponding address size 10KB to the process pr4 through the virtual memory sharing unit D41 - 4 .

[0357] S1713: The destination process queries the corresponding physical address from the page table by computing the shared virtual address between nodes, and accesses the shared virtual memory through the queried physical address.

[0358] When the destination process wants to access the shared virtual memory in the host computing node, it can query the corresponding physical address from the page table of the destination process through the shared virtual address between computing nodes, and then use the chip interconnect bus to access the corresponding shared virtual memory space through the physical address.

[0359] For example, in the situation shown in FIG18 , if process pr2 wants to access the data of process pr1 in memory 13, it can query the physical address PA0 corresponding to the shared virtual address VA_S between computing nodes from MMU 221 or SMMU 251 through the shared virtual address VA_S between computing nodes, and then directly access the data through the physical address PA0.

[0360] For another example, in the scenario shown in FIG19 , if process pr2 wants to access the data of process pr1 in memory 16 , it can query the physical address PA01 corresponding to the shared virtual address VA_S1 from SMMU 251 through the shared virtual address VA_S1 between computing nodes, and then directly access the data through the physical address PA01.

[0361] For another example, in the situation shown in FIG20 , if process pr4 wants to access the data of process pr3 in memory 33, it can query the physical address PA02 corresponding to the shared virtual address VA_S2 between computing nodes from MMU 421 or SMMU 3521 through the shared virtual address VA_S2 between computing nodes, and then directly access the data through the physical address PA02.

[0362] For example, in the situation shown in FIG21 , if process pr4 wants to access the data of process pr3 in memory 36, it can query the physical address PA03 corresponding to the shared virtual address VA_S3 from SMMU 3521 through the shared virtual address VA_S3 between computing nodes, and then directly access the data through the physical address PA03.

[0363] Through the method provided in the embodiment of the present application, the destination process can access the data in the memory of the host computing node by using the chip interconnect bus through the shared virtual address between computing nodes, without the need for data transmission through the network, thereby improving the efficiency of data access.

[0364] In the aforementioned embodiments, the page tables of the destination process and the host process map physical addresses using the same inter-node shared virtual address. For example, in the scenario shown in FIG18 , the page tables of processes pr1 and pr2 both establish a mapping relationship with the physical address PA0 using the inter-node shared virtual address VA_S. In the scenario shown in FIG19 , the page tables of processes pr1 and pr2 both establish a mapping relationship with the physical address PA01 using the inter-node shared virtual address VA_S1. In the scenario shown in FIG20 , the page tables of processes pr3 and pr4 both establish a mapping relationship with the physical address PA02 using the inter-node shared virtual address VA_S2. In the scenario shown in FIG21 , the page tables of processes pr3 and pr4 both establish a mapping relationship with the physical address PA03 using the inter-node shared virtual address VA_S3.

[0365] However, in some scenarios, the inter-node shared virtual address provided by the host process to the destination process may have already been used by the destination process. Therefore, this application also provides another data processing method, in which a physical address is mapped using different inter-node shared virtual addresses in the page tables of the destination process and the host process.

[0366] Specifically, FIG22 shows a schematic diagram of an interactive process of a data processing method according to some embodiments of the present application. As shown in FIG22 , the interactive process includes the following steps:

[0367] S2201: The host process in the host computing node sends a shared virtual memory request to the page table management unit.

[0368] When the host process of the host computing node needs to communicate data with other processes, it can send a shared virtual memory request to the page table management unit in the host computing node. The request can include the size of the memory to be shared.

[0369] It should be understood that if the shared virtual memory requested by the host process is the memory in the host computing node's host, the host process may request the shared virtual memory from the page table management unit in the host.

[0370] It should be understood that if the shared virtual memory requested by the host process is the memory in the AI ​​chip of the host computing node, the host process can request the shared virtual memory from the page table management unit in the AI ​​chip. For details, please refer to the aforementioned step S1701 and will not be repeated here.

[0371] S2202: The page table management unit in the host computing node allocates a shared virtual address, and adds a mapping relationship between the allocated first shared virtual address and the corresponding physical address to the page table of the host process.

[0372] After receiving a shared virtual memory request from a host process, the page table management unit in the host computing node allocates a virtual address segment from the host process's shared virtual address space as a first shared virtual address based on the shared virtual address size specified in the request. It also allocates a physical address in the host computing node's memory that corresponds to the first shared virtual address size. The mapping between the allocated first shared virtual address and the physical address is then inserted into the page table of the processor corresponding to the physical address.

[0373] It should be understood that, compared to the aforementioned embodiments, the manner in which the virtual address space of the application process is divided in the embodiments of the present application is also different. For example, referring to FIG23 , when the application in the computing node initializes the shared virtual memory function in the shared virtual memory management and initializes the virtual address space of the application process, it is not necessary to divide the virtual address space of the process into three (the process private virtual address space, the shared virtual address space within the computing node, and the shared virtual address space between computing nodes). Instead, it is divided into two virtual address spaces, such as the process private virtual address space and the shared virtual address space (the virtual address space accessible to other processes).

[0374] For example, referring to FIG. 24 , corresponding to computing system 2 shown in FIG. 7 and FIG. 11A , the host computing node may be computing node P1, and the host process may be process pr1 of application program AP1. If the virtual memory to be shared by process pr1 is memory in host computer 11, page table management unit D11-3 may allocate a first shared virtual address VA_S4, and the physical address corresponding to the first shared virtual address VA_S4 is physical address PA04 in memory 13. The mapping relationship (VA_S4→PA04) between the first shared virtual address VA_S4 and the physical address PA04 is then inserted into the page table of process pr1, and the page table is stored in MMU 121 of the processor (CPU 12) corresponding to memory 13.

[0375] For another example, referring to FIG25 , corresponding to the computing system 2 shown in FIG7 and FIG11A , if the virtual memory to be shared by process pr1 is the memory in AI chip 14, page table management unit D14-3 may allocate a first shared virtual address VA_S5, and the physical address corresponding to the first shared virtual address VA_S5 is the physical address PA05 in memory 16. Then, the mapping relationship (VA_S5→PA05) between the first shared virtual address VA_S5 and the physical address PA05 is inserted into the page table of process pr1, and the page table is stored in SMMU 151 of accelerator 15.

[0376] For another example, referring to FIG26 , corresponding to the computing system 3 shown in FIG8 and FIG11B , the host computing node may be computing node P3, and the host process may be process pr3 of application program AP3. If the virtual memory to be shared by process pr3 is memory in host computer 31, page table management unit D31-3 may allocate a first shared virtual address VA_S6, and the physical address corresponding to shared virtual address VA_S6 is physical address PA06 in memory 33. The mapping relationship (VA_S6→PA06) between the first shared virtual address VA_S6 and physical address PA06 is then inserted into the page table of process pr3, and the page table is stored in the MMU 321 of the processor (CPU 32) corresponding to memory 33.

[0377] For another example, referring to FIG27 , corresponding to the computing system 3 shown in FIG8 and FIG11B , if the virtual memory to be shared by process pr3 is the memory in the AI ​​chip 34, the page table management unit D34-3 may allocate a first shared virtual address VA_S7, and the physical address corresponding to the first shared virtual address VA_S7 is the physical address PA07 in the memory 36. Then, the mapping relationship (VA_S7→PA07) between the first shared virtual address VA_S7 and the physical address PA07 is inserted into the page table of process pr3, and the page table is stored in the SMMU 3511 of the acceleration unit 351.

[0378] S2203: The page table management unit in the host computing node sends the allocation result to the host process.

[0379] After allocating the first shared virtual address, the page table management unit in the host computing node can send the allocation result to the host process. The allocation result may include the first shared virtual address, the physical address corresponding to the first shared virtual address, the address size of the first shared virtual address, etc.

[0380] For example, corresponding to the situation shown in FIG. 24 , the page table management unit D11 - 3 may send the first shared virtual address VA_S4 , the size of the first shared virtual address VA_S4 (eg, 10 KB), and the physical address PA04 corresponding to VA_S4 to the process pr1 .

[0381] For another example, corresponding to the situation shown in FIG. 25 , the page table management unit D14 - 3 may send the first shared virtual address VA_S5 , the size of the first shared virtual address VA_S5 (eg, 10 KB), and the physical address PA05 corresponding to VA_S5 to the process pr1 .

[0382] For another example, corresponding to the situation shown in FIG26 , the page table management unit D31 - 3 may send the first shared virtual address VA_S6 , the size of the first shared virtual address VA_S6 (eg, 10 KB), and the physical address PA06 corresponding to VA_S6 to the process pr3 .

[0383] For example, corresponding to the situation shown in Figure 27, the page table management unit D34-3 can send the first shared virtual address VA_S7, the size of the first shared virtual address VA_S7 (for example, 10KB), and the physical address PA07 corresponding to VA_S7 to the process pr3.

[0384] S2204: The host process sends a request to the host virtual memory sharing engine to establish a shared virtual memory queue.

[0385] After receiving the allocation result, if the host process wants to communicate data with other computing nodes, it can send a request to the host virtual memory sharing engine to establish a shared virtual memory queue. The request may include the sharing name, the host computing node identifier (such as the device identifier of the host and AI chip in the host computing node), and the destination computing node identifier (such as the device identifier of the host and AI chip in the host computing node).

[0386] It should be understood that in some embodiments, the host process can send a request to establish a shared virtual memory queue to the virtual sharing engine in the AI ​​chip through the virtual memory sharing unit in the host.

[0387] For example, for the situations shown in Figures 24 and 25, process pr1 can send a request to establish a shared virtual memory queue to the virtual memory sharing engine D14-4 through the virtual memory sharing unit D11-4. The request includes the shared virtual memory queue name Q1, the host computing node identifier (device identifier D11 of the host 11, device identifier D14 of the AI ​​chip 14), and the destination computing node identifier (device identifier D21 of the host 21, device identifier D24 of the AI ​​chip 24).

[0388] For another example, in the scenarios shown in Figures 26 and 27, process pr3 can send a request to establish a shared virtual memory queue to virtual memory sharing engine D34-4 via virtual memory sharing unit D31-4. The request includes the shared virtual memory queue name Q2, the host computing node identifier (device identifier D31 of host 31 and device identifier D34 of AI chip 34), and the destination computing node identifier (device identifier D41 of host 41 and device identifier D34 of AI chip 34).

[0389] It should be understood that the destination computing node may be one or more.

[0390] It should be understood that if a shared virtual memory queue has been established between the host process and the destination process (ie, the process in the destination computing node that communicates data with the host process), steps S2204 to S2206 may be skipped.

[0391] S2205: The host virtual memory sharing engine establishes a shared virtual memory queue and synchronizes it to the destination virtual memory sharing engine.

[0392] After receiving the request to establish virtual memory sharing, the host virtual memory sharing engine can respond to the request, establish a corresponding shared virtual memory queue, and synchronize it to each destination virtual memory sharing engine (the virtual memory sharing engine on the destination computing node).

[0393] For example, corresponding to the scenarios shown in Figures 24 and 25, virtual memory sharing engine D14-4 can establish a shared virtual memory queue Q1, corresponding to device nodes D11, D21, D14, and D24. Virtual memory sharing engine D14-4 then synchronizes shared virtual memory queue Q1 to virtual memory sharing engine D24-4.

[0394] For another example, corresponding to the situations shown in Figures 26 and 27, the shared virtual memory queue Q2 can be established by the virtual memory sharing engine D34-4, and the device nodes corresponding to the shared virtual memory queue Q2 are identified by D31, D41 and D34. Since the host virtual memory sharing engine and the destination virtual memory sharing engine are the same, there is no need to synchronize

[0395] It should be understood that in some embodiments, the host virtual memory sharing engine may also forward the virtual memory sharing establishment request to the destination virtual memory sharing engine instead of sending the established shared virtual memory queue, which is not limited here.

[0396] It should be understood that the process of the host virtual memory sharing engine synchronizing the shared virtual memory queue to the destination virtual memory sharing engine can be achieved through data transmission through the chip interconnection bus, or through other forms, such as data transmission through the network. Here, there is no limitation on the data transmission method in the process of the host virtual memory sharing engine synchronizing the shared virtual memory queue to the destination virtual memory sharing engine.

[0397] S2206: The destination virtual memory sharing engine establishes a shared virtual memory queue.

[0398] After receiving the shared virtual memory queue or the request to establish a shared virtual memory queue sent by the host virtual memory sharing engine, the destination virtual memory sharing engine may establish the same shared virtual memory queue as that in the host virtual memory sharing engine.

[0399] For example, corresponding to the situations shown in FIG24 and FIG25 , when the virtual memory sharing engine D24 - 4 receives the shared virtual memory queue Q1 , it can establish the same shared virtual memory queue Q1 , which includes device node identifiers D31 , D41 , and D34 .

[0400] S2207: The host process sends a virtual memory sharing information enqueue request to the host virtual memory sharing engine.

[0401] When the host process needs to communicate with other computing nodes in the established shared virtual memory queue, it can send a virtual memory sharing information enqueue request to the host virtual memory sharing engine. The request may include the first shared virtual address, the size of the first shared virtual address, the physical address corresponding to the first shared virtual address, and the shared virtual memory queue name, etc.

[0402] It should be understood that in some embodiments, the host process can send a request to establish a shared virtual memory queue to the virtual sharing engine in the AI ​​chip through the virtual memory sharing unit in the host.

[0403] For example, corresponding to the situation in Figure 24, process pr1 can send a virtual memory sharing information enqueue request to the virtual memory sharing engine D14-4 through the virtual memory sharing unit D11-4. The request may include the shared virtual memory queue name Q1, the first shared virtual address VA_S4, the first shared virtual address size 10KB, and the physical address PA04 corresponding to the first shared virtual address VA_S4.

[0404] For another example, corresponding to the situation in Figure 25, process pr1 can send a virtual memory sharing information enqueue request to the virtual memory sharing engine D14-4 through the virtual memory sharing unit D11-4. The request may include the shared virtual memory queue name Q1, the first shared virtual address VA_S5, the first shared virtual address size 10KB, and the physical address PA05 corresponding to the first shared virtual address VA_S5.

[0405] For another example, corresponding to the situation in Figure 26, process pr3 can send a virtual memory sharing information enqueue request to the virtual memory sharing engine D34-4 through the virtual memory sharing unit D31-4. The request may include the shared virtual memory queue name Q2, the first shared virtual address VA_S6, the first shared virtual address size 10KB, and the physical address PA06 corresponding to the first shared virtual address VA_S6.

[0406] For example, corresponding to the situation in Figure 26 or Figure 27, process pr3 can send a virtual memory sharing information enqueue request to the virtual memory sharing engine D34-4 through the virtual memory sharing unit D31-4. The request may include the shared virtual memory queue name Q2, the first shared virtual address VA_S7, the first shared virtual address size 10KB, and the physical address PA07 corresponding to the first shared virtual address VA_S7.

[0407] It should be understood that in some embodiments, the destination process may also add virtual memory sharing information to the shared virtual memory queue by sending a virtual memory sharing information enqueue request to the destination virtual memory sharing engine, so that the host process or other destination processes can access the memory in the computing node where the destination process is located.

[0408] S2208: The host virtual memory sharing engine adds the virtual memory sharing information to the shared virtual memory queue and synchronizes it to the destination virtual memory sharing engine.

[0409] The host virtual memory sharing engine responds to the virtual memory information enqueue request, adds the virtual memory sharing information in the virtual memory sharing information enqueue request, such as the first shared virtual address, the size of the first shared virtual address and the physical address corresponding to the first shared virtual address, to the corresponding shared virtual memory queue, and synchronizes it to the destination virtual memory sharing engine in the queue.

[0410] It should be understood that after receiving the virtual memory sharing information, the destination virtual memory sharing engine may add the virtual memory sharing information to the shared virtual memory queue of the destination computing node.

[0411] For example, in the scenario shown in FIG24 , virtual memory sharing engine D14-4 may add virtual memory sharing information (the first shared virtual address VA_S4, the physical address PA04 corresponding to the first shared virtual address VA_S4, and the size of the first shared virtual address VA_S4, which is 10 KB) to shared virtual memory queue Q1. Virtual memory sharing engine D14-4 may then synchronize the virtual memory sharing information with virtual memory sharing engine D24-4.

[0412] For another example, in the scenario shown in FIG25 , virtual memory sharing engine D14-4 may add virtual memory sharing information (first shared virtual address VA_S5, physical address PA05 corresponding to first shared virtual address VA_S5, and size of first shared virtual address VA_S5, 10 KB) to shared virtual memory queue Q1. Virtual memory sharing engine D14-4 may then synchronize the virtual memory sharing information with virtual memory sharing engine D24-4.

[0413] For another example, for the situation shown in Figure 26, the virtual memory sharing engine D34-4 can add the first shared virtual address VA_S6, the physical address PA06 corresponding to the first shared virtual address VA_S6, and the size of the first shared virtual address VA_S6 (10KB) to the shared virtual memory queue Q2.

[0414] For example, for the situation shown in Figure 27, the virtual memory sharing engine D34-4 can add the first shared virtual address VA_S7, the physical address PA07 corresponding to the first shared virtual address VA_S7, and the size of the first shared virtual address VA_S7 (10KB) to the shared virtual memory queue Q2.

[0415] It should be understood that in the scenarios shown in FIG. 26 and FIG. 27 , since both the host virtual memory sharing engine and the destination host virtual memory sharing engine are virtual memory sharing engines D34 - 4 , there is no need to synchronize the information of the shared virtual memory queue Q2 to the destination virtual memory sharing engine.

[0416] It should be understood that the process of the host virtual memory sharing engine synchronizing the virtual memory sharing information to the destination virtual memory sharing engine can be achieved through data transmission through the chip interconnection bus, or through other forms, such as data transmission through the network. Here, there is no limitation on the data transmission method in the process of the host virtual memory sharing engine synchronizing the virtual memory sharing information to the destination virtual memory sharing engine.

[0417] S2209: The host virtual memory sharing engine adds the mapping relationship between the first shared virtual address and the corresponding physical address to the page table of the host process in the host computing node.

[0418] The host virtual memory sharing engine adds the mapping relationship between the first shared virtual address and the corresponding physical address to the page table of the host process in the host computing node. If the first corresponding physical address is a physical address in the host's memory, the host virtual memory sharing engine may store the page table in the host's MMU and the AI ​​chip's SMMU; if the physical address corresponding to the first shared virtual address is a physical address in the AI ​​chip's memory, the host virtual memory sharing engine may store the page table in the AI ​​chip's SMMU.

[0419] For example, corresponding to the situation shown in Figure 24, the virtual memory sharing engine D14-4 can add the mapping relationship "VA_S4→PA04" between the first shared virtual address VA_S and the physical address PA4 to the page table of the process pr1, for example, stored in the SMMU 151 corresponding to the accelerator 15 and the MMU 121 corresponding to the CPU 12.

[0420] For another example, corresponding to the situation shown in FIG25 , the virtual memory sharing engine D14 - 4 may add the mapping relationship “VA_S5→PA05” between the first shared virtual address VA_S5 and the physical address PA05 to the page table of the process pr1 , for example, stored in the MMU 151 corresponding to the accelerator 15 .

[0421] For another example, corresponding to the situation shown in Figure 26, the virtual memory sharing engine D34-4 can add the mapping relationship "VA_S6→PA06" between the first shared virtual address VA_S6 and the physical address PA06 to the page table of process pr3, for example, stored in the SMMU3511 corresponding to the acceleration unit 351 and the MMU 321 corresponding to the CPU 32.

[0422] For example, corresponding to the situation shown in Figure 27, the virtual memory sharing engine D34-4 can add the mapping relationship "VA_S7→PA07" between the first shared virtual address VA_S7 and the physical address PA07 to the page table of process pr3, for example, stored in the SMMU3511 corresponding to the acceleration unit 351.

[0423] It should be understood that in some embodiments, if the physical address corresponding to the first shared virtual address is less than or equal to 4KB, the host virtual memory sharing engine may establish a page table entry to store the mapping relationship between the first shared virtual address and the corresponding physical address. If the physical address corresponding to the first shared virtual address is greater than 4KB, the host virtual memory sharing engine may establish a page table entry for storing the mapping relationship between the first shared virtual address and the corresponding physical address based on a granularity of 4KB per physical page. In other words, the mapping relationship between the first shared virtual address and the corresponding physical address may be stored via one or more page table entries.

[0424] S2210: The destination process sends a virtual memory sharing information dequeue request to the destination virtual memory sharing engine.

[0425] The destination process sends a virtual memory sharing information dequeue request to the destination virtual memory sharing engine. The request may include the name of the shared virtual memory queue.

[0426] It should be understood that the destination process can send a virtual memory sharing information dequeue request to the destination virtual memory sharing engine when it detects a change in the information in the shared virtual memory queue, or receives a message from the host process requiring data communication, or wants to obtain data from the host process, or when other methods agreed upon with the host process are met.

[0427] It should be understood that, in some embodiments, the destination process may send a virtual memory sharing information dequeue request to the destination virtual memory sharing engine via a virtual memory sharing unit running in the destination computing node.

[0428] For example, for the situations shown in Figures 24 and 25, the process pr2 of the application AP2 can send a virtual memory sharing information dequeue request to the virtual memory sharing engine D24-4 through the virtual memory sharing unit D21-4, and the request may include the name Q1 of the shared virtual memory queue.

[0429] For another example, for the situations shown in Figures 26 and 27, the process pr4 of the application AP2 can send a virtual memory sharing information dequeue request to the virtual memory sharing engine D34-4 through the virtual memory sharing unit D41-4, and the request can include the name Q2 of the shared virtual memory queue.

[0430] S2211: The destination virtual memory sharing engine sends virtual memory sharing information to the virtual memory sharing unit in the destination computing node.

[0431] When the destination virtual memory sharing engine receives a virtual memory sharing information dequeue request, it responds to the request, finds the corresponding shared virtual memory information according to the shared virtual memory queue name in the request, and sends the shared virtual memory information (such as the first shared virtual address, the physical address corresponding to the first shared virtual address, the size of the first shared virtual address, etc.) to the virtual memory sharing unit in the destination computing node.

[0432] For example, corresponding to the scenario shown in FIG24 , the virtual memory sharing engine D24-4 can query the shared virtual memory queue according to the shared virtual memory queue name Q1 to obtain the first shared virtual address VA_S4 and the corresponding physical address PA04. The shared virtual address VA_S4, the size of the shared virtual address VA_S4 (10 KB), and the physical address PA04 are then sent to the virtual memory sharing unit D21-4.

[0433] For another example, corresponding to the scenario shown in FIG25 , the virtual memory sharing engine D24-4 can query the shared virtual memory queue according to the shared virtual memory queue name Q1 to obtain the first shared virtual address VA_S5 and the corresponding physical address PA05. The shared virtual address VA_S5, the size of the shared virtual address VA_S5 (10 KB), and the physical address PA05 are then sent to the virtual memory sharing unit D21-4.

[0434] For another example, corresponding to the scenario shown in FIG26 , the virtual memory sharing engine D34-4 can query the shared virtual memory queue according to the shared virtual memory queue name Q2 to obtain the first shared virtual address VA_S6 and the corresponding physical address PA06. The shared virtual address VA_S6, the size of the shared virtual address VA_S6 (10 KB), and the physical address PA06 are then sent to the virtual memory sharing unit D41-4.

[0435] For example, corresponding to the scenario shown in FIG27 , the virtual memory sharing engine D34-4 can query the shared virtual memory queue according to the shared virtual memory queue name Q2 to obtain the first shared virtual address VA_S7 and the corresponding physical address PA07. The shared virtual address VA_S7, the size of the shared virtual address VA_S7 (10 KB), and the physical address PA07 are then sent to the virtual memory sharing unit D41-4.

[0436] S2212: The virtual memory sharing unit in the destination computing node applies for a second virtual memory shared address from the shared virtual address space of the destination process according to the virtual memory sharing information.

[0437] After receiving the virtual memory sharing information, the virtual memory sharing unit in the destination computing node may apply for a second shared virtual address having the same size as the first shared virtual address according to the size of the first shared virtual address in the virtual memory sharing information.

[0438] For example, corresponding to the situation shown in FIG. 24 , the virtual memory sharing unit D21 - 4 may apply for a second shared virtual address VA_S4 ′, and the size of the second shared virtual address VA_S4 ′ is the same as that of the first shared virtual address VA_S4 .

[0439] For another example, corresponding to the situation shown in FIG. 25 , the virtual memory sharing unit D21 - 4 may apply for a second shared virtual address VA_S5 ′, and the size of the second shared virtual address VA_S5 ′ is the same as that of the first shared virtual address VA_S5 .

[0440] For another example, corresponding to the situation shown in FIG. 26 , the virtual memory sharing unit D41 - 4 may apply for a second shared virtual address VA_S6 ′, and the size of the second shared virtual address VA_S6 ′ is the same as that of the first shared virtual address VA_S6 .

[0441] For another example, corresponding to the situation shown in FIG. 27 , the virtual memory sharing unit D41 - 4 may apply for a second shared virtual address VA_S7 ′, and the size of the second shared virtual address VA_S7 ′ is the same as that of the first shared virtual address VA_S7 .

[0442] S2213: The virtual memory sharing unit in the destination computing node sends a shared virtual memory queue update request to the destination virtual memory sharing engine.

[0443] After applying for the second shared virtual address, the virtual memory sharing unit in the destination computing node can send a shared virtual memory queue update request to the destination virtual memory sharing engine. The update request may include the name of the shared virtual memory queue to be updated, the first shared virtual address, the second shared virtual address, the size of the first shared virtual address and / or the second shared virtual address, the physical address corresponding to the first shared virtual address, etc.

[0444] For example, corresponding to the situation shown in Figure 24, the virtual memory sharing unit D21-4 can send a shared virtual memory queue update request to the virtual memory sharing engine D24-4. The update request may include: the name Q1 of the shared virtual memory queue, the first shared virtual address VA_S4, the second shared virtual address VA_S4', the size of the first shared virtual address VA_S4 (10KB), and the physical address PA04 corresponding to the first shared virtual address VA_S4.

[0445] For another example, corresponding to the situation shown in Figure 25, the virtual memory sharing unit D21-4 can send a shared virtual memory queue update request to the virtual memory sharing engine D24-4. The update request may include: the name Q1 of the shared virtual memory queue, the first shared virtual address VA_S5, the second shared virtual address VA_S5', the size of the first shared virtual address VA_S5 (10KB), and the physical address PA05 corresponding to the first shared virtual address VA_S5.

[0446] For another example, corresponding to the situation shown in Figure 26, the virtual memory sharing unit D41-4 can send a shared virtual memory queue update request to the virtual memory sharing engine D34-4. The update request may include: the name Q2 of the shared virtual memory queue, the first shared virtual address VA_S6, the second shared virtual address VA_S6', the size of the first shared virtual address VA_S6 (10KB), and the physical address PA06 corresponding to the first shared virtual address VA_S6.

[0447] For example, corresponding to the situation shown in Figure 27, the virtual memory sharing unit D41-4 can send a shared virtual memory queue update request to the virtual memory sharing engine D34-4. The update request may include: the name Q2 of the shared virtual memory queue, the first shared virtual address VA_S7, the second shared virtual address VA_S7', the size of the first shared virtual address VA_S7 (10KB), and the physical address PA07 corresponding to the first shared virtual address VA_S7.

[0448] S2214: The destination virtual memory sharing engine updates the shared virtual memory queue, and adds the mapping relationship between the second shared virtual address and the corresponding physical address to the page table of the destination process in the destination computing node.

[0449] The destination virtual memory sharing engine responds to the shared virtual memory queue update request and updates the corresponding shared virtual memory queue according to the virtual memory sharing information in the request. Then, in the destination computing node, a mapping relationship between the second shared virtual address and the corresponding physical address is added to the page table of the destination process.

[0450] For example, corresponding to the scenario shown in FIG24 , the virtual memory sharing engine D24-4 may replace the first shared virtual address VA_S4 in the shared virtual memory queue Q1 with the second shared virtual address VA_S4′. The mapping relationship between the second shared virtual address VA_S4′ and the physical address PA04 is then added to the page table of process pr2. The page table may be stored in the SMMU 251 corresponding to the accelerator 25 and the MMU 221 corresponding to the CPU 12.

[0451] For another example, corresponding to the scenario shown in FIG25 , the virtual memory sharing engine D24-4 may replace the first shared virtual address VA_S5 in the shared virtual memory queue Q1 with the second shared virtual address VA_S5′. The mapping relationship between the second shared virtual address VA_S5′ and the physical address PA05 is then added to the page table of process pr2, which may be stored in the SMMU 251 corresponding to the accelerator 25.

[0452] For another example, corresponding to the scenario shown in FIG26 , the virtual memory sharing engine D34-4 may replace the first shared virtual address VA_S6 in the shared virtual memory queue Q2 with the second shared virtual address VA_S6′. The mapping relationship between the second shared virtual address VA_S6′ and the physical address PA06 is then added to the page table of the process pr4. The page table may be stored in the SMMU 3521 corresponding to the acceleration unit 352 and the MMU 421 corresponding to the CPU 42.

[0453] For another example, corresponding to the scenario shown in FIG27 , the virtual memory sharing engine D34-4 may replace the first shared virtual address VA_S7 in the shared virtual memory queue Q2 with the second shared virtual address VA_S7′. The mapping relationship between the second shared virtual address VA_S7′ and the physical address PA07 is then added to the page table of the process pr4. The page table may be stored in the SMMU 3521 corresponding to the acceleration unit 352.

[0454] It should be understood that in some embodiments, if the physical address corresponding to the second shared virtual address is less than or equal to 4KB, the host virtual memory sharing engine may establish a page table entry to store the mapping relationship between the second shared virtual address and the corresponding physical address. If the physical address corresponding to the second shared virtual address is greater than 4KB, the host virtual memory sharing engine may establish a page table entry for storing the mapping relationship between the second shared virtual address and the corresponding physical address based on a granularity of 4KB per physical page. In other words, the mapping relationship between the second shared virtual address and the corresponding physical address may be stored via one or more page table entries.

[0455] S2215: The destination virtual memory sharing engine sends the second shared virtual address and the corresponding address size to the destination process.

[0456] After adding the mapping relationship between the second shared virtual address and the corresponding physical address to the page table of the destination process, the destination virtual memory sharing engine may send the second shared virtual address and the corresponding address size to the destination process.

[0457] It should be understood that in some embodiments, the destination virtual memory sharing engine may send the second shared virtual address and the corresponding address size to the destination process via the virtual memory sharing unit in the destination computing node.

[0458] For example, for the situation shown in FIG24 , the virtual memory sharing engine D24 - 4 may send the inter-computing node shared virtual address VA_S4 ′ and the corresponding address size 10 KB to the process pr2 through the virtual memory sharing unit D21 - 4 .

[0459] For another example, in the scenario shown in FIG. 25 , the virtual memory sharing engine D24 - 4 may send the inter-computing node shared virtual address VA_S5 ′ and the corresponding address size 10 KB to the process pr2 through the virtual memory sharing unit D21 - 4 .

[0460] For another example, in the situation shown in FIG. 26 , the virtual memory sharing engine D34 - 4 may send the inter-computing node shared virtual address VA_S6 ′ and the corresponding address size 10 KB to the process pr4 through the virtual memory sharing unit D41 - 4 .

[0461] For example, in the scenario shown in FIG. 27 , the virtual memory sharing engine D34 - 4 may send the inter-computing node shared virtual address VA_S7 ′ and the corresponding address size 10 KB to the process pr4 through the virtual memory sharing unit D41 - 4 .

[0462] S2216: The destination process queries the corresponding physical address from the page table through the second shared virtual address, and accesses the shared virtual memory through the queried physical address.

[0463] When the destination process wants to access the shared virtual memory in the host computing node, it can query the corresponding physical address from the page table of the destination process through the second shared virtual address, and then use the chip interconnect bus to access the corresponding shared virtual memory space through the physical address.

[0464] For example, in the situation shown in FIG24 , if process pr2 wants to access the data of process pr1 in memory 13, it can query the physical address PA04 corresponding to the second shared virtual address VA_S4′ from MMU 221 or SMMU 251 through the second shared virtual address VA_S4′, and then directly access the data through the physical address PA04.

[0465] For another example, in the scenario shown in FIG25 , if process pr2 wants to access the data of process pr1 in memory 16, it can query the physical address PA05 corresponding to the second shared virtual address VA_S5′ from SMMU 251 through the second shared virtual address VA_S5′, and then directly access the data through the physical address PA05.

[0466] For another example, in the situation shown in FIG26 , if process pr4 wants to access the data of process pr3 in memory 33, it can query the physical address PA06 corresponding to the second shared virtual address VA_S6′ from MMU 421 or SMMU 3521 through the second shared virtual address VA_S6′, and then directly access the data through the physical address PA06.

[0467] For example, in the situation shown in FIG27 , if process pr4 wants to access the data of process pr3 in memory 36, it can query the physical address PA07 corresponding to the second shared virtual address VA_S7′ from SMMU 3521 through the second shared virtual address VA_S7′, and then directly access the data through the physical address PA07.

[0468] Through the method provided in the embodiment of the present application, the destination process can access data in the memory of the host computing node using the chip interconnect bus through the second shared virtual address without the need for data transmission through the network, thereby improving the efficiency of data access.

[0469] Furthermore, an embodiment of the present application also provides a server 100 for implementing one or more of the aforementioned computing nodes.

[0470] Specifically, FIG28 shows a schematic structural diagram of a server 100 according to some embodiments of the present application.

[0471] As shown in FIG. 28 , the server 100 includes at least one general-purpose processor 101 , at least one memory 102 , at least one heterogeneous processor 103 , and a bus 104 .

[0472] The general processor 101 may be a CPU, configured to execute instructions corresponding to the aforementioned application programs P1, P2, P3, P4, the system hypervisor, and each shared virtual memory manager to implement the data processing methods provided in each embodiment of the present application.

[0473] It should be understood that each general purpose processor can be connected to at least one memory 102 for storing execution and data during the operation of the server 100. In some embodiments, the memory 102 can be a volatile memory, such as random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), direct rambus RAM (DR RAM), etc.

[0474] The heterogeneous processor 103 may include one or more of GPU, FPGA, ASIC, NPU, and AI chip, which is used to execute the instructions corresponding to the aforementioned application P1, application P2, application P3, application P4, and each shared virtual memory management agent to implement the data processing method provided in each embodiment of the present application.

[0475] It should be understood that in some embodiments, at least one memory 1031 is connected to each heterogeneous processor 103, and the memory 1031 may include a volatile memory, such as RAM, SRAM, DRAM, SDRAM, DDR SDRAM, ESDRAM, SLDRAM, DR RAM, high bandwidth memory (HBM), etc.

[0476] It should be understood that in some embodiments, the memory 1031 may also be set inside the heterogeneous processor 103.

[0477] The bus 104 may include the aforementioned PCIe bus, CXL bus, QPI bus, etc.

[0478] It should be understood that the bus 104 may also include other buses for communication between modules in the server 100, buses for communication between the server 100 and input / output devices, buses for communication between the server 100 and other devices (eg, other servers), and the like.

[0479] It should be understood that when the server 100 is used to implement each computing node in the aforementioned computing system 1, computing system 2, and computing system 3, the heterogeneous processor 103 is optional.

[0480] It should be understood that at least one computing node among the aforementioned computing system 1, computing system 2, and computing system 3 can be deployed in the server 100.

[0481] It should be understood that the general-purpose processor 101 and the heterogeneous processor 103 in the server 100 can be connected via the aforementioned chip interconnect bus.

[0482] It should be understood that the server 100 may further include an MMU corresponding to the general processor 101 or a computing unit in the general processor 101, for storing a page table of a process.

[0483] It should be understood that the server 100 may further include an SMMU corresponding to a computing unit in the heterogeneous processor 103 or the heterogeneous processor 101, for storing a page table of a process.

[0484] Optionally, in some embodiments, the server 100 may further include a non-volatile memory for storing instructions and data involved in the methods provided in the embodiments of the present application.

[0485] Optionally, in some embodiments, the server 100 may further include a communication interface, such as a network interface, for communicating with other electronic devices (eg, other servers).

[0486] It should be understood that the structure of the server 100 shown in FIG28 is only an example. In other embodiments, the server 100 may also include more or fewer modules, which is not limited here.

[0487] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0488] Program code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor, a microcontroller, an application specific integrated circuit, or a microprocessor.

[0489] Program code can be implemented with a high-level programming language or an object-oriented programming language to communicate with the processing system. Where necessary, program code can also be implemented in assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0490] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to a floppy disk, an optical disk, an optical disc, a read-only memory, a magneto-optical disk, a random access memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, a magnetic card or an optical card, a flash memory, or a tangible machine-readable memory for transmitting information (e.g., a carrier wave, an infrared signal, a digital signal, etc.) using the Internet in an electrical, optical, acoustic, or other form of propagation signal. Therefore, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0491] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order may not be required. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of a structural or method feature in a particular figure does not imply that such feature is required in all embodiments, and in some embodiments, such features may not be included or may be combined with other features.

[0492] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems raised by this application. In addition, in order to highlight the innovative part of this application, the above-mentioned device embodiments of this application do not introduce units / modules that are not closely related to solving the technical problems raised by this application. This does not mean that other units / modules do not exist in the above-mentioned device embodiments.

[0493] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a" do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

Claims

1. A data processing method, applied to a computing system, characterized in that: The computing system comprises a first computing node and a second computing node, wherein the first computing node comprises a first processor, the second computing node comprises a second processor and a first memory, and the first processor and the second processor are connected via a chip interconnect bus; and the method comprises: The first processor detects a cross-node access request, where the cross-node access request is used to access the first memory; The first processor accesses the first memory through the chip interconnect bus.

2. The method according to claim 1, characterized in that The cross-node access request includes a first virtual address, and the first processor accesses the first memory through a chip interconnect bus, including: The first processor determines a first physical address corresponding to the first virtual address; The first processor accesses a first physical storage space corresponding to the first physical address in the first memory through the chip interconnection bus.

3. The method according to claim 2, characterized in that The first processor runs a first process, the first virtual address is a virtual address of the first process; and determining a first physical address corresponding to the first virtual address includes: The first processor searches a first page table for the first physical address through a first virtual address, wherein the first page table is a page table of the first process, and the first page table includes a first mapping relationship between the first virtual address and the first physical address.

4. The method according to claim 3, characterized in that The second processor runs a second process, and the first physical address is a physical address corresponding to a second virtual address of the second process.

5. The method according to claim 4, characterized in that The first mapping relationship is determined in the following manner: The second processor sends a second mapping relationship between the second virtual address and the first physical address to the first processor; The first processor adds the first mapping relationship between the first virtual address and the first physical address to the first page table according to the second mapping relationship.

6. The method according to claim 5, characterized in that The first computing node runs a first virtual memory sharing engine, and the second computing node runs a second virtual memory sharing engine; Furthermore, the second processor sends a second mapping relationship between the second virtual address and the first physical address to the first processor, including: After the second processor adds the second mapping relationship to the first queue, the second processor uses the second virtual memory sharing engine to synchronize the first queue to the first virtual memory sharing engine; The first queue is a shared virtual memory queue corresponding to the first computing node and the second computing node, and the first virtual memory sharing engine and the second virtual memory sharing engine both store the first queue.

7. The method according to claim 6, characterized in that The first virtual address and the second virtual address are the same.

8. The method according to claim 6, characterized in that The first virtual address and the second virtual address are different.

9. The method according to any one of claims 6 to 8, characterized in that The adding the first mapping relationship between the first virtual address and the first physical address to the first page table includes: The first processor obtains the second mapping relationship from the first queue of the second virtual memory sharing engine; The first processor determines the first mapping relationship based on the second mapping relationship, and adds the first mapping relationship to the first page table.

10. The method according to any one of claims 6 to 8, characterized in that The first computing node and the second computing node are computing nodes in a first association group.

11. The method according to claim 10, characterized in that A node topology management unit is running in the second computing node, and the first association group is determined in the following manner: The second computing node establishes a topological relationship of computing nodes in the computing system using the node topology management unit; The second computing node establishes the first association group based on the topological relationship, and the first association group includes the device identifier of each device node in the first computing node and the second computing node.

12. The method according to any one of claims 1 to 11, characterized in that The first memory is the memory corresponding to the second processor.

13. The method according to any one of claims 1 to 11, characterized in that The second computing node also includes a third processor, and the first memory is a memory corresponding to the third processor.

14. The method according to claim 1, characterized in that The first processor or the second processor includes any one of a central processing unit CPU, a graphics processing unit GPU, a field programmable gate array FPGA, an application specific integrated circuit ASIC, a neural network processing unit NPU, and an artificial intelligence chip.

15. The method according to any one of claims 1 to 14, characterized in that The chip interconnect bus includes at least one of the following buses: a high-speed serial computer expansion bus standard PCIe bus, a computing express interconnect CXL bus, and a quick path interconnect QPI bus.

16. The method according to claim 1, characterized in that At least one of the first computing node and the second computing node is a heterogeneous computing node.

17. A computing system, characterized in that: The computing system comprises a first computing node and a second computing node, wherein the first computing node comprises a first processor, the second computing node comprises a second processor and a first memory, and the first processor and the second processor are connected via a chip interconnect bus; The first processor is configured to access the first memory through the chip interconnect bus when a cross-node access request to access the first memory is detected.

18. A device, characterized in that include: A memory for storing instructions; At least one processor, configured to instruct the instructions to enable the device to implement the method according to any one of claims 1 to 16.

19. A readable storage medium, characterized in that: The readable storage medium stores instructions, and when the instructions are executed by a device, the device implements the method according to any one of claims 1 to 16.

20. A computer program product, characterized in that When the computer program product is executed on a device, the device is caused to implement the method according to any one of claims 1 to 16.

Citation Information

Patent Citations

  • Computing system and method for sharing device memories of different computing devices

    CN114385516A

  • Request processing method, device and system

    CN116010296A

  • Memory access method, chip, electronic equipment and computer readable storage medium

    CN116136826A

  • Remote direct memory access with copy-on-write support

    US20150089009A1

  • Using Shared Virtual Memory Resources for Performing Memory-Mapping

    US20150134930A1

Cited By

  • Computing system and computing method

    CN120980077A

  • Resource access method, computing network, and related device

    WO2026157313A1