Expansion method of interconnection topology, electronic equipment and storage medium
By using switches to connect all processing units in the GPU interconnect topology and using memory management strategies for interleaving and deinterleaving operations, the problem of the inability to scale the GPU interconnect topology in the prior art is solved, and more efficient interconnection performance is achieved.
Patent Information
- Application Number
- CN202411344118.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-25
- Publication Date
- 2025-04-15
AI Technical Summary
The scale of the GPU interconnect topology supported by existing hardware cannot be expanded, resulting in the inability to scale the interconnect scale between GPUs.
By obtaining the physical topology of the switch connecting all processing units and configuring the memory management policy of the master program, interleaving and deinterleaving operations are adopted to enable the port of each processing unit to route requests from different processing units.
The scale of the processing unit interconnection scale is achieved, which reduces latency and improves bandwidth, reliability and load balancing performance.
Smart Images
Figure CN119829479A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of chip design, and in particular to an expansion method of interconnection topology, electronic equipment and storage medium. Background Art
[0002] For the interconnection topology between multiple GPUs directly through physical connections, data access is fast and efficient. For example, for hardware with 8 fully interconnected GPUs, each GPU corresponds to a memory address range. For example, the local memory size of each GPU is 64GB. If the memory access address issued by the engine on GPU0 falls within the address range of GPU1, for example, the index bit of the GPU is [38:36], and the value of [38:36] is 1, it will be routed to the 4th port port4 on GPU0 by the on-chip bus system, and sent to the 6th port port6 of GPU1, and then enter GPU1. This process does not require software participation. The software only needs to know the address range corresponding to each GPU and perform corresponding memory resource management. The data access process is fast and efficient.
[0003] Since the routing between GPUs in the current hardware-supported interconnection topology is fixed, it is impossible to expand the scale of GPU interconnection. Summary of the invention
[0004] In view of the above technical problems, the technical solution adopted by the present invention is: a method for expanding an interconnection topology, the method comprising the following steps:
[0005] A physical topology is obtained, wherein the physical topology includes a switch and a plurality of processing units, wherein all processing units in the physical topology are interconnected with the switch; wherein each processing unit has N ports.
[0006] Configure the memory management strategy of the master control program so that each port of each processing unit accesses the memory address space of all processing units in the physical topology according to the non-local target physical address routing; the memory management strategy includes interleaving operations and deinterleaving operations. The interleaving operation includes: obtaining the original base address of the target physical address, moving the target bit of the page index in the original base address to before the unit index bit to obtain the interleaving base address, and the interleaving base address is used to establish a page table so that the master control program indexes the port of the current processing unit through the target bit. The deinterleaving operation includes: restoring the target bit in the interleaving base address to the original position through the switch to obtain the actual base address, so that the master control program accesses the target processing unit and its memory address space through the actual base address routing.
[0007] In addition, the present invention also provides a non-transitory computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the above method.
[0008] In addition, the present invention also provides an electronic device, including a processor and the above-mentioned non-transitory computer-readable storage medium.
[0009] The present invention has at least the following beneficial effects:
[0010] The embodiment of the present invention provides an interconnection topology expansion method, electronic device and storage medium, which obtains the physical topology of all processing units connected by a switch and combines the interleaving operation and deinterleaving operation in the memory management unit, so that the same port can simultaneously route access requests to different processors, so as to achieve the purpose of the same port of a processing unit communicating with all other processing units, that is, to achieve the purpose of expanding the interconnection scale of the processing units. In addition, since a processing unit can access another processing unit from different ports at the same time in parallel, it reduces latency and improves performance such as bandwidth, reliability and load balancing. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0012] Figure 1 A flow chart of a method for expanding an interconnection topology provided by an embodiment of the present invention;
[0013] Figure 2 A schematic diagram of a physical topology provided by an embodiment of the present invention;
[0014] Figure 3 A schematic diagram of the relationship between address spaces of GPU 3 and a main control program from different perspectives provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0016] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present invention have the same meanings as commonly understood by those skilled in the art.
[0017] See also Figure 1 , which shows a method for expanding an interconnection topology, the method comprising the following steps:
[0018] S100, obtaining a physical topology, wherein the physical topology includes at least one switch and a plurality of processing units, wherein all processing units in the physical topology are interconnected with the switch; wherein each processing unit has N ports.
[0019] In one embodiment, the processing unit is a graphics processing unit (Graphic Processing Unit, GPU), a general purpose computing on a graphics processor (General Purpose Computing on GPU, GPGPU) or a neural network processor (Neural-network Processing Units, NPU), and other processing units for parallel computing also fall within the scope of protection of the present invention.
[0020] In one embodiment, the switch is a PCIe switch, a CXL switch, an Ethernet switch, or an NVLink switch. Other types of switches used for multi-GPU interconnection communication also fall within the protection scope of the present invention.
[0021] In one embodiment, the physical topology includes multiple stages of switches, through which all processing units are interconnected.
[0022] As an example, see Figure 2 , which shows a schematic diagram of a physical topology. The figure includes M processing units and a switch group, and the value of M is greater than or equal to 2. The switch group may include one switch or multiple switches. When a switch can support M processing units, the switch group includes one switch; when the number of ports of a switch is insufficient to support M processing units, the switch group includes multiple switches, and the multiple switches can form a multi-stage switch structure. In one embodiment, when multiple switches are included, the ports with the same number of each GPU are connected to the same switch. Other ways of connecting a GPU to a switch also fall within the scope of protection of the present invention.
[0023] It should be noted that in the prior art, the mapping relationship between the port of each processing unit and the processing unit connected to the port is fixed, and the inside of the processing unit is directly routed to the corresponding port and accesses the target processing unit according to the unit index of the target processing unit in the physical address. This strategy enables each port of each processing unit to communicate only with the processing unit connected to it, limiting the interconnection scale. In order to expand the interconnection scale, the present invention connects all processing units through a switch, and then cooperates with the memory management strategy so that each port of each processing unit connected to the switch can be interconnected with all switches to achieve the purpose of expanding the scale. The specific implementation steps of the memory management strategy are described below.
[0024] S200, configuring the memory management strategy of the main control program so that each port of each processing unit accesses the memory address space of all processing units in the physical topology according to the non-local target physical address routing; the memory management strategy includes interleaving operation and deinterleaving operation.
[0025] The main control program is used to manage the memory of all processing units. For example, it is responsible for allocating, recycling, and mapping memory space. Mapping memory space is to convert the virtual address of the memory into a physical address, and the page table is a key data structure for implementing the conversion from virtual address to physical address. Each process has one or more page tables to record the mapping relationship between its virtual memory pages and physical memory pages.
[0026] The page table includes a base address of a physical address, and the base address includes a unit index bit and a page index bit.
[0027] Each value of the unit index bit corresponds to a specific target processing unit, and there is a one-to-one mapping relationship between the unit index bit and the target processing unit. The number of binary digits of the unit index bit is determined according to the number of processing units, and the target processing unit is determined by the value of the unit index bit.
[0028] As an example, the [39:36]th bit of the physical address is the unit index bit. When the unit index bit is 0000, it indicates that the target processing unit is GPU0, and the on-chip bus of the current processing unit will be routed to the port connected to GPU0 according to the unit index bit; when the index bit is 0010, it indicates that the target processing unit is GPU2, and the on-chip bus of the current processing unit will be routed to the port connected to GPU2 according to the unit index bit; and so on.
[0029] Each value of the page index bit corresponds to a certain physical page, that is, the value of the page index bit is the number of the physical page, and different values correspond to physical pages with different numbers.
[0030] The non-local target physical address refers to the memory address accessed by the current access request being the physical address of another processing unit. Before the interleaving operation is performed, the target physical address and the local physical address have the same composition. In order to conveniently distinguish between the local physical address and the non-local physical address, the non-local physical address is referred to as the target physical address, which will not be stated below. The steps for determining the non-local target physical address include:
[0031] S210, acquiring the physical address of the target processing unit and extracting the original base address of the physical address.
[0032] In order to conveniently distinguish the impact of different operations on the base address, the base address before the interleaving operation is called the original base address, the base address after the interleaving operation is called the interleaving base address, and the base address after the deinterleaving operation is called the actual base address, which will not be further stated below.
[0033] S220, obtaining the unit index bit of the original base address.
[0034] In one embodiment, when the storage space size of each target processing unit is a power of 2, the position of the unit index bit in the physical address is fixed, so the unit index bit can be obtained by obtaining the address bit range determined in the physical address. Otherwise, the relationship between the physical address addr, the storage space size size and the unit index bit Index satisfies: Indicates a round-down operation, and the unit of size is bytes. Divide the physical address by the size of each storage space to get the index value of the unit where the physical address is located, and if the result is not an integer, round it down to get the nearest integer index.
[0035] S230, when the unit index bit of the original base address is the same as the unit index bit of the physical address of the current processing unit, the target physical address is a local physical address; otherwise, the target physical address is a non-local target physical address.
[0036] As an example, if the unit index bit of the physical address of the current processing unit is 0011, and if the unit index bit of the physical address of the target processing unit is also 0011, then it is determined that the unit index bit of the original base address is the same as the unit index bit of the physical address of the current processing unit. If the unit index bit of the physical address of the target processing unit is 0010, then it is determined that the unit index bit of the original base address is different from the unit index bit of the physical address of the current processing unit.
[0037] For a local physical address, the memory management strategy is: resetting the unit index bit in the physical address to a local preset value to obtain a new physical address, and the local preset value is used to index the local memory.
[0038] The local preset value is 0. Other values of the local preset value also fall within the protection scope of the present invention.
[0039] Further, the interleaving operation includes: obtaining the original base address of the target physical address, moving the target bit of the page index in the original base address to before the unit index bit to obtain the interleaving base address, the interleaving base address is used to establish a page table, so that the main control program indexes the port of the current processing unit through the target bit; wherein the target bit is K address bits of the page index, the in Indicates rounding up.
[0040] The physical address includes a base address and an offset address within a page, and the base address includes a unit index bit, a page index bit, and other address bit information.
[0041] In one embodiment, the target bit of the page index is the low-order address of the page index. For example, when the number of ports in each processing unit is N=8, then That is, the 3 address bits with the lowest page index are taken as the target bits, and 8 ports are indexed by 3 address bits. For another example, when the number of ports is N = 3, the 3 address bits with the lowest page index are taken as the target bits. The address bits are used as the target bits, and the three ports are indexed by 2 address bits.
[0042] The page table includes multiple entries, each of which includes an interleaving base address. When an access request is used to access the memory, the interleaving base address of the physical address can be obtained by searching the entry in the page table, and then access is performed through the interleaving base address.
[0043] Among them, the target bit in the interleaved base address occupies the position of the unit index bit. Therefore, after looking up the page table, the on-chip bus will use the target bit of the page index as the unit index to route the access request to the port pointed to by the page index, thereby enabling each port of each GPU in the physical topology to communicate with other GPUs and achieve the purpose of logical interconnection.
[0044] As an example, if the current processing unit includes 8 ports and needs to write 64KB of data to the target processing unit, the current processing unit will initiate an access request sequence to the target processing unit, which includes accessing the unit index and its physical page of different target processing units. Each unit index maps a specified port. If no interleaving operation is performed, the access requests with the same unit index will be routed to the target processor through the same port, and the access requests with different unit indexes will be routed to different target processors. If an interleaving operation is performed, the access requests corresponding to the same physical address of the index page will be routed to the same port, and different index pages will be routed to different ports, so that each port can route the access requests of all other processors, that is, each port can establish a logical connection relationship with all other processors. Therefore, the current processing unit will divide the 64KB address range into 8 parts, and the 8 access requests will be distributed to their corresponding ports.
[0045] The effect of the interleaving operation is equivalent to assigning physical pages with different numbers to different ports, regardless of the actual unit index of the target processing unit, because the page index of the physical page occupies the position of the unit index, so it will be routed to the corresponding port according to the page index. In this way, addresses of different physical pages pointing to the same unit index can be routed from different ports, that is, physical addresses pointing to different unit indexes but the same physical page index can be routed from the same port, so as to achieve the purpose of logically interconnecting each port with other processing units.
[0046] Furthermore, the deinterleaving operation includes: restoring the target bit in the interleaving base address to its original position through a switch to obtain an actual base address, so that the main control program accesses the target processing unit and its memory address space through the actual base address routing.
[0047] Among them, after the interleaving operation, the access request can be routed out from the corresponding port of the current processing unit. The access request routed to the switch needs to be correctly routed to the target processing unit. Before routing to the target processing unit, a deinterleaving operation is required to restore the correct physical address and index the target processing unit according to the original unit index.
[0048] Take GPU3 as an example, see Figure 3 , Figure 3The figure shows the relationship between the address spaces of GPU3 and the main control program from different perspectives. GPU0 Mem is the address space of GPU numbered 0 from the perspective of the main control program, GPU1 Mem is the address space of GPU numbered 1 from the perspective of the main control program, and so on; the memory space of each GPU is continuous from the perspective of the main control program. From the perspective of GPU3's memory, the local memory of GPU3, that is, Local Mem in the figure, may be continuous; but after the interleaving operation, the memory of GPU3 is mapped to different ports according to the index of the physical page.
[0049] As an example, if interleaving and deinterleaving operations are not performed, when the physical address of the access request sent by GPU3 includes the unit index of GPU2, the master control program routes the access request to the fifth port port5 of the current processing unit, and routes it to GPU2 through port5; similarly, when the physical address of the access request sent by GPU3 includes the unit index of GPU1, the master control program routes the access request to the fourth port port4 of the current processing unit, and routes it to GPU1 through port4. That is, each port is only used to access a fixed GPU, and 7 ports are connected to 7 GPUs. For example, please refer again Figure 3 , if the method provided by the present invention is used to perform interleaving and deinterleaving operations on the base address, after the interleaving operation, when the physical address of the access request issued by GPU3 includes the unit index of GPU2, if the page index it accesses is 7, the main control program routes the access request to the 7th port port7 of the current processing unit, and after performing the deinterleaving operation, the access request is routed to GPU2; similarly, when the physical address of the access request issued by GPU3 includes the unit index of GPU1, if the page index it accesses is 7, the main control program routes the access request to the 7th port port7 of the current processing unit, and after performing the deinterleaving operation, the access request is routed to GPU1. That is, the purpose of routing different access requests of two target processing units through the same port7 is achieved, that is, the purpose of logically connecting different processing units with one port is achieved.
[0050] In one implementation, when the total number of processing units in the physical topology is greater than N, the interleaving operation further includes: obtaining the next address bit adjacent to the unit index bit as the highest address bit, and moving the highest address bit to before the highest bit of the base address.
[0051] In one implementation, the deinterleaving operation further includes a post-processing operation: moving the highest bit in the actual base address obtained by deinterleaving to between the unit index bit and the next adjacent address bit in the actual base address and configuring it as a preset value.
[0052] As an example, see Figure 3 , Figure 3 GPU3 has N=7 ports. When the switch is connected to 16 GPUs, the total number of processing units in the physical topology is greater than N. The main control program manages the memory space Mem of the 16 GPUs. From the memory perspective of the main control program, the memory space of each GPU is continuous. If the main control program supports 512GB of memory space, the memory space allocated to each GPU is 512 / 16=32GB, and the size of each memory page is 4KB. If the target physical address of the current access request in GPU3 is: 0x22_3456_1000, the page offset address of the target physical address is the [11:0]th bit, and the original base address of the target physical address is: 0x22_3456_1. The unit index bit of the original base address is the [39:36]th bit: 0010, and the target bit of the page index of the original base address is the [14:12]th bit: 001. Then the target bit is moved to occupy the [38:36]th bit before the unit index bit, and the
[35] th bit in the original base address is moved to the front of the highest bit, and the interleaved base address 0x12_468A_C is obtained. According to the interleaved base address, the current processing unit is routed out through the port numbered 1. Then, the switch deinterleaves the interleaved base address, so the [38:36]th bit in the interleaved base address is restored to the original [14:12]th bit, the highest bit of the interleaved base address is removed, and the value of the next address adjacent to the unit index bit in the original base address is configured as a preset value, for example, the preset value is 0, then 0x22_3456_1 is obtained, and according to the [39:36]th bit being "0x2", the corresponding access request is sent to GPU2.
[0053] As another example, the physical address is: 0xCA_3456_0000, which is located on GPU12; the interleaving base address is 0xC_468A_C; the base address after the switch deinterleaving is: 0xC2_3456_0 (binary representation: 1100 001000110100 0101 0110 0000). The highest bit of the switch address needs to be set to 1 to restore the original base address, that is, the highest bit of the binary representation "0010" corresponding to 0x2 is configured as the preset value 1, so the final base address obtained after the post-processing operation is: 0xCA_3456_0 (binary representation: 1100 1010 0011 0100 0101 0110 0000). According to the [39:36] bit being "0xC", the corresponding access request is sent to GPU12.
[0054] In summary, an embodiment of the present invention provides a method for expanding an interconnection topology. The method obtains the physical topology of a switch connecting all processing units, and combines the interleaving operation and deinterleaving operation in a memory management unit, so that the same port can simultaneously route access requests to different processors, thereby achieving the purpose of the same port of a processing unit communicating with all other processing units, that is, achieving the purpose of expanding the scale of interconnection of processing units.
[0055] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to implementing a method in a method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the method provided in the above embodiment.
[0056] An embodiment of the present invention further provides an electronic device, comprising a processor and the aforementioned non-transitory computer-readable storage medium.
[0057] An embodiment of the present invention further provides a computer program product, which includes program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the method according to various exemplary embodiments of the present invention described above in this specification.
[0058] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0059] Although some specific embodiments of the present invention have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It should also be understood by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.
Claims
1. A method for expanding an interconnection topology, characterized in that: The method comprises the following steps: Acquire a physical topology, wherein the physical topology includes at least one switch and a plurality of processing units, and all processing units in the physical topology are interconnected with the switch; wherein each processing unit has N ports; Configuring a memory management strategy of the main control program so that each port of each processing unit accesses the memory address space of all processing units in the physical topology according to a non-local target physical address routing; the memory management strategy includes an interleaving operation and a de-interleaving operation; The interleaving operation includes: obtaining the original base address of the target physical address, moving the target bit of the page index in the original base address to before the unit index bit to obtain the interleaving base address, and the interleaving base address is used to establish a page table so that the main control program indexes the port of the current processing unit through the target bit; The deinterleaving operation includes: restoring the target bit in the interleaving base address to the original position through a switch to obtain an actual base address, so that the main control program accesses the target processing unit and its memory address space through the actual base address routing.
2. The method according to claim 1, characterized in that The target bits are K address bits of the page index. in Indicates rounding up.
3. The method according to claim 1, characterized in that The target bit of the page index is the low-order address of the page index.
4. The method according to claim 1, characterized in that When the total number of processing units in the physical topology is greater than N, the interleaving operation further includes: obtaining the next address bit adjacent to the unit index bit as the highest address bit, and moving the highest address bit to before the highest bit of the base address.
5. The method according to claim 3, characterized in that: The de-interleaving operation also includes a post-processing operation: moving the highest bit in the actual base address obtained by de-interleaving to between the unit index bit and the next adjacent address bit in the actual base address and configuring it as a preset value.
6. The method according to claim 1, characterized in that The step of determining the non-local target physical address includes: Obtaining the physical address of the target processing unit and extracting the original base address of the physical address; Obtaining the unit index bit of the original base address; When the unit index bit of the original base address is the same as the unit index bit of the physical address of the current processing unit, the target physical address is a local physical address; otherwise, the target physical address is a non-local target physical address.
7. The method according to claim 6, characterized in that For a local physical address, the memory management strategy is: resetting the unit index bit in the physical address to a local preset value to obtain a new physical address, and the local preset value is used to index the local memory.
8. The method according to claim 1, characterized in that The switch is a PCIe switch, a CXL switch, an Ethernet switch or an NVLink switch.
9. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the storage medium, characterized in that: The at least one instruction or the at least one program is loaded and executed by the processor to implement the method according to any one of claims 1 to 8.
10. An electronic device, characterized in that: The invention comprises a processor and the non-transitory computer-readable storage medium as claimed in claim 9.