Method for scalable block crossbar switch

Abstract

The invention relates to the technical field of crossbar switches, and discloses a method for an extensible block crossbar switch, which performs grouping and mapping based on the number and layout of SM cores of a GPGPU (General Purpose Graphics Processing Unit), and constructs a plurality of sub-internetworks to ensure smooth transmission of data. Through optimization and test implementation, parameters are adjusted to improve performance. According to the method, a position-aware storage scheme is adopted, the data caching and transmission consumption is reduced, the long-distance communication requirement is reduced, the performance of the GPGPU is greatly optimized, and the operation efficiency and the response speed of the GPGPU are improved. Through the method, the interconnection challenge between the SM core and the L2 cache in the GPGPU can be more effectively handled, and the sustainable development of related technologies is promoted.

Description

Technical Field

[0001] The present application relates to the technical field of crossbar switches, for example, to a method for an expandable block crossbar switch. Background Art

[0002] GPGPU performs well in processing high-data-volume, computationally intensive tasks with its powerful computing power, ultra-high parallel data processing efficiency, and huge data throughput. Its core architecture includes multiple stream multiprocessors (SMs) arranged in an array, which interact with L2 cache through an efficient on-chip interconnect network to ensure smooth execution of computing tasks.

[0003] Considering the high independence between computing units and little data interaction of GPGPU, the design of its interconnection network mainly focuses on optimizing the connection efficiency between computing units and shared L2 cache. In this context, crossbar switches are widely used as a non-blocking topology to improve data transmission efficiency.

[0004] However, traditional crossbar switch designs are limited by VLSI scalability issues, especially the limitations of pin count and routing area, which becomes a significant challenge in GPGPU, especially when there are a large number of SM cores and a large L2 memory space.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0007] The embodiments of the present disclosure provide a method for a scalable block crossbar switch, thereby improving the performance of a GPGPU.

[0008] In some embodiments, the method comprises: Grouping and mapping implementation: Determine the grouping parameters based on the number and layout of SM cores in the GPGPU, and map the SM cores to the L2 cache groups; Internetwork construction and implementation: Design and build multiple sub-internets, each of which contains an appropriate number of data links and crossbar switch configurations, and deploy off-network computing nodes and off-network storage nodes on the SM side and L2 cache side respectively to ensure smooth data transmission; Optimization and test implementation: Conduct simulation tests on the Internet and optimize and adjust grouping parameters and cross-switch configurations based on the test results.

[0009] Optionally, the grouping and mapping implementation includes grouping management, and the grouping management includes: For a total of K SM cores in GPGPU, each adjacent k SM cores are divided into a SM group, which is divided into m groups in total, where k×m=K; Correspondingly, the entire L2 cache address space is also divided into m groups based on their physical locations to ensure that for the i-th SM group, i∈[0,m], the i-th L2 cache group corresponding to it has the shortest physical distance among all cache groups and the lowest communication cost.

[0010] Optionally, data and instruction information involved in the i-th SM group are mainly stored in the i-th L2 cache group, so that data interaction between the SM group and the L2 cache is mainly completed on the shortest data link.

[0011] Optionally, an indirect data link is formed based on a series connection of crossbar switches.

[0012] Optionally, direct data links between non-corresponding groups are removed, and data is exchanged between non-corresponding groups through indirect data links.

[0013] Optionally, the Internet network construction implementation includes: Divide the entire internetwork into m sub-internetworks; Each sub-network adopts a crossbar switch network topology and contains n data links, each of which corresponds to a portion of the address space of the L2 cache group, where n <m。

[0014] Optionally, the SM group has a total of n nodes connected to the Internet; the n SM group-side nodes are further grouped, with two nodes in each group; the group node can only initiate memory access requests to the corresponding two cache group nodes; the mapping relationship between the m SM cores and the n nodes is determined by the intra-group network and the arbitrator.

[0015] Optionally, on the SM side, the sub-interconnect network also includes an off-network computing node for initiating a memory access request from other SM groups, and the sub-interconnect network controller is configured so that the off-network computing node and the other n port nodes are equivalent to each other and have the same priority; on the L2 cache side, it includes n port nodes of the current cache group and an off-network storage node, and the off-network storage node is used to process memory access requests initiated to other cache groups; for all memory access tasks where the target cache group is located outside the sub-interconnect network, the requested data leaves the sub-interconnect network through the off-network storage node, and this port node is equivalent to other storage-side port nodes.

[0016] Optionally, each interconnection network undertakes (n+1)×(n+1) data communication tasks.

[0017] Optionally, the cross switch includes a 2×3 cross switch and a 1×(n+1) cross switch; the 2×3 cross switch is used to undertake the interconnection task between each group of nodes and the corresponding cache group nodes; the 1×(n+1) cross switch is used for the interconnection between the off-network computing node and any cache group node or the next sub-interconnection network.

[0018] The method for scalable block crossbar switch provided by the embodiment of the present disclosure can achieve the following technical effects: The present invention determines grouping parameters based on the number and layout of SM cores in GPGPU and maps SM cores to L2 cache groups, innovatively proposes a location-aware storage solution, and designs an expandable block crossbar switch based on this.

[0019] By performing operations close to the storage space, the resource consumption in the data cache and transmission process is reduced, further optimizing the performance of GPGPU. As the number of SM cores increases, the fully connected interconnection network mode will bring huge resource consumption and network complexity. However, this solution effectively alleviates the pressure on the L2 cache and the interconnection network between SM cores through optimized storage design and scalable block crossbar switches.

[0020] It is worth mentioning that even in a fully connected crossbar switch network, the communication between SM cores and L2 cache addresses that are physically far apart needs to go through long-distance, high-loss, and high-latency data links. This solution reduces the need for such long-distance communication through a location-aware storage strategy, further improving the computing efficiency and response speed of GPGPU. This innovative design not only improves the performance of GPGPU, but also demonstrates its practicality and industry advantages in processing large amounts of data and computationally intensive tasks.

[0021] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein: Figure 1 It is a schematic diagram of an SM core and L2 cache near memory provided by an embodiment of the present disclosure; Figure 2 It is a schematic diagram of an Internet network provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0024] The terms "first", "second", etc. in the embodiments of the present disclosure are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so as to describe the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0025] Unless otherwise stated, the term "plurality" means two or more.

[0026] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0027] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0028] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0029] Breaking the memory wall has always been one of the main goals of modern computer architecture. Near-memory computing refers to performing operations in logic close to the storage space to reduce resource consumption in data caching and transmission, and save on-chip communication resources.

[0030] GPGPU is mainly aimed at computing-intensive tasks. The interconnection network between L2 cache and SM cores is under great pressure. As the number of SM cores increases, the fully connected interconnection network mode will generate huge resource consumption and extremely high network complexity.

[0031] There will inevitably be a large physical distance between the SM core and the L2 cache. Even in a fully connected crossbar switch network, communication between the SM core and the L2 cache address that are physically far apart on the chip requires a long-distance, high-loss, and high-latency data link.

[0032] Based on this background, the present invention aims to provide an efficient and scalable interconnection network design, which reduces the complexity and cost of data communication and improves the performance of GPGPU by combining a location-aware storage solution and a block crossbar switch. Specifically, it includes the following contents: A grouping management strategy based on physical location is used to reasonably group SM cores and L2 caches in GPGPU to reduce communication delay and cost.

[0033] A cache-based storage strategy ensures that data and instruction information are stored in the corresponding L2 cache group as much as possible, reducing the use of long-distance data links.

[0034] A scalable block crossbar switch interconnection network design improves the scalability and flexibility of the interconnection network by building multiple small-scale crossbar switch components.

[0035] The implementation method is as follows: A method for a scalable block crossbar switch includes: Grouping and mapping implementation: According to the number and layout of SM cores in GPGPU, determine the reasonable grouping parameters and map the SM cores to the L2 cache groups.

[0036] Internetwork construction and implementation: Design and build multiple sub-internets, each of which contains an appropriate number of data links and crossbar switch configurations. Deploy off-network computing nodes and off-network storage nodes on the SM side and L2 cache side respectively to ensure smooth data transmission.

[0037] Optimization and test implementation: Conduct simulation tests on the Internet to verify its scalability, communication efficiency, resource consumption and other indicators. According to the test results, optimize and adjust the grouping parameters, crossbar switch configuration, etc. to achieve the best performance.

[0038] The present invention effectively solves the scalability problem faced by traditional crossbar switches in GPGPU by combining a location-aware storage solution and an extensible block crossbar switch design. This design not only reduces resource consumption and complexity, but also improves the communication efficiency and extensibility of the Internet. This solution provides strong support for the development of high-performance computing devices such as GPGPU, and has broad application prospects and market value.

[0039] Specifically, the grouping and mapping implementation includes grouping management, such as Figure 1As shown, for a total of K SM cores in the GPGPU, every adjacent k SM cores are divided into one SM group, and a total of m groups are formed, where k × m = K. Correspondingly, the entire L2 cache address space is also divided into m groups based on its physical location, so as to ensure that for the i-th SM group, i ∈ [0, m]. The i-th L2 cache group corresponding to it has the closest physical distance to it among all cache groups. This strategy can significantly reduce the communication cost and latency, thereby improving the efficiency of data interaction.

[0040] In addition, in the present invention, for the i-th SM group, all the data and instruction materials involved should be stored in the i-th L2 cache group as much as possible. This can minimize the use of long-distance and high-cost data links, improve the locality of data access, and enable the data interaction between the SM group and the L2 cache to be mainly completed on the shortest data link.

[0041] In the traditional crossbar design, there may be direct data links between any two nodes. However, in the present invention, the direct data links between non-corresponding groups are discarded and replaced with indirect data links formed by cascading crossbars. This design can further reduce the complexity of the interconnection network and save hardware resources.

[0042] In the interconnection network design, the entire interconnection network is divided into m sub-interconnection networks. Each sub-network adopts a crossbar network topology and contains n data links (where n < m). These data links correspond to a partial address space of the L2 cache group to ensure that each SM group can efficiently access its corresponding L2 cache group.

[0043] For the SM group side, to further reduce the crossbar complexity, the SM group has a total of n nodes accessing the interconnection network. The n SM group side nodes are further grouped, with two nodes in each group. Each group of nodes can only initiate memory access requests to the corresponding two cache group nodes. The mapping relationship between the m SM cores and the n nodes is determined by the intra-group network and the arbiter.

[0044] As Figure 2 shown, to further improve the flexibility and scalability of the interconnection network, off-network computing nodes and off-network storage nodes are introduced. These nodes are used to process memory access requests from other sub-interconnection networks and achieve the cascading between sub-interconnection networks. Through this design, the scale of the interconnection network can be easily expanded to adapt to different numbers and configurations of GPGPUs.

[0045] As an example, on the SM side, the sub-interconnect network also includes an off-network computing node for initiating a memory access request from other SM groups, and the sub-interconnect network controller is configured so that the off-network computing node and the other n port nodes are equivalent to each other and have the same priority; on the L2 cache side, it includes n port nodes of the current cache group and an off-network storage node, and the off-network storage node is used to process memory access requests initiated to other cache groups; for all memory access tasks where the target cache group is located outside the sub-interconnect network, the requested data leaves the sub-interconnect network through the off-network storage node, and this port node is equivalent to other storage-side port nodes.

[0046] The addition of off-network computing nodes and off-network storage nodes enables each sub-interconnection network to be connected in series. A data link is established for non-corresponding SMs and cache groups. However, since the data link contains multiple strings of cross switches, the data link is an indirect blocked data link.

[0047] In terms of the configuration of the crossbar switch, the present invention uses two different types of crossbar switches: 2×3 crossbar switches and 1×(n+1) crossbar switches. The former is used to undertake the interconnection task between each group of nodes and the corresponding cache group nodes; the latter is used to interconnect the off-network computing node with any cache group node or the next sub-internet. This configuration method can ensure the efficient operation of the internetwork and provide good scalability.

[0048] It can be understood that for each interconnection network, it needs to undertake (n+1)×(n+1) data communication tasks. Based on the above node correspondence: each group of nodes only processes memory access requests initiated to the corresponding two cache group nodes. At the same time, the target node may not be located in the current cache group, so it is also necessary to establish a data path with the off-network storage node. This design uses a 2×3 cross-point-based cross switch to undertake the interconnection task. For the off-network computing node, it initiates memory access requests from other sub-interconnection networks, and may interact with any cache group node or continue to pass the memory access request to the next sub-interconnection network. Therefore, it requires a 1×(n+1) cross switch to undertake the interconnection task. Therefore, the network body of each sub-interconnection network is composed of 2×3 and 1×(n+1) crossbar switches, totaling ( ) switching components.

[0049] The network structure of multiple switching components solves the inherent scalability problem of crossbar switches and greatly improves their deployment flexibility. In addition, the location-aware storage solution and cache corresponding design strategy reduce the complexity of the Internet network, save hardware resources, and have a minimal impact on the Internet network communication efficiency.

[0050] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the scope of protection. As used in the description in the text, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method or device including the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0051] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. The technicians may clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0052] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically separately, or two or more units may be integrated in one unit.

[0053] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for a scalable block crossbar switch, characterized in that: include: Grouping and mapping implementation: Determine the grouping parameters based on the number and layout of SM cores in the GPGPU, and map the SM cores to the L2 cache groups; Internetwork construction and implementation: Design and build multiple sub-internets, each of which contains an appropriate number of data links and crossbar switch configurations, and deploy off-network computing nodes and off-network storage nodes on the SM side and L2 cache side respectively to ensure smooth data transmission; Optimization and test implementation: Conduct simulation tests on the Internet and optimize and adjust grouping parameters and cross-switch configurations based on the test results.

2. The method according to claim 1, characterized in that The grouping and mapping implementation includes grouping management, and the grouping management includes: For a total of K SM cores in GPGPU, each adjacent k SM cores are divided into a SM group, which is divided into m groups in total, where k×m=K; Correspondingly, the entire L2 cache address space is also divided into m groups based on their physical locations to ensure that for the i-th SM group, i∈[0,m], the i-th L2 cache group corresponding to it has the shortest physical distance among all cache groups and the lowest communication cost.

3. The method according to claim 2, characterized in that The data and instruction information involved in the i-th SM group are mainly stored in the i-th L2 cache group, so that the data interaction between the SM group and the L2 cache is mainly completed on the shortest data link.

4. The method according to claim 3, characterized in that An indirect data link is formed based on a series connection of crossbar switches.

5. The method according to claim 4, characterized in that The direct data links between non-corresponding groups are removed, and data is exchanged between non-corresponding groups through indirect data links.

6. The method according to any one of claims 1 to 5, characterized in that: The Internet network construction implementation includes: Divide the entire internetwork into m sub-internetworks; Each sub-network adopts a crossbar switch network topology and contains n data links, each of which corresponds to a portion of the address space of the L2 cache group, where n <m。 7. The method according to claim 6, characterized in that The SM group has a total of n nodes connected to the Internet; The n SM group side nodes are further grouped into two nodes per group; A group node can only initiate memory access requests to the corresponding two cache group nodes; The mapping relationship between m SM cores and n nodes is determined by the intra-group network and arbitrator.

8. The method according to claim 7, characterized in that On the SM side, the sub-interconnect network also includes an off-network computing node to initiate a memory access request from other SM groups. The sub-interconnect network controller is configured so that the off-network computing node and other n port nodes are equivalent to each other and have the same priority. On the L2 cache side, it includes n port nodes of the current cache group and an off-network storage node, which is used to process memory access requests to other cache groups; For all memory access tasks where the target cache group is located outside the sub-interconnect network, the requested data leaves the sub-interconnect network through the off-network storage node, which is equivalent to other storage-side port nodes.

9. The method according to claim 7, characterized in that: Each interconnection network undertakes (n+1)×(n+1) data communication tasks.

10. The method according to claim 7, characterized in that The crossbar switch includes a 2×3 crossbar switch and a 1×(n+1) crossbar switch; The 2×3 crossbar switch is used to undertake the interconnection task between each group of nodes and the corresponding cache group nodes; The 1×(n+1) crossbar switch is used to interconnect the off-network computing node with any cache group node or the next sub-interconnection network.

Images