Processor / endpoint communication coupling configuration system
By introducing a communication coupling component configuration engine in the information processing system, dynamically identifying and configuring communication resources between the processor and the endpoint, the fixity problem of the communication coupling component between the processor and the endpoint in the prior art is solved, and the server device is improved Scalability and performance.
Patent Information
- Application Number
- CN202080030030.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-01-13
- Filing Date
- 2020-04-24
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2040-04-24
AI Technical Summary
Fixed/static communication couplings between processors and endpoints in existing information processing systems result in poor scalability of server devices, reduced throughput and increased latency, and different hardware devices are required to implement different processor/endpoint configurations.
Provides an information processing system, including a communication coupling component configuration engine, capable of identifying and configuring multiple communication coupling components, determining their configuration capabilities, and dynamically configures the processor based on the hardware subsystem, communication configuration capabilities and the capabilities of a multi-endpoint adapter device. Communication resources with endpoints, enabling configurable communication couplings.
Enhanced scalability of server devices, allowing processor addition without additional hardware, optimizes the throughput and latency of processor interconnects, and supports multiple processing subsystem/endpoint configurations.
Smart Images

Figure CN113748416B_ABST
Abstract
Description
Background Art
[0001] The present disclosure relates generally to information handling systems and, more particularly, to configuring a communications coupling between a processor and an endpoint in an information handling system.
[0002] As the value and use of information continue to grow, individuals and businesses seek additional ways to process and store information. One option available to users is an information processing system. An information processing system typically processes, compiles, stores and / or transmits information or data for commercial, personal or other purposes, allowing users to take advantage of the value of this information. Because technology and information processing requirements and needs are different between different users or applications, information processing systems may also differ in the following aspects: what information to process, how to process information, how much information to process, store or transmit, and how quickly and efficiently information can be processed, stored or transmitted. Changes in information processing systems allow information processing systems to be general or configured for specific users or specific purposes (such as financial transaction processing, airline reservations, enterprise data storage or global communications). In addition, an information processing system may include a variety of hardware and software components that can be configured to process, store and transmit information, and may include one or more computer systems, data storage systems and networking systems.
[0003] An information handling system (such as, for example, a server device) may be configured to include multiple processors and utilize a non-uniform memory access (NUMA) computer memory design in which each processor is provided with its own rapidly accessible local memory, and in which the processors are interconnected via a processor interconnect (e.g., as implemented in the NUMA computer memory design provided by the Santa Clara, California, USA). The processor interconnects are coupled together using a hyperpath interconnect (UPI) available in processing systems provided by the company, which allows processors to access memory local to other processors. In addition, such server devices may be provided with a multi-endpoint adapter device that provides multiple endpoints (e.g., PCIe endpoints). Conventionally, processors are coupled to the multi-endpoint adapter device via hardware using fixed / static communication couplings to allow those processors to communicate with the endpoints to utilize resources available via those endpoints. The use of fixed / static communication couplings between processors and endpoints prevents scaling of server devices (e.g., to provide additional processors in the server device) and does not take into account endpoint devices connected to the processors. In addition, fixed / static communication couplings may result in the use of processor interconnects in endpoint communications, which may reduce throughput via processor interconnects (processor interconnects are intended to allow processors to access adjacent processor root complexes) and increase latency in those communications. Further, fixed / static communication couplings in conventional systems require different hardware devices (e.g., motherboards and riser devices) for server devices in order to implement different processor / endpoint configurations.
[0004] Therefore, it would be desirable to provide a processor / endpoint communications coupling configuration system. Summary of the invention
[0005] According to one embodiment, an information handling system (IHS) includes: a first processing system; and a first memory system, the first memory system coupled to the first processing system and including instructions that, when executed by the first processing system, cause the first processing system to provide a communication coupling configuration engine, the communication coupling configuration engine being configured to: identify at least one hardware subsystem including a plurality of communication couplings coupling a plurality of second processing subsystems to a first multi-point adapter device; determine at least one communication coupling configuration capability of the plurality of communication couplings coupling the plurality of second processing subsystems to the first multi-point adapter device; determine at least one first multi-point adapter device capability of the first multi-point adapter device; and configure the plurality of communication couplings based on the at least one hardware subsystem, the at least one communication configuration capability, and the at least one first multi-point adapter device capability, wherein the configuration of the plurality of communication couplings provides at least one communication resource between at least one of the plurality of second processing subsystems and at least one endpoint on the first multi-point adapter device. BRIEF DESCRIPTION OF THE DRAWINGS
[0006] Figure 1 is a schematic diagram illustrating one embodiment of an information handling system (IHS);
[0007] Figure 2is a schematic diagram showing one embodiment of a server device, which provides an example of a processor / endpoint communication coupling configuration system of the present disclosure;
[0008] Figure 3 is a schematic diagram showing one embodiment of a server device, which provides an example of a processor / endpoint communication coupling configuration system of the present disclosure;
[0009] Figure 4 is a flow chart illustrating one embodiment of a method for configuring a communication coupling between a processor and an endpoint;
[0010] Figure 5A It is shown in Figure 4 Method period Figure 2 A schematic diagram of one embodiment of a processor / endpoint communication coupling configuration system in a server device of;
[0011] Figure 5B It is shown in Figure 4 Method period Figure 2 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0012] Figure 5C It is shown in Figure 4 Method period Figure 2 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0013] Fig. 6A It is shown in Figure 4 Method period Figure 2 A schematic diagram of one embodiment of a processor / endpoint communication coupling configuration system in a server device of;
[0014] Figure 6B It is shown in Figure 4 Method period Figure 2 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0015] Figure 6C It is shown in Figure 4 Method period Figure 2 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0016] Fig. 7A It is shown in Figure 4 Method period Figure 3A schematic diagram of one embodiment of a processor / endpoint communication coupling configuration system in a server device of;
[0017] Figure 7B It is shown in Figure 4 Method period Figure 3 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0018] Figure 7C It is shown in Figure 4 Method period Figure 3 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0019] Fig.7D It is shown in Figure 4 Method period Figure 3 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0020] Fig. 7E It is shown in Figure 4 Method period Figure 3 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0021] Figure 7F It is shown in Figure 4 Method period Figure 3 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0022] Figure 7G It is shown in Figure 4 Method period Figure 3 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0023] Fig. 8A It is shown in Figure 4 Method period Figure 3 A schematic diagram of one embodiment of a processor / endpoint communication coupling configuration system in a server device of;
[0024] Figure 8B It is shown in Figure 4 Method period Figure 3 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0025] Figure 8C It is shown in Figure 4 Method period Figure 3 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0026] Fig.8D It is shown in Figure 4 Method period Figure 3 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0027] Fig. 8E It is shown in Figure 4 Method period Figure 3 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0028] Fig.9A It is shown in Figure 4 Method period Figure 3 A schematic diagram of one embodiment of a processor / endpoint communication coupling configuration system in a server device of;
[0029] Fig. 9B It is shown in Figure 4 Method period Figure 3 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0030] Fig. 9C It is shown in Figure 4 Method period Figure 3 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0031] Fig.9D It is shown in Figure 4 Method period Figure 3 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device;
[0032] Fig.9E It is shown in Figure 4 Method period Figure 3 A schematic diagram of configuring a communication coupling according to an embodiment of a processor / endpoint communication coupling configuration system in a server device. DETAILED DESCRIPTION
[0033] For purposes of this disclosure, an information handling system may include any tool or collection of tools operable to compute, calculate, determine, classify, process, transmit, receive, retrieve, initiate, switch, store, display, transmit, indicate, detect, record, reproduce, process, or utilize any form of information, intelligence, or data for commercial, scientific, control, or other purposes. For example, an information handling system may be a personal computer (e.g., a desktop computer or laptop computer), a tablet computer, a mobile device (e.g., a personal digital assistant (PDA) or a smart phone), a server (e.g., a blade server or a rack server), a network storage device, or any other suitable device, and may vary in size, shape, performance, functionality, and price. An information handling system may include random access memory (RAM), one or more processing resources (such as a central processing unit (CPU) or hardware or software control logic), ROM, and / or other types of non-volatile memory. Additional components of an information handling system may include one or more disk drives, one or more network ports for communicating with external devices, and various input and output (I / O) devices (such as a keyboard, mouse, touch screen, and / or video display). The information handling system may also include one or more buses operable to transmit communications between the various hardware components.
[0034] In one embodiment, IHS 100 ( Figure 1 ) includes a processor 102 connected to a bus 104. The bus 104 serves as a connection between the processor 102 and other components of the IHS 100. An input device 106 is coupled to the processor 102 to provide input to the processor 102. Examples of input devices may include a keyboard, a touch screen, a pointing device (such as a mouse, a trackball, and a touch pad), and / or various other input devices known in the art. Programs and data are stored on a mass storage device 108, which is coupled to the processor 102. Examples of mass storage devices may include a hard disk, an optical disk, a magneto-optical disk, a solid-state storage device, and / or various other mass storage devices known in the art. The IHS 100 also includes a display 110, which is coupled to the processor 102 through a video controller 112. A system memory 114 is coupled to the processor 102 to provide fast storage to the processor to facilitate the execution of computer programs by the processor 102. Examples of system memory may include random access memory (RAM) devices, such as dynamic RAM (DRAM), synchronous DRAM (SDRAM), solid-state memory devices, and / or various other memory devices known in the art. In one embodiment, chassis 116 houses some or all of the components of IHS 100. It should be understood that other buses and intermediate circuits may be deployed between the above-mentioned components and processor 102 to facilitate interconnection between the components and processor 102.
[0035] Reference now Figure 2 , shows an embodiment of a server device 200, which can provide a processor / endpoint communication coupling configuration system of the present disclosure. In one embodiment, the server device 200 can be configured by the above reference Figure 1 The IHS 100 discussed provides and / or may include some or all of the components of the IHS 100. In addition, although shown and discussed as a server device 200, those skilled in the art who understand the present disclosure will recognize that the functionality of the server device 200 discussed below may be provided by other devices that are configured to operate similarly as discussed below. In the illustrated embodiment, the server device 200 includes a chassis 202 that houses the components of the server device 200, only some of which are shown below. For example, the chassis 202 may house a board 204, such as, for example, a motherboard and / or other circuit boards that will be apparent to those skilled in the art who understand the present disclosure. In at least some of the examples provided below, the board 204 (or at least some portions of the board 204) may be viewed as a hardware subsystem in the server device 200 that provides a communication coupling (e.g., traces, wiring, and / or other communication couplings known in the art) between the processing subsystem discussed below and the endpoints.
[0036] A plurality of processing subsystems are disposed on board 204 and interconnected via processing subsystems (such as, for example, in a computer system provided by Santa Clara, California, USA). The Ultra Path Interconnect (UPI) available on the processing system provided by the company is coupled together. For some examples provided below, Figure 2 The embodiment of the server device 200 shown in provides an example of a dual processor system having a processing subsystem 206a and a processing subsystem 208a included on a board 204 and coupled together via a UPI 210. In addition, each of the processing subsystems 206a and 208a is provided with and connected to a corresponding memory subsystem 206b and 208b. For example, the processing subsystems and memory subsystems in the server device 200 may utilize a non-uniform memory access (NUMA) computer memory design, wherein each corresponding processing subsystem is connected to a corresponding local memory subsystem to provide a corresponding NUMA node (i.e., the processing subsystem 206a and its local memory subsystem 206b provide a first NUMA node, and the processing subsystem 208a and its local memory subsystem 208b provide a second NUMA node). However, although in Figure 2Two processing subsystems / memory subsystems (e.g., two NUMA nodes) are shown in the figure, but as discussed below, more processing subsystems / memory subsystems (e.g., NUMA nodes) may be provided in accordance with the teachings of the present disclosure and are also within the scope of the present disclosure.
[0037] In the illustrated embodiment, each of the processing subsystems 206a and 208a is coupled to a control / bridge subsystem 212, which may be disposed on the board 204. In various examples, the control / bridge subsystem 212 may be provided by an analog bus exchange switch, a PCIe packet switch, and / or various other control / bridge subsystems that will be apparent to those skilled in the art who understand the present disclosure. Although shown and described as a control / bridge subsystem, those skilled in the art who understand the present disclosure will recognize that other subsystems may perform the functionality of the control / bridge subsystem 212 discussed below while also remaining within the scope of the present disclosure. In the illustrated embodiment, the control / bridge subsystem 212 may be provided by hardware on the board 204, which is coupled to a riser device 214 included on the board 204 via, for example, traces and / or other communication couplings known in the art. In at least some of the examples provided below, the control / bridging subsystem 212 (or at least some portions of the control / bridging subsystem 212), the riser card device 214 (or at least some portions of the riser card device 214), and the board 204 coupling them together can be viewed as a hardware subsystem in the server device 200 that provides communication couplings (e.g., traces, wiring, and / or other communication couplings known in the art) between the processing subsystems 206a and 208a discussed below and the endpoints.
[0038] In other embodiments, the control / bridge subsystem 212 may be provided as part of the riser card device 214, included on a device (e.g., an expansion card) coupled to the board 204, and / or provided in various other ways while still falling within the scope of the present disclosure. In some examples, as discussed in more detail below, the communication coupling between the control / bridge subsystem 212 and the riser card device 214 may be configured by the control / bridge subsystem 212 and / or the riser card device 214 (e.g., by targeting a Figure 2 In the illustrated embodiment, the riser device 214 may include two x16 connectors (eg, PCIe connectors) that provide Figure 2, eight x4 connectors are shown in FIG. 2 (e.g., x4 connectors numbered “1,” “2,” “3,” and “4” for a first x16 connector, and x4 connectors “5,” “6,” “7,” and “8” for a second x16 connector). In some embodiments, riser device 214 can be configured to describe one or more connection topologies on board 204 to endpoints that are coupled to riser device 214.
[0039] In the illustrated embodiment, a baseboard management controller (BMC) device 216 is included in the chassis 202 and is coupled to the control / bridging subsystem 212. For example, the BMC device 216 may be manufactured by a manufacturer such as the AT&T Inc. The company's integrated Remote access controller (iDRAC) is provided, but other remote access controller devices and / or BMCs will also fall within the scope of the present disclosure. In the embodiments discussed below, the BMC device 216 includes a BMC processing system coupled to a BMC memory system, and the BMC memory system includes instructions that, when executed by the BMC processing system, cause the BMC processing system to provide a communication coupling configuration engine, and the communication coupling configuration engine is configured to perform the functionality of the communication coupling configuration engine and the BMC device discussed below. However, those skilled in the art who understand the present disclosure will recognize that the communication coupling configuration engine of the present disclosure may be provided by other hardware / software subsystems in the server device 200, and / or may be connected to the server device 200 (for example, in a network-connected management system), while still falling within the scope of the present disclosure. Although a particular server device 200 has been shown, those skilled in the art who understand the present disclosure will recognize that the server device (or other device operating in accordance with the teachings of the present disclosure in a manner similar to that described below for server device 200) may include a variety of components and / or component configurations for providing conventional server device functionality as well as the functionality discussed below, while also remaining within the scope of the present disclosure.
[0040] refer to Figure 3 , shows another embodiment of the server device 200. For some examples provided below, Figure 4The embodiment of the server device 200 shown in provides an example of a four-processor system having a processing subsystem 300a connected to the processing subsystem 206a via a UP 304 and a processing subsystem 302a connected to the processing subsystem 208a via a UPI 306. Similar to the processing subsystems 206a and 208a, each of the processing subsystems 300a and 302a is provided with and connected to a respective memory subsystem 300b and 302. As discussed above, the processing subsystems and memory subsystems in the server device 200 may utilize a non-uniform memory access (NUMA) computer memory design, wherein each respective processing subsystem is connected to a respective local memory subsystem to provide a respective NUMA node (i.e., the processing subsystem 300a and its local memory subsystem 300b provide a third NUMA node, and the processing subsystem 302a and its local memory subsystem 302b provide a fourth NUMA node). However, although in Figure 3 , four processing subsystems / memory subsystems (e.g., four NUMA nodes) are shown, but those skilled in the art who understand the present disclosure will recognize that more processing subsystems / memory subsystems (e.g., NUMA nodes) may be provided in accordance with the teachings of the present disclosure, which also fall within the scope of the present disclosure. In the illustrated embodiment, each of the processing subsystems 300a and 302a and the processing subsystems 206a and 208a are coupled to the control / bridging subsystem 212.
[0041] Reference now Figure 4, showing one embodiment of a method 400 for configuring a communication coupling between a processor and an endpoint. As discussed below, the systems and methods of the present disclosure configure communication couplings between one or more processing subsystems in a server device and one or more endpoints coupled to those processing subsystems based on the hardware subsystems providing those communication couplings, the communication coupling configuration capabilities of those communication couplings, and the capabilities of a multi-endpoint adapter device providing the endpoints. In some embodiments, the configuration of the communication couplings between one or more processing subsystems in the server device and one or more endpoints coupled to those processing subsystems may also be based on a retrieved user configuration policy. In some embodiments, the configuration of the communication couplings between the one or more processing subsystems and one or more endpoints coupled to those processing subsystems may be performed and applied during runtime of the server device, while in other embodiments, the configuration of the communication couplings between the one or more processing subsystems and one or more endpoints coupled to those processing subsystems may require a boot operation by the server device. Thus, providing configurable communications couplings between processing subsystems and endpoints enhances the scalability of server devices (e.g., to provide additional processing subsystems in the server devices), while taking into account the endpoint devices connected to those processing subsystems, and allows a single motherboard and riser card device to implement many different processing subsystem / endpoint configurations.
[0042] In the embodiments discussed below, a multi-port adapter device may be coupled to riser card device 214. FIG. 5A to FIG. 5C In the example shown and described, multi-port adapter device 500 is coupled to riser card device 214 (e.g., via one of the x16 connectors provided on riser card device 214 using x4 connectors “1”, “2”, “3”, and “4”) and includes a plurality of endpoints coupled to riser card device 214. For example, and as discussed in more detail below, the coupling between the endpoints provided on multi-port adapter device 500 and riser card 214 may be configurable (e.g., by providing a plurality of connectors for each of the plurality of ports). Figure 5A In one specific example, the endpoints provided on the multi-endpoint adapter device 500 may be physical or virtual peripheral component interconnect express (PCIe) endpoints (such as, for example, Figure 5A5. The server device 200 is provided with a network interface controller (NIC) 500a shown in FIG. 5 on the multi-endpoint adapter device 500. As discussed in more detail below, couplings between the control / bridging subsystem 212, the riser card device 214, and the NIC 500a provide one or more hardware subsystems that include a plurality of communication couplings that can couple each of the plurality of endpoints (e.g., NIC 500a) to at least one of the plurality of processing subsystems 206a and 208a to provide a respective subset of available communication resources in the server device 200 to each of the plurality of processing subsystems 206a and 208a.
[0043] In one particular example, NIC 500a provides a "multi-PCIe endpoint" that is configured to allow the resources of the NIC to be split between multiple processing subsystems. For example, and as discussed below, NIC 500a can be configured to split its corresponding x16 connection to riser device 214 into two x8 connections to a pair of processing subsystems, or four x4 connections to each of four processing subsystems. However, while specific examples are described, those skilled in the art who understand the present disclosure will recognize that other endpoints with other configurable connection capabilities will also fall within the scope of the present disclosure. Figure 5A Also shown is how another device 502 (e.g., a non-multipoint adapter device) can be coupled to the riser card device 214 and how a device subsystem 502a can be provided that is also coupled to the riser card device 214 through a configurable connection (e.g., through a configurable interface for the device subsystem 502a). Figure 5A ). Additionally, in the illustrated embodiment, the BMC device 216 is coupled to each of the NIC 500a and the device subsystem 502a.
[0044] Reference below FIG. 6A to FIG. 6C In the example shown and described, a pair of devices 600 and 602 (e.g., non-multi-port adapter devices, such as conventional expansion cards) may be coupled to a riser card device 214 (e.g., via two x16 connectors provided on the riser card device 214 using x4 connectors "1," "2," "3," and "4," and x4 connectors "5," "6," "7," and "8"), and include device subsystems 600a and 602a, respectively, coupled to the riser card device 214. For example, and as discussed in more detail below, the coupling between the device subsystems 600a and 602a provided on the devices 600 and 602 and the riser card device 214 may be configurable (e.g., by providing a configuration for each of the device subsystems 600a and 602a). Fig. 6A). However, while specific examples are described, those skilled in the art who understand the present disclosure will recognize that other device subsystems with other configurable connection capabilities will also fall within the scope of the present disclosure. As discussed in more detail below, the couplings between the control / bridging subsystem 212, the riser card device 214, and the device subsystems 600a and 602a provide one or more hardware subsystems, which include multiple communication couplings that can couple each of the multiple device subsystems 600a and 602a to at least one of the multiple processing subsystems 206a and 208a so as to provide a corresponding subset of the available communication resources in the server device 200 to each of the multiple processing subsystems 206a and 208a. In addition, in the illustrated embodiment, the BMC device 216 is coupled to each of the device subsystems 600a and 602a.
[0045] Reference below FIG. 7A to FIG. 7G In the example shown and described, a pair of multi-endpoint adapter devices 700 and 702 are coupled to riser card device 214 (e.g., via two x16 connectors provided on riser card device 214 using x4 connectors “1”, “2”, “3”, and “4” and x4 connectors “5”, “6”, “7”, and “8”) and include a plurality of endpoints coupled to riser card device 214. For example, and as discussed in more detail below, the coupling between the endpoints provided on multi-endpoint adapter devices 700 and 702 and riser card 214 may be configurable (e.g., by providing a plurality of connectors for each of the plurality of adapter devices 700 and 702). Fig. 7A In one specific example, the endpoints provided on multi-endpoint adapter devices 700 and 702 may be physical or virtual peripheral component interconnect express (PCIe) endpoints (such as, for example, Fig. 7A 200 and 702a). As discussed in more detail below, couplings between the control / bridging subsystem 212, the riser card device 214, and the NICs 700a and 702a provide one or more hardware subsystems that include a plurality of communication couplings that can couple each of the plurality of endpoints (e.g., NICs 700a and 702a) to at least one of the plurality of processing subsystems 206a, 208a, 300a, and 302a so as to provide a respective subset of available communication resources in the server device 200 to each of the plurality of processing subsystems 206a, 208a, 300a, and 302a.
[0046] In one particular example, NICs 700a and 702a provide a "multi-PCIe endpoint device" that is configured to allow the resources of the NIC to be partitioned between multiple processing subsystems. For example, and as discussed below, each of NICs 700a and 702a can be configured to partition their respective x16 connections to riser device 214 into two x8 connections to a pair of processing subsystems, or four x4 connections to each of four processing subsystems. However, while specific examples are described, those skilled in the art who understand the present disclosure will recognize that other endpoints with other configurable connection capabilities will also fall within the scope of the present disclosure. Additionally, in the illustrated embodiment, BMC device 216 is coupled to each of NICs 700a and 702a.
[0047] Reference below FIG. 8A to FIG. 8E In the example shown and described, a pair of multi-endpoint adapter devices 800 and 802 are coupled to riser card device 214 (e.g., via two x16 connectors provided on riser card device 214 using x4 connectors “1”, “2”, “3”, and “4” and x4 connectors “5”, “6”, “7”, and “8”) and include a plurality of endpoints coupled to riser card device 214. For example, and as discussed in more detail below, the coupling between the endpoints provided on multi-endpoint adapter devices 800 and 802 and riser card 214 may be configurable (e.g., by providing a plurality of connectors for each of the plurality of adapter devices 800 and 802). Fig. 8A In one specific example, the endpoints provided on multi-endpoint adapter devices 800 and 802 may be physical or virtual peripheral component interconnect express (PCIe) endpoints (such as, for example, Fig. 8A 2. As discussed in more detail below, couplings between the control / bridging subsystem 212, the riser card device 214, and the NICs 800a and 802a provide one or more hardware subsystems that include a plurality of communication couplings that can couple each of the plurality of endpoints (e.g., NICs 800a and 802a) to at least one of the plurality of processing subsystems 206a, 208a, 300a, and 302a so as to provide a respective subset of available communication resources in the server device 200 to each of the plurality of processing subsystems 206a, 208a, 300a, and 302a.
[0048] In a particular example, NICs 800a and 802a provide a "multi-PCIe endpoint device" that is configured to allow the resources of the NIC to be partitioned between multiple processing subsystems. For example, and as discussed below, each of NICs 800a and 802a can be configured to partition their respective x16 connections to riser device 214 into two x8 connections to a pair of processing subsystems, or four x4 connections to each of four processing subsystems. However, while specific examples are described, those skilled in the art who understand the present disclosure will recognize that other endpoints with other configurable connection capabilities will also fall within the scope of the present disclosure. Additionally, in the illustrated embodiment, BMC device 216 is coupled to each of NICs 800a and 802a.
[0049] Reference below 9A to 9E In the example shown and described, multi-endpoint adapter device 900 is coupled to riser card device 214 (e.g., via one of the x16 connectors provided on riser card device 214 using x4 connectors “1,” “2,” “3,” and “4”) and includes a plurality of endpoints coupled to riser card device 214. For example, and as discussed in more detail below, the coupling between the endpoints provided on multi-endpoint adapter device 900 and riser card 214 may be configurable (e.g., by providing a plurality of Fig.9A In one specific example, the endpoints provided on the multi-endpoint adapter device 900 may be physical or virtual peripheral component interconnect express (PCIe) endpoints (such as, for example, Fig.9A 900a) on the multi-endpoint adapter device 900. As discussed in more detail below, couplings between the control / bridging subsystem 212, the riser card device 214, and the NIC 900a provide one or more hardware subsystems that include a plurality of communication couplings that can couple each of the plurality of endpoints (e.g., the NIC 900a) to at least one of the plurality of processing subsystems 206a, 208a, 300a, and 302a to provide a respective subset of available communication resources in the server device 200 to each of the plurality of processing subsystems 206a, 208a, 300a, and 302a.
[0050] In one particular example, the NIC 900a provides a "multi-PCIe endpoint device" that is configured to allow the resources of the NIC to be partitioned between multiple processing subsystems. For example, and as discussed below, the NIC 900a can be configured to partition its corresponding x16 connection to the riser device 214 into two x8 connections to a pair of processing subsystems, or four x4 connections to each of four processing subsystems. However, while specific examples are described, those skilled in the art who understand the present disclosure will recognize that other endpoints with other configurable connection capabilities will also fall within the scope of the present disclosure. Fig.9A Also shown is how another device 902 (e.g., a non-multipoint adapter device) can be coupled to the riser card device 214 and how a device subsystem 902a can be provided that is also coupled to the riser card device 214 through a configurable connection (e.g., through a configurable interface for the device subsystem 902a). Fig.9A 900a and device subsystem 902a). Additionally, in the illustrated embodiment, BMC device 216 is coupled to each of NIC 900a and device subsystem 902a.
[0051] Method 400 begins at block 402, where a hardware subsystem including a communication coupling that couples a processing subsystem to a multi-point adapter device is identified. In one embodiment, at block 402, a communication coupling configuration engine provided by a BMC device 216 operates to identify a hardware subsystem including a communication coupling that couples a processing subsystem to a multi-point adapter device. In one specific example, the BMC device 216 may be started before the server device 200 is powered on or otherwise initialized, and may be operable to perform an inventory operation to identify any or all hardware subsystems in the server device 200. Thus, at block 402, the communication coupling configuration engine in the BMC device 216 may identify the presence of a host central processing unit (CPU), port availability, motherboard-CPU-riser card / connector, and / or other hardware subsystems that will be apparent to one of ordinary skill in the art who understands the present disclosure. Reference Figure 5A and Fig. 6A In the example provided in FIG. 4 , at box 402, the communication coupling configuration engine in the BMC device 216 can identify the memory subsystems 206a and 208b, the processing subsystems 206a and 208a, the UPI 210 between the processing subsystems 206a and 208a, the control bridge subsystem 212, the coupling between the processing subsystems 206a / 208a and the control bridge subsystem 212, the riser card device 214, and the coupling between the control bridge subsystem 212 and the riser card device 214.
[0052] Similarly, refer to Fig. 7A , Fig. 8A and Fig.9A In the example provided in, at box 402, the communication coupling configuration engine in the BMC device 216 can identify the memory subsystems 206a, 208b, 300b and 302b; the processing subsystems 206a, 208a, 300a and 302a; the UPI 210 between the processing subsystems 206a and 208a; the UPI 304 between the processing subsystems 300a and 206a; the UPI 306 between the processing subsystems 208a and 302a; the control bridge subsystem 212; the coupling between the processing subsystems 206a / 208a / 300a / 302a and the control bridge subsystem 212; the riser card device 214; and the coupling between the control bridge subsystem 212 and the riser card device 214. However, while specific examples are described, those skilled in the art who understand the present disclosure will recognize how a BMC device may perform inventory operations to identify any hardware subsystem that provides a communication coupling between a processing subsystem and an endpoint while also remaining within the scope of the present disclosure.
[0053] Method 400 then proceeds to block 404, where the communication coupling configuration capabilities of the communication coupling are determined. In one embodiment, at block 404, a communication coupling configuration engine provided by the BMC device 216 operates to determine the communication coupling configuration capabilities of the communication coupling that couples the processing subsystem to the multi-end adapter device. As discussed above, in one particular example, the BMC device 216 may be started before the server device is powered on or otherwise initialized, and may be operable to perform an inventory operation to determine the communication coupling configuration capabilities of the communication coupling included in the hardware subsystem identified at block 402. Thus, at block 402, the communication coupling configuration engine in the BMC device 216 may determine the installed riser card type, trace configuration, wiring configuration (e.g., elongated PCIe wiring configuration), and / or other communication coupling configuration information that will be apparent to one of skill in the art who understands the present disclosure. Thus, with reference to FIG. Figure 5A and Fig. 6A In the example provided in, at box 402, the communication coupling configuration engine in the BMC device 216 can determine the coupling between the processing subsystem 206a / 208a and the control bridge subsystem 212, the coupling between the control bridge subsystem 212 and the riser card device 214, and the configuration capabilities of the riser card device 214.
[0054] Similarly, reference Fig. 7A , Fig. 8A and Fig.9A, at block 402, the communication coupling configuration engine in the BMC device 216 may determine the configuration capabilities of the couplings between the processing subsystem 206a / 208a / 300a / 302a and the control bridge subsystem 212, the couplings between the control bridge subsystem 212 and the riser card device 214, and the riser card device 214. However, while specific examples are described, those skilled in the art who understand the present disclosure will recognize how the BMC device may perform inventory operations to determine the configuration capabilities of various communication couplings between the processing subsystems and the endpoints while still falling within the scope of the present disclosure.
[0055] The method 400 then proceeds to block 406, where the multi-point adapter device capabilities are determined. In one embodiment, at block 406, a communication coupling configuration engine provided by the BMC device 216 operates to determine the capabilities of any multi-point adapter devices provided in the server device 200. As discussed above, in one particular example, the BMC device 216 may be started before the server device is powered on or otherwise initialized, and may be operable to perform an inventory operation to determine the multi-point adapter device capabilities of the multi-point adapter devices coupled to the riser card device 214. Thus, at block 406, the communication coupling configuration engine in the BMC device 216 may perform a read operation on an electrically erasable programmable read-only memory (EEPROM) subsystem in any multi-point adapter device to which it is connected in order to determine the capabilities of the multi-point adapter, which may include determining the endpoint type, endpoint capabilities, endpoint communication coupling configuration (e.g., the configuration of the communication coupling between the endpoint and the riser card device 214), the configuration capabilities of the endpoint communication coupling, and / or other multi-point adapter device information that will be apparent to those skilled in the art who understand the present disclosure. Thus, reference is made to Figure 5A , at block 406, the communications coupling configuration engine in the BMC device 216 may determine the capabilities of the multi-point adapter device 500 and may determine that the device 502 is not a multi-point adapter device.
[0056] Similarly, refer to Fig. 6A In the example provided in FIG. 4 , at block 406, the communication coupling configuration engine in the BMC device 216 may determine that the devices 600 and 602 are not multi-endpoint adapter devices. Fig. 7A In the example provided in FIG. 4 , at block 406, the communication coupling configuration engine in the BMC device 216 may determine the capabilities of the multi-endpoint adapter devices 700 and 702. Similarly, referring to FIG. Fig. 8A In the example provided in FIG. 4 , at block 406, the communication coupling configuration engine in the BMC device 216 may determine the capabilities of the multi-endpoint adapter devices 800 and 902. Similarly, referring to FIG. Fig.9A, at block 406, the communication coupling configuration engine in the BMC device 216 may determine the capabilities of the multi-point adapter device 900, and may determine that the device 902 is not a multi-point adapter device. However, although a specific example is described, those skilled in the art who understand the present disclosure will recognize how the BMC device may perform inventory operations to determine the various capabilities of the multi-point adapter devices included in the server device while still falling within the scope of the present disclosure.
[0057] The method 400 may then proceed to optional block 408, where a user configuration policy may be retrieved. In one embodiment, at optional block 408, a communication coupling configuration engine provided by the BMC device 216 may be operable to retrieve a user configuration policy. For example, an administrator or other user of the server device 200 may be given the option to define a user configuration policy detailing the desired operation of the server device 200, the multi-end adapter device provided in the server device 200, the resources to be allocated to applications provided by the processing subsystem in the server device 200, and / or any other user configuration policy information that would be apparent to one of ordinary skill in the art understanding the present disclosure (e.g., via an F2 menu selection provided during booting of the server device 200, via an out-of-band method known in the art, etc.).
[0058] The method 400 then proceeds to block 410, where configuration information is generated. In one embodiment, at block 410, a communication coupling configuration engine provided by the BMC device 216 is operable to determine and generate configuration information based on: at least one of the hardware subsystems identified at block 402; the configuration capabilities of the communication coupling determined at block 404; the capabilities of the multi-end adapter device determined at block 406; and, in some embodiments, the user configuration policy retrieved at optional block 408. For example, in embodiments where a user configuration policy is not retrieved, the communication coupling configuration engine provided by the BMC device 216 is operable to determine a "best fit" configuration for the communication coupling, the "best fit" configuration being based on at least one of the hardware subsystems identified at block 402; the configuration capabilities of the communication coupling determined at block 404; and the capabilities of the multi-end adapter device determined at block 406. However, in embodiments where a user configuration policy is retrieved, a communication coupling configuration engine provided by the BMC device 216 is operable to determine a "best fit" configuration for the communication coupling based on at least one of the hardware subsystems identified at block 402; the configuration capabilities of the communication coupling determined at block 404; the capabilities of the multi-endpoint adapter device determined at block 406; and the user configuration policy. In either case, the communication coupling configuration engine provided by the BMC device 216 may have knowledge of the hardware subsystems and I / O topology in the server device 200 and generate a mapping of the hardware subsystems to the processing subsystems and endpoint links. As will be appreciated by those skilled in the art who understand the present disclosure, the discovery of the configuration capabilities of the BMC device 216 for the source and destination may be modified by the user as necessary.
[0059] As will be appreciated by those skilled in the art who understand the present disclosure, the configuration information generated at block 410 may be any information that provides configuration of the communication coupling in the examples discussed below. Thus, the configuration information generated at block 410 may include any information that provides the specific communication resource allocations discussed in the examples below, as well as any other communication resource allocations that will be apparent to those skilled in the art who understand the present disclosure.
[0060] Furthermore, in some embodiments, at least some of the configuration information may be transmitted to the control / bridging subsystem 212 and multi-port adapter devices included in the server device 200 via a communication coupling configuration engine provided by the BMC device 216, so as to, for example, provide for configuring the control / bridging subsystem 212 and those multi-port adapter devices in a manner corresponding to the configuration of the communication coupling so as to allocate communication resources via those configurations. For example, at block 410, the communication coupling configuration engine provided by the BMC device 216 may be operable to configure the control / bridging subsystem 212 and those multi-port adapter devices via a sideband interface (e.g., an integrated circuit bus (ICB)). 2C) sideband interface) to provide flexible input / output (I / O) configuration information to a multi-point adapter device disposed in the server device 200. In such an example, the flexible I / O configuration information may be configured to cause an adapter device I / O expander in the multi-point adapter device to convert an endpoint (e.g., a NIC in the example below) from acting as a single endpoint (e.g., a single NIC) utilizing an x16 connection to the riser card device 214 to acting as two endpoints (e.g., two NICs) each utilizing an x8 connection to the riser card device 214. However, while specific examples are provided, those skilled in the art who understand the present disclosure will recognize that various configuration information may be provided to the multi-point adapter device to provide various configurations that would also fall within the scope of the present disclosure.
[0061] The method 400 may then proceed to optional block 412, where a boot operation may be initiated. In one embodiment, at optional block 412, a communication coupling configuration engine provided by the BMC device 216 may be operable to initiate a boot operation of the server device 200. As discussed above, in some specific examples, the BMC device 216 may be initiated before the server device 200 is powered on or otherwise initialized, and may then be operable to perform blocks 402, 404, 406, optional block 408, and block 410 of the method 400, followed by optional block 412, in which booting of the server device 200 is initiated to allow configuration discussed below with respect to block 414. However, in other embodiments, operation of the BMC device 216 and its communication coupling configuration engine (or other communication coupling configuration subsystem provided as discussed above) may occur during runtime operation of the server devices 200, and thus optional block 412 may be skipped, and configuration discussed below with respect to block 414 may also be performed during runtime operation of those server devices. However, in embodiments in which server device 200 performs a reboot operation at optional block 412, server device 200 may power on and, in response, perform a power-on self-test (POST).
[0062] The method 400 then proceeds to block 414, where the communication coupling is configured to provide at least one communication resource between at least one processing subsystem on the board and at least one endpoint on the multi-endpoint adapter device. In one embodiment, at block 414, the communication coupling configuration engine provided by the BMC device 216 is operable to provide elastic I / O configuration information to a basic input / output system (BIOS) in the server device 200, which can be configured to allow the BIOS to perform CPU link purposes that may include configuration operations, split operations, and / or other CPU link purposes that will be apparent to those skilled in the art who understand the present disclosure. In addition, any endpoint on the multi-endpoint adapter device provided in the server device 200 may be released due to a reset, and link training may be performed, and the adapter device I / O expander in the multi-endpoint adapter device may read the requested I / O expander signal state to configure its endpoint and split accordingly. As will be understood by those skilled in the art who understand the present disclosure, when the endpoint is a PCIe endpoint as part of a single chip package, there may be a single set of PHY pins, and bus exchange switching may be performed at an analog pre-SERDES level or a digital post-PHY level. Although specific examples are provided, those skilled in the art who understand the present disclosure will recognize that the BMC device 216 can communicate with the BIOS in the server device 200 in various ways to control endpoint signals so that the expected topology and memory configuration type are set to achieve appropriate startup and runtime use. Some exemplary configurations will now be discussed with reference to the figures, but those skilled in the art who understand the present disclosure will recognize that any of the various configurations can also be implemented through the teachings of the present disclosure.
[0063] refer to Figure 5A , Figure 5B and Figure 5C The example shown in Figure 5B 2 shows how a communication coupling configuration engine provided by the BMC device 216 can provide communication resources in the server device 200 to the processing subsystem 206a by configuring the communication couplings 504a, 504b and 504c (or providing a configuration of the communication couplings) so as to couple the processing subsystem 206a to the NIC 500a via an x8 connection (provided by the x4 connectors "1" and "2" on the riser card device 214) and configuring the communication couplings 504a, 504d and 504e so as to couple the processing subsystem 206a to the device subsystem 502a via an x8 connection (provided by the x4 connectors "5" and "6" on the riser card device 214). Figure 5C2 shows how a communication coupling configuration engine provided by the BMC device 216 can provide communication resources in the server device 200 to the processing subsystem 208a by configuring the communication couplings 506a, 506b, and 506c (or providing a configuration of the communication couplings) to couple the processing subsystem 208a to the NIC 500a via an x8 connection (provided by the x4 connectors "3" and "4" on the riser card device 214) and configuring the communication couplings 506a and 506d to provide the processing subsystem 208a with access to multiple endpoints (e.g., NIC 500a and device subsystem 502a). For example, such a configuration can avoid the need for switches (e.g., PCIe switches) on the adapter devices 500 and 502 to enable the available endpoints (which can appear to the operating system as multiple PCIe devices, device drivers, and corresponding stacks) to be utilized while providing logical system setup advantages (e.g., for assigning specific endpoints to specific virtual machines). Thus, Figure 5A , Figure 5B and Figure 5C The example in shows how each processing subsystem 206a and 206b can be assigned communication resources (in this example, an x8 communication coupling) to NIC 500a on the multi-end adapter device 500, while the device subsystem 502a can be assigned communication resources (in this example, an x8 communication coupling) to a single processing subsystem 206a.
[0064] refer to Fig. 6A , Figure 6B and Figure 6C The example shown in Figure 6B 2 shows how a communications coupling configuration engine provided by the BMC device 216 may provide communications resources in the server device 200 to the processing subsystem 206a by configuring the communications couplings 604a, 604b, and 604c (or providing a configuration of the communications couplings) so as to couple the processing subsystem 206a to the device subsystem 600a via x16 connections (provided by x4 connectors "1", "2", "3", and "4" on the riser card device 214). Figure 6C 2 shows how the communication coupling configuration engine provided by the BMC device 216 can provide communication resources in the server device 200 to the processing subsystem 208a by configuring the communication couplings 606a, 606b, and 606c (or providing a configuration of the communication couplings) to couple the processing subsystem 208a to the device subsystem 602a via x16 connections (provided by x4 connectors "5", "6", "7", and "8" on the riser card device 214). Thus, Fig. 6A , Figure 6B and Figure 6CThe example in FIG. 2 shows how each processing subsystem 206a and 208a may be assigned communication resources (in this example, x16 communication couplings) to corresponding device subsystems 600a and 602a on devices 600 and 602. For example, when a non-multipoint adapter device is connected to riser card device 214, a communication coupling configuration engine provided by BMC device 216 may recognize and provide a maximum bandwidth connection (e.g., x16 in this example) between each processing subsystem and a corresponding one of the non-multipoint adapter devices.
[0065] refer to Fig. 7A , Figure 7B , Figure 7C , Fig.7D and Fig. 7E The example shown in Figure 7B 2 shows how a communications coupling configuration engine provided by the BMC device 216 may provide communications resources in the server device 200 to the processing subsystem 300a by configuring the communications couplings 704a, 704b, and 704c (or providing a configuration of the communications couplings) so as to couple the processing subsystem 300a to the NIC 700a via an x8 connection (provided by the x4 connectors "1" and "2" on the riser card device 214). Figure 7C 2 shows how a communications coupling configuration engine provided by the BMC device 216 may provide communications resources in the server device 200 to the processing subsystem 206a by configuring the communications couplings 706a, 706b, and 706c (or providing a configuration of the communications couplings) so as to couple the processing subsystem 206a to the NIC 700a via an x8 connection (provided by x4 connectors "3" and "4" on the riser card device 214).
[0066] Fig.7D 2 shows how a communications coupling configuration engine provided by the BMC device 216 can provide communications resources in the server device 200 to the processing subsystem 208a by configuring the communications couplings 708a, 708b, and 708c (or providing a configuration of the communications couplings) so as to couple the processing subsystem 208a to the NIC 702a via an x8 connection (provided by the x4 connectors "5" and "6" on the riser card device 214). Fig. 7E 2 shows how the communication coupling configuration engine provided by the BMC device 216 can provide communication resources in the server device 200 to the processing subsystem 302a by configuring the communication couplings 710a, 710b, and 710c (or providing a configuration of the communication couplings) to couple the processing subsystem 302a to the NIC 702a via an x8 connection (provided by the x4 connectors "7" and "8" on the riser card device 214). Thus, Fig. 7A , Figure 7B , Figure 7C , Fig.7D and Fig. 7E The example in shows how each processing subsystem 206a and 300a can be assigned communication resources (in this example, a corresponding x8 communication coupling) to one of NIC 700a, while each processing subsystem 208a and 302a can be assigned communication resources (in this example, a corresponding x8 communication coupling) to the other of NIC 702a. For example, when multiple multi-point adapter devices are connected to riser card device 214, the communication resources (e.g., in this example, x8 connections for each processing subsystem) can be evenly divided to ensure that each processing subsystem has direct I / O access without having to traverse UPI 210, 304, or 306.
[0067] Continue with the above reference Fig. 7A , Figure 7B , Figure 7C , Fig.7D and Fig. 7E Examples discussed and referenced Figure 7F and Figure 7G , the control / bridging subsystem 212 and the communication coupling configuration engine provided by the BMC device 216 can be configured to understand data traffic patterns and provide quality of service (QoS) functions, which can be accomplished via user preference settings, automatic traffic pattern adjustments, and / or using various other techniques that will be apparent to those skilled in the art who understand the present disclosure. For example, Figure 7F and Figure 7G 2 shows how the control / bridging subsystem 212 and / or the communication coupling configuration engine provided by the BMC device 216 can reallocate communication resources in the server device 200 to the processing subsystem 206a by configuring the communication couplings 712a, 712b and 712c (or providing a configuration of the communication couplings) so as to couple the processing subsystem 206a to the NIC 702a via an x8 connection (provided by the x4 connectors "5" and "6" on the riser card device 214), and how the control / bridging subsystem 212 and / or the communication coupling configuration engine provided by the BMC device 216 can reallocate communication resources in the server device 200 to the processing subsystem 208a by configuring the communication couplings 714a, 714b and 714c (or providing a configuration of the communication couplings) so as to couple the processing subsystem 208a to the NIC 700a via an x8 connection (provided by the x4 connectors "3" and "4" on the riser card device 214). Thus, the communication resources provided between a particular set of processing subsystems and endpoints (e.g., x8 connections in this example) may be swapped to meet QoS considerations. FIG. 7A to FIG. 7GIn the example provided in , a x16 connection may be provided between certain processing subsystems and corresponding endpoints, while a x4 connection may be provided between the remaining processing subsystems and their corresponding endpoints so as to also meet QoS considerations.
[0068] refer to Fig. 8A , Figure 8B , Figure 8C , Fig.8D and Fig. 8E The example shown in Figure 8B 2 shows how a communication coupling configuration engine provided by the BMC device 216 can provide communication resources in the server device 200 to the processing subsystem 300a by configuring the communication couplings 804a, 804b and 804c (or providing a configuration of the communication couplings) so as to couple the processing subsystem 300a to the NIC 800a via an x4 connection (provided by the x4 connector "1" on the riser card device 214) and by configuring the communication couplings 804a, 804d and 804e (or providing a configuration of the communication couplings) so as to couple the processing subsystem 300a to the NIC 802a via an x4 connection (provided by the x4 connector "5" on the riser card device 214). Figure 8C 802a. The diagram shows how a communications coupling configuration engine provided by the BMC device 216 can provide communications resources in the server device 200 to the processing subsystem 206a by configuring communications couplings 806a, 806b, and 806c (or providing a configuration of the communications couplings) so as to couple the processing subsystem 206aa to the NIC 800a via an x4 connection (provided by the x4 connector “2” on the riser card device 214) and by configuring communications couplings 806a, 806d, and 806e (or providing a configuration of the communications couplings) so as to couple the processing subsystem 206a to the NIC 802a via an x4 connection (provided by the x4 connector “6” on the riser card device 214).
[0069] Fig.8D 8. The diagram shows how a communications coupling configuration engine provided by the BMC device 216 may provide communications resources in the server device 200 to the processing subsystem 208a by configuring communications couplings 808a, 808b, and 808c (or providing a configuration of the communications couplings) so as to couple the processing subsystem 208a to the NIC 800a via an x4 connection (provided by the x4 connector "3" on the riser card device 214) and by configuring communications couplings 808a, 808d, and 808e (or providing a configuration of the communications couplings) so as to couple the processing subsystem 208a to the NIC 802a via an x4 connection (provided by the x4 connector "7" on the riser card device 214). Fig. 8E2 shows how the communication coupling configuration engine provided by the BMC device 216 can provide communication resources in the server device 200 to the processing subsystem 302a by configuring the communication couplings 810a, 810b, and 810c (or providing a configuration of the communication couplings) to couple the processing subsystem 302a to the NIC 800a via an x4 connection (provided by the x4 connector "4" on the riser card device 214) and by configuring the communication couplings 810a, 810d, and 810e (or providing a configuration of the communication couplings) to couple the processing subsystem 302a to the NIC 802a via an x4 connection (provided by the x4 connector "8" on the riser card device 214). Thus, Fig. 8A , Figure 8B , Figure 8C , Fig.8D and Fig. 8E The example in shows how each processing subsystem 206a, 208a, 300a, and 302a can be assigned redundant communication resources (in this example, a pair of x4 communication couplings) to each of the NICs 800a and 802a. Thus, the control / bridging subsystem 212 and / or the BMC device 216 can be configured to recognize a NIC failure, an unavailable link to one of the NICs, and / or other communication failures associated with a particular processing subsystem, and in response, can implement redundant communication couplings to the processing subsystems.
[0070] refer to Fig.9A , Fig. 9B , Fig. 9C , Fig.9D and Fig.9E The example shown in Fig. 9B 2 shows how a communication coupling configuration engine provided by the BMC device 216 can provide communication resources in the server device 200 to the processing subsystem 300a by configuring the communication couplings 904a, 904b and 904c (or providing a configuration of the communication couplings) so as to couple the processing subsystem 300a to the NIC 900a via an x4 connection (provided by the x4 connector "1" on the riser card device 214) and by configuring the communication couplings 904a, 904d and 904e (or providing a configuration of the communication couplings) so as to couple the processing subsystem 300a to the device subsystem 902a via an x4 connection (provided by the x4 connector "5" on the riser card device 214). Fig. 9C2 shows how a communication coupling configuration engine provided by BMC device 216 can provide communication resources in server device 200 to processing subsystem 206a by configuring communication couplings 906a, 906b, and 906c (or providing a configuration of the communication couplings) to couple processing subsystem 206a to NIC 900a via an x4 connection (provided by x4 connector "2" on riser device 214) and by configuring communication couplings 906a and 906d (or providing a configuration of the communication couplings) to provide processing subsystem 206a with access to multiple endpoints (e.g., NIC 900a and device subsystem 902a), which can provide communication resources similar to those described above with reference to FIG. Figure 5C Advantages similar to those discussed.
[0071] Fig.9D 2 shows how a communication coupling configuration engine provided by BMC device 216 can provide communication resources in server device 200 to processing subsystem 208a by configuring communication couplings 908a, 908b, and 908c (or providing a configuration of the communication couplings) to couple processing subsystem 208a to NIC 900a via an x4 connection (provided by x4 connector "3" on riser device 214) and by configuring communication couplings 908a and 908d (or providing a configuration of the communication couplings) to provide processing subsystem 208a with access to multiple endpoints (e.g., NIC 900a and device subsystem 902a), which can provide similar communication resources to those described above with reference to FIG. Figure 5C Advantages similar to those discussed. Fig.9E 2 shows how a communication coupling configuration engine provided by BMC device 216 can provide communication resources in server device 200 to processing subsystem 302a by configuring communication couplings 910a, 910b, and 910c (or providing a configuration of the communication couplings) to couple processing subsystem 302a to NIC 900a via an x4 connection (provided by x4 connector "4" on riser device 214) and by configuring communication couplings 910a and 910d (or providing a configuration of the communication couplings) to provide processing subsystem 302a with access to multiple endpoints (e.g., NIC 900a and device subsystem 902a), which can provide communication resources similar to those described above with reference to FIG. Figure 5C Advantages similar to those discussed above. Fig.9A , Fig. 9B , Fig. 9C , Fig.9D and Fig.9EThe example in shows how each processing subsystem 206a, 208a, 300a and 302a can be assigned communication resources (in this example, a x4 communication coupling) to NIC 900a, while the device subsystem 902a can be assigned communication resources (in this example, a x4 communication coupling) to a single processing subsystem 300a.
[0072] Thus, systems and methods have been described for configuring communication couplings between one or more processing subsystems in a server device and one or more endpoints coupled to those processing subsystems based on hardware providing those communication couplings, communication coupling configuration capabilities of those communication couplings, and a multi-endpoint adapter device providing the endpoints. In some embodiments, the configuration of the communication couplings between one or more processing subsystems in the server device and one or more endpoints coupled to those processing subsystems may also be based on retrieved user configuration policies. In some embodiments, the configuration of the communication couplings between the one or more processing subsystems and one or more endpoints coupled to those processing subsystems may be performed and applied during runtime of the server device, while in other embodiments, the configuration of the communication couplings between the one or more processing subsystems and one or more endpoints coupled to those processing subsystems may require a startup operation by the server device. Thus, configurable communication couplings between processing subsystems and endpoints are provided, which enhance the scalability of the server device (e.g., to provide additional processing subsystems in the server device) while taking into account the endpoint devices connected to those processing subsystems, thereby allowing the hardware subsystems and their associated communication couplings to support a variety of processing subsystem / endpoint configurations.
[0073] In addition, the present disclosure is filed together with the following: U.S. Patent Application No. 16 / 396,022 filed on April 26, 2019 and relating to virtual machine deployment technology, attorney docket number 16356.2030US01; U.S. Patent Application No. 16 / 396,200 filed on April 26, 2019 and relating to data segmentation technology, attorney docket number 16356.2031US01; U.S. Patent Application No. 16 / 396,320 filed on April 26, 2019 and relating to data replication technology, attorney docket number 16356.2032US01; U.S. Patent Application No. 16 / 396,330 filed on April 26, 2019 and relating to data replication technology, attorney docket number 16356.2032US01; U.S. Patent Application No. 16 / 396,340 filed on April 26, 2019 and relating to data replication technology, attorney docket number 16356.2034US01; U.S. Patent Application No. 16 / 396,453, attorney docket number 16356.2033US01, relating to packet routing technology; U.S. Patent Application No. 16 / 396,521, attorney docket number 16356.2035US01, filed on April 26, 2019, and relating to connection configuration technology; and U.S. Patent Application No. 16 / 395,584, attorney docket number 16356.2036US01, filed on April 26, 2019, and relating to sideband communication technology; each of which includes an implementation utilizing a multi-processor / multi-endpoint system described in some implementations included in the present disclosure. Those skilled in the art who understand the present disclosure will recognize how the implementations of the present disclosure may be combined with some or all of the disclosures discussed above, and therefore those disclosures are incorporated herein by reference in their entirety.
[0074] Although exemplary embodiments have been shown and described, extensive modifications, changes and substitutions are contemplated in the foregoing disclosure, and in some cases, some features of the described embodiments may be employed without corresponding use of other features. It should therefore be understood that the appended claims are to be construed broadly and in a manner consistent with the scope of the embodiments disclosed herein.
Claims
1. A processor and endpoint communication coupling configuration system, comprising: a plurality of processing subsystems coupled together via at least one processor interconnect, the at least one processor interconnect allowing a first processing subsystem included in the plurality of processing subsystems to access a local memory of a second processing subsystem included in the plurality of processing subsystems via the at least one processor interconnect; a first multi-point adapter device; at least one hardware subsystem, the at least one hardware subsystem comprising a plurality of communication couplings coupling the plurality of processing subsystems to the first multi-point adapter device; as well as a communication coupling configuration engine coupled to the first multi-endpoint adapter device and the at least one hardware subsystem, wherein the communication coupling configuration engine is configured to: identifying at least one hardware subsystem comprising each of the plurality of communication couplings coupling the plurality of processing subsystems to the first multi-point adapter device; determining at least one first multi-point adapter device capability of the first multi-point adapter device; retrieving at least one first user configuration policy; as well as The plurality of communication couplings are configured based on the at least one hardware subsystem, the at least one first user-configured policy, and the at least one first multi-point adapter device capabilities, wherein the configuration of the plurality of communication couplings provides at least one communication resource between at least one of the plurality of processing subsystems and at least one endpoint on the first multi-point adapter device without using the at least one processor interconnect to communicate between the at least one of the plurality of processing subsystems and the at least one endpoint on the first multi-point adapter device.
2. The system of claim 1, wherein the communication coupling configuration engine is configured to: At least one communication coupling configuration capability of the plurality of communication couplings coupling the plurality of processing subsystems to the first multi-point adapter device is determined, wherein the configuration of the plurality of communication couplings is further based on the at least one communication coupling configuration capability.
3. The system of claim 1 , wherein the communication coupling configuration engine is configured to: causing at least one startup operation to be performed subsequent to the configuration of the plurality of communication couplings, wherein the configuration of the plurality of communication couplings that provide the at least one communication resource between the at least one of the plurality of processing subsystems and the at least one endpoint on the first multi-endpoint adapter device is effective subsequent to the startup operation.
4. The system of claim 1, wherein the communication coupling configuration engine is configured to: The configuration of the plurality of communication couplings is performed during runtime operation, wherein the configuration of the plurality of communication couplings that provide the at least one communication resource between the at least one of the plurality of processing subsystems and the at least one endpoint on the first multi-endpoint adapter device is effective during the runtime operation.
5. The system according to claim 1, further comprising: a second multi-point adapter device, the second multi-point adapter device being coupled to the plurality of processing subsystems via the plurality of communication couplings included in the at least one hardware subsystem, wherein the communication coupling configuration engine is configured to: identifying at least one hardware subsystem comprising each of the plurality of communication couplings coupling the plurality of processing subsystems to the second multi-port adapter device; determining at least one second multi-point adapter device capability of the second multi-point adapter device; retrieving at least one second user configuration policy; as well as The plurality of communication couplings are configured based on the at least one hardware subsystem, the at least one second user-configured policy, and the at least one second multi-point adapter device capabilities, wherein the configuration of the plurality of communication couplings provides at least one communication resource between at least one of the plurality of processing subsystems and at least one endpoint on the second multi-point adapter device.
6. The system of claim 1, further comprising: A baseboard management controller (BMC) device is coupled to the first multi-point adapter device and the at least one hardware subsystem, wherein the BMC device provides the communication coupling configuration engine.
7. An information processing system, comprising: First treatment system; as well as a first memory system coupled to the first processing system and comprising instructions that, when executed by the first processing system, cause the first processing system to provide a communication coupling configuration engine configured to: identifying at least one hardware subsystem comprising a plurality of communication couplings coupling a plurality of second processing subsystems to a first multi-port adapter device, wherein the plurality of second processing subsystems are coupled together via at least one processor interconnect that allows a first processing subsystem in the first processing system to access a respective local memory contained in each second processing subsystem in the plurality of second processing subsystems; retrieving at least one first user configuration policy; determining at least one first multi-point adapter device capability of the first multi-point adapter device; and The plurality of communication couplings are configured based on the at least one hardware subsystem, the at least one first user-configured policy, and the at least one first multi-point adapter device capability, wherein the configuration of the plurality of communication couplings provides at least one communication resource between at least one of the plurality of second processing subsystems and at least one endpoint on the first multi-point adapter device without using the at least one processor interconnect to communicate between the at least one of the plurality of second processing subsystems and the at least one endpoint on the first multi-point adapter device.
8. The information processing system of claim 7, wherein the communication coupling configuration engine is configured to: At least one communication coupling configuration capability of the plurality of communication couplings coupling the plurality of second processing subsystems to the first multi-point adapter device is determined, wherein the configuration of the plurality of communication couplings is further based on the at least one communication coupling configuration capability.
9. The information processing system of claim 7, wherein the communication coupling configuration engine is configured to: causing at least one startup operation to be performed subsequent to the configuration of the plurality of communication couplings, wherein the configuration of the plurality of communication couplings that provide the at least one communication resource between the at least one of the plurality of second processing subsystems and the at least one endpoint on the first multi-endpoint adapter device takes effect subsequent to the startup operation.
10. The information handling system of claim 7, wherein the communication coupling configuration engine is configured to: The configuration of the plurality of communication couplings is performed during runtime operation, wherein the configuration of the plurality of communication couplings that provide the at least one communication resource between the at least one of the plurality of second processing subsystems and the at least one endpoint on the first multi-endpoint adapter device is effective during the runtime operation.
11. The information processing system of claim 7, wherein the communication coupling configuration engine is configured to: identifying the at least one hardware subsystem comprising the plurality of communication couplings coupling the plurality of second processing subsystems to a second multi-point adapter device; retrieving at least one second user configuration policy; determining at least one second multi-point adapter device capability of the second multi-point adapter device; as well as The plurality of communication couplings are configured based on the at least one hardware subsystem, the at least one second user-configured policy, and the at least one second multi-point adapter device capabilities, wherein the configuration of the plurality of communication couplings provides at least one communication resource between at least one of the plurality of second processing subsystems and at least one endpoint on the second multi-point adapter device. 12 . The information processing system according to claim 7 , wherein the first processing system and the first memory system are provided in a baseboard management controller (BMC) device.
13. The information processing system of claim 7 , wherein the at least one communication resource provided between the at least one of the plurality of second processing subsystems and the at least one endpoint on the first multi-endpoint adapter device comprises providing redundant communication resources of each of the plurality of second processing subsystems to at least two endpoints.
14. A method for configuring a communication coupling between a processor and an endpoint, comprising: identifying, by the communication coupling configuration subsystem, at least one hardware subsystem comprising a plurality of communication couplings coupling a plurality of processing subsystems to a first multi-port adapter device, wherein the plurality of processing subsystems are coupled together via at least one processor interconnect, the at least one processor interconnect allowing a first processing subsystem included in the plurality of processing subsystems to access a local memory of a second processing subsystem included in the plurality of processing subsystems; retrieving, by the communication coupling configuration subsystem, at least one first user configuration policy; determining, by the communications coupling configuration subsystem, at least one first multi-point adapter device capability of the first multi-point adapter device; and The plurality of communication couplings are configured by the communication coupling configuration subsystem based on the at least one hardware subsystem, the at least one first user configuration policy, and the at least one first multi-point adapter device capabilities, wherein the configuration of the plurality of communication couplings provides at least one communication resource between at least one of the plurality of processing subsystems and at least one endpoint on the first multi-point adapter device without using the at least one processor interconnect to communicate between the at least one of the plurality of processing subsystems and the at least one endpoint on the first multi-point adapter device.
15. The method according to claim 14, further comprising: At least one communication coupling configuration capability of the plurality of communication couplings coupling the plurality of processing subsystems to the first multi-point adapter device is determined by the communication coupling configuration subsystem, wherein the configuration of the plurality of communication couplings is further based on the at least one communication coupling configuration capability.
16. The method according to claim 14, further comprising: At least one startup operation is performed by the communications coupling configuration subsystem after the configuration of the plurality of communications couplings, wherein the configuration of the plurality of communications couplings that provide the at least one communications resource between the at least one of the plurality of processing subsystems and the at least one endpoint on the first multi-endpoint adapter device takes effect after the startup operation.
17. The method according to claim 14, further comprising: The configuration of the plurality of communication couplings is performed by the communication coupling configuration subsystem during runtime operation, wherein the configuration of the plurality of communication couplings that provide the at least one communication resource between the at least one of the plurality of processing subsystems and the at least one endpoint on the first multi-endpoint adapter device is effective during the runtime operation.
18. The method of claim 14, further comprising: identifying, by the communications coupling configuration subsystem, the at least one hardware subsystem comprising the plurality of communications couplings coupling the plurality of processing subsystems to a second multi-port adapter device; retrieving, by the communication coupling configuration subsystem, at least one second user configuration policy; determining, by the communications coupling configuration subsystem, at least one second multi-point adapter device capability of the second multi-point adapter device; as well as The plurality of communication couplings are configured by the communication coupling configuration subsystem based on the at least one hardware subsystem, the at least one second user configuration policy, and the at least one second multi-endpoint adapter device capability, wherein the configuration of the plurality of communication couplings provides at least one communication resource between at least one of the plurality of processing subsystems and at least one endpoint on the second multi-endpoint adapter device.
19. The method of claim 14, wherein a baseboard management controller (BMC) device provides the communication coupling configuration engine.
20. The method of claim 14, wherein the at least one communication resource provided between the at least one of the plurality of processing subsystems and the at least one endpoint on the first multi-endpoint adapter device comprises providing redundant communication resources of each of the plurality of processing subsystems to at least two endpoints.
