Communication method, system, device, product and storage medium of computer system

Abstract

The application discloses a communication method, system, device, product and storage medium of a computer system, and relates to the computer field, and aims at solving the problem that a single-channel architecture becomes a bottleneck of system performance. The method comprises the following steps: identifying a plurality of direct memory access controllers and a plurality of non-transparent bridge devices in a first computer system; determining a current target non-transparent bridge device corresponding to each direct memory access controller in the plurality of non-transparent bridge devices; for each direct memory access controller, performing a binding operation on the direct memory access controller and the corresponding current target non-transparent bridge device, so that the direct memory access controller accesses a second computer system through the corresponding current target non-transparent bridge device. The application can disperse data transmission load to a plurality of communication links, avoid transmission of all data through the same communication link, improve the throughput of the system and reduce the possibility of a bottleneck, and improve the data processing capacity and performance of the whole system.

Description

Technical Field

[0001] This invention relates to the field of computers, and in particular to communication methods, systems, devices, products and storage media for computer systems. Background Technology

[0002] With the rapid development of computer technology, high-speed interconnection between systems has become particularly important. Non-Transparent Bridge (NTB), as a highly efficient interconnection technology, allows two independent computer systems to communicate directly, bypassing complex network protocol stacks and enabling direct memory-level access and data transfer, thus significantly improving performance. NTB technology is often used in conjunction with Direct Memory Access (DMA) technology, which allows peripheral devices to directly access system memory without CPU (Central Processing Unit) intervention, effectively reducing processor load. However, existing NTB technologies are mainly based on a single-channel architecture. In this architecture, two independent computer domains are interconnected through a single NTB link. As data transfer demands increase, the single-channel NTB link can become a bottleneck for system performance, limiting the overall system's data processing capabilities.

[0003] Therefore, how to provide a solution to the above-mentioned technical problems is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to provide a communication method, system, device, product, and storage medium for a computer system, which enables the data transmission load to be distributed across multiple communication links, thereby avoiding all data being transmitted through the same communication link, thus improving system throughput, reducing the possibility of bottlenecks, and enhancing the data processing capability and performance of the entire system.

[0005] To address the aforementioned technical problems, this invention provides a communication method for a computer system, wherein the computer system includes multiple non-transparent bridging devices, and the communication method of the computer system includes:

[0006] Identify multiple direct memory access controllers and multiple non-transparent bridging devices in the first computer system;

[0007] Among the plurality of non-transparent bridging devices, determine the current target non-transparent bridging device corresponding to each of the direct memory access controllers; at least two of the direct memory access controllers have different current target non-transparent bridging devices.

[0008] For each of the direct memory access controllers, a binding operation is performed between the direct memory access controller and the corresponding current target non-transparent bridging device, so that the direct memory access controller can access the second computer system through the corresponding current target non-transparent bridging device. The binding operation includes configuring the target address space of the direct memory access controller to the base address register address range of the corresponding current target non-transparent bridging device.

[0009] Optionally, the communication method of the computer system further includes:

[0010] For each of the current target non-transparent bridging devices, the bandwidth utilization of the current target non-transparent bridging device is monitored, and in response to the bandwidth utilization meeting the rebinding condition, a target direct memory access controller is determined from at least one of the direct memory access controllers bound to the current target non-transparent bridging device.

[0011] For each of the target direct memory access controllers, a new target device is determined among the plurality of non-transparent bridging devices, and the target direct memory access controller is rebound to the corresponding new target device so that the target direct memory access controller can access the second computer system through the corresponding new target device. The rebinding operation includes configuring the target address space of the target direct memory access controller to the base address register address range of the corresponding new target device.

[0012] Optionally, the communication method of the computer system further includes:

[0013] Determine the upper and lower limits of bandwidth utilization.

[0014] When the bandwidth utilization of the current target non-transparent bridging device is greater than the upper limit threshold of bandwidth utilization, it is determined that the bandwidth utilization meets the rebinding condition;

[0015] The process of determining a new target device for the target direct memory access controller among the plurality of said non-transparent bridging devices includes:

[0016] A new target device is determined for the target direct memory access controller among the current target non-transparent bridging devices whose bandwidth utilization is less than the lower limit threshold of bandwidth utilization.

[0017] Optionally, the process of determining a new target device for the target direct memory access controller among the plurality of said non-transparent bridging devices includes:

[0018] Determine the available bandwidth of the multiple non-transparent bridging devices;

[0019] For each of the target direct memory access controllers, determine the load transfer requirements of the target direct memory access controller, and determine a new target device for the target direct memory access controller among the non-transparent bridging devices whose available bandwidth meets the load transfer requirements.

[0020] Optionally, the communication method of the computer system includes:

[0021] Determine the minimum rebalancing time interval for the first computer system;

[0022] The process of monitoring the bandwidth utilization of the current target non-transparent bridging device includes:

[0023] Monitor the bandwidth utilization of the current target non-transparent bridging device at the minimum rebalancing time interval.

[0024] Optionally, the process of monitoring the bandwidth utilization of the current target non-transparent bridging device includes:

[0025] Obtain the transmission rate of each of the direct access controllers bound to the current target non-transparent bridging device;

[0026] The bandwidth utilization of the current target non-transparent bridging device is obtained based on the rate of each of the aforementioned transmission rates and the preset time interval.

[0027] Optionally, the process of obtaining the transmission rate of each of the direct access controllers includes:

[0028] For each of the direct access controllers, obtain the descriptor table of the direct access controller, and obtain the transmission rate of the direct access controller based on the accumulated value of the length field in the descriptor table and the preset time interval.

[0029] Optionally, the communication method of the computer system further includes:

[0030] Obtain load balancing performance data for the first computer system;

[0031] A performance report log file is generated based on the load balancing effect data and the bandwidth utilization of each of the current target non-transparent bridging devices.

[0032] Optionally, the process of generating a performance report log file based on the load balancing effect data and the bandwidth utilization of each of the current target non-transparent bridging devices includes:

[0033] A trend chart is generated from the load balancing effect data of the first computer system obtained according to a preset aggregation period and the bandwidth utilization of each of the current target non-transparent bridging devices; the trend chart represents the change of the load distribution of the first computer system over time.

[0034] A performance report log file is generated based on the trend chart.

[0035] Optionally, after identifying multiple direct memory access controllers and multiple non-transparent bridging devices in the first computer system, the communication method of the computer system includes:

[0036] A distance matrix is ​​established based on multiple direct memory access controllers and multiple non-transparent bridging devices, wherein the elements of the distance matrix are used to characterize the logical distance between the corresponding combinations of direct memory access controllers and non-transparent bridging devices;

[0037] The process of determining the current target non-transparent bridge device corresponding to each of the multiple non-transparent bridge devices includes:

[0038] For each of the direct memory access controllers, based on the distance matrix, the non-transparent bridging device with the smallest logical distance to the direct memory access controller is selected as the current target non-transparent bridging device of the direct memory access controller.

[0039] Optionally, the process of establishing a distance matrix based on the multiple direct memory access controllers and the multiple non-transparent bridging devices includes:

[0040] Obtain the communication topology of the first computer system;

[0041] For each of the direct memory access controllers, the logical distance between the direct memory access controller and each of the non-transparent bridging devices is determined according to the communication topology, and the logical distance is the number of hops between the direct memory access controller and the non-transparent bridging device;

[0042] A distance matrix is ​​established based on the combination of each direct memory access controller and non-transparent bridging device and their corresponding logical distances.

[0043] To address the aforementioned technical problems, the present invention also provides a communication system for a computer system, the computer system including multiple non-transparent bridging devices, the communication system of the computer system comprising:

[0044] An identification module is used to identify multiple direct memory access controllers and multiple non-transparent bridging devices in the first computer system;

[0045] A determination module is used to determine the current target non-transparent bridge device corresponding to each of the multiple non-transparent bridge devices; at least two of the current target non-transparent bridge devices corresponding to the direct memory access controllers are different;

[0046] A binding module is used to configure the target address space of each direct memory access controller to the base address register address range of the corresponding current target non-transparent bridging device, so that the direct memory access controller can access the second computer system through the corresponding current target non-transparent bridging device.

[0047] To address the aforementioned technical problems, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the communication method of the computer system as described in any of the preceding claims.

[0048] To address the aforementioned technical problems, the present invention also provides an electronic device, comprising:

[0049] Memory, used to store computer programs;

[0050] A processor, configured to implement the communication method of any of the preceding computer systems when executing the computer program.

[0051] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the communication method of any of the computer systems described above.

[0052] This application provides a communication method for a computer system. The computer system includes multiple non-transparent bridging devices. By binding multiple direct memory access controllers to different non-transparent bridging devices, the data transmission load is distributed across multiple communication links, thereby avoiding overload on a single communication link, improving system throughput, and reducing the possibility of bottlenecks. When multiple direct memory access controllers transmit data through different non-transparent bridging devices, the full bandwidth of the transmission link of each non-transparent bridging device can be utilized, improving the data processing capability and performance of the entire system.

[0053] This application also provides a communication system for a computer system, a computer program product, an electronic device, and a computer-readable storage medium, which have the same beneficial effects as the communication method of the aforementioned computer system. Attached Figure Description

[0054] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1A flowchart illustrating the steps of a communication method for a computer system provided by the present invention;

[0056] Figure 2 This invention provides a multi-channel hardware topology diagram of a computer system.

[0057] Figure 3 A schematic diagram illustrating the binding relationship between a non-transparent bridging device and a direct memory access controller provided by the present invention;

[0058] Figure 4 This invention provides a schematic diagram of the structure of a communication system for a computer system.

[0059] Figure 5 This is a schematic diagram of the structure of an electronic device provided by the present invention;

[0060] Figure 6 This is a schematic diagram of the structure of a computer-readable storage medium provided by the present invention. Detailed Implementation

[0061] The core of this invention is to provide a communication method, system, device, product, and storage medium for a computer system, which enables the data transmission load to be distributed across multiple communication links, thereby avoiding all data being transmitted through the same communication link, thus improving system throughput, reducing the possibility of bottlenecks, and enhancing the data processing capability and performance of the entire system.

[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Firstly, please refer to Figure 1 This invention provides a communication method for a computer system, comprising:

[0064] S101: Identify multiple direct memory access controllers and multiple non-transparent bridging devices in the first computer system;

[0065] To facilitate understanding of the solution in this embodiment, firstly based on Figure 2 , Figure 2This diagram illustrates the NTB multi-channel hardware topology between two computer systems (host domains). The computer systems in this embodiment are described, including main memory, a root complex (RC), a switch, and multiple non-transparent bridge devices (NTB devices). The two computer systems are interconnected via multiple PCIe (Peripheral Component Interconnect Express) links. NTB devices are located at both ends of each PCIe link, used for performing operations such as local memory mapping and address translation. Specifically, the NTB devices are mounted on the root complex (RC), which is directly connected to the main memory. This configuration allows the NTB devices to communicate with each other via the PCIe links between the two NTB devices through the host's internal bus.

[0066] In this embodiment, the accessing party in the two computer systems is identified as the first computer system, and the accessed party is identified as the second computer system.

[0067] During system startup, the DMA-NTB binding initialization kernel module is run. This module is responsible for identifying all DMA controllers (i.e., direct memory access controllers) and all NTB devices in the first computer system, and reading the PCIe link status register to obtain the link width and speed information of each NTB device so as to understand the transmission capacity of each NTB transmission link later. It also reads the BAR (Base Address Register) configuration of each NTB device and records the BAR address range and attributes. The BAR address is the key information for NTB devices to perform memory mapping. Recording the above information is necessary to correctly configure the target address space of the DMA controller later.

[0068] S102: Determine the current target non-transparent bridge device corresponding to each direct memory access controller among multiple non-transparent bridge devices; at least two direct memory access controllers have different current target non-transparent bridge devices.

[0069] In this embodiment, for each DMA controller, its corresponding current target non-transparent bridging device is determined from among multiple non-transparent bridging devices. The DMA controllers and non-transparent bridging devices can have a one-to-one correspondence, reducing contention and latency, or a many-to-one relationship, where multiple DMA controllers correspond to the same non-transparent bridging device. This configuration is suitable for scenarios with low data traffic or low real-time requirements, saving hardware resources. When determining the corresponding current target non-transparent bridging device for each DMA controller, at least two DMA controllers must have different current target non-transparent bridging devices. This is to distribute data flow, preventing all DMA controllers from transmitting data through the same non-transparent bridging device, thereby reducing the load on individual non-transparent bridging devices and improving the overall system throughput.

[0070] It is understandable that by ensuring that there are at least two DMA controllers corresponding to different NTB devices, the system can more effectively utilize the multi-channel capabilities provided by multiple NTB devices, thereby improving data processing capabilities and reducing the risk of a single NTB link becoming a performance bottleneck.

[0071] S103: For each direct memory access controller, perform a binding operation between the direct memory access controller and the corresponding current target non-transparent bridge device, so that the direct memory access controller can access the second computer system through the corresponding current target non-transparent bridge device. The binding operation includes configuring the target address space of the direct memory access controller to the base address register address range of the corresponding current target non-transparent bridge device.

[0072] In this embodiment, after determining the current target non-transparent bridge device to be bound to the DMA controller, it is necessary to bind the DMA controller and the current target non-transparent bridge device. The binding operation includes configuring the target address space of the DMA controller, that is, setting the target address space of the DMA to the base address register (BAR) address range of the corresponding NTB device, that is, mapping the target address space of the DMA to the BAR address range of the NTB device. After the configuration is completed, the binding of the DMA and the NTB device is realized. When the DMA performs a memory access operation, it is actually accessing the memory of the second computer system.

[0073] Furthermore, this embodiment will also configure the necessary address translation logic to ensure that the address generated by the DMA controller can be correctly translated into a valid address in the second computer system.

[0074] After configuring the target address space and address translation, the DMA controller will be enabled, allowing it to begin transferring data to the second computer system. It can also monitor the performance of DMA transfers and adjust binding relationships and address space configurations as needed to optimize performance and respond to system changes.

[0075] Through the above steps, the DMA controller can directly access the memory of the second computer system, thereby achieving efficient data transfer. This binding operation reduces CPU intervention, lowers latency, increases data transfer rate, and enables the system to better utilize the multi-channel capabilities provided by multiple NTB devices, avoiding the performance bottleneck that a single NTB link might become.

[0076] As can be seen, in this embodiment, the computer system includes multiple non-transparent bridging devices. By binding multiple direct memory access controllers to different non-transparent bridging devices, the data transmission load is distributed across multiple communication links, thereby avoiding overload on a single communication link, improving system throughput and reducing the possibility of bottlenecks. When multiple direct memory access controllers transmit data through different non-transparent bridging devices, the full bandwidth of the transmission link of each non-transparent bridging device can be utilized, improving the data processing capability and performance of the entire system.

[0077] Based on the above embodiments:

[0078] In one exemplary embodiment, the communication method of the computer system further includes:

[0079] For each current target non-transparent bridging device, monitor the bandwidth utilization of the current target non-transparent bridging device, and in response to the bandwidth utilization meeting the rebinding condition, determine the target direct memory access controller among at least one direct memory access controller bound to the current target non-transparent bridging device.

[0080] For each target direct memory access controller, a new target device is determined among multiple non-transparent bridging devices. The target direct memory access controller is then rebound to the corresponding new target device so that the target direct memory access controller can access the second computer system through the corresponding new target device. The rebinding operation includes configuring the target address space of the target direct memory access controller to the base address register address range of the corresponding new target device.

[0081] In this embodiment, the bandwidth utilization of each NTB device bound to the DMA controller is monitored. This can be achieved by analyzing the status registers of the NTB devices or through network management tools. The purpose of monitoring is to determine when the bandwidth utilization reaches a threshold, requiring rebinding to optimize performance.

[0082] Specifically, the rebinding process is triggered when bandwidth utilization meets preset rebinding conditions. These conditions may include high bandwidth utilization, increased data transmission latency, or other performance metrics. Among at least one DMA controller bound to the current target non-transparent bridging device, a target direct memory access controller is determined, including but not limited to DMA controllers significantly affected by bandwidth limitations. For each target direct memory access controller, a new target device is determined from among multiple NTB devices. The new target device differs from the current target non-transparent bridging device; specifically, the optimal new target device is selected based on the performance and load of each NTB device. The target direct memory access controller is then rebinded to the new NTB device, including configuring the target address space of the target direct memory access controller to the corresponding base address register (BAR) address range of the new NTB device, i.e., updating the DMA controller's configuration to point to the BAR address range of the new NTB device.

[0083] Through the rebinding operation described above, the target direct memory access controller can access the second computer system through the new NTB device, potentially reducing data transfer latency and increasing data transfer rates. The new binding relationship and bandwidth utilization will continue to be monitored to ensure data transfer remains optimal. Adjustments will be made again if necessary to accommodate evolving data transfer requirements.

[0084] In this way, the binding relationship between the DMA controller and the NTB device can be dynamically adjusted to respond to changes in bandwidth utilization and optimize data transmission performance, which helps to improve the system's flexibility and responsiveness and ensures that efficient data transmission can still be maintained under high load conditions.

[0085] In one exemplary embodiment, the communication method of the computer system further includes:

[0086] Determine the upper and lower limits of bandwidth utilization.

[0087] When the bandwidth utilization of the current target non-transparent bridging device is greater than the upper limit threshold of bandwidth utilization, it is determined that the bandwidth utilization meets the rebinding condition.

[0088] The process of determining a new target device for a target direct memory access controller among multiple non-transparent bridging devices includes:

[0089] In the current target non-transparent bridging device where the bandwidth utilization is less than the lower limit threshold, a new target device is determined for the target direct memory access controller.

[0090] In this embodiment, an upper and lower threshold for bandwidth utilization are first set, for example, 80% and 20%. These thresholds identify which NTB devices are under high or low load. When bandwidth utilization exceeds the upper threshold, rebinding is deemed to be met to alleviate the burden on overloaded devices. For DMA controllers bound to overloaded NTB devices, NTB devices with bandwidth utilization below the lower threshold are identified. New target devices are then reassigned to the overloaded DMA controller. Underutilized NTB devices bound to other DMA controllers in the first computer system are used as alternative NTB devices for the overloaded DMA controller, allowing for more efficient use of existing resources and improved overall resource utilization. Furthermore, considering that the bound NTB devices have already undergone initial configuration, only the existing binding relationships need to be adjusted, reducing configuration complexity. Under high load conditions, rapid reassignment of bound NTB devices can quickly alleviate the pressure on the overloaded DMA controller without waiting for new NTB devices to initialize and configure.

[0091] Through the above steps, the system can effectively manage data transmission between the DMA controller and the NTB device, optimize bandwidth utilization, and improve overall system performance.

[0092] In one exemplary embodiment, the process of determining a new target device for a target direct memory access controller among a plurality of non-transparent bridging devices includes:

[0093] Determine the available bandwidth for multiple non-transparent bridging devices;

[0094] For each target direct memory access controller, determine the load transfer requirements of the target direct memory access controller, and identify a new target device for the target direct memory access controller among non-transparent bridging devices with available bandwidth that meet the load transfer requirements.

[0095] In this embodiment, the available bandwidth of each NTB device can also be obtained, and a new target device can be selected for the overloaded DMA controller based on this, which helps to achieve a more balanced load distribution, thereby reducing the performance bottleneck caused by overload of a single NTB device.

[0096] In one exemplary embodiment, the communication method of the computer system includes:

[0097] Determine the minimum rebalancing time interval for the first computer system;

[0098] The process of monitoring the bandwidth utilization of the current target non-transparent bridging device includes:

[0099] Monitor the bandwidth utilization of the current target non-transparent bridging device at the minimum rebalancing interval.

[0100] In this embodiment, the minimum rebalancing time interval is a time threshold set in the system to avoid triggering excessive rebalancing operations due to frequent load changes. Within this time interval, even if the system load changes, the system will not perform rebalancing operations. Setting this interval can reduce the system overhead and performance impact caused by frequent rebalancing. By setting a minimum rebalancing time interval and monitoring bandwidth utilization accordingly, the resource allocation of the computer system can be effectively managed and optimized while maintaining system stability.

[0101] In one exemplary embodiment, the process of monitoring the bandwidth utilization of the current target non-transparent bridging device includes:

[0102] Obtain the transfer rate of each direct access controller bound to the current target non-transparent bridging device;

[0103] The bandwidth utilization of the current target non-transparent bridging device is obtained based on the rate of each transmission rate and the preset time interval.

[0104] In this embodiment, the ratio of the transmission rate of each direct access controller bound to the current target non-transparent bridging device to a preset time interval is used as the bandwidth utilization of the current target non-transparent bridging device. It can be understood that the transmission rate of DMA control represents the DMA controller's ability to transmit data within a certain time period. The specific calculation formula is: Bandwidth utilization = (Sum of rates / Preset time interval).

[0105] This embodiment summarizes how to effectively monitor and calculate the bandwidth utilization of the current target non-transparent bridging device, thereby providing data support for system performance optimization and resource management, helping to ensure the efficient use of network resources, and timely identifying and resolving potential performance bottlenecks.

[0106] In one exemplary embodiment, the process of obtaining the transmission rate of each direct access controller includes:

[0107] For each direct access controller, obtain the descriptor table of the direct access controller, and obtain the transmission rate of the direct access controller based on the accumulated value of the length field in the descriptor table and the preset time interval.

[0108] In this embodiment, for each DMA controller, its descriptor table is first obtained. The descriptor table is a data structure used to store transmission information (including but not limited to target address space, length, etc.) in DMA transmission. The value of the length field in the descriptor table is extracted as the data volume and accumulated. That is, the descriptor list of the DMA controller is obtained according to a preset scan cycle. When the status flag of the descriptor is incomplete, the value of the length field is ensured to be within a reasonable range. The values ​​of the length field extracted in multiple scan cycles are accumulated to obtain the accumulated value. The ratio of the accumulated value to the preset time interval is used as the transmission rate of the DMA controller.

[0109] The accumulation process is described as follows: A cumulative data volume variable and the current time are initialized, with an initial value of 0. The extracted length field value is added to the cumulative data volume variable, and the relevant timestamp information is updated. The completion time of the data volume calculation is recorded. When calculating the DMA transfer rate, the cumulative data volume at the end is subtracted from the cumulative data volume at the beginning to obtain the data transfer volume within a preset time interval. The data transfer volume is divided by the length of the preset time interval to obtain the average transfer rate. The calculated rate is then converted to an appropriate unit (such as MB / s or Gb / s).

[0110] Before performing this step, the following operations are also included: When the system starts, the initialization program of the traffic monitoring module allocates the necessary memory space to store monitoring data and intermediate calculation results, initializes counters and timestamp variables, which will be used to track the amount and time of data transmission, obtains the memory location of the descriptor table through the corresponding DMA driver, verifies the validity of the obtained address to ensure that it has appropriate read permissions, and configures monitoring parameters, including defining the scanning frequency of the descriptor table, determining the time interval of traffic calculation, and configuring the frequency and format of report generation.

[0111] In this embodiment, the transmission rate of each DMA controller can be accurately monitored and calculated, which facilitates network management and performance optimization, and ensures the efficiency and reliability of data transmission.

[0112] In one exemplary embodiment, the communication method of the computer system further includes:

[0113] Obtain load balancing performance data for the first computer system;

[0114] Performance report log files are generated based on load balancing performance data and bandwidth utilization of each current target non-transparent bridging device. Specifically, load balancing performance data of the first computer system, which can be obtained according to a preset aggregation period, and bandwidth utilization of each current target non-transparent bridging device are used to generate trend charts. These trend charts represent the changes in load distribution of the first computer system over time, and performance report log files are generated based on these trend charts.

[0115] In this embodiment, during the communication process between the first computer system and the second computer system, the load balancing effect data of the first computer system is recorded, and the load balancing effect data and the bandwidth utilization of the NTB device are periodically summarized to generate a trend chart. The trend chart represents the change of the load distribution of the first computer system over time, so as to effectively monitor the load balancing effect of the first computer system and evaluate and optimize system performance.

[0116] In one exemplary embodiment, after identifying a plurality of direct memory access controllers and a plurality of non-transparent bridging devices in a first computer system, the communication method of the computer system includes:

[0117] A distance matrix is ​​established based on multiple direct memory access controllers and multiple non-transparent bridging devices. The elements of the distance matrix are used to characterize the logical distance between the corresponding combinations of direct memory access controllers and non-transparent bridging devices.

[0118] The process of determining the current target non-transparent bridge device corresponding to each direct memory access controller among multiple non-transparent bridge devices includes:

[0119] For each direct memory access controller, based on the distance matrix, the non-transparent bridging device with the smallest logical distance to the direct memory access controller is selected as the current target non-transparent bridging device of the direct memory access controller.

[0120] In this embodiment, a distance matrix is ​​first established based on multiple direct memory access controllers and multiple non-transparent bridging devices. For example, the rows of the distance matrix can be DMA controllers, and the columns can be NTB devices. The element a in this distance matrix... ij This represents the logical distance between the i-th DMA controller and the j-th NTB device. When selecting the initial binding relationship, for each DMA controller, the NTB device with the shortest logical distance is selected as the current target non-transparent bridging device for binding, as shown in the reference. Figure 3 As shown, the initial binding relationship can then be recorded in the binding table. It's understandable that a shorter logical distance means fewer network devices traversed during data transmission between the DMA controller and the NTB device, ensuring an optimized data transmission path, thereby reducing data transmission latency and improving system response speed. Furthermore, this proximity-based binding mechanism reduces the number of network devices along the data transmission path between the DMA controller and the bound NTB device, resulting in a relatively smaller amount of data that each device needs to process, thus reducing the possibility of network congestion and improving overall network bandwidth utilization.

[0121] In one exemplary embodiment, the process of establishing a distance matrix based on multiple direct memory access controllers and multiple non-transparent bridging devices includes:

[0122] Obtain the communication topology of the first computer system;

[0123] For each direct memory access controller, the logical distance between the direct memory access controller and each non-transparent bridging device is determined according to the communication topology. The logical distance is the number of hops between the direct memory access controller and the non-transparent bridging device.

[0124] A distance matrix is ​​established based on the combination of each direct memory access controller and non-transparent bridging device and their corresponding logical distances.

[0125] In this embodiment, system management tools or the PCIe enumeration API can be used to identify all PCIe devices in the system, including DMA controllers and NTB devices. Based on the enumerated information, a device tree representing the PCIe topology is constructed. Nodes in the device tree represent PCIe devices, and edges represent PCIe links. All DMA controllers and NTB devices are identified in the PCIe device tree.

[0126] For each DMA controller, a tree traversal algorithm (such as Depth-First Search, DFS) is used to calculate the PCIe switch hop count to reach each NTB device. The hop count refers to the number of switches traversed in the PCIe topology from the DMA controller to the NTB device. The PCIe switch hop count from each DMA controller to each NTB device is recorded. A matrix is ​​created where rows represent DMA controllers and columns represent NTB devices, with the matrix dimension equal to the number of DMA controllers multiplied by the number of NTB devices. Based on the recorded PCIe switch hop counts, the distance from each DMA controller to each NTB device is entered into the corresponding position in the matrix. If the hop count between DMA controller i and NTB device j is n, then n is entered in the i-th row and j-th column of the matrix. If there is no direct PCIe connection between a DMA controller and an NTB device, the corresponding element in the matrix can be marked as infinity or a preset high value, indicating unreachability. The generated DMA-NTB distance matrix is ​​stored in the system's memory or disk for use in subsequent binding decision processes.

[0127] Through the above steps, the PCIe topology can be effectively analyzed, logical distances calculated, and a distance matrix generated to optimize the binding relationship between the DMA controller and NTB devices. This matrix provides system administrators or automation scripts with a clear view to select the optimal DMA-NTB binding strategy. In summary, this invention significantly improves data transmission bandwidth between systems by implementing multiple parallel PCIe links. The multi-channel architecture can simultaneously support multiple DMA transmission operations, effectively alleviating the bandwidth bottleneck caused by a single channel. The multi-channel design provides the system with greater flexibility, adapting to more complex network topologies. A dynamic load balancing algorithm is introduced to achieve intelligent allocation of NTB channels and DMA resources. Based on real-time monitoring data, the system can automatically adjust resource allocation strategies to maximize resource utilization. Under high load conditions, the load balancing mechanism can distribute tasks across multiple channels, avoiding overload of a single channel and improving overall throughput. Intelligent binding between the DMA controller and NTB devices is achieved, ensuring optimal data transmission paths. By allocating DMA resources based on proximity, data transmission latency is significantly reduced.

[0128] Secondly, please refer to Figure 4 The present invention also provides a communication system for a computer system, the computer system including multiple non-transparent bridging devices, the communication system of the computer system including:

[0129] Identification module 11 is used to identify multiple direct memory access controllers and multiple non-transparent bridging devices in the first computer system;

[0130] The determination module 12 is used to determine the current target non-transparent bridge device corresponding to each direct memory access controller among multiple non-transparent bridge devices; at least two direct memory access controllers have different current target non-transparent bridge devices.

[0131] Binding module 13 is used to configure the target address space of each direct memory access controller to the base address register address range of the corresponding current target non-transparent bridge device, so that the direct memory access controller can access the second computer system through the corresponding current target non-transparent bridge device.

[0132] For a description of the communication system of the computer system provided by the present invention, please refer to the above embodiments; the present invention will not be described again here.

[0133] The communication system of the computer system provided by the present invention has the same beneficial effects as the communication method of the computer system described above.

[0134] Thirdly, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the communication method of any of the computer systems described above.

[0135] For a description of the communication system of the computer program product provided by the present invention, please refer to the above embodiments; the present invention will not be described again here.

[0136] The communication system for a computer program product provided by this invention has the same beneficial effects as the communication method of the aforementioned computer system.

[0137] Fourthly, please refer to Figure 5 The present invention also provides an electronic device, comprising:

[0138] Memory 21 is used to store computer programs;

[0139] The processor 22 is configured to implement the steps of the communication method of the computer system described in any of the above embodiments when executing a computer program.

[0140] The electronic device also includes:

[0141] Input interface 23, connected to processor 22 via communication bus 26, is used to acquire externally imported computer programs, parameters, and instructions, and save them to memory 21 under the control of processor 22. This input interface can be connected to an input device to receive parameters or instructions manually entered by the user. This input device can be a touch layer covering the display screen, or buttons, a trackball, or a touchpad mounted on the terminal casing.

[0142] Display unit 24 is connected to processor 22 via communication bus 26 and is used to display data sent by processor 22. This display unit can be a liquid crystal display screen or an electronic ink display screen, etc.

[0143] Network port 25 is connected to processor 22 via communication bus 26 and is used for communication with external terminal devices. The communication technology used for this connection can be wired or wireless communication technology, such as mobile high-definition link technology, universal serial bus, high-definition multimedia interface, wireless fidelity technology, Bluetooth communication technology, Bluetooth low power communication technology, and communication technology based on IEEE 802.11s.

[0144] For a description of the electronic device provided by the present invention, please refer to the above embodiments; the present invention will not be described again here.

[0145] The electronic device provided by this invention has the same beneficial effects as the communication method of the computer system described above.

[0146] Fifthly, please refer to Figure 6 The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the communication methods of the computer system described above.

[0147] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0148] For a description of the computer-readable storage medium provided by the present invention, please refer to the above embodiments; the present invention will not be described again here.

[0149] The computer-readable storage medium provided by the present invention has the same beneficial effects as the communication method of the computer system described above.

[0150] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0151] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A communication method for a computer system, characterized in that, The computer system includes multiple non-transparent bridging devices, and the communication method of the computer system includes: Identify multiple direct memory access controllers and multiple non-transparent bridging devices in the first computer system; Among the plurality of non-transparent bridging devices, determine the current target non-transparent bridging device corresponding to each of the direct memory access controllers; at least two of the direct memory access controllers have different current target non-transparent bridging devices. For each of the direct memory access controllers, a binding operation is performed between the direct memory access controller and the corresponding current target non-transparent bridging device, so that the direct memory access controller can access the second computer system through the corresponding current target non-transparent bridging device. The binding operation includes configuring the target address space of the direct memory access controller to the base address register address range of the corresponding current target non-transparent bridging device. The communication method of the computer system further includes: For each of the current target non-transparent bridging devices, the bandwidth utilization of the current target non-transparent bridging device is monitored, and in response to the bandwidth utilization meeting the rebinding condition, a target direct memory access controller is determined from at least one of the direct memory access controllers bound to the current target non-transparent bridging device. For each of the target direct memory access controllers, a new target device is determined among the plurality of non-transparent bridging devices, and the target direct memory access controller is rebound to the corresponding new target device so that the target direct memory access controller can access the second computer system through the corresponding new target device. The rebinding operation includes configuring the target address space of the target direct memory access controller to the base address register address range of the corresponding new target device.

2. The communication method of the computer system according to claim 1, characterized in that, The communication method of the computer system further includes: Determine the upper and lower limits of bandwidth utilization. When the bandwidth utilization of the current target non-transparent bridging device is greater than the upper limit threshold of bandwidth utilization, it is determined that the bandwidth utilization meets the rebinding condition; The process of determining a new target device for the target direct memory access controller among the plurality of said non-transparent bridging devices includes: A new target device is determined for the target direct memory access controller among the current target non-transparent bridging devices whose bandwidth utilization is less than the lower limit threshold of bandwidth utilization.

3. The communication method of the computer system according to claim 1, characterized in that, The process of determining a new target device for the target direct memory access controller among the plurality of said non-transparent bridging devices includes: Determine the available bandwidth of the multiple non-transparent bridging devices; For each of the target direct memory access controllers, determine the load transfer requirements of the target direct memory access controller, and determine a new target device for the target direct memory access controller among the non-transparent bridging devices whose available bandwidth meets the load transfer requirements.

4. The communication method of the computer system according to claim 1, characterized in that, The communication method of the computer system includes: Determine the minimum rebalancing time interval for the first computer system; The process of monitoring the bandwidth utilization of the current target non-transparent bridging device includes: Monitor the bandwidth utilization of the current target non-transparent bridging device at the minimum rebalancing time interval.

5. The communication method of the computer system according to claim 1, characterized in that, The process of monitoring the bandwidth utilization of the current target non-transparent bridging device includes: Obtain the transfer rate of each of the direct memory access controllers bound to the current target non-transparent bridging device; The bandwidth utilization of the current target non-transparent bridging device is obtained based on the rate of each of the aforementioned transmission rates and the preset time interval.

6. The communication method of the computer system according to claim 5, characterized in that, The process of obtaining the transfer rate of each of the aforementioned direct memory access controllers includes: For each of the direct memory access controllers, the descriptor table of the direct memory access controller is obtained, and the transmission rate of the direct memory access controller is obtained based on the accumulated value of the length field in the descriptor table and the preset time interval.

7. The communication method of the computer system according to claim 1, characterized in that, The communication method of the computer system further includes: Obtain load balancing performance data for the first computer system; A performance report log file is generated based on the load balancing effect data and the bandwidth utilization of each of the current target non-transparent bridging devices.

8. The communication method of the computer system according to claim 7, characterized in that, The process of generating a performance report log file based on the load balancing effect data and the bandwidth utilization of each of the current target non-transparent bridging devices includes: A trend chart is generated from the load balancing effect data of the first computer system obtained according to a preset aggregation period and the bandwidth utilization of each of the current target non-transparent bridging devices; the trend chart represents the change of the load distribution of the first computer system over time. A performance report log file is generated based on the trend chart.

9. The communication method of the computer system according to any one of claims 1-8, characterized in that, After identifying multiple direct memory access controllers and multiple non-transparent bridging devices in the first computer system, the communication method of the computer system includes: A distance matrix is ​​established based on multiple direct memory access controllers and multiple non-transparent bridging devices, wherein the elements of the distance matrix are used to characterize the logical distance between the corresponding combinations of direct memory access controllers and non-transparent bridging devices; The process of determining the current target non-transparent bridge device corresponding to each of the multiple non-transparent bridge devices includes: For each of the direct memory access controllers, based on the distance matrix, the non-transparent bridging device with the smallest logical distance to the direct memory access controller is selected as the current target non-transparent bridging device of the direct memory access controller.

10. The communication method of the computer system according to claim 9, characterized in that, The process of establishing a distance matrix based on the multiple direct memory access controllers and the multiple non-transparent bridging devices includes: Obtain the communication topology of the first computer system; For each of the direct memory access controllers, the logical distance between the direct memory access controller and each of the non-transparent bridging devices is determined according to the communication topology, and the logical distance is the number of hops between the direct memory access controller and the non-transparent bridging device; A distance matrix is ​​established based on the combination of each direct memory access controller and non-transparent bridging device and their corresponding logical distances.

11. A communication system for a computer system, characterized in that, The computer system includes multiple non-transparent bridging devices, and the communication system of the computer system includes: An identification module is used to identify multiple direct memory access controllers and multiple non-transparent bridging devices in the first computer system; A determination module is used to determine the current target non-transparent bridge device corresponding to each of the multiple non-transparent bridge devices; at least two of the current target non-transparent bridge devices corresponding to the direct memory access controllers are different; A binding module is used to configure the target address space of each direct memory access controller to the base address register address range of the corresponding current target non-transparent bridging device, so that the direct memory access controller can access the second computer system through the corresponding current target non-transparent bridging device. The communication system of the computer system is also used for: For each of the current target non-transparent bridging devices, the bandwidth utilization of the current target non-transparent bridging device is monitored, and in response to the bandwidth utilization meeting the rebinding condition, a target direct memory access controller is determined from at least one of the direct memory access controllers bound to the current target non-transparent bridging device. For each of the target direct memory access controllers, a new target device is determined among the plurality of non-transparent bridging devices, and the target direct memory access controller is rebound to the corresponding new target device so that the target direct memory access controller can access the second computer system through the corresponding new target device. The rebinding operation includes configuring the target address space of the target direct memory access controller to the base address register address range of the corresponding new target device.

12. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the communication method of the computer system as described in any one of claims 1-10.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the communication method of the computer system according to any one of claims 1-10 when executing the computer program.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the communication method of the computer system according to any one of claims 1-10.

Images