Communication method, system and device of computer system, product and storage medium

Abstract

The invention discloses a communication method and system of a computer system, equipment, a product and a storage medium, relates to the field of computers, and aims to solve the problem that a single-channel architecture can become a bottleneck of system performance, the method comprises the following steps: identifying a plurality of direct memory access controllers and a plurality of non-transparent bridging equipment in a first computer system; determining a current target non-transparent bridging device corresponding to each direct memory access controller in the plurality of non-transparent bridging devices; and for each direct memory access controller, carrying out binding operation on the direct memory access controller and the corresponding current target non-transparent bridging equipment, so that the direct memory access controller accesses the second computer system through the corresponding current target non-transparent bridging equipment. According to the invention, the data transmission load can be dispersed to a plurality of communication links, and all data are prevented from being transmitted through the same communication link, so that the throughput of the system is improved, the possibility of bottleneck is reduced, and the data processing capability and performance of the whole system are improved.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a communication method, system, device, product and storage medium of a computer system. Background Art

[0002] With the rapid development of computer technology, high-speed interconnection between systems has become particularly important. Non-Transparent Bridge (NTB), as an efficient interconnection technology, allows two independent computer systems to communicate directly, bypassing the complex network protocol stack, and achieves direct memory-level access and data transmission, thereby significantly improving performance. NTB technology is often used in conjunction with Direct Memory Access (DMA) technology, which allows peripheral devices to directly access system memory without CPU (Central Processing Unit) intervention, effectively reducing processor load. However, existing NTB technology is mainly based on a single-channel architecture, in which two independent computer domains are interconnected through an NTB link. As data transmission requirements grow, the single-channel NTB link may become a bottleneck for system performance, limiting the data processing capabilities of the entire system.

[0003] Therefore, how to provide a solution to the above technical problems is a problem that those skilled in the art need to solve at present. Summary of the invention

[0004] The purpose of the present invention is to provide a communication method, system, device, product and storage medium for a computer system, which can disperse the data transmission load to multiple communication links, thereby avoiding that all data is transmitted through the same communication link, thereby improving the system throughput, reducing the possibility of bottlenecks, and improving the data processing capability and performance of the entire system.

[0005] In order to solve the above technical problems, the present invention provides a communication method of a computer system, wherein the computer system includes a plurality of non-transparent bridge devices, and the communication method of the computer system includes:

[0006] identifying a plurality of direct memory access controllers and a plurality of said non-transparent bridge devices in a first computer system;

[0007] Determining a current target non-transparent bridge device corresponding to each of the direct memory access controllers among the plurality of non-transparent bridge devices; the current target non-transparent bridge devices corresponding to at least two of the direct memory access controllers are different;

[0008] For each of the direct memory access controllers, a binding operation is performed on the direct memory access controller and the corresponding current target non-transparent bridge device so that the direct memory access controller accesses the second computer system through the corresponding current target non-transparent bridge device, and the binding operation includes configuring the target address space of the direct memory access controller to be the base address register address range of the corresponding current target non-transparent bridge device.

[0009] Optionally, the communication method of the computer system further includes:

[0010] For each of the current target non-transparent bridge devices, monitor the bandwidth utilization of the current target non-transparent bridge device, and in response to the bandwidth utilization satisfying a rebinding condition, determine a target direct memory access controller from at least one direct memory access controller bound to the current target non-transparent bridge device;

[0011] For each of the target direct memory access controllers, a new target device is determined for the target direct memory access controller in the plurality of the non-transparent bridge devices, and a rebinding operation is performed on the target direct memory access controller and the corresponding new target device so that the target direct memory access controller accesses the second computer system through the corresponding new target device, wherein the rebinding operation includes configuring the target address space of the target direct memory access controller to be the base address register address range of the corresponding new target device.

[0012] Optionally, the communication method of the computer system further includes:

[0013] Determine an upper bandwidth utilization threshold and a lower bandwidth utilization threshold;

[0014] When the bandwidth utilization of the current target non-transparent bridge device is greater than the bandwidth utilization upper limit threshold, determining that the bandwidth utilization meets the rebinding condition;

[0015] The process of determining a new target device for the target direct memory access controller in the plurality of non-transparent bridge devices comprises:

[0016] A new target device is determined for the target direct memory access controller from among the current target non-transparent bridge devices whose bandwidth utilization is less than the bandwidth utilization lower limit threshold.

[0017] Optionally, the process of determining a new target device for the target direct memory access controller in the plurality of non-transparent bridge devices includes:

[0018] determining available bandwidth of a plurality of said non-transparent bridging devices;

[0019] For each of the target direct memory access controllers, a load transmission requirement of the target direct memory access controller is determined, and a new target device is determined for the target direct memory access controller in the non-transparent bridge device whose available bandwidth meets the load transmission requirement.

[0020] Optionally, the communication method of the computer system includes:

[0021] determining a minimum rebalancing time interval for the first computer system;

[0022] The process of monitoring the bandwidth utilization of the current target non-transparent bridge device includes:

[0023] The bandwidth utilization of the current target non-transparent bridge device is monitored according to the minimum rebalancing time interval.

[0024] Optionally, the process of monitoring the bandwidth utilization of the current target non-transparent bridge device includes:

[0025] Acquire the transmission rate of each direct access controller bound to the current target non-transparent bridge device;

[0026] The bandwidth utilization of the current target non-transparent bridge device is obtained based on the rate of each transmission rate and the preset time interval.

[0027] Optionally, the process of acquiring the transmission rate of each direct access controller includes:

[0028] For each of the direct access controllers, a descriptor table of the direct access controller is obtained, and a transmission rate of the direct access controller is obtained based on an accumulated value of a length field in the descriptor table and the preset time interval.

[0029] Optionally, the communication method of the computer system further includes:

[0030] Obtaining load balancing effect data of the first computer system;

[0031] A performance report log file is generated based on the load balancing effect data and the bandwidth utilization of each of the current target non-transparent bridge devices.

[0032] Optionally, the process of generating a performance report log file based on the load balancing effect data and the bandwidth utilization of each of the current target non-transparent bridging devices includes:

[0033] Generate a trend graph based on the load balancing effect data of the first computer system and the bandwidth utilization rate of each of the current target non-transparent bridge devices acquired according to a preset summary period; the trend graph represents the change of the load distribution of the first computer system over time;

[0034] A performance report log file is generated based on the trend graph.

[0035] Optionally, after identifying a plurality of direct memory access controllers and a plurality of non-transparent bridge devices in the first computer system, the communication method of the computer system includes:

[0036] Establishing a distance matrix based on the plurality of direct memory access controllers and the plurality of non-transparent bridge devices, wherein the elements of the distance matrix are used to represent the logical distances between the corresponding combinations of the direct memory access controllers and the non-transparent bridge devices;

[0037] The process of determining the current target non-transparent bridge device corresponding to each direct memory access controller among the plurality of non-transparent bridge devices comprises:

[0038] For each of the direct memory access controllers, based on the distance matrix, a non-transparent bridge device with the smallest logical distance to the direct memory access controller is selected as a current target non-transparent bridge device of the direct memory access controller.

[0039] Optionally, the process of establishing a distance matrix based on the plurality of direct memory access controllers and the plurality of non-transparent bridge devices includes:

[0040] Acquire a communication topology structure of the first computer system;

[0041] For each of the direct memory access controllers, determining a logical distance between the direct memory access controller and each of the non-transparent bridge devices according to the communication topology structure, wherein the logical distance is the number of hops between the direct memory access controller and the non-transparent bridge device;

[0042] A distance matrix is ​​established based on each DMA controller and non-transparent bridge device combination and their corresponding logical distances.

[0043] In order to solve the above technical problems, the present invention further provides a communication system of a computer system, wherein the computer system includes a plurality of non-transparent bridge devices, and the communication system of the computer system includes:

[0044] An identification module, configured to identify a plurality of direct memory access controllers and a plurality of the non-transparent bridge devices in a first computer system;

[0045] A determination module, used for determining a current target non-transparent bridge device corresponding to each of the direct memory access controllers among the plurality of the non-transparent bridge devices; the current target non-transparent bridge devices corresponding to at least two of the direct memory access controllers are different;

[0046] The binding module is used to configure the target address space of each direct memory access controller to the base address register address range of the corresponding current target non-transparent bridge device, so that the direct memory access controller can access the second computer system through the corresponding current target non-transparent bridge device.

[0047] To solve the above technical problem, the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the communication method of the computer system as described in any one of the above items.

[0048] In order to solve the above technical problems, the present invention further provides an electronic device, comprising:

[0049] Memory for storing computer programs;

[0050] A processor is used to implement the steps of the communication method of the computer system described in any one of the above when executing the computer program.

[0051] To solve the above technical problems, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the communication method of the computer system described in any one of the above items are implemented.

[0052] The present application provides a communication method for a computer system, wherein the computer system includes multiple non-transparent bridge devices. By binding multiple direct memory access controllers to different non-transparent bridge devices, the data transmission load is dispersed to multiple communication links, thereby avoiding the overload of a single communication link, improving the system throughput and reducing the possibility of bottlenecks. When multiple direct memory access controllers transmit data through different non-transparent bridge devices, the full bandwidth of the transmission link of each non-transparent bridge device can be utilized, thereby improving the data processing capability and performance of the entire system.

[0053] The present application also provides a communication system of a computer system, a computer program product, an electronic device and a computer-readable storage medium, which have the same beneficial effects as the communication method of the computer system described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1A flow chart of the steps of a communication method of a computer system provided by the present invention;

[0056] Figure 2 A multi-channel hardware topology diagram of a computer system provided by the present invention;

[0057] Figure 3 A schematic diagram of the binding relationship between a non-transparent bridge device and a direct memory access controller provided by the present invention;

[0058] Figure 4 A schematic diagram of the structure of a communication system of a computer system provided by the present invention;

[0059] Figure 5 A schematic diagram of the structure of an electronic device provided by the present invention;

[0060] Figure 6 This is a schematic diagram of the structure of a computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0061] The core of the present invention is to provide a communication method, system, device, product and storage medium for a computer system, which can disperse the data transmission load to multiple communication links, thereby avoiding all data from being transmitted through the same communication link, thereby improving the system throughput, reducing the possibility of bottlenecks, and improving the data processing capability and performance of the entire system.

[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] First, please refer to Figure 1 The present invention provides a communication method of a computer system, comprising:

[0064] S101: Identify multiple direct memory access controllers and multiple non-transparent bridge devices in a first computer system;

[0065] To facilitate understanding of the solution of this embodiment, firstly based on Figure 2 , Figure 2The NTB multi-channel hardware topology diagram between two computer systems (host domains) is shown, and the computer system in this embodiment is explained. The computer system includes a device main memory, an RC (Root Complex), a switch, and multiple non-transparent bridge devices (i.e., NTB devices). The two computer systems are interconnected through multiple PCIe (Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard) links. NTB devices are respectively set at both ends of each PCIe link to perform operations such as local memory mapping and address translation. Specifically, the NTB device is mounted on the root complex (RC), and the RC is directly connected to the device main memory. This configuration enables the NTB device to communicate data with the PCIe link between the two NTB devices through the bus inside the host.

[0066] In this embodiment, the accessing party of the two computer systems is determined to be the first computer system, and the accessed party is determined to be the second computer system.

[0067] During the system startup phase, the DMA-NTB binding initialization kernel module is run. This module is responsible for identifying all DMA controllers (i.e., direct memory access controllers) and all NTB devices in the first computer system, and reading the PCIe link status register to obtain the link width and speed information of each NTB device so as to subsequently understand the transmission capacity of each NTB transmission link, read the BAR (Base Address Register) configuration of each NTB device, and record the BAR address range and attributes. The BAR address is the key information for the NTB device to perform memory mapping. The above information is recorded so that the target address space of the DMA controller can be correctly configured later.

[0068] S102: determining a current target non-transparent bridge device corresponding to each direct memory access controller among a plurality of non-transparent bridge devices; the current target non-transparent bridge devices corresponding to at least two direct memory access controllers are different;

[0069] In this embodiment, for each DMA controller, its corresponding current target non-transparent bridge device is determined in multiple non-transparent bridge devices. The DMA controller and the non-transparent bridge device can be in a one-to-one relationship, thereby reducing competition and delay, or in a many-to-one relationship, that is, multiple DMA controllers correspond to the same non-transparent bridge device. This configuration is suitable for scenarios with small data flow or low real-time requirements, and can save hardware resources. When determining the corresponding current target non-transparent bridge device for each DMA controller, there are at least two DMA controllers corresponding to different current target non-transparent bridge devices. The purpose is to disperse the data flow and avoid all DMA controllers from transmitting data through the same non-transparent bridge device, thereby reducing the load of a single non-transparent bridge device and improving the throughput of the overall system.

[0070] It can be understood that by ensuring that there are at least two DMA controllers corresponding to different NTB devices, the system can more effectively utilize the multi-channel capabilities provided by multiple NTB devices, thereby improving data processing capabilities and reducing the risk of a single NTB link becoming a performance bottleneck.

[0071] S103: For each direct memory access controller, a binding operation is performed on the direct memory access controller and the corresponding current target non-transparent bridge device so that the direct memory access controller accesses the second computer system through the corresponding current target non-transparent bridge device, and the binding operation includes configuring the target address space of the direct memory access controller to be the base address register address range of the corresponding current target non-transparent bridge device.

[0072] In this embodiment, after determining the current target non-transparent bridge device for binding with the DMA controller, it is necessary to bind the DMA controller and the current target non-transparent bridge device. The binding operation includes configuring the target address space of the DMA controller, that is, setting the target address space of the DMA to the base address register (BAR) address range of the corresponding NTB device, that is, mapping the target address space of the DMA to the BAR address range of the NTB device. After the configuration is completed, the binding of the DMA and the NTB device is realized. When the DMA performs a memory access operation, it is actually accessing the memory of the second computer system.

[0073] Furthermore, this embodiment also configures necessary address conversion logic to ensure that the address generated by the DMA controller can be correctly converted into a valid address in the second computer system.

[0074] After the target address space and address translation are configured, the DMA controller is enabled so that it can begin transferring data to the second computer system. The performance of the DMA transfer can also be monitored, and the binding relationship and address space configuration can be adjusted as needed to optimize performance and respond to system changes.

[0075] Through the above steps, the DMA controller can directly access the memory of the second computer system, thereby achieving efficient data transmission. This binding operation reduces CPU intervention, reduces latency, improves data transmission rate, and enables the system to better utilize the multi-channel capabilities provided by multiple NTB devices, avoiding the performance bottleneck that a single NTB link may become.

[0076] It can be seen that in this embodiment, the computer system includes multiple non-transparent bridge devices. By binding multiple direct memory access controllers to different non-transparent bridge devices, the data transmission load is dispersed to multiple communication links, thereby avoiding the overload of a single communication link, improving the system throughput and reducing the possibility of bottlenecks. When multiple direct memory access controllers transmit data through different non-transparent bridge devices, the full bandwidth of the transmission link of each non-transparent bridge device can be used, thereby improving the data processing capability and performance of the entire system.

[0077] Based on the above embodiments:

[0078] In an exemplary embodiment, the communication method of the computer system further includes:

[0079] For each current target non-transparent bridge device, monitoring the bandwidth utilization of the current target non-transparent bridge device, and in response to the bandwidth utilization satisfying a rebinding condition, determining a target direct memory access controller from at least one direct memory access controller bound to the current target non-transparent bridge device;

[0080] For each target direct memory access controller, a new target device is determined for the target direct memory access controller in multiple non-transparent bridge devices, and a rebinding operation is performed on the target direct memory access controller and the corresponding new target device so that the target direct memory access controller accesses the second computer system through the corresponding new target device. The rebinding operation includes configuring the target address space of the target direct memory access controller to be the base address register address range of the corresponding new target device.

[0081] In this embodiment, the bandwidth utilization of each NTB device bound to the DMA controller is monitored, which can be achieved by analyzing the status register of the NTB device or through a network management tool. The purpose of monitoring is to determine when the bandwidth utilization reaches a threshold and needs to be re-bound to optimize performance.

[0082] Specifically, when the bandwidth utilization meets the preset rebinding conditions, the rebinding process is triggered. These conditions may include high bandwidth utilization, increased data transmission delay or other performance indicators. In at least one DMA controller bound to the current target non-transparent bridge device, determine the target direct memory access controller, including but not limited to the DMA controller that is greatly affected by the bandwidth limitation. For each target direct memory access controller, determine a new target device for it in multiple NTB devices. The new target device is different from the current target non-transparent bridge device. Specifically, the best new target device can be selected based on the performance and load of each NTB device. Rebind the target direct memory access controller with the new NTB device, including configuring the target address space of the target direct memory access controller to the base address register address range of the corresponding new NTB device, that is, updating the configuration of the DMA controller to point to the BAR address range of the new NTB device.

[0083] Through the above rebinding operation, the target direct memory access controller can access the second computer system through the new NTB device, which may reduce the delay of data transmission and improve the data transmission rate. And continue to monitor the new binding relationship and bandwidth utilization to ensure that data transmission is always kept in the optimal state. If necessary, it will be adjusted again to adapt to the changing data transmission needs.

[0084] In this way, the binding relationship between the DMA controller and the NTB device can be dynamically adjusted to respond to changes in bandwidth utilization and optimize data transfer performance, which helps to improve the flexibility and responsiveness of the system and ensure that efficient data transfer can be maintained under high load conditions.

[0085] In an exemplary embodiment, the communication method of the computer system further includes:

[0086] Determine an upper bandwidth utilization threshold and a lower bandwidth utilization threshold;

[0087] When the bandwidth utilization of the current target non-transparent bridge device is greater than the bandwidth utilization upper limit threshold, it is determined that the bandwidth utilization meets the rebinding condition;

[0088] The process of determining a new target device for a target direct memory access controller across multiple non-transparent bridge devices includes:

[0089] A new target device is determined for a target direct memory access controller among current target non-transparent bridge devices having a bandwidth utilization less than a bandwidth utilization lower threshold.

[0090] In this embodiment, the upper and lower thresholds of bandwidth utilization are first set, such as 80% and 20%. The upper and lower thresholds of bandwidth utilization are used to identify which NTB devices are in a high load or low load state. When the bandwidth utilization exceeds the upper threshold, it is determined that the rebinding conditions are met to reduce the burden of the overloaded device. For the DMA controller bound to the overloaded NTB device, the NTB device whose bandwidth utilization is lower than the lower threshold can be determined, and a new target device can be reallocated to the overloaded DMA controller. The NTB device that is bound to other DMA controllers in the first computer system but is not fully utilized is used as an alternative NTB device for the overloaded DMA controller, which can more effectively utilize existing resources and improve the overall resource utilization. Moreover, considering that the bound NTB device has completed the preliminary configuration, it is only necessary to adjust the existing binding relationship to reduce the configuration complexity. Under high load conditions, the rapid reallocation of the bound NTB device can quickly relieve the pressure of the overloaded DMA controller without having to wait for the new NTB device to be initialized and configured.

[0091] Through the above steps, the system can effectively manage data transmission between the DMA controller and the NTB device, optimize bandwidth utilization, and improve overall system performance.

[0092] In an exemplary embodiment, a process of determining a new target device for a target direct memory access controller in a plurality of non-transparent bridge devices includes:

[0093] Determine available bandwidth across multiple non-transparent bridge devices;

[0094] For each target direct memory access controller, a load transmission requirement of the target direct memory access controller is determined, and a new target device is determined for the target direct memory access controller in a non-transparent bridge device whose available bandwidth meets the load transmission requirement.

[0095] In this embodiment, the available bandwidth of each NTB device can also be obtained, and based on this, a new target device is selected for the overloaded DMA controller, which helps to achieve a more balanced load distribution, thereby reducing the performance bottleneck caused by the overload of a single NTB device.

[0096] In an exemplary embodiment, a communication method of a computer system includes:

[0097] determining a minimum rebalancing time interval for the first computer system;

[0098] The process of monitoring the bandwidth utilization of the current target non-transparent bridge device includes:

[0099] Monitors the bandwidth utilization of the current target non-transparent bridge device at the minimum rebalance interval.

[0100] In this embodiment, the minimum rebalancing time interval refers to a time threshold set in the system to avoid triggering too many rebalancing operations due to frequent load changes. During this time interval, even if the system load changes, the system will not perform a rebalancing operation. The setting of this interval can reduce the system overhead and performance impact caused by frequent rebalancing. By setting the minimum rebalancing time interval and monitoring the bandwidth utilization accordingly, the resource allocation of the computer system can be effectively managed and optimized while maintaining system stability.

[0101] In an exemplary embodiment, the process of monitoring the bandwidth utilization of the current target non-transparent bridge device includes:

[0102] Get the transmission rate of each direct access controller bound to the current target non-transparent bridge device;

[0103] The bandwidth utilization of the current target non-transparent bridge device is obtained based on the rate of each transmission rate and the preset time interval.

[0104] In this embodiment, the ratio of the rate sum of the transmission rates of each direct access controller bound to the current target non-transparent bridge device to the preset time interval is used as the bandwidth utilization of the current target non-transparent bridge device. It can be understood that the transmission rate controlled by DMA represents the ability of the DMA controller to transmit data within a certain period of time, and the specific calculation formula is: bandwidth utilization = (rate sum / preset time interval).

[0105] In summary, this embodiment can effectively monitor and calculate the bandwidth utilization of the current target non-transparent bridging device, thereby providing data support for system performance optimization and resource management, helping to ensure efficient utilization of network resources, and timely discovering and solving potential performance bottleneck problems.

[0106] In an exemplary embodiment, the process of obtaining the transmission rate of each direct access controller includes:

[0107] For each direct access controller, a descriptor table of the direct access controller is obtained, and a transmission rate of the direct access controller is obtained based on an accumulated value of a length field in the descriptor table and a preset time interval.

[0108] In this embodiment, for each DMA controller, its descriptor table is first obtained. The descriptor table is a data structure used to store transmission information (including but not limited to the target address space, length, etc.) in DMA transmission. The value of the length field in the descriptor table is extracted as the data amount for accumulation, that is, the descriptor list of the DMA controller is obtained according to the preset scanning cycle. When the status flag of the descriptor is an incomplete state, it is ensured that the value of the length field is within a reasonable range, and the values ​​of the length field extracted from multiple scanning cycles are accumulated to obtain an accumulated value. The ratio of the accumulated value to the preset time interval is used as the transmission rate of the DMA controller.

[0109] The accumulation process is explained. A cumulative data volume variable and the current time are initialized, and the initial value is set to 0. The value of the extracted length field is added to the cumulative data volume variable, and the relevant timestamp information is updated to record the completion time of the data volume calculation. When calculating the DMA transfer rate, the cumulative data volume at the end is subtracted from the cumulative data volume at the start to obtain the data transfer volume within the preset time interval. The data transfer volume is divided by the length of the preset time interval to obtain the average transfer rate. The calculated rate is converted into an appropriate unit (such as MB / s or Gb / s).

[0110] Before executing this step, the following operations are also included: when the system starts, the initialization program of the traffic monitoring module will allocate the necessary memory space to store monitoring data and intermediate calculation results, initialize counter and timestamp variables, which will be used to track the amount and time of data transmission, obtain the memory location of the descriptor table through the corresponding DMA driver, and verify the validity of the obtained address to ensure that it has appropriate read permissions, configure monitoring parameters, including defining the scanning frequency of the descriptor table, determining the time interval for traffic calculation, and configuring the frequency and format of report generation.

[0111] In this embodiment, the transmission rate of each DMA controller can be accurately monitored and calculated, which facilitates network management and performance optimization and ensures the efficiency and reliability of data transmission.

[0112] In an exemplary embodiment, the communication method of the computer system further includes:

[0113] Obtaining load balancing effect data of the first computer system;

[0114] A performance report log file is generated based on the load balancing effect data and the bandwidth utilization of each current target non-transparent bridge device. A trend graph can be generated based on the load balancing effect data of the first computer system and the bandwidth utilization of each current target non-transparent bridge device acquired according to a preset summary period; the trend graph represents the change of the load distribution of the first computer system over time, and a performance report log file is generated based on the trend graph.

[0115] In this embodiment, during the communication between the first computer system and the second computer system, the load balancing effect data of the first computer system is recorded, and the load balancing effect data and the bandwidth utilization of the NTB device are periodically summarized to generate a trend graph; the trend graph represents the change of the load distribution of the first computer system over time, so as to effectively monitor the load balancing effect of the first computer system and evaluate and optimize the system performance.

[0116] In an exemplary embodiment, after identifying a plurality of direct memory access controllers and a plurality of non-transparent bridge devices in a first computer system, a communication method of the computer system includes:

[0117] A distance matrix is ​​established based on multiple direct memory access controllers and multiple non-transparent bridge devices, wherein elements of the distance matrix are used to represent logical distances of corresponding combinations of direct memory access controllers and non-transparent bridge devices;

[0118] The process of determining a current target non-transparent bridge device corresponding to each direct memory access controller among a plurality of non-transparent bridge devices includes:

[0119] For each direct memory access controller, based on the distance matrix, a non-transparent bridge device with the smallest logical distance to the direct memory access controller is selected as the current target non-transparent bridge device of the direct memory access controller.

[0120] In this embodiment, a distance matrix is ​​first established based on multiple direct memory access controllers and multiple non-transparent bridge devices. For example, the rows of the distance matrix can be DMA controllers, and the columns of the distance matrix can be NTB devices. The element a in the distance matrix is ij Indicates the logical distance between the i-th DMA controller and the j-th NTB device. When selecting the initial binding relationship, for each DMA controller, select an NTB device with the shortest logical distance as the current target non-transparent bridge device for binding. Figure 3 As shown, the initial binding relationship can then be recorded in the binding table. It can be understood that a shorter logical distance means that the data passes through fewer network devices when it is transmitted between the DMA controller and the NTB device, ensuring the optimization of the data transmission path, thereby reducing the delay in data transmission and improving the response speed of the system. In addition, this proximity binding mechanism reduces the number of network devices on the data transmission path between the DMA controller and the bound NTB device, and the amount of data that each device needs to process is relatively small, thereby reducing the possibility of network congestion and improving the bandwidth utilization of the overall network.

[0121] In an exemplary embodiment, a process of establishing a distance matrix based on a plurality of direct memory access controllers and a plurality of non-transparent bridge devices includes:

[0122] Acquire a communication topology structure of the first computer system;

[0123] For each direct memory access controller, determining a logical distance between the direct memory access controller and each non-transparent bridge device according to a communication topology structure, wherein the logical distance is the number of hops between the direct memory access controller and the non-transparent bridge device;

[0124] A distance matrix is ​​established based on each DMA controller and non-transparent bridge device combination and their corresponding logical distances.

[0125] In this embodiment, a system management tool or PCIe enumeration API can be used to identify all PCIe devices in the system, including DMA controllers and NTB devices. Based on the enumerated information, a device tree representing the PCIe topology is constructed. The nodes in the device tree represent PCIe devices, and the edges represent PCIe links. All DMA controllers and NTB devices are identified in the PCIe device tree.

[0126] For each DMA controller, use a tree traversal algorithm (such as depth-first search DFS) to calculate the number of PCIe switch hops to reach each NTB device. The number of hops refers to the number of switches that need to be traversed in the PCIe topology from the DMA controller to the NTB device. Record the number of PCIe switch hops from each DMA controller to each NTB device. Create a matrix with rows representing DMA controllers and columns representing NTB devices, and the dimension of the matrix is ​​the number of DMA controllers multiplied by the number of NTB devices. Fill in the corresponding position of the matrix with the distance from each DMA controller to each NTB device based on the recorded PCIe switch hops. If the number of hops between DMA controller i and NTB device j is n, fill in n in the position of the i-th row and j-th column of the matrix. If there is no direct PCIe connection between a DMA controller and the NTB device, the corresponding element in the matrix can be marked as infinity or a preset high value to indicate unreachable. Store the generated DMA-NTB distance matrix in the system's memory or disk for use in subsequent binding decision processes.

[0127] Through the above steps, the PCIe topology can be effectively analyzed, the logical distance can be calculated, and a distance matrix for optimizing the binding relationship between the DMA controller and the NTB device can be generated. This matrix provides a clear view for the system administrator or the automated script to select the optimal DMA-NTB binding strategy. In summary, the present invention greatly improves the data transmission bandwidth between systems by realizing multiple parallel PCIe links. The multi-channel architecture can simultaneously support multiple DMA transmission operations, effectively alleviating the bandwidth bottleneck caused by a single channel. The multi-channel design provides the system with higher flexibility and can adapt to more complex network topologies. A dynamic load balancing algorithm is introduced to realize the intelligent allocation of NTB channels and DMA resources. Based on real-time monitoring data, the system can automatically adjust the resource allocation strategy to maximize resource utilization. Under high load conditions, the load balancing mechanism can distribute tasks to multiple channels to avoid overloading a single channel and improve the overall throughput. The intelligent binding of the DMA controller and the NTB device is realized to ensure the optimization of the data transmission path. The DMA resources are allocated by the principle of proximity, which significantly reduces the delay of data transmission.

[0128] Second, please refer to Figure 4 The present invention also provides a communication system of a computer system, the computer system includes a plurality of non-transparent bridge devices, and the communication system of the computer system includes:

[0129] An identification module 11, used to identify a plurality of direct memory access controllers and a plurality of non-transparent bridge devices in a first computer system;

[0130] A determination module 12 is used to determine the current target non-transparent bridge device corresponding to each direct memory access controller in a plurality of non-transparent bridge devices; the current target non-transparent bridge devices corresponding to at least two direct memory access controllers are different;

[0131] The binding module 13 is used to configure the target address space of each direct memory access controller to the base address register address range of the corresponding current target non-transparent bridge device, so that the direct memory access controller can access the second computer system through the corresponding current target non-transparent bridge device.

[0132] For an introduction to a communication system of a computer system provided by the present invention, please refer to the above embodiments, and the present invention will not be described in detail here.

[0133] A communication system for a computer system provided by the present invention has the same beneficial effects as the communication method for the computer system described above.

[0134] In a third aspect, the present invention further provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of any of the above-mentioned computer system communication methods.

[0135] For an introduction to a communication system of a computer program product provided by the present invention, please refer to the above embodiments, and the present invention will not be described in detail herein.

[0136] The communication system of a computer program product provided by the present invention has the same beneficial effects as the communication method of the computer system mentioned above.

[0137] Fourth, please refer to Figure 5 The present invention also provides an electronic device, comprising:

[0138] A memory 21, used for storing computer programs;

[0139] The processor 22 is configured to implement the steps of the computer system communication method described in any of the above embodiments when executing a computer program.

[0140] The electronic device also includes:

[0141] The input interface 23 is connected to the processor 22 via the communication bus 26, and is used to obtain the computer programs, parameters and instructions imported from the outside, and save them in the memory 21 under the control of the processor 22. The input interface can be connected to an input device to receive parameters or instructions manually input by the user. The input device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the terminal housing.

[0142] The display unit 24 is connected to the processor 22 via the communication bus 26 and is used to display the data sent by the processor 22. The display unit can be a liquid crystal display or an electronic ink display.

[0143] The network port 25 is connected to the processor 22 via the communication bus 26, and is used to communicate with various external terminal devices. The communication technology used in the communication connection can be a wired communication technology or a wireless communication technology, such as mobile high-definition link technology, universal serial bus, high-definition multimedia interface, wireless fidelity technology, Bluetooth communication technology, low-power Bluetooth communication technology, communication technology based on IEEE802.11s, etc.

[0144] For an introduction to an electronic device provided by the present invention, please refer to the above embodiments, and the present invention will not be described in detail here.

[0145] The electronic device provided by the present invention has the same beneficial effects as the communication method of the computer system mentioned above.

[0146] Fifth, please refer to Figure 6 The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned computer system communication methods are implemented.

[0147] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0148] For an introduction to a computer-readable storage medium provided by the present invention, please refer to the above embodiments, and the present invention will not be described in detail here.

[0149] A computer-readable storage medium provided by the present invention has the same beneficial effects as the above-mentioned communication method of the computer system.

[0150] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprising a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0151] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A communication method of a computer system, characterized in that: The computer system includes a plurality of non-transparent bridge devices, and the communication method of the computer system includes: identifying a plurality of direct memory access controllers and a plurality of said non-transparent bridge devices in a first computer system; Determining a current target non-transparent bridge device corresponding to each of the direct memory access controllers among the plurality of non-transparent bridge devices; the current target non-transparent bridge devices corresponding to at least two of the direct memory access controllers are different; For each of the direct memory access controllers, a binding operation is performed on the direct memory access controller and the corresponding current target non-transparent bridge device so that the direct memory access controller accesses the second computer system through the corresponding current target non-transparent bridge device, and the binding operation includes configuring the target address space of the direct memory access controller to be the base address register address range of the corresponding current target non-transparent bridge device.

2. The computer system communication method according to claim 1, characterized in that: The communication method of the computer system further includes: For each of the current target non-transparent bridge devices, monitor the bandwidth utilization of the current target non-transparent bridge device, and in response to the bandwidth utilization satisfying a rebinding condition, determine a target direct memory access controller from at least one direct memory access controller bound to the current target non-transparent bridge device; For each of the target direct memory access controllers, a new target device is determined for the target direct memory access controller in the plurality of the non-transparent bridge devices, and a rebinding operation is performed on the target direct memory access controller and the corresponding new target device so that the target direct memory access controller accesses the second computer system through the corresponding new target device, wherein the rebinding operation includes configuring the target address space of the target direct memory access controller to be the base address register address range of the corresponding new target device.

3. The communication method of a computer system according to claim 2, characterized in that: The communication method of the computer system further includes: Determine an upper bandwidth utilization threshold and a lower bandwidth utilization threshold; When the bandwidth utilization of the current target non-transparent bridge device is greater than the bandwidth utilization upper limit threshold, determining that the bandwidth utilization meets the rebinding condition; The process of determining a new target device for the target direct memory access controller in the plurality of non-transparent bridge devices comprises: A new target device is determined for the target direct memory access controller among the current target non-transparent bridge devices whose bandwidth utilization is less than the bandwidth utilization lower limit threshold.

4. The communication method of the computing system according to claim 2, characterized in that: The process of determining a new target device for the target direct memory access controller in the plurality of non-transparent bridge devices comprises: determining available bandwidth of a plurality of said non-transparent bridging devices; For each of the target direct memory access controllers, a load transmission requirement of the target direct memory access controller is determined, and a new target device is determined for the target direct memory access controller in the non-transparent bridge device whose available bandwidth meets the load transmission requirement.

5. The computer system communication method according to claim 2, characterized in that: The communication method of the computer system comprises: determining a minimum rebalancing time interval for the first computer system; The process of monitoring the bandwidth utilization of the current target non-transparent bridge device includes: The bandwidth utilization of the current target non-transparent bridge device is monitored according to the minimum rebalancing time interval.

6. The computer system communication method according to claim 2, characterized in that: The process of monitoring the bandwidth utilization of the current target non-transparent bridge device includes: Acquire the transmission rate of each direct access controller bound to the current target non-transparent bridge device; The bandwidth utilization of the current target non-transparent bridge device is obtained based on the rate of each transmission rate and the preset time interval.

7. The computer system communication method according to claim 6, characterized in that: The process of obtaining the transmission rate of each direct access controller includes: For each of the direct access controllers, a descriptor table of the direct access controller is obtained, and a transmission rate of the direct access controller is obtained based on an accumulated value of a length field in the descriptor table and the preset time interval.

8. The computer system communication method according to claim 2, characterized in that: The communication method of the computer system further includes: Obtaining load balancing effect data of the first computer system; A performance report log file is generated based on the load balancing effect data and the bandwidth utilization of each of the current target non-transparent bridge devices.

9. The computer system communication method according to claim 8, characterized in that: The process of generating a performance report log file based on the load balancing effect data and the bandwidth utilization of each of the current target non-transparent bridge devices includes: Generate a trend graph based on the load balancing effect data of the first computer system and the bandwidth utilization rate of each of the current target non-transparent bridge devices acquired according to a preset summary period; the trend graph represents the change of the load distribution of the first computer system over time; A performance report log file is generated based on the trend graph.

10. The computer system communication method according to any one of claims 1 to 9, characterized in that: After identifying a plurality of direct memory access controllers and a plurality of non-transparent bridge devices in a first computer system, the communication method of the computer system includes: Establishing a distance matrix based on the plurality of direct memory access controllers and the plurality of non-transparent bridge devices, wherein the elements of the distance matrix are used to represent the logical distances between the corresponding combinations of the direct memory access controllers and the non-transparent bridge devices; The process of determining the current target non-transparent bridge device corresponding to each direct memory access controller among the plurality of non-transparent bridge devices comprises: For each of the direct memory access controllers, based on the distance matrix, a non-transparent bridge device with the smallest logical distance to the direct memory access controller is selected as a current target non-transparent bridge device of the direct memory access controller.

11. The computer system communication method according to claim 10, characterized in that: The process of establishing a distance matrix based on the plurality of direct memory access controllers and the plurality of non-transparent bridge devices comprises: Acquire a communication topology structure of the first computer system; For each of the direct memory access controllers, determining a logical distance between the direct memory access controller and each of the non-transparent bridge devices according to the communication topology structure, wherein the logical distance is the number of hops between the direct memory access controller and the non-transparent bridge device; A distance matrix is ​​established based on each DMA controller and non-transparent bridge device combination and their corresponding logical distances.

12. A communication system for a computer system, characterized in that: The computer system includes a plurality of non-transparent bridge devices, and the communication system of the computer system includes: An identification module, configured to identify a plurality of direct memory access controllers and a plurality of the non-transparent bridge devices in a first computer system; A determination module, used for determining a current target non-transparent bridge device corresponding to each of the direct memory access controllers among the plurality of the non-transparent bridge devices; the current target non-transparent bridge devices corresponding to at least two of the direct memory access controllers are different; The binding module is used to configure the target address space of each direct memory access controller to the base address register address range of the corresponding current target non-transparent bridge device, so that the direct memory access controller can access the second computer system through the corresponding current target non-transparent bridge device.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the communication method of the computer system as claimed in any one of claims 1 to 11 are implemented.

14. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the communication method of the computer system according to any one of claims 1 to 11 when executing the computer program.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the communication method of the computer system according to any one of claims 1 to 11 are implemented.

Images