Computing power server detection method and device, computer device, and storage medium

Abstract

The application relates to a computing power server detection method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a sequence set; for each sequence, determining the estimated communication bandwidth between computing power cards in a server according to the query bandwidth between each two connected computing power cards in the sequence; performing traversal on the sequence, determining the estimated inter-server communication bandwidth corresponding to the sequence currently traversed, stopping the traversal when a traversal stop condition is met, and screening sample sequences from the traversed sequences; determining a standard inter-server communication bandwidth according to the sample sequences; acquiring the measured value of the communication bandwidth between the computing power cards in the server and the measured value of the inter-server communication bandwidth, and generating a computing power server detection report according to the standard inter-server communication bandwidth, the measured value of the communication bandwidth between the computing power cards in the server and the measured value of the inter-server communication bandwidth. The method can detect whether the computing power server has a problem.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a computing power server detection method and device, computer equipment, storage medium and computer program product. BACKGROUND

[0002] In the field of supercomputing, multiple computing power cards work together to complete computing, and multiple computing power servers work together to complete distributed computing scenarios. Computing power card is a hardware device specially used to improve the computing power of servers. It usually includes high-performance processing units such as GPU (Graphics Processing Unit) or TPU (Tensor Processing Unit) to accelerate the computing tasks of servers. Computing power server refers to a server equipped with computing power cards.

[0003] The cooperative work between computing power cards has complex requirements for the interconnection and intercommunication between computing power cards from hardware to software. In the multi-card cooperation, especially in the distributed computing scenario, multiple computing power cards form a ring communication. In order to avoid the communication from appearing the bucket effect, the installation topology of the cards on the server must be guaranteed, and the hardware ratio must be balanced. This hardware ratio and topology design is complex, involving multiple cooperative links, and errors in each link will cause the computing power server to have high configuration but low performance. The prior art lacks an automatic detection tool to check the assembled computing power server to determine whether the computing power server has problems. SUMMARY

[0004] Therefore, it is necessary to provide a computing power server detection method, device, computer equipment, computer readable storage medium and computer program product capable of checking problems of a computing power server.

[0005] In a first aspect, the present application provides a computing power server detection method. The method comprises:

[0006] Obtaining a sequence set corresponding to each computing power server in a computing power server cluster; each sequence in each sequence set comprises a computing power card and a communication card deployed on the corresponding computing power server;

[0007] For each sequence, determining an estimated inter-server communication bandwidth corresponding to the sequence according to the query bandwidth between each two connected computing power cards in the sequence;

[0008] According to the order of the estimated inter-server communication bandwidth from large to small, traversing the sequence, determining the estimated inter-server communication bandwidth corresponding to the sequence currently traversed, and stopping traversal when the traversal stop condition is met, and screening a sample sequence from the traversed sequence;

[0009] determine the standard inter-server communication bandwidth according to the estimated inter-server communication bandwidth and the estimated inter-processor card communication bandwidth corresponding to the sample sequence;

[0010] obtain the measured inter-processor card communication bandwidth and the measured inter-server communication bandwidth obtained by performing the stress test, and generate a computing power server detection report according to the standard inter-server communication bandwidth, the measured inter-processor card communication bandwidth, and the measured inter-server communication bandwidth.

[0011] In a second aspect, the present application also provides a computing power server detection device. The device comprises:

[0012] an estimated inter-processor card communication bandwidth module, configured to obtain a sequence set corresponding to each computing power server in a computing power server cluster; each sequence in each sequence set comprises a computing power card and a communication card deployed on the corresponding computing power server; for each sequence, determine an estimated inter-processor card communication bandwidth corresponding to the sequence according to the query bandwidth between each two connected computing power cards in the sequence;

[0013] a standard inter-server communication bandwidth determination module, configured to traverse the sequences in descending order of the estimated inter-processor card communication bandwidth, determine an estimated inter-server communication bandwidth corresponding to the sequence currently traversed, and stop the traversal when a traversal stop condition is met, and filter out a sample sequence from the traversed sequences; determine the standard inter-server communication bandwidth according to the estimated inter-processor card communication bandwidth and the estimated inter-server communication bandwidth corresponding to the sample sequence;

[0014] a report generation module, configured to obtain a measured inter-processor card communication bandwidth and a measured inter-server communication bandwidth obtained by performing a stress test, and generate a computing power server detection report according to the standard inter-server communication bandwidth, the measured inter-processor card communication bandwidth, and the measured inter-server communication bandwidth.

[0015] In a third aspect, the present application also provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements any of the computing power server detection methods in the embodiments of the present application when executing the computer program.

[0016] In a fourth aspect, the present application also provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement any of the computing power server detection methods in the embodiments of the present application.

[0017] In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements any of the computing power server detection methods in the embodiments of the present application.

[0018] The computing power server detection method, device, computer device, storage medium and computer program product described above determine a plurality of sequences, and for each sequence, the estimated inter-server communication bandwidth corresponding to the sequence can be determined. By determining the estimated inter-server communication bandwidth, each sequence can be traversed in descending order of the estimated inter-server communication bandwidth, so as to determine the estimated inter-server communication bandwidth corresponding to the sequence traversed. When the currently traversed sequence meets the traversal stop condition, the traversal is stopped, and the sample sequence whose communication card bandwidth performance can be fully utilized under the condition that the estimated inter-server communication bandwidth is as large as possible is obtained. By determining the sample sequence, the smaller value between the estimated inter-server communication bandwidth and the estimated inter-server communication bandwidth corresponding to the sample sequence is taken as the standard inter-server communication bandwidth for subsequent bandwidth comparison. The standard inter-server communication bandwidth is a bandwidth that the computing power server cluster can theoretically reach under normal conditions. Therefore, by obtaining the measured values of the inter-server communication bandwidth and the inter-server communication bandwidth based on the stress test, the measured values of the inter-server communication bandwidth and the inter-server communication bandwidth can be compared with the standard inter-server communication bandwidth, so as to determine whether the computing power server cluster has a problem based on the comparison result, and generate a server detection report marked with the comparison result. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 FIG. 1 is a diagram of an application environment of the computing power server detection method in an embodiment;

[0020] Figure 2 FIG. 2 is a flowchart of the computing power server detection method in an embodiment;

[0021] Figure 3 FIG. 3 is a diagram for determining a sample sequence in an embodiment;

[0022] Figure 4 FIG. 4 is a diagram of the overall flow of the computing power server detection in an embodiment;

[0023] Figure 5 FIG. 5 is a structural block diagram of the computing power server detection device in an embodiment;

[0024] Figure 6 FIG. 6 is an internal structure diagram of the computer device in an embodiment. DETAILED DESCRIPTION

[0025] In order to make the purposes, technical solutions and advantages of the present application clearer, further detailed description will be given below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0026] The computing power server detection method provided by the embodiments of the present application can be applied to an application environment as shown in Figure 1 . The detection server 102 and the computing power server 104 communicate through the network. The data storage system can store the data that needs to be processed by the computing power server 104. The data storage system can be integrated on the computing power server 104, or placed on the cloud or other servers. The detection server 102 can generate a sequence set corresponding to each computing power server 104 respectively, and process based on the sequence set to obtain a processing result, and generate a computing power server detection report based on the processing result. The computing power server 104 can be, but is not limited to, various desktop computers, notebook computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things device can be a smart speaker, a smart TV, a smart air conditioner, a smart vehicle device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc.

[0027] In one embodiment, as shown in Figure 2 , a computing power server detection method is provided. The method is described by taking the detection server in Figure 1 as an example. The detection server can be one of the computing power servers in the computing power server cluster, or a server other than the computing power server cluster. The computing power server detection method includes the following steps:

[0028] Step 202, obtaining a sequence set corresponding to each computing power server in the computing power server cluster respectively; each sequence in each sequence set includes the computing power card and the communication card deployed on the corresponding computing power server.

[0029] Specifically, the detection server can determine the computing power card and the communication card deployed on each computing power server, and collect the topology relationship of the computing power card on the computing power server, and the hardware connection information of the computing power card on the computing power server and other computing power servers. The topology relationship of the computing power card on the computing power server can include the hardware connection information between the computing power cards. The communication card refers to the hardware device used for communication between servers.

[0030] Further, the topology analyzer in the detection server can enumerate all combination relations of topology sequences among the mining cards based on the collected information, and obtain a sequence set corresponding to each mining server in the mining server cluster. For example, for the mining server 1, the mining server 1 is deployed with 8 mining cards GPU0 to GPU7 and 2 communication cards NIC0 to NIC1. The detection server can enumerate the serial connection order of the 8 mining cards to obtain a plurality of mining card strings, and the detection server serially connects the mining card string with the communication card to obtain a sequence. For example, the mining card string can be GPU0-GPU1-GPU2-GPU3-GPU4-GPU5-GPU6-GPU7, which indicates that data flows from GPU0 to GPU1, from GPU1 to GPU2, and so on, until GPU7. Wherein, GPUi-GPUj indicates that GPUi and GPUj are connected by hardware. The hardware connection types include PCIE (peripheral component interconnect express, a high-speed serial computer expansion bus standard) and SMP (symmetric multiprocessing) across numa (Non Uniform Memory Access, non-uniform memory access architecture). Since data flows to another mining server through the communication card, the communication card of the mining server can be spliced to the mining card string to obtain a sequence GPU0-GPU1-GPU2-GPU3-GPU4-GPU5-GPU6-GPU7-NIC0-NIC1. For the mining server 1, another sequence can also be generated, for example, GPU0-GPU1-GPU2-GPU4-GPU5-GPU3-GPU6-GPU7-NIC0-NIC1. It is easy to understand that one mining server can have multiple sequences, which are obtained by enumerating the serial connection order of the mining cards on the mining server.

[0031] In step 204, for each sequence, the estimated inter-server mining card communication bandwidth corresponding to the sequence is determined according to the query bandwidth between each two connected mining cards in the sequence.

[0032] Specifically, for each sequence, the detection server performs the following operations. The detection server determines the query bandwidth between each two hardware-connected mining cards in the sequence. For example, for the sequence “GPU0-GPU1-GPU2-GPU3-GPU4-GPU5-GPU6-GPU7-NIC0-NIC1”, the detection server can determine the query bandwidth between GPU0 and GPU1, the query bandwidth between GPU1 and GPU2, and so on. The detection server determines the estimated inter-server mining card communication bandwidth corresponding to the sequence according to the query bandwidths.

[0033] In one embodiment, the short board effect, also known as the barrel principle or the water bucket effect, refers to how much water a bucket can hold, which does not depend on the longest board, but on the shortest board. Due to the existence of the water bucket effect, the bandwidth rate of a data transmission line refers to the bandwidth rate of a data transmission line, which does not depend on the maximum bandwidth rate, but on the minimum bandwidth rate, so the minimum query bandwidth is used as the corresponding estimated server inter-card communication bandwidth of the sequence.

[0034] In one embodiment, the query bandwidth between two connected cards can be obtained by a preset instruction. The server reading bandwidth instruction can be implemented by various command line tools, such as nload instruction, etc.

[0035] Step 206, according to the order of the estimated server inter-card communication bandwidth from large to small, the sequence is traversed, the estimated server inter-card communication bandwidth corresponding to the sequence is determined, and the traversal is stopped when the traversal stop condition is met, and the sample sequence is selected from the traversed sequence.

[0036] Specifically, after obtaining the estimated server inter-card communication bandwidth corresponding to each sequence, the sequences can be sorted according to the order of the estimated server inter-card communication bandwidth from large to small, and a sequence group is obtained. From the first sequence in the sequence group, the sequence in the sequence group is traversed, and for the sequence traversed at present, the bandwidth rate between the head card in the sequence traversed at present and the communication card, and the bandwidth rate between the tail card and the communication card are determined. The estimated server inter-card communication bandwidth corresponding to the sequence traversed at present. The server detects whether the sequence traversed at present meets the traversal stop condition, and when the traversal stop condition is met, the sample sequence is selected from the traversed sequence. The sample sequence can be the sequence corresponding to the correct data transmission path, or the sequence corresponding to the data transmission path with the least error. The correct data transmission path is a data transmission path that fully utilizes the bandwidth performance of the communication card.

[0037] Step 208, according to the estimated server inter-card communication bandwidth and the estimated server inter-card communication bandwidth corresponding to the sample sequence, the standard server inter-card communication bandwidth is determined.

[0038] Specifically, since the estimated inter-server communication bandwidth and the estimated inter-processor card communication bandwidth of the sample sequence are obtained, the detection server can determine the standard inter-server communication bandwidth according to the estimated inter-server communication bandwidth and the estimated inter-processor card communication bandwidth of the sample sequence. For example, the bandwidth rate of data flowing in the same power server is generally greater than the bandwidth rate of data transmission from one power server to another power server, so the smaller value of the estimated inter-server communication bandwidth and the estimated inter-processor card communication bandwidth is taken as the standard inter-server communication bandwidth.

[0039] In step 210, the measured inter-processor card communication bandwidth and the measured inter-server communication bandwidth obtained by the pressure test are obtained, and the power server detection report is generated according to the standard inter-server communication bandwidth, the measured inter-processor card communication bandwidth and the measured inter-server communication bandwidth.

[0040] Specifically, for each power server, the pressure test tool can determine an intra-server optimal data transmission path based on each inter-processor card in the power server, and perform pressure test on the intra-server optimal data transmission path to obtain the measured inter-processor card communication bandwidth corresponding to the power server. For the power server cluster, the pressure test tool can connect each inter-processor card and communication card in the power server cluster in series to obtain an inter-server optimal data transmission path, and perform pressure test based on the inter-server optimal data transmission path to obtain the measured inter-server communication bandwidth corresponding to the power server cluster. The detection server compares each measured inter-processor card communication bandwidth with the standard inter-server communication bandwidth. If the measured inter-processor card communication bandwidth is less than the standard inter-server communication bandwidth and is more than 80% lower than the standard inter-server communication bandwidth, the power server corresponding to the measured inter-processor card communication bandwidth is marked as error, and it is considered that the power server corresponding to the measured inter-processor card communication bandwidth has problems, such as unreasonable configuration of inter-processor cards and communication cards, or high configuration and low performance of the power server. If the measured inter-server communication bandwidth is less than the standard inter-server communication bandwidth and is more than 80% lower than the standard inter-server communication bandwidth, it is considered that the power server cluster has problems, such as possible installation error of the driver, or problem of not properly inserting the hardware.

[0041] Further, the detection server generates a power server detection report based on the standard inter-server communication bandwidth, the measured inter-processor card communication bandwidth and the measured inter-server communication bandwidth. For example, the standard inter-server communication bandwidth, the measured inter-processor card communication bandwidth and the measured inter-server communication bandwidth are written into the detection report, and the problem is written in the detection report when the problem is determined.

[0042] In the above computing power server detection method, by determining a plurality of sequences, for each sequence, the estimated inter-server computing power card communication bandwidth corresponding to the sequence can be determined. By determining the estimated inter-server computing power card communication bandwidth, each sequence can be traversed in descending order of the estimated inter-server computing power card communication bandwidth to determine the estimated inter-server communication bandwidth corresponding to the sequence traversed. When the currently traversed sequence meets the traversal stop condition, the traversal is stopped, and the sample sequence is obtained, in which the inter-server computing power card communication bandwidth is as large as possible, and the communication card bandwidth performance can be fully utilized. By determining the sample sequence, the smaller value of the estimated inter-server computing power card communication bandwidth and the estimated inter-server communication bandwidth corresponding to the sample sequence is taken as the standard inter-server communication bandwidth for subsequent bandwidth comparison. The standard inter-server communication bandwidth is a bandwidth that the computing power server cluster can theoretically achieve under normal conditions. Therefore, by obtaining the measured values of the inter-server computing power card communication bandwidth and the inter-server communication bandwidth obtained based on the stress test, the measured values of the inter-server computing power card communication bandwidth and the inter-server communication bandwidth are compared with the standard inter-server communication bandwidth, to determine whether there is a problem in the computing power server cluster based on the comparison result, and generate a server detection report marked with the comparison result.

[0043] In one embodiment, the estimated inter-server computing power card communication bandwidth corresponding to the sequence is determined according to the query bandwidth between each two connected computing power cards in the sequence, comprising: traversing each two connected computing power cards in the sequence; querying the query bandwidth between the two computing power cards currently traversed; determining the nominal bandwidth between the two computing power cards currently traversed according to the first hardware connection information between the two computing power cards; when the query bandwidth between the two computing power cards currently traversed is less than the nominal bandwidth, marking the two computing power cards currently traversed as errors; and after the traversal ends, taking the smallest query bandwidth as the estimated inter-server computing power card communication bandwidth corresponding to the sequence.

[0044] Specifically, the detection server can traverse each two connected computing power cards in the sequence, and for the two computing power cards currently traversed, the detection server can determine the first hardware connection information between the two computing power cards currently traversed, such as determining the hardware connection information between GPU0 and GPU1. Based on the first hardware connection information, the nominal bandwidth between the two computing power cards currently traversed is determined, such as the hardware connection information between GPU0 and GPU1 is SYS, SYS indicates that it needs to cross numa and Pcle exchange chip, the nominal bandwidth corresponding to SYS is 2GB / s. The hardware connection information between GPU2 and GPU3 is PXB, PXB indicates that it needs to cross Pcle exchange chip, the nominal bandwidth corresponding to PXB is 16GB / s, and so on.

[0045] Further, for the two computing cards currently traversed to, the detection server can also obtain the query bandwidth between the two computing cards currently traversed to. The query bandwidth is the bandwidth obtained by querying. Generally, the query bandwidth is equal to the nominal bandwidth. If the query bandwidth of the two computing cards currently traversed to is less than the nominal bandwidth, it can be considered that there is a problem with the two computing cards currently traversed to, and the two computing cards currently traversed to are marked as error. The marking of error can be recorded in the computing server detection report subsequently.

[0046] Further, when the traversal ends, the minimum query bandwidth is taken as the estimated inter-server computing card communication bandwidth corresponding to the sequence.

[0047] In the embodiment, by comparing the nominal bandwidth with the query bandwidth, it can be determined whether there is a problem with the computing card in the computing server based on the comparison result, and the computing card that may have a problem is marked as error when there is a problem. In this way, the engineer can troubleshoot the computing card that may have an error through the error marking in the computing server detection report. And due to the existence of the short board effect, the minimum query bandwidth is taken as the estimated inter-server computing card communication bandwidth corresponding to the sequence, so that the estimated inter-server computing card communication bandwidth is more accurate.

[0048] In one of the embodiments, determining the estimated inter-server communication bandwidth corresponding to the sequence currently traversed to includes: determining second hardware connection information between the head computing card in the sequence currently traversed to and the communication card in the sequence currently traversed to, and determining the estimated inter-card communication bandwidth between the head computing card and the communication card according to the second hardware connection information; determining third hardware connection information between the tail computing card in the sequence currently traversed to and the communication card in the sequence currently traversed to, and determining the estimated inter-card communication bandwidth between the tail computing card and the communication card according to the third hardware connection information; and determining the estimated inter-server communication bandwidth corresponding to the sequence currently traversed to according to the estimated inter-card communication bandwidth between the head computing card and the communication card and the estimated inter-card communication bandwidth between the tail computing card and the communication card.

[0049] Specifically, each sequence can be traversed in descending order of the estimated communication bandwidth between the computing cards in the server. For the sequence currently traversed, the detection server determines the second hardware connection information between the head computing card in the sequence currently traversed and the communication card in the sequence currently traversed. The head computing card refers to the first computing card in the sequence. The detection server can determine the estimated communication bandwidth between the head computing card and the communication card according to the second hardware connection information. Further, the detection server determines the third hardware connection information between the tail computing card in the sequence currently traversed and the communication card in the sequence currently traversed, and determines the estimated communication bandwidth between the tail computing card and the communication card according to the third hardware connection information. The tail computing card refers to the last computing card in the sequence.

[0050] When the estimated communication bandwidth between the head computing card and the communication card and the estimated communication bandwidth between the tail computing card and the communication card are determined, the estimated inter-server communication bandwidth corresponding to the sequence currently traversed can be determined according to the estimated communication bandwidth between the head computing card and the communication card and the estimated communication bandwidth between the tail computing card and the communication card. For example, the smaller one of the estimated communication bandwidth between the head computing card and the communication card and the estimated communication bandwidth between the tail computing card and the communication card is taken as the estimated inter-server communication bandwidth corresponding to the sequence currently traversed.

[0051] In one embodiment, determining the estimated communication bandwidth between the head computing card and the communication card according to the second hardware connection information comprises: for each communication card in the sequence currently traversed, determining the hardware bandwidth between the head computing card and the communication card according to the second hardware connection information between the head computing card and the communication card; determining the communication card bandwidth of the communication card, and marking the head computing card and the communication card as erroneous when the hardware bandwidth is less than the communication card bandwidth; and accumulating the communication card bandwidths of the communication cards in the sequence currently traversed to obtain the estimated communication bandwidth between the head computing card and the communication cards.

[0052] Specifically, the detection server takes out the head computing card in the sequence currently traversed to, queries the second hardware connection information between the head computing card and each communication card in the sequence currently traversed to; for each communication card in the sequence currently traversed to, the hardware bandwidth between the head computing card and the communication card is determined according to the second hardware connection information between the head computing card and the communication card. For example, when the second hardware connection information between the head computing card and the communication card is SYS, the hardware bandwidth between the head computing card and the communication card is determined to be 2 GB / S. The detection server determines the communication card bandwidth of the communication card, for example, the communication card has a nominal bandwidth, and the nominal bandwidth is taken as the communication card bandwidth of the communication card. The detection server judges whether the hardware bandwidth between the head computing card and the communication card is less than the communication card bandwidth of the communication card, and in the case of less than, the head computing card and the communication card are marked as errors, and it is considered that the performance of the communication card is not fully used.

[0053] Further, the detection server accumulates the communication card bandwidth of each communication card in the sequence currently traversed to to obtain the estimated communication bandwidth between the head computing card and the communication card.

[0054] In this embodiment, by comparing the hardware bandwidth and the communication bandwidth, it can be determined whether there is a problem between the head computing card and the communication card based on the comparison result, and the head computing card and the communication card with the problem are marked as errors when there is a problem. Therefore, the marked errors can be recorded in the computing power server detection report in the subsequent, and the engineers can troubleshoot the head computing card and the communication card with possible problems through the marked errors in the computing power server detection report. Moreover, since data generally flows into the head computing card in the sequence through the communication card, and then flows out to the communication card through the tail computing card in the sequence, the communication bandwidth between the head computing card and the communication card is estimated in this embodiment to obtain the estimated communication bandwidth between the head computing card and the communication card.

[0055] In one of the embodiments, the detection server takes out the tail computing card in the sequence currently traversed to, queries the third hardware connection information between the tail computing card and each communication card in the sequence currently traversed to; for each communication card in the sequence currently traversed to, the hardware bandwidth between the tail computing card and the communication card is determined according to the third hardware connection information between the tail computing card and the communication card. For example, when the third hardware connection information between the tail computing card and the communication card is SYS, the hardware bandwidth between the tail computing card and the communication card is determined as 2 GB / S. The detection server determines the communication card bandwidth of the communication card. The detection server judges whether the hardware bandwidth between the tail computing card and the communication card is less than the communication card bandwidth of the communication card, and in the case of being less, the tail computing card and the communication card are marked as error, and it is considered that the performance of the communication card is not fully used. Further, the detection server accumulates the communication card bandwidth of each communication card in the sequence currently traversed to to obtain the estimated communication bandwidth between the tail computing card and the communication card.

[0056] In one of the embodiments, the traversal is stopped when the traversal stopping condition is met, and a sample sequence is selected from the traversed sequences, including: when there is no marked error between the head computing card and each communication card in the sequence currently traversed to, and there is no marked error between the tail computing card and each communication card in the sequence currently traversed to, the traversal is stopped, and the sequence currently traversed to is taken as the sample sequence; or when all sequences are traversed, each sequence has a marked error between the head computing card and the communication card, or each sequence has a marked error between the tail computing card and the communication card, the sequence with the least marked error is taken as the sample sequence.

[0057] Specifically, when there is no marked error between the head computing card and each communication card in the sequence currently traversed to, and there is no marked error between the tail computing card and each communication card in the sequence currently traversed to, the traversal is stopped, and the sequence currently traversed to is taken as the sample sequence. If all sequences are traversed, and each sequence is marked with an error, the sequence with the least marked error is taken as the sample sequence. For example, referring to Figure 3 , in sequence 1, the head computing card and communication card 1 and communication card 2 have marked errors, and the tail computing card and communication card 1 have a marked error; in sequence 2, the tail computing card and communication card 3, communication card 4 and communication card 6 have marked errors; in sequence 3, the head computing card and communication card 7 have a marked error, and the tail computing card and communication card 9 have a marked error. Since sequence 3 has only 2 marked errors, and other sequences have 3 marked errors, sequence 3 is taken as the sample sequence. Figure 3 A determination diagram of the sample sequence in one of the embodiments is shown.

[0058] In the embodiment, the sequence without a marking error or with the least marking error is taken as a sample sequence, so that the sample sequence can be an optimal path of a data transmission, so that a bandwidth that can be reached by the cluster of computing power servers under normal conditions can be obtained based on the sample sequence, and the bandwidth is taken as a standard value for comparison with a measured value.

[0059] In one of the embodiments, the standard inter-server communication bandwidth is determined according to the estimated intra-server communication bandwidth of the computing power cards and the estimated inter-server communication bandwidth corresponding to the sample sequence, including: when the estimated intra-server communication bandwidth of the sample sequence is less than the estimated inter-server communication bandwidth of the sample sequence, and the difference between the estimated intra-server communication bandwidth of the sample sequence and the estimated inter-server communication bandwidth is greater than the communication bandwidth of any communication card in the sample sequence, marking an error for the sample sequence; and taking the smaller one of the estimated intra-server communication bandwidth of the sample sequence and the estimated inter-server communication bandwidth as the standard inter-server communication bandwidth.

[0060] Specifically, for the sample sequence, when the estimated intra-server communication bandwidth of the sample sequence is less than the estimated inter-server communication bandwidth of the sample sequence, and the difference is greater than the communication bandwidth of any communication card in the sample sequence, it can be considered that the difference between the estimated intra-server communication bandwidth and the estimated inter-server communication bandwidth is too large, at least one of the estimated intra-server communication bandwidth and the estimated inter-server communication bandwidth has a problem at this time, therefore, the sample sequence can be marked as an error, and the marking error is recorded in the computing power server detection report.

[0061] Further, the detection server can take the smaller one of the estimated intra-server communication bandwidth of the sample sequence and the estimated inter-server communication bandwidth as the standard inter-server communication bandwidth.

[0062] In the embodiment, compared with the transmission of data in the server, the bandwidth of the data transmission out of the server is smaller, therefore, the smaller one of the estimated intra-server communication bandwidth and the estimated inter-server communication bandwidth is taken as the standard inter-server communication bandwidth, so that the standard inter-server communication bandwidth is more accurate.

[0063] In one of the embodiments, the power server detection report is generated according to the standard inter-server communication bandwidth, the measured value of the inter-ASIC communication bandwidth in the server, and the measured value of the inter-server communication bandwidth, including: for each power server corresponding to the measured value of the inter-ASIC communication bandwidth in the server, when the measured value of the inter-ASIC communication bandwidth in the server is less than the standard inter-server communication bandwidth, and the difference between the measured value of the inter-ASIC communication bandwidth in the server and the standard inter-server communication bandwidth is greater than a preset difference, the power server corresponding to the inter-ASIC communication bandwidth in the server is marked as an error; when the measured value of the inter-server communication bandwidth is less than the standard inter-server communication bandwidth, and the difference between the measured value of the inter-server communication bandwidth and the standard inter-server communication bandwidth is greater than a preset difference, the power server cluster is marked as an error; and the power server detection report is generated based on the marked errors.

[0064] Specifically, each power server can be subjected to a stress test to obtain the measured value of the inter-ASIC communication bandwidth in the server corresponding to each power server. For each measured value of the inter-ASIC communication bandwidth in the server, the measured value of the inter-ASIC communication bandwidth in the server is compared with the standard inter-server communication bandwidth, and when the measured value of the inter-ASIC communication bandwidth in the server is less than the standard inter-server communication bandwidth, and the difference between the two is greater than a preset difference, for example, when the measured value of the inter-ASIC communication bandwidth in the server is less than 80% of the standard inter-server communication bandwidth, it is determined that the power server corresponding to the inter-ASIC communication bandwidth in the server may have a problem, and at this time, the power server corresponding to the inter-ASIC communication bandwidth in the server is marked as an error.

[0065] The power server cluster can be subjected to a stress test to obtain the measured value of the inter-server communication bandwidth. The measured value of the inter-server communication bandwidth is compared with the standard inter-server communication bandwidth by the detection server, and when the measured value of the inter-server communication bandwidth is less than the standard inter-server communication bandwidth, and the difference between the measured value of the inter-server communication bandwidth and the standard inter-server communication bandwidth is greater than a preset difference, for example, when the measured value of the inter-server communication bandwidth is less than 80% of the standard inter-server communication bandwidth, it is determined that the power server cluster may have a problem, and at this time, the power server cluster is marked as an error.

[0066] When the marked error is determined, each marked error can be recorded in the power server detection report, so that an engineer can troubleshoot problems based on the marked errors in the power server detection report.

[0067] In one of the embodiments, the power server detection report is generated according to the standard inter-server communication bandwidth, the measured value of the inter-ASIC communication bandwidth in the server, and the measured value of the inter-server communication bandwidth, including: for each power server corresponding to the measured value of the inter-ASIC communication bandwidth in the server, when the measured value of the inter-ASIC communication bandwidth in the server is less than the standard inter-server communication bandwidth, and the difference between the measured value of the inter-ASIC communication bandwidth in the server and the standard inter-server communication bandwidth is greater than a preset difference, the power server corresponding to the inter-ASIC communication bandwidth in the server is marked as an error; when the measured value of the inter-server communication bandwidth is less than the standard inter-server communication bandwidth, and the difference between the measured value of the inter-server communication bandwidth and the standard inter-server communication bandwidth is greater than a preset difference, the power server cluster is marked as an error; and the power server detection report is generated based on the marked errors. Figure 4 , Figure 4A schematic diagram of the overall flow of computing power server detection in an embodiment is shown. Step 1: The user configures the communication card name between the computing power servers, NIC0 and NIC1. The collector completes the computing power card topology information collection on the server, including the computing power card name NVIDI A4090, the number of computing power cards 8, the hardware connection topology between each computing power card, and the hardware connection topology between the inter-server communication card and the computing power card. The hardware connection types are PCIE and SMP across numa, and the PCIE is 3.0 16GB / s bandwidth. Step 2: The topology analyzer traverses all sequences of the computing power card, and for each sequence traversed, the bandwidth rate of the hardware connection between each two cards is compared in order one by one, and the minimum query bandwidth is calculated as the estimated inter-server computing power card communication bandwidth corresponding to the sequence traversed. For example, for the sequence GPU0-GPU1-GPU2-GPU3-GPU4-GPU5-GPU6-GPU7, the estimated rate between GPU3 and GPU4 across numa communication is the lowest at 10GB / s, and the other two types are PCIE3.0 rate 16GB / s. Take 10GB / s as the estimated inter-server computing power card communication bandwidth corresponding to the sequence, and other sequences are similar.

[0068] Third step: from the sequence with the maximum communication bandwidth between the computing cards in the estimation server, the sequence is traversed in the order of the communication bandwidth between the computing cards in the server from large to small, and the communication bandwidth between the computing cards in the estimation server is calculated. Fourth step: taking out the head computing card in the sequence currently traversed, querying the connection hardware information of the head computing card and each communication card, querying the hardware bandwidth and communication card bandwidth recorded in the data storage, and marking an error if the hardware bandwidth is less than the communication card bandwidth, for example, the performance of two communication cards is 25 GB / s, but the hardware connection between the computing card and the communication card has a problem of 10 GB / s, and two errors are marked. Finally, the communication card bandwidth 50 GB / s is accumulated as the estimated communication bandwidth between the head computing card and the communication card of the sequence currently traversed. Fifth step: taking out the tail computing card in the sequence currently traversed, repeating the calculation method of the fourth step, and the estimated communication bandwidth between the tail computing card and the communication card is also 50 GB / s, but there are 2 error marks. Comparing the two sets of communication bandwidths in the fourth and fifth steps, since the two sets of communication bandwidths are the same in the above example, 50 GB / s is taken as the standard inter-server communication bandwidth. Sixth step: determining the sample sequence, since in the above example, there are error marks after the traversal is completed, and the number of error marks is the same, the first sequence can be taken as the sample sequence. Seventh step: comparing the estimated communication bandwidth between the computing cards in the server 10 GB / s obtained in the second step and the estimated inter-server communication bandwidth 50 GB / s obtained in the fifth step according to the sample sequence obtained in the sixth step, since the estimated communication bandwidth between the computing cards in the server 10 GB / s is much smaller than the estimated inter-server communication bandwidth 50 GB / s, and the difference between the two exceeds the bandwidth of one communication card, an error is marked; taking the smaller value 10 GB / s as the standard inter-server communication bandwidth.

[0069] The eighth step is to test the computing power card in the computing power server, and obtain the measured value of the communication bandwidth between the computing power cards in the server, for example, 9.7 GB / s. The measured value of the communication bandwidth between the servers is obtained by testing the computing power server cluster, for example, 9 GB / s. Since the two measured values are close to the standard server communication bandwidth of 10 GB / s, no error is marked. The ninth step is to generate a computing power server detection report, and write the computing power card topology information in the computing power server detection report. The estimated communication bandwidth between the computing power cards in the server is 10 GB / s, the standard communication bandwidth between the servers is 10 GB / s, the measured communication bandwidth between the servers is 9 GB / s, and the measured communication bandwidth between the computing power cards in the server is 9.7 GB / s. The change suggestion can also be written in the computing power server detection report. For example, if there is a marked error in the second step above, output the error information, the nominal rate and the query rate, and suggest checking the server startup parameters; if the sequence obtained in the sixth step above is a marked error, output the error information, and the hardware bandwidth and the communication card bandwidth, and suggest adjusting the communication card slot; if there is a marked error in the seventh step above, output the error information, and suggest reducing the communication card or increasing the communication bandwidth between the computing power cards; if there is a problem in the eighth step above, output the error information, and the measured value and the estimated value, and suggest checking the server startup parameters, whether the hardware is inserted correctly, and whether the driver installation is correct.

[0070] As can be seen from the above embodiments, the communication bandwidth of the server topology can be quickly analyzed by the method, and the estimated value and the measured value are compared, the detection efficiency is improved, the consistency of the detection result is improved, and the reliability of the server assembly is improved; the problem link is clearly output by the report, the troubleshooting suggestion is provided, the complexity of manual troubleshooting is reduced, and the problem troubleshooting efficiency is improved.

[0071] It should be understood that although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0072] The application greatly improves the detection efficiency, the consistency of the detection result, and the reliability of the server assembly by comparing the theoretical value of the communication bandwidth after algorithm analysis of the server topology with the measured value. The complexity of manual troubleshooting is reduced, and the problem troubleshooting efficiency is improved by outputting a report and providing guiding troubleshooting suggestions for various problems. The detection efficiency, the consistency of the detection result, and the reliability of the server assembly are improved by comparing the theoretical value of the communication bandwidth after algorithm analysis of the automatic server topology with the measured value. The complexity of manual troubleshooting is reduced, and the problem troubleshooting efficiency is improved by outputting a report and providing guiding troubleshooting suggestions for various problems.

[0073] Based on the same inventive concept, the embodiments of the application also provide a computing power server detection device for implementing the computing power server detection method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more computing power server detection device embodiments provided below can be referred to the limitations of the computing power server detection method in the foregoing, which will not be described here again.

[0074] In one embodiment, as shown in Figure 5 A computing power server detection device 500 is provided, which includes a server internal algorithm card intercommunication bandwidth estimation module 502, a standard server intercommunication bandwidth determination module 504, and a report generation module 506, wherein:

[0075] The server internal algorithm card intercommunication bandwidth estimation module 502 is configured to obtain a sequence set corresponding to each algorithm server in the computing power server cluster; each sequence in each sequence set includes an algorithm card and a communication card deployed on the corresponding algorithm server; for each sequence, the estimated server internal algorithm card intercommunication bandwidth corresponding to the sequence is determined according to the query bandwidth between each two connected algorithm cards in the sequence.

[0076] The standard server intercommunication bandwidth determination module 504 is configured to traverse the sequences in descending order of the estimated server internal algorithm card intercommunication bandwidth, determine the estimated server intercommunication bandwidth corresponding to the sequence currently traversed, and stop the traversal when the traversal stop condition is met, and filter out a sample sequence from the traversed sequences; and determine the standard server intercommunication bandwidth according to the estimated server internal algorithm card intercommunication bandwidth and the estimated server intercommunication bandwidth corresponding to the sample sequence.

[0077] The report generation module 506 is configured to obtain the measured value of the server internal algorithm card intercommunication bandwidth and the measured value of the server intercommunication bandwidth obtained by pressure testing, and generate a computing power server detection report according to the standard server intercommunication bandwidth, the measured value of the server internal algorithm card intercommunication bandwidth, and the measured value of the server intercommunication bandwidth.

[0078] In one of the embodiments, the server intra-miner communication bandwidth estimation module 502 is further configured to traverse each two connected miners in the sequence, query a query bandwidth between the two miners currently traversed to; determine a nominal bandwidth between the two miners currently traversed to according to first hardware connection information between the two miners currently traversed to; mark an error for the two miners currently traversed to when the query bandwidth between the two miners currently traversed to is less than the nominal bandwidth; and after the traversal ends, take the smallest query bandwidth as the estimated server intra-miner communication bandwidth corresponding to the sequence.

[0079] In one of the embodiments, the standard server inter-communication bandwidth determination module 504 is further configured to determine second hardware connection information between a head miner in the sequence currently traversed to and a communication card in the sequence currently traversed to, and determine an estimated head miner-communication card inter-communication bandwidth according to the second hardware connection information; determine third hardware connection information between a tail miner in the sequence currently traversed to and the communication card in the sequence currently traversed to, and determine an estimated tail miner-communication card inter-communication bandwidth according to the third hardware connection information; and determine an estimated server inter-communication bandwidth corresponding to the sequence currently traversed to according to the estimated head miner-communication card inter-communication bandwidth and the estimated tail miner-communication card inter-communication bandwidth.

[0080] In one of the embodiments, the standard server inter-communication bandwidth determination module 504 is further configured to, for each communication card in the sequence currently traversed to, determine a hardware bandwidth between the head miner and the communication card according to the second hardware connection information between the head miner and the communication card; query a communication card bandwidth of the communication card, and mark an error for the head miner and the communication card when the hardware bandwidth is less than the communication card bandwidth; and accumulate the communication card bandwidths of the communication cards in the sequence currently traversed to to obtain the estimated head miner-communication card inter-communication bandwidth.

[0081] In one of the embodiments, the standard server inter-communication bandwidth determination module 504 is further configured to, when there is no marked error between the head miner and each communication card in the sequence currently traversed to and between the tail miner and each communication card in the sequence currently traversed to, stop the traversal and take the sequence currently traversed to as a sample sequence; or, when all the sequences are traversed, each sequence has a marked error between the head miner and the communication card or each sequence has a marked error between the tail miner and the communication card, take a sequence with the least marked error as the sample sequence.

[0082] In one of the embodiments, the standard inter-server communication bandwidth determination module 504 is further configured to, when the estimated inter-graphics card communication bandwidth of the sample sequence in the sample sequence is less than the estimated inter-server communication bandwidth of the sample sequence, and the difference between the estimated inter-graphics card communication bandwidth of the sample sequence and the estimated inter-server communication bandwidth of the sample sequence is greater than the communication bandwidth of any one communication card in the sample sequence, mark the sample sequence as incorrect; and take the smaller one of the estimated inter-graphics card communication bandwidth of the sample sequence and the estimated inter-server communication bandwidth of the sample sequence as the standard inter-server communication bandwidth.

[0083] In one of the embodiments, the report generation module 506 is further configured to, for each graphics card in the graphics server, when the measured inter-graphics card communication bandwidth is less than the standard inter-server communication bandwidth, and the difference between the measured inter-graphics card communication bandwidth and the standard inter-server communication bandwidth is greater than a preset difference value, mark the graphics server corresponding to the inter-graphics card communication bandwidth as incorrect; when the measured inter-server communication bandwidth is less than the standard inter-server communication bandwidth, and the difference between the measured inter-server communication bandwidth and the standard inter-server communication bandwidth is greater than the preset difference value, mark the graphics server cluster as incorrect; and generate a graphics server detection report based on each marked incorrect graphics server.

[0084] The above-mentioned modules in the graphics server detection device can be all or partially implemented by software, hardware, and combinations thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above-mentioned modules.

[0085] In one embodiment, a computer device is provided, which can be a server, and the internal structure diagram thereof can be as shown in Figure 6 The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store graphics server detection data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a graphics server detection method.

[0086] Those skilled in the art can understand that, Figure 6 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0087] In one embodiment, a computer device is provided, including a memory and a processor, the memory stores a computer program, and the processor implements any of the computing power server detection methods in the embodiments of the present application when executing the computer program.

[0088] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement any of the computing power server detection methods in the embodiments of the present application.

[0089] In one embodiment, a computer program product is provided, which includes a computer program, and the computer program is executed by a processor to implement any of the computing power server detection methods in the embodiments of the present application.

[0090] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0091] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0092] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0093] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for detecting computing power servers, characterized in that, The method includes: Obtain the sequence set corresponding to each computing server in the computing server cluster; each sequence in each sequence set includes the computing card and communication card deployed on the corresponding computing server; For each sequence, the estimated inter-server communication bandwidth of the corresponding sequence is determined based on the query bandwidth between every two connected computing cards in the sequence. According to the estimated inter-server communication bandwidth of computing power cards in descending order, the sequence is traversed to determine the estimated inter-server communication bandwidth corresponding to the currently traversed sequence, and the traversal is stopped when the traversal stop condition is met, and sample sequences are selected from the traversed sequences. Based on the estimated inter-server computing power card communication bandwidth and the estimated inter-server communication bandwidth corresponding to the sample sequence, the standard inter-server communication bandwidth is determined. Obtain the measured values ​​of communication bandwidth between computing cards within the server and communication bandwidth between servers obtained from stress testing. Generate a computing server test report based on the standard inter-server communication bandwidth, the measured values ​​of communication bandwidth between computing cards within the server, and the measured values ​​of communication bandwidth between servers. The step of determining the estimated inter-server communication bandwidth corresponding to the sequence based on the query bandwidth between every two connected computing cards in the sequence includes: For each pair of connected computing cards in the sequence, the query bandwidth between the two computing cards currently being traversed is queried. The nominal bandwidth between the two computing cards currently being traversed is determined based on the first hardware connection information between them. If the query bandwidth between the two currently traversed computing cards is less than the nominal bandwidth, mark the two currently traversed computing cards as errors; After the traversal is completed, the minimum query bandwidth is used as the estimated inter-server computing power card communication bandwidth corresponding to the sequence.

2. The method according to claim 1, characterized in that, The communication card refers to the hardware device used for communication between servers.

3. The method according to claim 1, characterized in that, Determining the estimated inter-server communication bandwidth corresponding to the currently traversed sequence includes: Determine the second hardware connection information between the head computing power card and the communication card in the currently traversed sequence, and determine the estimated communication bandwidth between the head computing power card and the communication card based on the second hardware connection information. The third hardware connection information between the tail computing power card in the currently traversed sequence and the communication card in the currently traversed sequence is determined, and the estimated communication bandwidth between the tail computing power card and the communication card is determined based on the third hardware connection information. Based on the estimated communication bandwidth between the head computing card and the communication card and the estimated communication bandwidth between the tail computing card and the communication card, the estimated inter-server communication bandwidth corresponding to the currently traversed sequence is determined.

4. The method according to claim 3, characterized in that, The step of determining the estimated communication bandwidth between the head computing card and the communication card based on the second hardware connection information includes: For each communication card in the currently traversed sequence, the hardware bandwidth between the head computing power card and the targeted communication card is determined based on the second hardware connection information between the head computing power card and the targeted communication card. Query the communication card bandwidth of the target communication card, and mark the head computing power card and the target communication card as incorrect when the hardware bandwidth is less than the communication card bandwidth; The communication card bandwidths of each communication card in the currently traversed sequence are summed to obtain the estimated communication bandwidth between the head computing power card and the communication card.

5. The method according to claim 4, characterized in that, The step of stopping traversal when the traversal stopping condition is met, and selecting sample sequences from the traversed sequences, includes: If there are no marking errors between the head computing card and each communication card in the currently traversed sequence, and if there are no marking errors between the tail computing card and each communication card in the currently traversed sequence, then the traversal stops, and the currently traversed sequence is taken as the sample sequence; or After all sequences have been traversed, if each sequence has a marking error between the head computing card and the communication card, or if each sequence has a marking error between the tail computing card and the communication card, the sequence with the fewest marking errors will be used as the sample sequence.

6. The method according to claim 1, characterized in that, The step of determining the standard inter-server communication bandwidth based on the estimated inter-card communication bandwidth within the server and the estimated inter-server communication bandwidth corresponding to the sample sequence includes: If the communication bandwidth between computing power cards in the estimated server of the sample sequence is less than the communication bandwidth between estimated servers of the sample sequence, and the difference between the communication bandwidth between computing power cards in the estimated server of the sample sequence and the communication bandwidth between estimated servers is greater than the communication bandwidth of any communication card in the sample sequence, then the sample sequence is marked as incorrect. The smaller of the estimated intra-server computing card communication bandwidth and the estimated inter-server communication bandwidth of the sample sequence is taken as the standard inter-server communication bandwidth.

7. The method according to claim 1, characterized in that, The process of generating a computing server detection report based on the standard inter-server communication bandwidth, the measured inter-server computing card communication bandwidth, and the measured inter-server communication bandwidth includes: For each of the computing power servers, if the measured value of the inter-card communication bandwidth within the server is less than the standard inter-server communication bandwidth, and the difference between the measured value of the inter-card communication bandwidth within the server and the standard inter-server communication bandwidth is greater than a preset difference, the computing power server corresponding to the inter-card communication bandwidth within the server is marked as incorrect. When the measured value of the inter-server communication bandwidth is less than the standard inter-server communication bandwidth, and the difference between the measured value of the inter-server communication bandwidth and the standard inter-server communication bandwidth is greater than a preset difference, the computing server cluster is marked as incorrect. A computing server detection report is generated based on each tag error.

8. A computing server detection device, characterized in that, The device includes: The server-side inter-hardware communication bandwidth estimation module is used to obtain the sequence set corresponding to each computing server in the computing server cluster. Each sequence in each sequence set includes computing cards and communication cards deployed on the corresponding computing server. For each sequence, the module iterates through every two connected computing cards in the sequence and queries the query bandwidth between the two currently traversed computing cards. Based on the first hardware connection information between the two currently traversed computing cards, the module determines the nominal bandwidth between the two currently traversed computing cards. When the query bandwidth between the two currently traversed computing cards is less than the nominal bandwidth, the two currently traversed computing cards are marked as incorrect. After the traversal is completed, the minimum query bandwidth is taken as the estimated inter-hardware communication bandwidth of the server corresponding to the sequence. The standard inter-server communication bandwidth determination module is used to traverse the sequence according to the estimated inter-server communication bandwidth of computing power cards in descending order, determine the estimated inter-server communication bandwidth corresponding to the currently traversed sequence, stop traversal when the traversal stop condition is met, and filter out sample sequences from the traversed sequences; and determine the standard inter-server communication bandwidth based on the estimated inter-server communication bandwidth of computing power cards and the estimated inter-server communication bandwidth corresponding to the sample sequence. The report generation module is used to obtain the measured values ​​of communication bandwidth between computing cards within the server and the measured values ​​of communication bandwidth between servers obtained from stress testing, and to generate a computing server test report based on the standard inter-server communication bandwidth, the measured values ​​of communication bandwidth between computing cards within the server, and the measured values ​​of communication bandwidth between servers.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images