Whole cabinet configuration state detection method and device, electronic equipment and storage medium

Abstract

The rack configuration status detection method, device, electronic equipment, and storage medium provided in this application first acquire the GPU frequency signal received by each CPU interface, the physical port connection information between each GPU and the switching chip, and the slot connection information between each CPU and GPU, providing a data foundation for subsequent configuration status detection. Then, based on the GPU frequency signal received by each CPU interface, the first configuration information between each CPU and its connected GPU is determined. Based on the physical port connection information, the second configuration status between the GPU and its connected switching chip is determined. Based on the slot connection information and the first configuration information, the third configuration status between each CPU and its connected GPU is determined, thereby determining the overall rack configuration status. This replaces the traditional manual troubleshooting mode, achieving rapid and automatic configuration status detection, accurately locating abnormal configuration points, reducing troubleshooting time, significantly improving detection efficiency, and significantly reducing operation and maintenance costs.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, and in particular to a whole cabinet configuration state detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the explosive growth of computing power demand in the fields of artificial intelligence and high-performance computing, integrated whole cabinets integrating CPU nodes, GPU nodes and switch chips have become the mainstream deployment form. The whole cabinet usually adopts an expansion architecture of connecting multiple GPU nodes with one CPU to realize flexible expansion of computing power, but lacks a method for automatically detecting hardware connection and communication topology of CPU, GPU and switch chip.

[0003] In the related art, it is impossible to quickly distinguish whether the problem is a hardware connection fault, a configuration logic error or a topology mismatch of CPU, GPU and switch chip, the efficiency of whole cabinet configuration detection is low, and the operation and maintenance cost is high.

[0004] Therefore, how to improve the efficiency of whole cabinet configuration state detection and reduce the operation and maintenance cost has become a technical problem to be solved. SUMMARY

[0005] Embodiments of the present application provide a whole cabinet configuration state detection method and device, electronic equipment and storage medium to solve the problems of low efficiency and high operation and maintenance cost of whole cabinet configuration state detection in the related art.

[0006] In a first aspect, the embodiments of the present application provide a whole cabinet configuration state detection method, the whole cabinet comprising at least one CPU, at least one GPU and at least one switch chip, the CPU being connected with the GPU, and the GPU being connected with the switch chip, and the method comprising:

[0007] obtaining GPU frequency signals received by an interface of each CPU, physical port connection information of each GPU and switch chip, and slot connection information of each CPU and GPU;

[0008] determining first configuration information between each CPU and the connected GPU according to the GPU frequency signals received by the interface of each CPU;

[0009] determining second configuration state between each GPU and the connected switch chip according to the physical port connection information;

[0010] determining third configuration state between each CPU and the connected GPU according to the slot connection information and the first configuration information;

[0011] determining the configuration state of the whole cabinet according to the second configuration state and the third configuration state.

[0012] In one possible implementation, based on the GPU frequency signal received by each CPU's interface, first configuration information between each CPU and the connected GPU is determined, including:

[0013] The first GPU frequency signal received by the first interface of the target CPU is compared with the second GPU frequency signal received by the second interface of the target CPU; the target CPU is any CPU in the rack; the first interface is the CPU interface in the target CPU that is in an odd position; the second interface is the CPU interface in the target CPU that is in an even position.

[0014] If the first GPU frequency signal is the same as the second GPU frequency signal, the first configuration information is determined to be that the target CPU connects to 2n GPUs; n is an integer not less than 2.

[0015] If the first GPU frequency signal is different from the second GPU frequency signal, the first configuration information is determined to be that the target CPU is connected to n GPUs.

[0016] In one possible implementation, a second configuration state between the GPU and the connected switching chip is determined based on the physical port connection information, including:

[0017] If the physical port connection information of the target GPU is successfully matched with the preset physical port connection table, the second configuration state between the target GPU and the connected switching chip is determined to be normal; the target GPU is any GPU connected to the target CPU.

[0018] Otherwise, the second configuration state is determined to be a configuration error.

[0019] In one possible implementation, a third configuration state between each CPU and the connected GPU is determined based on the slot connection information and the first configuration information, including:

[0020] Based on the first configuration information, when the slot connection information of the target CPU and the target GPU is successfully compared with the preset physical slot connection table, the third configuration state between the target CPU and the target GPU is determined to be normal.

[0021] Otherwise, the third configuration state is determined to be a configuration anomaly.

[0022] In one possible implementation, determining the configuration state of the entire rack based on the second configuration state and the third configuration state includes:

[0023] If both the second and third configuration states are normal, the configuration status of the entire rack is determined to be normal.

[0024] Otherwise, based on the first configuration information, the second configuration status, and the third configuration status, an abnormal configuration status log for the entire rack is generated.

[0025] In one possible implementation, the first configuration information includes the number of GPUs connected to the target CPU and the communication topology information between the target CPU and the GPU.

[0026] Secondly, embodiments of this application provide a rack configuration status detection device, the rack including at least one CPU, at least one GPU, and at least one switching chip, the CPU being connected to the GPU; the GPU being connected to the switching chip; including:

[0027] The acquisition module is used to acquire the GPU frequency signal received by the interface of each CPU, the physical port connection information between each GPU and the switching chip, and the slot connection information between each CPU and GPU.

[0028] The determination module is used to determine the first configuration information between each CPU and the connected GPU based on the GPU frequency signal received by each CPU's interface.

[0029] The determination module is also used to determine the second configuration state between the GPU and the connected switching chip based on the connection information of each physical port;

[0030] The determination module is also used to determine the third configuration state between each CPU and the connected GPU based on the connection information of each slot and the first configuration information;

[0031] The determination module is also used to determine the configuration status of the entire cabinet based on the second and third configuration statuses.

[0032] Thirdly, embodiments of this application provide an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0033] The memory stores the instructions that the computer executes;

[0034] The processor executes computer-executable instructions stored in memory to implement the method as described in the first aspect or any of the above.

[0035] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first aspect or any of the above-mentioned methods.

[0036] Fifthly, embodiments of this application provide a computer program. The computer program product includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the methods described in the first aspect or any of the above-mentioned methods.

[0037] The rack configuration status detection method, apparatus, electronic device, and storage medium provided in this application embodiment first acquire the GPU frequency signal received by each CPU interface, the physical port connection information between each GPU and the switching chip, and the slot connection information between each CPU and GPU. This automated collection of multi-dimensional core information provides a data foundation for subsequent configuration status detection. Then, based on the GPU frequency signal received by each CPU interface, the method determines the first configuration information between each CPU and its connected GPU. Based on the physical port connection information, it determines the second configuration status between the GPU and its connected switching chip. Based on the slot connection information and the first configuration information, it determines the third configuration status between each CPU and its connected GPU. Finally, based on the second and third configuration statuses, the overall rack configuration status is determined. This method performs layered configuration identification of CPUs and GPUs, GPU and switch port verification, and CPU and GPU slot matching verification, ultimately jointly determining the overall rack configuration status. This replaces the traditional manual troubleshooting mode, achieving rapid and automatic configuration status detection, accurately locating abnormal configuration points, reducing troubleshooting time, significantly improving detection efficiency, and significantly reducing operation and maintenance costs. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0039] Figure 1 A schematic diagram of an architecture where a CPU connects to two GPUs, provided for an embodiment of this application;

[0040] Figure 2 A schematic diagram of an architecture with one CPU connected to four GPUs is provided for an embodiment of this application;

[0041] Figure 3 A flowchart illustrating the rack configuration status detection method provided in this application embodiment. Figure 1 ;

[0042] Figure 4 A flowchart illustrating the rack configuration status detection method provided in this application embodiment. Figure 2 ;

[0043] Figure 5A schematic diagram of the hardware architecture for rack configuration status detection provided in an embodiment of this application;

[0044] Figure 6 This is a schematic diagram of the structure of the rack configuration status detection device provided in the embodiments of this application;

[0045] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0046] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] Before introducing the embodiments of this application, the application background of the embodiments of this application will be explained first:

[0049] With the explosive growth in computing power demand in fields such as artificial intelligence and high-performance computing, integrated racks that combine CPU nodes, GPU nodes, and switching chips have become the mainstream deployment form. These racks typically employ an expansion architecture where one CPU connects to multiple GPU nodes to achieve flexible expansion of computing power. However, they lack a method for automatically detecting the hardware connections and communication topology between the CPU, GPU, and switching chip.

[0050] In related technologies, the main problems with identifying the configuration status of a complete server rack are as follows:

[0051] First, the rack supports a variety of expansion configurations, but the hardware connections and logical topology of the CPU, GPU and switching chips lack automatic detection methods, making it difficult for maintenance personnel to quickly confirm whether the current actual configuration is consistent with the design.

[0052] Traditional methods require manually recording the slot correspondence between the CPU and GPU, as well as the port binding information between the GPU and the switch. Manual recording is not only time-consuming, but also prone to configuration errors due to human error, which can lead to communication failures in the computing cluster.

[0053] Secondly, when communication anomalies occur in the entire rack, the relevant solutions cannot quickly distinguish whether it is a hardware connection failure, a configuration logic error, or a topology mismatch. It is necessary to check the hardware links of the CPU, GPU, and switch one by one, which results in high maintenance costs and long fault recovery time.

[0054] In addition, under different computing power requirements, the rack will switch to configurations such as 1-to-2 (i.e., one CPU connected to two GPUs) and 1-to-4 (i.e., one CPU connected to four GPUs), but it lacks a unified configuration status detection mechanism and cannot automatically adapt to the CPU and GPU configuration information, topology relationship, and GPU and switch port binding rules under different configurations, which limits the rack's flexible expansion capabilities.

[0055] Therefore, improving the efficiency of rack configuration status detection and reducing operation and maintenance costs have become urgent technical problems to be solved.

[0056] To address the technical problems existing in related technologies, the inventors of this application propose the following solution: For the issues of low efficiency and high maintenance costs in detecting the configuration status of the entire rack, this application first receives frequency signals sent from the GPU's Complex Programmable Logic Device (CPLD) to various interfaces of the CPU. By comparing the frequency signals received by the CPU's odd-even position interfaces, the number of connections and communication topology between the CPU and GPU (i.e., the first configuration information) are automatically identified, replacing manual statistics of configuration types. Secondly, the physical port binding information between the GPU and the switching chip is obtained and compared with a preset port table to determine the configuration status of the GPU and the switch. Simultaneously, the slot connection information between the CPU and GPU is combined with the first configuration information to verify the CPU-GPU slot matching, thereby determining the third configuration status. Finally, through the joint determination of the second and third configuration statuses, the result of whether the entire rack configuration is normal is output; in case of an anomaly, a fault log containing the first configuration information is automatically generated. By automatically identifying configurations using hardware frequency signals, verifying multi-dimensional information linkage, and automatically generating abnormal logs, the system achieves rapid and automatic detection of the entire rack's configuration status. Configuration identification and fault location can be completed without manual intervention, significantly improving detection efficiency while reducing labor costs and troubleshooting time in operation and maintenance.

[0057] Specifically, Figure 1 A schematic diagram of an architecture connecting a CPU to two GPUs is provided as an embodiment of this application, as shown below. Figure 1 As shown, a CPU includes 4 interfaces (such as...) Figure 1(CON0, CON1, CON2, CON3) A GPU includes two interfaces. A CPU connects two GPUs. The CPU's first and second interfaces receive the CPU frequency signals FREQ1 and FREQ2, respectively. The CPU's third and fourth interfaces receive the CPU frequency signals FREQ1 and FREQ2, respectively.

[0058] Figure 2 A schematic diagram of an architecture connecting a CPU to four GPUs is provided for an embodiment of this application, as shown below. Figure 2 As shown, a CPU includes 4 interfaces (such as...) Figure 1 (CON0, CON1, CON2, CON3) One GPU includes two interfaces, and one CPU connects to four GPUs. The CPU frequency signal received by the first, second, third, and fourth interfaces of the CPU is the FREQ1 signal.

[0059] The parts not described in detail are disclosed in the following embodiments.

[0060] The technical solution of this application will now be described in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0061] It is worth noting that the application fields of the methods, apparatus, electronic devices and storage media provided in this application are not limited.

[0062] The subject of this application is an electronic device, which may specifically be a server, terminal device, etc.

[0063] Figure 3 A flowchart illustrating the rack configuration status detection method provided in this application embodiment. Figure 1 The entire rack includes at least one CPU, at least one GPU, and at least one switching chip; the CPU is connected to the GPU; the GPU is connected to the switching chip; such as Figure 3 As shown, the method may include the following steps:

[0064] Step 31: Obtain the GPU frequency signal received by the interface of each CPU, the physical port connection information between each GPU and the switching chip, and the slot connection information between each CPU and GPU.

[0065] Step 32: Determine the first configuration information between each CPU and the connected GPU based on the GPU frequency signal received by each CPU's interface.

[0066] In the above steps, the controller of the whole cabinet, as the core execution entity, first completes the collection and summarization of the three key information in step 31 through the hardware communication link of the whole cabinet, so as to provide a data foundation for subsequent configuration status determination.

[0067] Specifically, the GPU frequency signal is generated by the CPLD of each GPU node. This signal carries the type characteristics of the GPU node and is transmitted to the corresponding CPU interface through the GPU interface. The controller collects the frequency signals obtained by the CPLD of the CPU node and records the CPU interface location corresponding to each frequency signal. The physical port connection information between the GPU and the switching chip is obtained by the target GPU chip from the switching chip through in-band communication, and then synchronized to the CPLD of the target GPU node through the IIC bus. The controller further reads the connection information of each port of each GPU node from the CPLD of each GPU node through a hardware channel. The slot connection information between the CPU and GPU is generated by the CPLD of the target GPU node based on its own deployed physical slots, and then reported to the CPLD of the corresponding CPU node through a hardware channel. The controller extracts the slot connection information between the target GPU and the CPU from the CPLD of the CPU node and establishes the association between the target GPU and the corresponding CPU.

[0068] Then, the controller determines the number of GPUs connected to each CPU based on the GPU frequency signal received by each CPU's interface, which is to determine the first configuration information between each CPU and the connected GPU.

[0069] For example, if the entire rack adopts a 1-to-4 configuration (that is, 1 CPU connected to 4 GPUs), the controller will collect the 4 GPU frequency signals received by the 4 interfaces of the CPU respectively, and then determine the first configuration information between the CPU and the connected GPUs based on whether the 4 GPU frequency signals are the same.

[0070] Step 33: Determine the second configuration state between the GPU and the connected switching chip based on the connection information of each physical port.

[0071] Specifically, step 33 may include the following implementation methods:

[0072] If the physical port connection information of the target GPU is successfully matched with the preset physical port connection table, the second configuration status between the target GPU and the connected switching chip is determined to be normal.

[0073] The target GPU is any GPU connected to the target CPU.

[0074] Otherwise, the second configuration state is determined to be a configuration error.

[0075] In this step, the controller first calls a pre-stored physical port connection table. This table is based on the rack design specifications and records the standard binding relationships between the physical ports of each GPU node and the switching chip under different configuration types. The controller compares the actual physical port connection information of each target GPU with the corresponding entries in the pre-stored table one by one. If the actual bound port of a target GPU matches the pre-stored table perfectly, the port configuration between the GPU and the switching chip is considered normal; if there are cases where the actual bound port does not match any entries in the pre-stored table, or no port binding information is detected, the configuration is considered abnormal.

[0076] For example, in a 1-to-4 configuration, the preset physical port connection table clearly defines the P0 port of the switching chip corresponding to GPU0, and the four switching ports of GPU0 are connected to the P0 port of the switching chip; the four switching ports of GPU1 are connected to the P1 port of the switching chip.

[0077] When the target GPU is GPU0, the actual physical port connection information collected is that 3 switching ports of GPU0 are connected to the P0 port of the switching chip, and 1 switching port of GPU0 is connected to the P1 port of the switching chip. Since the matching with the preset physical port connection table fails, the controller determines that the second configuration state between the target GPU and the switching chip is abnormal.

[0078] Step 34: Determine the third configuration state between each CPU and the connected GPU based on the connection information of each slot and the first configuration information.

[0079] Specifically, step 34 may include the following implementation methods:

[0080] Based on the first configuration information, when the slot connection information of the target CPU and the target GPU is successfully compared with the preset physical slot connection table, the third configuration state between the target CPU and the target GPU is determined to be normal.

[0081] Otherwise, the third configuration state is determined to be a configuration anomaly.

[0082] In this step, the first configuration information clearly defines the number of GPUs connected to the target CPU and the communication topology. Based on this first configuration information, the controller matches the corresponding standard slot connection rules from the preset physical slot connection table. If the actual slot connection information fully conforms to the standard rules, that is, the slot of the target CPU, the number of slots of the connected GPUs, and the specific slot number are all consistent with the preset, then the third configuration state between the target CPU and the target GPU is determined to be normal. If there are situations such as mismatch in the number of slots (e.g., the first configuration information indicates 4 GPUs to be connected, but only 3 GPU connections are actually detected) or mismatch in the slot numbers, then the configuration is determined to be abnormal.

[0083] For example, if the first configuration information is "target CPU2 connects to 2 GPU nodes", the standard slots corresponding to this configuration in the preset physical slot connection table are CPU2 connecting to GPU0 slot 0 and GPU1 slot 1, but the actual collected slot connection information is CPU2 connecting to GPU0 slot 0 and GPU2 slot 2, then the controller determines that the third configuration state between the target CPU2 and the corresponding GPU node is a configuration abnormality.

[0084] Step 35: Determine the configuration status of the entire cabinet based on the second and third configuration statuses.

[0085] Specifically, step 35 may also include the following implementation methods:

[0086] If both the second and third configuration states are normal, the configuration status of the entire cabinet is determined to be normal.

[0087] Otherwise, based on the first configuration information, the second configuration status, and the third configuration status, an abnormal configuration status log for the entire rack is generated.

[0088] In this step, the controller performs a global summary judgment on the second configuration status of all GPUs and switching chips, and the third configuration status of all CPUs and their corresponding GPUs. The controller will only output a conclusion that the overall rack configuration status is normal if the second configuration status of all target GPUs within the rack is normal, and the third configuration status of all target CPUs and their corresponding GPUs is normal. If any second or third configuration status is abnormal, the overall rack configuration status is determined to be abnormal, and an exception log generation process is triggered.

[0089] The exception log will contain detailed information about the number of CPU and GPU connections and the topology, the second and third configuration states of the exception, and the time of the exception and the corresponding node identifier, so that operation and maintenance personnel can quickly locate the location of the exception and investigate the cause of the exception.

[0090] For example, if the second configuration state of GPU1 is detected to be abnormal while the other states are normal, the controller determines that the entire rack configuration is abnormal and generates an abnormal log containing "First configuration information: CPU1 is connected to 4 GPUs in a star topology; Abnormal type: GPU and switch port configuration abnormal; Abnormal details: GPU1's switch port 4 is actually bound to port P1, which is the standard P0 port".

[0091] The first configuration information mentioned above includes the number of GPUs connected to the target CPU and the communication topology information between the target CPU and the GPU.

[0092] The rack configuration status detection method provided in this application first acquires the GPU frequency signal received by each CPU interface, the physical port connection information between each GPU and the switching chip, and the slot connection information between each CPU and GPU. This automated collection of multi-dimensional core information provides a data foundation for subsequent configuration status detection. Then, based on the GPU frequency signal received by each CPU interface, the method determines the first configuration information between each CPU and its connected GPU. Based on the physical port connection information, it determines the second configuration status between the GPU and its connected switching chip. Based on the slot connection information and the first configuration information, it determines the third configuration status between each CPU and its connected GPU. Finally, based on the second and third configuration statuses, the method determines the overall rack configuration status. This method performs layered configuration identification of CPUs and GPUs, GPU and switch port verification, and CPU and GPU slot matching verification, ultimately jointly determining the overall rack configuration status. This replaces the traditional manual troubleshooting method, enabling rapid and automatic detection of configuration status, accurately locating abnormal configuration points, reducing troubleshooting time, significantly improving detection efficiency, and significantly reducing operation and maintenance costs.

[0093] Based on the above embodiments, Figure 4 A flowchart illustrating the rack configuration status detection method provided in this application embodiment. Figure 2 ,like Figure 4 As shown, step 32 may also include the following steps:

[0094] Step 41: Compare the first GPU frequency signal received by the first interface of the target CPU with the second GPU frequency signal received by the second interface of the target CPU.

[0095] The target CPU is any CPU in the entire rack. The first interface is the CPU interface located in the odd-numbered position among the target CPUs. The second interface is the CPU interface located in the even-numbered position among the target CPUs.

[0096] Step 42: If the first GPU frequency signal is the same as the second GPU frequency signal, determine the first configuration information as the target CPU connecting 2n GPUs.

[0097] Where n is an integer not less than 2.

[0098] Step 43: The first GPU frequency signal is different from the second GPU frequency signal. Therefore, the first configuration information is determined to be that the target CPU is connected to n GPUs.

[0099] In the above steps, the controller first defines the positions of the target CPU's interfaces: based on the interface arrangement order of the target CPU, counting starts from the first interface closest to the edge of the CPU chip. Interfaces numbered 1, 3, 5... are defined as first interfaces (i.e., odd-numbered interfaces), and interfaces numbered 2, 4, 6... are defined as second interfaces (i.e., even-numbered interfaces). Subsequently, the controller extracts the first GPU frequency signal received by each first interface and the second GPU frequency signal received by each second interface from the CPLD of the target CPU. The frequency signals are generated by the CPLD of the GPUs connected to the CPU. The controller performs feature comparison on the extracted first and second GPU frequency signals through a built-in frequency detection module. The comparison dimensions include signal frequency value, signal duty cycle, and signal period stability. Finally, the number of GPUs connected to the target CPU is determined based on the comparison results.

[0100] For example, a target CPU integrates four high-speed interfaces, with interfaces 1 and 3 being the first interfaces and interfaces 2 and 4 being the second interfaces. If the entire rack supports both 1-to-2 and 1-to-4 configurations, the default rule is that "in the 1-to-2 configuration, the odd-numbered interfaces of the same target CPU receive different GPU frequency signals than their corresponding even-numbered interfaces; in the 1-to-4 configuration, the odd-numbered interfaces receive the same GPU frequency signals as their even-numbered interfaces." When the target CPU is CPU0, the controller extracts the first GPU frequency signal from its first interface, which is FREQ2 signal with a frequency of 1KHz. The second GPU frequency signal from its second interface is also FREQ2 signal with a frequency of 1KHz. After comparison, it is found that the first and second frequency signals are the same, and the number of interfaces is 4 high-speed interfaces. At this time, the first configuration information is determined to be "target CPU0 connects 2n=4 GPUs", and the corresponding communication topology is "1-to-4 star topology". If the controller extracts the first GPU frequency signal from the first interface of the target CPU0 as FREQ1 signal with a frequency of 1KHz, and the second GPU frequency signal from the second interface as FREQ2 signal with a frequency of 10KHz, the comparison results are different, and the number of interfaces is 4 high-speed interfaces. Then, the first configuration information is determined to be "target CPU0 connects n=2 GPUs", and the corresponding communication topology is "1-to-2 star topology".

[0101] In the above example, if a target CPU integrates 6 high-speed interfaces, with interfaces 1, 3, and 5 being the first interfaces and interfaces 2, 4, and 6 being the second interfaces, when the target CPU is CPU1, the controller extracts the first GPU frequency signal from its first interface, which is FREQ2 (1kHz), and the second GPU frequency signal from its second interface, which is also FREQ2 (1kHz). After comparison, the first and second frequency signals are found to be the same, and the number of interfaces is 6 high-speed interfaces. At this time, the first configuration information is determined to be "target CPU0 connects to 2n=6 GPUs", and the corresponding communication topology is "1-to-6 star topology". If the controller extracts the first GPU frequency signal from the first interface of target CPU1 as FREQ1 (1kHz) and the second GPU frequency signal from the second interface as FREQ2 (10kHz), the comparison results are different, and the number of interfaces is 6 high-speed interfaces, then the first configuration information is determined to be "target CPU0 connects to n=3 GPUs", and the corresponding communication topology is "1-to-3 star topology".

[0102] It should be noted that the value of n in this step matches the hardware design specifications of the entire rack. Different racks can preset the range of n values ​​(e.g., n=2, 3, 4, etc.) according to their maximum expansion capabilities. The controller can obtain the selectable values ​​of n by reading the preset configuration file, and then accurately output the number of GPU connections in the first configuration information. At the same time, the communication topology information in the first configuration information is indirectly determined by the comparison results of frequency signals. When the first and second frequency signals are the same, it corresponds to a "symmetrical 1-to-2n star topology"; when the signals are different, it corresponds to a "distributed 1-to-n star topology", realizing the synchronous identification of the number of connections and topology information.

[0103] The rack configuration status detection method provided in this application first compares the first GPU frequency signal received by the first interface of the target CPU with the second GPU frequency signal received by the second interface of the target CPU. The target CPU is any CPU in the rack, the first interface is the CPU interface in the odd-numbered position of the target CPU, and the second interface is the CPU interface in the even-numbered position of the target CPU. By comparing the GPU frequency signals received by the odd and even interfaces of the CPU, the number of CPU and GPU connections and the communication topology are automatically identified, eliminating the need for manual statistics or configuration, thus achieving rapid and automatic identification of configuration information and significantly improving detection efficiency. If the first GPU frequency signal and the second GPU frequency signal are the same, the first configuration information is determined to be that the target CPU is connected to 2n GPUs; where n is an integer not less than 2. If the first GPU frequency signal and the second GPU frequency signal are different, the first configuration information is determined to be that the target CPU is connected to n GPUs. The determination of the first configuration information is completed based on the native frequency signals of the hardware, reducing errors caused by manual intervention, lowering the cost of fault diagnosis and maintenance, and adapting to the flexible detection needs of multiple configuration scenarios.

[0104] In one possible implementation, Figure 5 This is a schematic diagram of the hardware architecture for rack configuration status detection provided in an embodiment of this application, as shown below. Figure 5 As shown, the controller, as the execution entity, first collects two types of core information through the CPLD of CPU node 0: one is the GPU frequency signal transmitted from the CPLDs of GPU nodes 0 to 7 to the CPU node interface via a hardware link; the other is the physical port connection information between the GPU chips of each GPU node and the switching chip node 0, which is obtained by the GPU chips through in-band communication. This information is synchronized by the GPU chips to their own CPLD via the IIC bus and then reported to the CPLD of the CPU node; at the same time, the slot connection information between each GPU node and the CPU node is collected.

[0105] Subsequently, the controller compares the GPU frequency signals received by the odd and even interfaces of the CPU node to determine the number of connections and communication topology between the CPU node 0 and the GPU node (i.e., the first configuration information); then it compares the physical port connection information of the GPU node with the preset port table to determine the configuration status of the GPU and the switching chip (i.e., the second configuration status); and combines the first configuration information with the slot connection information to verify the slot matching between the CPU and the GPU (i.e., the third configuration status).

[0106] Finally, the controller makes a joint judgment on the second and third configuration states: if both are normal, it outputs the conclusion that the entire rack configuration is normal; if either state is abnormal, it generates a log containing abnormal nodes and abnormal types by associating the first configuration information, thus completing the automated detection of the entire rack configuration status.

[0107] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0108] Figure 6 This is a schematic diagram of the rack configuration status detection device provided in an embodiment of this application. Figure 6 As shown, the entire cabinet includes at least one CPU, at least one GPU, and at least one switching chip; the CPU is connected to the GPU; the GPU is connected to the switching chip; the device includes:

[0109] The acquisition module 61 is used to acquire the GPU frequency signal received by the interface of each CPU, the physical port connection information between each GPU and the switching chip, and the slot connection information between each CPU and GPU.

[0110] The determination module 62 is used to determine the first configuration information between each CPU and the connected GPU based on the GPU frequency signal received by each CPU's interface.

[0111] The determination module 62 is also used to determine the second configuration state between the GPU and the connected switching chip based on the connection information of each physical port;

[0112] The determination module 62 is also used to determine the third configuration state between each CPU and the connected GPU based on the connection information of each slot and the first configuration information;

[0113] The determination module 62 is also used to determine the configuration status of the entire cabinet based on the second configuration status and the third configuration status.

[0114] In one possible implementation, the determining module 62 is specifically used for:

[0115] The first GPU frequency signal received by the first interface of the target CPU is compared with the second GPU frequency signal received by the second interface of the target CPU; the target CPU is any CPU in the rack; the first interface is the CPU interface in the target CPU that is in an odd position; the second interface is the CPU interface in the target CPU that is in an even position.

[0116] If the first GPU frequency signal is the same as the second GPU frequency signal, the first configuration information is determined to be that the target CPU connects to 2n GPUs; n is an integer not less than 2.

[0117] If the first GPU frequency signal is different from the second GPU frequency signal, the first configuration information is determined to be that the target CPU is connected to n GPUs.

[0118] In one possible implementation, based on the physical port connection information, a second configuration state between the GPU and the connected switching chip is determined, and the determining module 62 is further configured to:

[0119] If the physical port connection information of the target GPU is successfully matched with the preset physical port connection table, the second configuration state between the target GPU and the connected switching chip is determined to be normal; the target GPU is any GPU connected to the target CPU.

[0120] Otherwise, the second configuration state is determined to be a configuration error.

[0121] In one possible implementation, based on the slot connection information and the first configuration information, a third configuration state between each CPU and the connected GPU is determined, and the determining module 62 is further configured to:

[0122] Based on the first configuration information, when the slot connection information of the target CPU and the target GPU is successfully compared with the preset physical slot connection table, the third configuration state between the target CPU and the target GPU is determined to be normal.

[0123] Otherwise, the third configuration state is determined to be a configuration anomaly.

[0124] In one possible implementation, the configuration state of the entire rack is determined based on the second configuration state and the third configuration state, and the determining module 62 is further configured to:

[0125] If both the second and third configuration states are normal, the configuration status of the entire rack is determined to be normal.

[0126] Otherwise, based on the first configuration information, the second configuration status, and the third configuration status, an abnormal configuration status log for the entire rack is generated.

[0127] In one possible implementation, the first configuration information includes the number of GPUs connected to the target CPU and the communication topology information between the target CPU and the GPU.

[0128] The apparatus provided in this application embodiment can be used to execute the determination method in any of the above embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0129] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented in software via processing element calls, while others are implemented in hardware. Additionally, these modules can be fully or partially integrated together, or implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through the integrated logic circuits in the hardware of the processor element or through software instructions.

[0130] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 7 As shown, the electronic device may include: a processor 71, a memory 72, and computer program instructions stored in the memory 72 and executable on the processor 71. When the processor 71 executes the computer program instructions, it implements the method provided in any of the foregoing embodiments.

[0131] Optionally, the various components of the electronic device can be connected via a system bus.

[0132] The memory 72 can be a separate memory unit or a memory unit integrated into the processor 71. The number of processors 71 can be one or more.

[0133] It should be understood that the processor 71 can be a Central Processing Unit (CPU), or other general-purpose processors 71, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor 71 can be a microprocessor 71, or any conventional processor 71. The steps of the method disclosed in this application can be directly manifested as being executed by the hardware processor 71, or being executed by a combination of hardware and software modules within the processor 71.

[0134] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. Memory 72 may include random access memory (RAM) 72, and may also include non-volatile memory (NVM) 72, such as at least one disk storage device 72.

[0135] All or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable memory 72. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned memory 72 (storage medium) includes: read-only memory 72 (ROM), RAM, flash memory 72, hard disk, solid-state hard disk, magnetic tape, floppy disk, optical disk, and any combination thereof.

[0136] The electronic device provided in this application embodiment can be used to execute the method provided in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0137] This application provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the above-described method.

[0138] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0139] Optionally, a readable storage medium can be coupled to the processor, enabling the processor to read information from and write information to the readable storage medium. Alternatively, the readable storage medium can be an integral part of the processor. The processor and the readable storage medium can reside in application-specific integrated circuits (ASICs). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0140] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and the at least one processor can implement the above-described method when executing the computer program.

[0141] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for detecting the configuration status of a server rack, wherein the server rack includes at least one CPU, at least one GPU, and at least one switching chip, wherein the CPU is connected to the GPU; and the GPU is connected to the switching chip; characterized in that, include: The GPU frequency signal received by the interface of each CPU, the physical port connection information between each GPU and the switching chip, and the slot connection information between each CPU and the GPU are obtained. Based on the GPU frequency signal received by the interface of each CPU, determine the first configuration information between each CPU and the connected GPU; Based on the physical port connection information, a second configuration state between the GPU and the connected switching chip is determined; Based on the slot connection information and the first configuration information, a third configuration state between each CPU and the connected GPU is determined; The configuration status of the entire cabinet is determined based on the second configuration status and the third configuration status.

2. The method according to claim 1, characterized in that, The step of determining the first configuration information between each CPU and the connected GPU based on the GPU frequency signal received from the interface of each CPU includes: The first GPU frequency signal received by the first interface of the target CPU is compared with the second GPU frequency signal received by the second interface of the target CPU; the target CPU is any CPU in the rack; the first interface is the CPU interface in the odd-numbered position of the target CPU; the second interface is the CPU interface in the even-numbered position of the target CPU; If the first GPU frequency signal is the same as the second GPU frequency signal, the first configuration information is determined to be that the target CPU is connected to 2n GPUs; where n is an integer not less than 2. If the first GPU frequency signal is different from the second GPU frequency signal, the first configuration information is determined to be that the target CPU is connected to n GPUs.

3. The method according to claim 2, characterized in that, Determining the second configuration state between the GPU and the connected switching chip based on the physical port connection information includes: If the physical port connection information of the target GPU is successfully compared with the preset physical port connection table, the second configuration state between the target GPU and the connected switching chip is determined to be normal; the target GPU is any GPU connected to the target CPU. Otherwise, the second configuration state is determined to be a configuration error.

4. The method according to claim 3, characterized in that, The step of determining the third configuration state between each CPU and the connected GPU based on the slot connection information and the first configuration information includes: Based on the first configuration information, when the slot connection information of the target CPU and the target GPU is successfully compared with the preset physical slot connection table, the third configuration state between the target CPU and the target GPU is determined to be normal configuration. Otherwise, the third configuration state is determined to be a configuration anomaly.

5. The method according to claim 4, characterized in that, Determining the configuration state of the entire cabinet based on the second configuration state and the third configuration state includes: If both the second configuration state and the third configuration state are normal, the configuration state of the entire cabinet is determined to be normal. Otherwise, based on the first configuration information, the second configuration status, and the third configuration status, an abnormal configuration status log for the entire cabinet is generated.

6. The method according to claim 2, characterized in that, The first configuration information includes the number of GPUs connected to the target CPU, and the communication topology information between the target CPU and the GPU.

7. A rack configuration status detection device, the rack comprising at least one CPU, at least one GPU, and at least one switching chip, wherein the CPU is connected to the GPU; and the GPU is connected to the switching chip; characterized in that, include: The acquisition module is used to acquire the GPU frequency signal received by the interface of each CPU, the physical port connection information between each GPU and the switching chip, and the slot connection information between each CPU and the GPU. The determining module is used to determine first configuration information between each CPU and the connected GPU based on the GPU frequency signal received by the interface of each CPU; The determining module is further configured to determine a second configuration state between the GPU and the connected switching chip based on the connection information of each physical port; The determining module is further configured to determine a third configuration state between each CPU and the connected GPU based on the slot connection information and the first configuration information. The determining module is further configured to determine the configuration state of the entire cabinet based on the second configuration state and the third configuration state.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 6.

10. A computer program, characterized in that, The computer program includes a computer program stored in a computer-readable storage medium, which at least one processor can read from the computer-readable storage medium, and which, when executing the computer program, can implement the method described in any one of claims 1 to 6.

Images