Fault detection method, storage medium and electronic device

Abstract

The application discloses a fault detection method, a storage medium and an electronic device, and relates to the technical field of electronics. The method comprises the following steps: when it is determined that a starting mode is a target restart mode, and it is determined that an interconnection link is running at a predetermined data transmission rate range, generating a fault detection result based on register data corresponding to a physical interface used for realizing inter-processor interconnection communication in each processor, a processor model, and topology information of an electronic device to be detected. The method can avoid misjudging normal speed reduction, bandwidth reduction and the like as faults, thereby improving fault detection accuracy, thereby solving the technical problem of low fault detection accuracy in related methods, and achieving the technical effect of accurately detecting faults.

Description

Technical Field

[0001] This application relates to the field of electronic technology, and in particular to fault detection methods, storage media, and electronic devices. Background Technology

[0002] Servers and other electronic devices can employ modular architectures. For example, two Central Processing Units (CPUs) can form a dual-socket module (i.e., a dual-socket motherboard). Multiple dual-socket modules can be interconnected via Ultra Path Interconnect (UPI) links. Any reduction in speed or bandwidth in any UPI cable will affect server performance or the number of CPUs that can operate normally. During the power-on startup process of servers and other electronic devices, the interconnect links need to undergo an initialization phase, switching from a low-speed training state to a full-speed operating state. During the low-speed training phase, the speed and bandwidth of the interconnect links have not yet reached their rated values. Related UPI link fault detection methods may misjudge the situation where the speed and bandwidth of the interconnect links have not yet reached their rated values ​​during the initialization phase as a link fault, resulting in low fault detection accuracy. Summary of the Invention

[0003] This application provides a fault detection method, storage medium, and electronic device to at least address the problem of low fault detection accuracy in related technologies.

[0004] This application provides a fault detection method, comprising: determining the boot mode of an electronic device under test, wherein the electronic device under test includes multiple processors, and the multiple processors transmit data through interconnect links; when the boot mode is determined to be a target restart mode, and the interconnect links are determined to be operating at a predetermined data transmission rate range, generating a fault detection result based on the register data corresponding to the physical interface used to realize inter-processor interconnect communication in each processor, the processor model, and the topology information of the electronic device under test, wherein the topology information includes processor presence information and physical interface connection relationships, and the presence information represents the installation status of the processor in the slot.

[0005] This application also provides a fault detection device, including: a first determining module and a first generating module. The first determining module is used to determine the startup mode of an electronic device under test, wherein the electronic device under test includes multiple processors, and the multiple processors transmit data through interconnection links; the first generating module is used to generate a fault detection result based on the register data corresponding to the physical interface used to realize inter-processor interconnection communication in each processor, the processor model, and the topology information of the electronic device under test, when the startup mode is determined to be a target restart mode and the interconnection link is determined to be operating at a predetermined data transmission rate range.

[0006] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described fault detection methods.

[0007] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described fault detection methods.

[0008] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described fault detection methods.

[0009] This application, by determining the boot mode as the target restart mode and the interconnection link as operating within a predetermined data transmission rate range, generates fault detection results based on the register data corresponding to the physical interface used to implement inter-processor communication in each processor, the processor model, and the topology information of the electronic device under test. This avoids misjudging normal speed reduction, bandwidth reduction, and other situations as faults, thereby improving the accuracy of fault detection. Thus, it solves the technical problem of low fault detection accuracy in related methods and achieves the technical effect of improving fault detection accuracy. Attached Figure Description

[0010] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of a dual-socket server system architecture provided in an embodiment of this application.

[0012] Figure 2 This is a schematic diagram of a four-way server system architecture provided in an embodiment of this application.

[0013] Figure 3 This is a schematic diagram of an eight-way server system architecture provided in an embodiment of this application.

[0014] Figure 4 This is a flowchart of a fault detection method provided in an embodiment of this application.

[0015] Figure 5 This is a flowchart of a bandwidth reduction detection method provided in an embodiment of this application.

[0016] Figure 6 This is a flowchart of a deceleration detection method provided in an embodiment of this application.

[0017] Figure 7 This is a flowchart illustrating a connection relationship detection method provided in an embodiment of this application.

[0018] Figure 8 A flowchart of a fault detection method provided in another embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0020] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0021] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Rack-mount servers, tower servers, edge servers, and other electronic devices are typically based on a dual-socket server architecture. A dual-socket server can only support a maximum of two CPUs, but it can also be configured as a single-socket server by installing only one CPU; that is, a dual-socket server is compatible with a single-socket server configuration. For a dual-socket server architecture, the number of CPUs supported is fixed, supporting a maximum of two CPUs, and each CPU has the same peripherals, such as a fixed number of supported links, a fixed number of supported memory slots, and the same supported peripheral buses. The only difference is that when a dual-socket server is used as a single-socket server, the UPI bus used for CPU interconnection can be freed up, thereby increasing the number of Peripheral Component Interconnect (PCI) buses to support more devices. However, since the number of bus links directly connecting the dual-socket server to the CPU is fixed, its bandwidth capability is also fixed.

[0023] Figure 1This is a schematic diagram of a dual-socket server system architecture provided in an embodiment of this application.

[0024] As shown in Figure 1, the server includes CPU0 and CPU1, as well as a Basic Input / Output System (BIOS), a Platform Controller Hub (PCH), and a Baseboard Management Controller (BMC). CPU0 integrates a Management Engine (ME) and also has General Purpose Input / Output (GPIO) signal units.

[0025] The BIOS communicates with CPU0 via a Low Pin Count (LPC) bus, Serial Peripheral Interface (SPI), and Quad Serial Peripheral Interface (QSPI) bus. The BIOS also outputs a POST_DONE signal to the BMC. The BMC communicates bidirectionally with CPU0 via the Platform Environment Control Interface (PECI), LPC / SPI / QSPI buses, and PCI. The BMC also interacts with CPU0 via a USB over LAN virtual universal serial bus. The BMC also supports bidirectional interaction with the BIOS via the Intelligent Platform Management Interface (IPMI) protocol and the Redfish hardware management protocol. The BIOS also establishes a signal connection with the PCH. CPU0 is equipped with a UPI physical interface, enabling high-speed interconnect communication with CPU1 via a UPI link.

[0026] Figure 2 This is a schematic diagram of a four-way server system architecture provided in an embodiment of this application.

[0027] contrast Figure 1 and Figure 2 It can be seen that, Figure 2 The addition of CPU2 and CPU3 compared to the previous version... Figure 1CPU0 and CPU1 are directly connected via a UPI link, forming the first dual-socket motherboard module; CPU2 and CPU3 are also directly connected via a UPI link, forming the second dual-socket motherboard module. The two dual-socket modules are interconnected via multiple external UPI links, forming multiple cross-connection channels. For example, CPU0 is interconnected with both CPU2 and CPU3 via UPI links; CPU1 is interconnected with both CPU2 and CPU3 via UPI links. For the communication and connection methods between the BIOS, BMC, PCH, and CPU0, please refer to the above text, and will not be repeated here.

[0028] To address the issue of fixed bandwidth capacity in dual-socket servers, multiple CPUs can be incorporated into the server design concept. This multiplies the number of CPUs, thereby doubling the number of PCI links and memory slots. Consequently, the bandwidth capacity of each interface is also multiplied as the number of interfaces increases.

[0029] Figure 3 This is a schematic diagram of an eight-way server system architecture provided in an embodiment of this application.

[0030] like Figure 3 The circles shown represent servers, with the numbers in the center indicating server IDs. "MB" represents a dual-path module. The "3" on one side of the circle (e.g., bottom left, bottom right, top left, top right) indicates that the CPU is currently using a total of 3 UPI links. An eight-path server contains 8 CPUs, numbered 0 to 7. Each CPU has UPI physical ports numbered 0, 1, and 2. Each CPU establishes interconnection links with other CPUs through three UPI physical interfaces. UPI links are connected in four ways: direct connection via blind-fit connectors and flying wires to the motherboard, through the dual-path motherboard printed circuit board, cross-connection via blind-fit connectors and flying wires to the motherboard, and motherboard interconnection via high-speed backplane connectors through the backplane.

[0031] For example, four dual-socket servers can be physically linked via UPI cables to support an eight-socket server. However, when the UPI cable or the UPI bus between motherboards fails, such as when UPI speed is reduced or bandwidth is decreased, the overall performance of the server will be severely affected, causing the eight-socket server to be downgraded to a four-socket server, a two-socket server, or a single-socket server. Therefore, it is necessary to report the location information of the UPI fault and the specific core information such as the CPU and UPI number in a timely and accurate manner. In addition, the server itself may experience speed reduction or bandwidth reduction at a certain fixed stage, which is a normal design feature. However, the relevant methods can usually only detect a single type of fault and may misjudge the above normal design as a link fault, resulting in low fault detection accuracy.

[0032] In view of this, embodiments of this application provide a fault detection method, and the fault detection method is described in detail below in conjunction with the execution flow of the fault detection method.

[0033] Figure 4 This is a flowchart illustrating a fault detection method provided in one embodiment of this application. Figure 4 As shown, an embodiment of this application provides a fault detection method including operations S410 and S420.

[0034] In operation S410, the startup mode of the electronic device under test is determined. The electronic device under test includes multiple processors, which transmit data via interconnect links.

[0035] In S420, when the boot mode is determined to be the target restart mode and the interconnection link is determined to be operating within a predetermined data transmission rate range, a fault detection result is generated based on the register data corresponding to the physical interface used to realize inter-processor interconnection communication in each processor, the processor model, and the topology information of the electronic device under test. The topology information includes the processor's presence information and the physical interface connection relationship. The presence information represents the processor's installation status in the slot.

[0036] For example, the electronic device under test may include multiple processors, such as a multi-processor storage device, a multi-processor switch, or a server. For instance, the electronic device under test may be an eight-way server, which can be connected via a UPI interface between two server boards and a UPI cable. The interconnection link may include a UPI cable.

[0037] The BIOS can read the server's boot mode. Only when the boot mode is determined to be the target restart mode and the interconnection link is operating within a predetermined data transfer rate range, can it generate fault detection results based on the register data corresponding to the physical interface used to implement inter-processor communication in each processor, the processor model, and the topology information of the electronic device under test.

[0038] The startup modes of the electronic device under test include a variety of modes. In some startup modes, the interconnection link may experience speed reduction and bandwidth reduction, but this is a normal state and not caused by a fault. The target startup mode can include startup modes other than the aforementioned startup modes.

[0039] After the BIOS is loaded, it can read the processor's built-in UPI register to identify the current operating level of each interconnect link. The current operating level can include the low-speed initialization level (corresponding to Slow mode) and the rated high-speed operating level (corresponding to Fast mode). Slow mode is the low-speed level for initial negotiation after UPI power-on or restart. It is a necessary pre-process for link communication establishment. Both cold restart (i.e., the whole machine is completely powered off and then powered on) and warm restart (i.e., the whole machine is restarted without powering on) will enter Slow mode and cannot be skipped. UPI will only switch to Fast mode after completing Slow mode. Therefore, the temporary speed reduction caused by Slow mode is normal. By generating fault detection results only when the interconnect link is determined to be operating within the predetermined data transmission rate range, the normal speed reduction phenomenon in Slow mode can be avoided from being misjudged as an interconnect link failure.

[0040] For example, the physical interface may include UPI ports on the processor, such as port 0, port 1, and port 2 of CPU0; the register data corresponding to the physical interface used to implement inter-processor interconnection communication can be read from the port-specific link status register; the register data may include link status values ​​recorded in the internal hardware registers of each processor, such as bandwidth, data transmission rate, etc.; the physical interface connection relationship can characterize the correspondence between the physical interfaces of each processor, such as port 1 of CPU0 can correspond to port 0 of CPU2.

[0041] Based on the register data corresponding to the physical interface used to realize inter-processor communication in each processor, the processor model, and the topology information of the electronic device under test, the generated fault detection results can perform comprehensive detection of fault types. The generated fault detection results can cover multiple fault types such as abnormal bandwidth and abnormal data transmission rate, thereby realizing multi-dimensional fault detection.

[0042] According to the embodiments of this application, by determining that the boot mode is the target restart mode and that the interconnection link is operating within a predetermined data transmission rate range, the fault detection result is generated based on the register data, processor model, and topology information of the electronic device under test corresponding to the physical interface used to implement inter-processor interconnection communication in each processor. This avoids misjudging normal speed reduction, bandwidth reduction, and other situations as faults, thereby improving the accuracy of fault detection.

[0043] According to an embodiment of this application, the target restart mode includes restarting the electronic device under test without power failure; the fault detection result includes at least one of the following: a first detection result indicating abnormal bandwidth of the interconnection link, a second detection result indicating abnormal data transmission speed of the interconnection link, and a third detection result indicating abnormal connection relationship of the interconnection link compared to historical connection relationship.

[0044] For example, the startup mode can include a warm start and a cold start. A warm start can be a restart operation performed while maintaining power supply and without power interruption, while a cold start can be the startup process of a server being powered on again after a power outage. During a cold start, the interconnect link is in a low-speed training state, and its data transmission speed is low, but this is a normal state and not an abnormal slowdown. However, if a slowdown or bandwidth reduction occurs during a warm start, it may be due to an interconnect link failure. Therefore, the operation of generating fault detection results based on the register data corresponding to the physical interface used to implement inter-processor communication in each processor, the processor model, and the topology information of the electronic device under test can only be performed under the premise that the target restart mode, including restarting the electronic device under test without power interruption. In the case of a cold start, this operation can be ignored to avoid false alarms.

[0045] Interconnect links can have various types of faults, such as speed reduction, bandwidth reduction, topology anomalies, etc. By generating a first detection result indicating bandwidth anomalies, a second detection result indicating data transmission speed anomalies, and a third detection result indicating abnormal connection relationships compared to historical connection relationships, multi-dimensional fault detection can be performed to achieve comprehensive fault monitoring.

[0046] According to embodiments of this application, the generation of fault detection results based on register data corresponding to the physical interface used to implement inter-processor interconnection communication in each processor, processor model, and topology information of the electronic device under test includes: determining a first deviation of the bandwidth of the interconnection link from a predetermined bandwidth and a second deviation of the data transmission speed of the interconnection link from a predetermined speed based on register data, processor model, and processor presence information; generating a first detection result based on the first deviation and a second detection result based on the second deviation; and generating a third detection result based on the current topology information obtained after the current startup of the electronic device under test and the historical topology information obtained after the last startup of the electronic device under test.

[0047] For example, the first deviation and the second deviation can be calculated based on register data, CPU model, and processor availability information, respectively. The first deviation characterizes the degree of deviation of the actual link width from the predetermined width, and the second deviation characterizes the degree of deviation of the actual link speed from the predetermined speed. A large first deviation (e.g., exceeding a first predetermined value set according to actual needs) indicates an abnormal bandwidth in the interconnection link; for example, if the bandwidth is significantly lower than the predetermined bandwidth, it indicates a bandwidth reduction abnormality in the interconnection link. Similarly, a large second deviation (e.g., exceeding a second predetermined value set according to actual needs) indicates an abnormal data transmission rate in the interconnection link; for example, if the data transmission rate is significantly lower than the predetermined speed, it indicates a speed reduction abnormality in the interconnection link. The predetermined bandwidth and predetermined speed can be set according to actual needs and are not limited here.

[0048] The current topology information can characterize the number of CPUs detected during this startup, the UPI port connection relationship between each CPU, etc. The historical topology information can characterize the baseline topology information recorded and stored during the last normal startup. The third detection result can characterize the abnormal changes in the interconnection link connection relationship compared to the past.

[0049] For example, the topology detected during this startup can be compared with the topology recorded during the last startup. If changes are found (such as a decrease in the number of CPUs), it indicates a topology anomaly, and a third detection result can be generated.

[0050] According to the embodiments of this application, by determining the first deviation of the interconnect link bandwidth from the predetermined bandwidth and the second deviation of the interconnect link data transmission speed from the predetermined speed based on register data, processor model and processor in-situ information, a first detection result and a second detection result are generated accordingly. A third detection result is generated based on the current topology information obtained after the current startup of the electronic device under test and the historical topology information obtained after the last startup of the electronic device under test. This can detect fault conditions in an all-round way and cover multiple fault types.

[0051] According to an embodiment of this application, determining the first deviation of the interconnect link bandwidth from a predetermined bandwidth based on register data, processor model, and processor availability information includes: reading interconnect link bandwidth data from target register data corresponding to the target processor when the processor configuration status is determined to be normal based on the processor model and processor availability information; and determining the first deviation based on the interconnect link bandwidth data and the predetermined bandwidth.

[0052] For example, a normal processor configuration includes, for instance, the processor being properly installed in its socket and being recognized by the system. Target register data can be read from hardware registers located within each CPU, specifically used to record the status of its interconnect links. Target register data may include physical parameters characterizing the current CPU state, such as bandwidth, data transfer rate, and connection status. The target processor can be the processor currently being detected.

[0053] If the bandwidth data of the interconnection link deviates significantly from the predetermined bandwidth, it indicates that there is a bandwidth anomaly in the interconnection link.

[0054] According to an embodiment of this application, the fault detection method further includes: traversing each processor to determine the processor presence information and processor model of each processor; for the currently traversed target processor, if the processor presence information of the target processor indicates that the processor is installed in the corresponding processor socket and is recognized by the system, and the processor model of the target processor conforms to a predetermined model, then the processor configuration status is determined to be normal.

[0055] For example, starting with CPU0, it can check one by one whether each CPU is inserted into the processor socket and recognized by the system, and check whether the CPU model matches the predetermined model.

[0056] According to the embodiments of this application, by determining whether the processor configuration status is abnormal, configuration abnormalities can be directly filtered out, avoiding subsequent invalid detection and preventing system problems caused by accessing non-existent hardware.

[0057] According to an embodiment of this application, the interconnection link includes at least one data transmission channel; the bandwidth data includes: the bandwidth of the interconnection link when at least one data transmission channel is fully open, and the bandwidth of the interconnection link when at least one data transmission channel is partially open.

[0058] For example, an interconnect link may include multiple parallel data transmission channels. When all data transmission channels are open (e.g., all data transmission channels are operational), the interconnect link's bandwidth reaches its maximum value, i.e., it reaches the L0 state. When only some data transmission channels are open, the interconnect link's bandwidth decreases, i.e., it reaches the L0p state.

[0059] The L0p state is a predefined normal energy-saving state that achieves energy saving by actively shutting down some channels. It is not a fault. Therefore, when a bandwidth reduction is detected in the interconnection link and it is determined that the interconnection link is in the L0p state, the detection result can be ignored and no fault report can be made.

[0060] The L0 state requires all data transmission channels to be operational. Therefore, if the interaction link still experiences bandwidth reduction in the L0 state, it indicates that the interaction link may be faulty and a fault report is required.

[0061] According to the embodiments of this application, by detecting both the fully open and partially open states of the data transmission channel, a fault report can be made when the bandwidth reduction phenomenon still occurs in the interactive link when the data transmission channel is fully open, and no fault report is made when the bandwidth reduction phenomenon occurs in the interactive link when the data transmission channel is partially open. This can effectively distinguish between normal energy-saving bandwidth reduction and abnormal fault bandwidth reduction, avoid invalid alarms caused by misjudgment, and improve the accuracy of fault detection.

[0062] According to an embodiment of this application, determining a second deviation of the data transmission speed of the interconnect link from a predetermined speed includes: reading the data transmission speed of the interconnect link from target register data when the processor configuration status is determined to be normal based on the processor model and processor in-situ information; and determining the second deviation based on the data transmission speed of the interconnect link and the predetermined speed.

[0063] For example, if the CPU is not present, the system may access a non-existent register, which may lead to system crash or invalid data. If the CPU model is incompatible, the different register address mappings may lead to incorrect data reading. Therefore, under the premise that the configuration status is normal, the system can then judge whether an anomaly has occurred based on the second deviation, thereby preventing the reading of non-existent or invalid register data and avoiding system crashes and false alarms.

[0064] For example, starting with the first CPU, we can determine its presence and type. If the CPU is present and matches the expected model, we can check its UPI ports and read the register values ​​of the UPI link. We can then analyze the read data to determine if a speed reduction issue has occurred. Different CPU models may support different numbers of UPI ports; we can determine which UPI ports on the CPU are actually present. If the UPI ports are present, we can then read the register values. If the second deviation exceeds a predetermined value, we can determine that the interconnect link has experienced a speed reduction anomaly. The predetermined value can be set according to actual needs and is not limited here.

[0065] According to embodiments of this application, by reading the data transmission speed of the interconnect link from the target register data and determining the second deviation only when the processor configuration status is determined to be normal based on the processor model and processor in-situ information, the safety of the detection operation and the validity of the data are ensured, thereby ensuring the reliability of the fault detection results.

[0066] According to an embodiment of this application, the fault detection result further includes a fourth detection result; the fault detection method further includes: generating a fourth detection result when it is determined that the processor's configuration state is abnormal based on the processor model and the processor's in-situ information, wherein the fourth detection result indicates the processor's number and physical port number in the abnormal configuration state.

[0067] For example, if the expected model of a CPU is A, but the actual read model is B, it means that the CPU does not meet the expected configuration. It is also possible that the CPU is not physically installed in the CPU socket of the motherboard, or although the CPU is physically installed, it cannot be correctly recognized by the system due to poor contact or other reasons. In the above cases, it can be determined that the processor configuration status is abnormal, and a fourth detection result can be generated. The fourth detection result may include the processor's serial number, the serial number of the physical port configured by the processor, etc.

[0068] According to the embodiments of this application, when it is determined that the processor's configuration status is abnormal based on the processor model and the processor's presence information, a fourth detection result is generated, which can accurately locate the faulty processor and facilitate subsequent accurate repair of the faulty processor, thereby improving the operational efficiency of fault detection.

[0069] According to an embodiment of this application, generating a third detection result based on the current topology information obtained after the current startup of the electronic device under test and the historical topology information obtained after the last startup of the electronic device under test includes: comparing the number of historical processors in the historical topology information with the number of current processors in the current topology information to obtain a first comparison result; comparing the historical physical interface connection relationship in the historical topology information with the current physical interface connection relationship in the current topology information to obtain a second comparison result; determining whether the connection relationship has changed based on the first comparison result and the second comparison result; and generating a third detection result when the connection relationship has changed.

[0070] For example, historical topology information from the last startup can be obtained, such as from non-volatile memory. This historical topology information may record the number of CPUs in operation at the time of the last startup, and which UPI ports connected the CPUs to each other.

[0071] The number of CPUs detected during this startup and the connection status of each UPI port can be compared item by item with historical records to determine if the topology has changed. For example, all CPUs can be traversed in order of their CPU numbers to count the total number of CPUs that the current system can recognize and communicate with normally. For CPUs that the current system can recognize and communicate normally, all their UPI physical ports can be traversed, and the link status register corresponding to each port can be read to confirm whether the corresponding UPI link has been physically established. If the CPUs and all their UPI ports are normal, it can be determined that the connection relationship has not been abnormal.

[0072] According to an embodiment of this application, a first comparison result is obtained by comparing the number of historical processors in the historical topology information with the number of current processors in the current topology information; a second comparison result is obtained by comparing the historical physical interface connection relationship in the historical topology information with the current physical interface connection relationship in the current topology information; based on the first comparison result and the second comparison result, it is determined whether the connection relationship has changed, which can accurately determine whether the connection relationship has changed and accurately identify the faulty CPU and faulty port.

[0073] According to an embodiment of this application, the fault detection method further includes: when the startup mode is that the electronic device under test is started after being powered off, the operation of generating fault detection results is blocked.

[0074] For example, in cold start mode, that is, in the mode where the electronic device under test is started after being powered off, the predetermined speed and predetermined bandwidth of the interconnect link have not yet reached the rated value. However, this is a normal state and is not a speed reduction and bandwidth reduction caused by a fault. Therefore, in the case where the startup mode is when the electronic device under test is started after being powered off, the operation of generating fault detection results can be blocked to avoid misjudging the normal situation as a fault state.

[0075] According to embodiments of this application, by disabling the operation of generating fault detection results when the startup mode is that the electronic device under test is started after power failure, it is possible to avoid misjudging normal speed reduction and bandwidth reduction as a fault state, thereby improving the accuracy of fault detection.

[0076] According to an embodiment of this application, the fault detection method further includes: when it is determined that the startup mode is the restart of the electronic device under test without power failure, and it is determined that the interconnection link is operating at a rate lower than a predetermined data transmission rate, periodically determining the data transmission rate of the interconnection link until a preset termination condition is reached, wherein the preset termination condition includes determining that the interconnection link is operating at a predetermined data transmission rate.

[0077] For example, if the electronic device under test restarts without being powered off, it indicates that the electronic device under test is in a hot start mode; if the interconnection link operates at a rate lower than the predetermined data transmission rate, it indicates that the interconnection link is in a slow mode.

[0078] To avoid misjudging Slow mode as a hardware fault, fault detection can be omitted in Slow mode. Instead, the data transmission rate of the interconnect link can be determined periodically to check whether the data transmission rate of the interconnect link has reached a predetermined data transmission rate range. Once the predetermined data transmission rate range has been reached, a fault detection result is generated based on the register data corresponding to the physical interface used to realize inter-processor communication in each processor, the processor model, and the topology information of the electronic device under test. The predetermined data transmission rate range can be set according to actual needs and is not limited here.

[0079] The time interval for periodically determining the data transmission rate of the interconnection link can be set according to actual needs, such as 100 milliseconds, 1 second, etc.

[0080] According to the embodiments of this application, by not performing the operation of generating fault detection results based on the register data, processor model, and topology information of the electronic device under test corresponding to the physical interface used to realize inter-processor interconnection communication in each processor when the startup mode is determined to be restarting without power failure and the interconnection link is determined to be operating at a rate lower than a predetermined data transmission rate, it is possible to avoid misjudging the normal speed reduction state as a fault state, thereby improving the accuracy of fault detection; by periodically determining the data transmission rate of the interconnection link, it is possible to detect the situation where the interconnection link is operating at a predetermined data transmission rate range in a timely manner, ensuring the operational efficiency of fault detection.

[0081] According to an embodiment of this application, the preset termination condition further includes: the interconnection link operates at a rate lower than the predetermined data transmission rate for a preset duration; the fault detection method further includes: when the interconnection link operates at a rate lower than the predetermined data transmission rate for a preset duration, determining that the data transmission speed of the interconnection link is abnormal.

[0082] For example, the preset duration can be set according to actual needs, such as 10 seconds, 30 seconds, 60 seconds, etc.

[0083] If the data transmission rate of the interconnection link is consistently lower than the predetermined data transmission rate range and this state is maintained for a preset duration, it indicates that there may be an abnormal data transmission speed in the interconnection link. Therefore, it is no longer necessary to wait for the link to recover, the periodic determination operation can be terminated directly, and the connection can be directly confirmed to have experienced a speed reduction fault.

[0084] According to the embodiments of this application, by directly determining that the data transmission speed of the interconnection link is abnormal when the interconnection link operates at a rate lower than the predetermined data transmission rate for a preset duration, the system can avoid waiting indefinitely for the interconnection link to recover, thereby ensuring that the fault can be detected and reported in a timely manner.

[0085] According to an embodiment of this application, the fault detection method further includes: determining the target model and target physical port number of the processor corresponding to the faulty interconnection link characterized by the fault detection result; and generating a fault log based on the target model and target physical port number.

[0086] For example, a faulty interconnect link may include at least one of the following: an interconnect link with abnormal data transmission speed, an interconnect link with abnormal bandwidth, or an interconnect link with an abnormal connection relationship compared to historical connection relationships.

[0087] For example, the model and port number of the server with the abnormal configuration status can also be determined, and a fault log can be generated accordingly.

[0088] After a speed reduction, bandwidth reduction, or connection abnormality occurs, the speed reduction information can be sent to the BMC via IPMI command. The CPU model and CPU port number of the speed reduction, bandwidth reduction, or connection abnormality are reported to the BMC for recording and display in the log.

[0089] According to the embodiments of this application, by determining the target model and target physical port number of the processor corresponding to the faulty interconnect link, and generating a fault log based on the target model and target physical port number, the location of the faulty link can be quickly located without having to check each one in a complex system containing multiple processors and multiple interconnect links, thereby improving the efficiency of fault location and maintenance.

[0090] Figure 5 This is a flowchart of a bandwidth reduction detection method provided in an embodiment of this application.

[0091] like Figure 5As shown, the electronic device under test can be a server. The number of processors in the current server can be read. Starting from the first processor, the presence and model of the current processor can be determined. It can be determined whether the processor is present and the model matches. If the processor is not present or does not match, the Fast Link Interconnect bandwidth reduction logic can be exited, and the Basic Input / Output System can continue to start in this case. If the processor is present and the model matches, the register value corresponding to the Fast Link Interconnect port (i.e., UPI port) of the current processor can be read. It can be determined whether the Fast Link Interconnect port register value has a bandwidth reduction problem. If no bandwidth reduction problem occurs, the bandwidth status of the next processor can be determined, and it can be determined whether it is the last processor. If it is determined to be the last processor, the Basic Input / Output System continues to start. If it is determined not to be the last processor, the bandwidth status of the next processor can be determined.

[0092] In the event of a bandwidth reduction problem, the processor model and port number of the processor experiencing the bandwidth reduction problem can be sent to the baseboard management controller via the intelligent platform management interface. The baseboard management controller can receive the log information and record it on the baseboard management controller interface for display. Then, it can determine whether it is the last processor. If it is confirmed to be the last processor, the basic input / output system continues to start. If it is not confirmed to be the last processor, it can continue to determine the bandwidth status of the next processor and perform the operation of determining whether the next processor is in place and the model matches.

[0093] Figure 6 This is a flowchart of a deceleration detection method provided in an embodiment of this application.

[0094] like Figure 6 As shown, the electronic device under test can be a server. The system can read the number of processors currently on the server and, starting with the first processor, determine its presence and model. It can determine if the processor is present and if the model matches. If the processor is not present or does not match, the system can exit the fast-channel interconnect speed-down logic, and the basic input / output system can continue to start in this case. If the processor is present and the model matches, the system can read the register value corresponding to the fast-channel interconnect port of the current processor. It can determine if there is a speed-down problem with the fast-channel interconnect port register value. If no speed-down problem occurs, the system can continue to determine the speed of the next processor (such as data transfer speed) and whether it is the last processor. If it is the last processor, the basic input / output system continues to start. If it is not the last processor, the system can continue to determine the speed status of the next processor.

[0095] In the event of a speed reduction issue, the processor model and port number of the processor experiencing the speed reduction problem can be sent to the baseboard management controller via the intelligent platform management interface command. For example, the processor model and port number corresponding to the slowed interconnect can be sent to the baseboard management controller. The baseboard management controller receives the log information and records it on the baseboard management controller interface for display. It can determine whether it is the last processor. If it is determined to be the last processor, the basic input / output system continues to start. If it is determined not to be the last processor, it can continue to determine the speed status of the next processor and perform the operation of determining whether the next processor is in place and the model matches.

[0096] Figure 7 This is a flowchart illustrating a connection relationship detection method provided in an embodiment of this application.

[0097] like Figure 7 As shown, the fast lane interconnect topology from the last startup can be retrieved. This can be performed during both cold and warm starts. It can also check whether the current fast lane interconnect topology is the same as the previous one. Figure 1 If the processors are in sync, the Basic Input / Output System (BIOS) will continue to boot. If they are not in sync, the system can determine if the processor is present and of a matching model. If the processor is not present or is mismatched, the BIOS can continue to boot. If the processor is present and of a matching model, the system can read the register value corresponding to the Fast Interconnect Port (UPI) of the current processor. It can then determine if there is an abnormal connection relationship in the UPI register value. If no abnormal connection relationship is found, the system can continue to determine the connection relationship of the next processor and whether it is the last processor. If it is the last processor, the BIOS will continue to boot. If it is not the last processor, the system can continue to determine the connection relationship of the next processor.

[0098] In the event of a connection error, the processor model and port number of the processor with the connection error can be sent to the baseboard management controller via the intelligent platform management interface. The baseboard management controller receives the log information and records it on the baseboard management controller interface for display. Then, it can determine whether it is the last processor. If it is the last processor, the basic input / output system continues to start. If it is not the last processor, it can continue to determine the connection status of the next processor and perform the operation of determining whether the next processor is in place and the model matches.

[0099] Figure 8 A flowchart of a fault detection method provided in another embodiment of this application.

[0100] like Figure 8 As shown, the electronic device under test can be a server. After the server starts, it can be determined whether the server is in a cold start. In the case of a cold start, the fast channel interconnect bandwidth reduction and speed reduction detection logic can be skipped, and the server can be restarted. In the case of a non-cold start, such as a warm start, it can be read whether the startup mode of the fast channel interconnect link is the first mode. The first mode can be the Slow mode. If it is the first mode, the fast channel interconnect bandwidth reduction and speed reduction detection logic can be skipped. If it is not the first mode, it can continue to read whether the startup mode of the fast channel interconnect link is the second mode. The second mode can be the Fast mode. If it is not the second mode, the fast channel interconnect bandwidth reduction and speed reduction detection logic can also be skipped, and the server can continue to start. If it is the second mode, it can be read whether the fast channel interconnect link is in the first state or the second state. The first state can be the L0 state, and the second state can be the L0p state. If it is not in the first state or the second state, the fast channel interconnect bandwidth reduction and speed reduction detection logic can be skipped. If it is in the first state or the second state, it can sequentially enter the fast channel interconnect topology module, the fast channel interconnect bandwidth reduction module, and the fast channel interconnect speed reduction module, and then the server can continue to start. The processing logic of the fast channel interconnect topology module can be found in [reference needed]. Figure 7 The flowchart for connection relationship detection is shown. The processing logic of the fast-channel interconnect bandwidth reduction module can be found in [reference needed]. Figure 5 The flowchart for bandwidth reduction detection is shown. The processing logic of the fast-channel interconnect speed reduction module can be found in [reference needed]. Figure 6 The flowchart for the speed reduction detection shown here will not be elaborated further on the processing logic of the fast channel interconnect topology module, the fast channel interconnect bandwidth reduction module, and the fast channel interconnect speed reduction module.

[0101] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0102] Embodiments of this application also provide a fault detection device, including a first determining module and a first generating module. The first determining module is used to determine the startup mode of an electronic device under test, wherein the electronic device under test includes multiple processors, and the multiple processors transmit data through interconnect links. The first generating module is used to generate a fault detection result based on the register data corresponding to the physical interface used to realize inter-processor interconnection communication in each processor, the processor model, and the topology information of the electronic device under test, when the startup mode is determined to be a target restart mode and the interconnect link is determined to be operating at a predetermined data transmission rate range.

[0103] For example, the first generation module includes: a determining submodule, used to determine a first deviation of the bandwidth of the interconnect link from a predetermined bandwidth and a second deviation of the data transmission speed of the interconnect link from a predetermined speed based on register data, processor model, and processor in-situ information; a first generation submodule, used to generate a first detection result based on the first deviation and a second detection result based on the second deviation; and a second generation submodule, used to generate a third detection result based on the current topology information obtained after the current startup of the electronic device under test and the historical topology information obtained after the last startup of the electronic device under test.

[0104] The determination submodule includes a first determination unit, which is used to read the bandwidth data of the interconnection link from the target register data corresponding to the target processor, provided that the processor configuration status is normal based on the processor model and the processor in-situ information; and to determine a first deviation based on the bandwidth data of the interconnection link and a predetermined bandwidth.

[0105] The determination submodule also includes a traversal unit, which is used to traverse each processor, determine the processor presence information and processor model of each processor; for the target processor currently being traversed, if the processor presence information of the target processor indicates that the processor is installed in the corresponding processor socket and is recognized by the system, and the processor model of the target processor matches the predetermined model, then the processor configuration status is determined to be normal.

[0106] The determining submodule may further include: a reading unit, used to read the data transmission speed of the interconnect link from the target register data when the processor configuration status is determined to be normal based on the processor model and processor in-situ information; and a second determining unit, used to determine a second deviation based on the data transmission speed of the interconnect link and a predetermined speed.

[0107] The second generation submodule may include a generation unit, which is used to compare the number of historical processors in the historical topology information with the number of current processors in the current topology information to obtain a first comparison result; compare the historical physical interface connection relationship in the historical topology information with the current physical interface connection relationship in the current topology information to obtain a second comparison result; determine whether the connection relationship has changed based on the first comparison result and the second comparison result; and generate a third detection result when the connection relationship has changed.

[0108] The fault detection device may also include a shielding module to shield the operation of generating fault detection results when the startup mode is that the electronic device under test is started after being powered off.

[0109] The fault detection device may further include a second determining module, used to periodically determine the data transmission rate of the interconnection link when the startup mode is determined to be restarting the electronic device under test without power failure, and the interconnection link is determined to be operating at a rate lower than a predetermined data transmission rate range, until a preset termination condition is reached, wherein the preset termination condition includes determining that the interconnection link is operating at a predetermined data transmission rate range.

[0110] The fault detection device may also include a third determining module, used to determine that the data transmission speed of the interconnection link is abnormal when the interconnection link operates at a rate lower than the predetermined data transmission rate for a preset duration.

[0111] The fault detection device may also include a second generation module, used to determine the target model and target physical port number of the processor corresponding to the faulty interconnection link characterized by the fault detection result; and to generate a fault log based on the target model and target physical port number.

[0112] For a description of the features in the embodiment corresponding to the fault detection device, please refer to the relevant description of the embodiment corresponding to the fault detection method, which will not be repeated here.

[0113] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above-described fault detection method embodiments.

[0114] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described fault detection method embodiments when it is run.

[0115] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0116] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described fault detection method embodiments.

[0117] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described fault detection method embodiments.

[0118] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be performed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0119] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0120] The above provides a detailed description of a fault detection method provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A fault detection method, characterized in that, include: The startup mode of the electronic device under test is determined, wherein the electronic device under test includes multiple processors, and the multiple processors transmit data to each other through interconnection links; If the startup mode is determined to be the target restart mode and the interconnection link is determined to be operating within a predetermined data transmission rate range, a fault detection result is generated based on the register data corresponding to the physical interface used to realize inter-processor interconnection communication in each processor, the processor model, and the topology information of the electronic device under test. The topology information includes processor presence information and physical interface connection relationship. The presence information represents the installation status of the processor in the slot. The target restart mode includes restarting the electronic device under test without power failure. The method further includes: When the startup mode is that the electronic device under test is started after a power failure, the operation of generating fault detection results is disabled. When it is determined that the startup mode is the restart of the electronic device under test without power failure, and when it is determined that the interconnection link is operating at a rate lower than the predetermined data transmission rate range, the data transmission rate of the interconnection link is periodically determined until a preset termination condition is reached. The preset termination condition includes determining that the interconnection link is operating at the predetermined data transmission rate range, and the preset termination condition also includes: the interconnection link operating at a rate lower than the predetermined data transmission rate range for a preset duration. If the interconnection link operates at a rate lower than the predetermined data transmission rate for a predetermined duration, the data transmission speed of the interconnection link is determined to be abnormal.

2. The fault detection method according to claim 1, characterized in that, The fault detection result includes at least one of the following: a first detection result indicating an abnormal bandwidth of the interconnection link, a second detection result indicating an abnormal data transmission speed of the interconnection link, and a third detection result indicating that the connection relationship of the interconnection link is abnormal compared to historical connection relationships.

3. The fault detection method according to claim 2, characterized in that, The generation of fault detection results based on the register data corresponding to the physical interface used for inter-processor communication in each of the processors, the processor model, and the topology information of the electronic device under test includes: Based on the register data, the processor model, and the processor in-situ information, a first deviation of the bandwidth of the interconnect link from a predetermined bandwidth is determined, and a second deviation of the data transmission speed of the interconnect link from a predetermined speed is determined. The first detection result is generated based on the first deviation, and the second detection result is generated based on the second deviation; The third detection result is generated based on the current topology information obtained after the current startup of the electronic device under test, and the historical topology information obtained after the previous startup of the electronic device under test.

4. The fault detection method according to claim 3, characterized in that, The determination of the first deviation of the interconnect link bandwidth from the predetermined bandwidth based on the register data, the processor model, and the processor availability information includes: If the processor configuration status is determined to be normal based on the processor model and the processor availability information, the bandwidth data of the interconnection link is read from the target register data corresponding to the target processor. The first deviation is determined based on the bandwidth data of the interconnection link and the predetermined bandwidth.

5. The fault detection method according to claim 4, characterized in that, The method further includes: Traverse each of the processors to determine the processor in-situ information and processor model of each processor; For the target processor currently being traversed, if the processor's processor presence information indicates that the processor is installed in the corresponding processor socket and is recognized by the system, and the processor model of the target processor matches a predetermined model, then the processor's configuration status is determined to be normal.

6. The fault detection method according to claim 4, characterized in that, The interconnection link includes at least one data transmission channel; The bandwidth data includes: the bandwidth of the interconnection link when all at least one data transmission channel is open, and the bandwidth of the interconnection link when at least one data transmission channel is partially open.

7. The fault detection method according to claim 4, characterized in that, The second deviation of the data transmission speed of the interconnection link from the predetermined speed includes: If the processor configuration is determined to be normal based on the processor model and the processor's availability information, the data transmission speed of the interconnect link is read from the target register data. The second deviation is determined based on the data transmission speed of the interconnection link and the predetermined speed.

8. The fault detection method according to claim 4, characterized in that, The fault detection results also include a fourth detection result; The method further includes: If, based on the processor model and the processor's availability information, it is determined that the processor's configuration status is abnormal, a fourth detection result is generated, wherein the fourth detection result indicates the processor's ID and physical port ID, indicating the abnormal configuration status.

9. The fault detection method according to claim 3, characterized in that, The generation of the third detection result based on the current topology information obtained after the current startup of the electronic device under test and the historical topology information obtained after the previous startup of the electronic device under test includes: The number of historical processors in the historical topology information is compared with the number of current processors in the current topology information to obtain a first comparison result; The historical physical interface connection relationships in the historical topology information are compared with the current physical interface connection relationships in the current topology information to obtain a second comparison result; Based on the first comparison result and the second comparison result, determine whether the connection relationship has changed; When the connection relationship changes, the third detection result is generated.

10. The fault detection method according to claim 1, characterized in that, The method further includes: Determine the target model and target physical port number of the processor corresponding to the faulty interconnect link characterized by the fault detection results; A fault log is generated based on the target model and the target physical port number.

11. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the fault detection method according to any one of claims 1 to 10.

12. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the fault detection method as described in any one of claims 1 to 10 when executing the computer program.

Images