Accelerator card fault determination method and device, equipment and medium
By monitoring and parsing the operating data of the accelerator card and combining the fault diagnosis rule table, the problem that BMC cannot collect and analyze the accelerator card errors of the server equipment is solved, and the accurate determination of the cause of the failure is achieved, reducing the risk of server downtime.
Patent Information
- Application Number
- CN202410866547.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-30
- Publication Date
- 2024-11-05
AI Technical Summary
BMC cannot collect and analyze various errors of server device accelerator cards in-band, and cannot determine the cause of the failure, resulting in out-of-band fault diagnosis that cannot cover all scenarios.
By monitoring the operating data of the accelerator card, determine whether there is a fault, and obtain the original data of the relevant registers in the operating system based on the operating system, analyze the data through the substrate management controller, and generate the fault analysis results in combination with the fault diagnosis rule table to determine the cause of the fault.
It realizes indirect acquisition of accelerator card-related register data and preliminary failure analysis in BMC, accurately analyzes health status and failure risks, reduces the probability of server downtime and improves work efficiency.
Smart Images

Figure CN118897747A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of server technology, and in particular to an acceleration card fault determination method, device, equipment and medium. Background Art
[0002] In traditional servers, equipment failures such as network card failure, central processing unit (CPU) failure, peripheral component interconnect express (PCIe) bus hang, graphics processing unit (GPU) failure, and accelerator card failure are inevitable. These failures may cause system downtime or restart, causing trouble for users in using the server.
[0003] Currently, the Baseboard Management Controller (BMC) supports the use of the Management Component Transport Protocol (MCTP) over the Inter-Integrated Circuit (I2C) protocol to obtain the health status information of the server's internal devices, and supports the acquisition of CPU register data out-of-band through the Platform Environment Control Interface (PECI), and performs fault location and analysis according to the fault diagnosis rule table. However, the BMC cannot collect and analyze the uncorrectable errors (UCE), correctable errors (CE), and FAULT faults of the server device accelerator card in-band, so it is impossible to monitor its health status according to the fault diagnosis rule table provided by the device manufacturer. This results in out-of-band fault diagnosis not being able to cover all scenarios.
[0004] In view of the above problems, how to solve the problem that the current BMC cannot collect and analyze multiple errors of the server device accelerator card in-band and cannot determine the cause of the failure is an urgent problem to be solved by technical personnel in this field. Summary of the invention
[0005] The purpose of the present invention is to provide an accelerator card fault determination method, device, equipment and medium to solve the problem that the current BMC cannot collect and analyze multiple errors of the server device accelerator card in-band and cannot determine the cause of the fault.
[0006] In order to solve the above technical problems, the present invention provides a method for determining an accelerator card fault, comprising:
[0007] Monitoring the operation data of the accelerator card, and determining whether the accelerator card has a fault according to the operation data of the accelerator card;
[0008] If the accelerator card fails, obtaining original data of each register related to the accelerator card based on the operating system in-band;
[0009] Parsing the raw data through a baseboard management controller to generate a first fault analysis result of the acceleration card;
[0010] Obtain a fault diagnosis rule table, and generate a second fault analysis result of the accelerator card according to the fault diagnosis rule table and the original data; wherein the fault diagnosis rule table is a rule table provided by the manufacturer of the accelerator card for analyzing the health status and fault risk of the accelerator card;
[0011] The fault cause of the accelerator card is determined according to the first fault analysis result and the second fault analysis result.
[0012] On the one hand, monitoring the operating data of the accelerator card and determining whether the accelerator card has a fault according to the operating data of the accelerator card includes:
[0013] Execute troubleshooting scripts based on the server's operating system;
[0014] Monitoring the operation data of the accelerator card through the fault diagnosis script; wherein the operation data at least includes uncorrectable errors, correctable errors, temperature, voltage and current;
[0015] Determine whether the number of times the uncorrectable errors are triggered and the number of times the correctable errors are triggered by the accelerator card reach corresponding thresholds;
[0016] If the number of times the uncorrectable errors of the accelerator card are triggered reaches a corresponding threshold, or the number of times the correctable errors of the accelerator card are triggered reaches a corresponding threshold, it is confirmed that the accelerator card is faulty, and the step of obtaining the original data of each register related to the accelerator card based on the operating system band is entered.
[0017] On the other hand, the obtaining of original data of each register related to the accelerator card based on the operating system in-band includes:
[0018] Through the script collection module under the operating system, the status information and description information of the acceleration card in the enhanced error reporting register, the machine check architecture register, the kernel controller error log register, the kernel status register, the first acceleration card status register and the second acceleration card status register are obtained to obtain the original data.
[0019] On the other hand, parsing the original data by a baseboard management controller to generate a first fault analysis result of the acceleration card includes:
[0020] Based on a preset communication protocol, calling a script sending module to transmit the original data to the baseboard management controller;
[0021] Parsing the original data through the fault diagnosis module of the baseboard management controller to generate the first fault analysis result;
[0022] The first fault analysis result and the original data are stored in a data log.
[0023] On the other hand, the acquiring of the fault diagnosis rule table and generating the second fault analysis result of the accelerator card according to the fault diagnosis rule table and the original data includes:
[0024] Determine a setting rule for the data bits of each register based on the fault diagnosis rule table;
[0025] Determine, according to the original data, whether the eighth bit in the kernel controller error log register, which is used to indicate the first fault source from the kernel, is 1;
[0026] If it is confirmed that the eighth bit in the kernel controller error log register for indicating the first fault source from the kernel is 1, it is confirmed that the micro control unit of the acceleration card fails, and it is determined according to the original data whether the seventh bit in the kernel controller error log register for indicating the second fault source from the kernel is 1;
[0027] If it is confirmed that the seventh bit in the core controller error log register indicating the second fault source from the core is not 1, it is confirmed that the core of the micro control unit has no fault;
[0028] If it is confirmed that the seventh bit in the kernel controller error log register indicating the second fault source from the kernel is 1, it is confirmed that the kernel of the micro control unit fails, and the cause of the kernel fault of the micro control unit is determined according to the kernel status register and the kernel diagnosis process;
[0029] If it is confirmed that the eighth bit in the kernel controller error log register for indicating the first fault source from the kernel is not 1, it is confirmed that the micro control unit of the acceleration card has not failed, and it is determined according to the original data whether the sixth bit in the kernel controller error log register for indicating the first fault source from the memory is 1;
[0030] If it is confirmed that the sixth bit in the kernel controller error log register indicating the first fault source from the memory is not 1, it is confirmed that the memory of the micro control unit has no fault;
[0031] If it is confirmed that the sixth bit in the core controller error log register indicating the first fault source from the memory is 1, it is confirmed that the memory of the micro control unit has a fault, and the cause of the memory fault of the micro control unit is determined based on other register data bits and input and output diagnostic processes.
[0032] On the other hand, the acquiring of the fault diagnosis rule table and generating the second fault analysis result of the accelerator card according to the fault diagnosis rule table and the original data includes:
[0033] Determine, according to the original data, whether the eighth bit in the first accelerator card status register, which is used to indicate a first fault source from a non-core, is 1;
[0034] If it is confirmed that the eighth bit in the first accelerator card status register indicating the first fault source from the non-core is not 1, it is confirmed that the accelerator card does not have an overheating or overcurrent fault;
[0035] If it is confirmed that the eighth bit in the first accelerator card status register indicating the first fault source from the non-core is 1, it is confirmed that the accelerator card has an overheating and overcurrent fault, and the values of the seventh and eighth bits in the second accelerator card status register are determined according to the original data;
[0036] When the eighth bit in the second accelerator card status register is 1, it is confirmed that the accelerator card has an overheating fault;
[0037] When the seventh bit in the second acceleration card status register is 1, it is confirmed that an overcurrent fault occurs in the acceleration card.
[0038] On the other hand, determining the fault cause of the accelerator card according to the first fault analysis result and the second fault analysis result includes:
[0039] Determining whether the first fault analysis result and the second fault analysis result are the same;
[0040] If it is confirmed that the first fault analysis result and the second fault analysis result are the same, the first fault analysis result or the second fault analysis result is used as the final fault result of the accelerator card, so as to determine the fault cause of the accelerator card according to the final fault result;
[0041] If it is confirmed that the first fault analysis result and the second fault analysis result are different, taking the second fault analysis result as the final fault result of the accelerator card;
[0042] The parsing method of the baseboard management controller for the original data is updated according to the final fault result.
[0043] In order to solve the above technical problems, the present invention further provides an accelerator card fault determination device, comprising:
[0044] A monitoring module, used for monitoring the operation data of the accelerator card, and judging whether the accelerator card has a fault according to the operation data of the accelerator card; if the accelerator card has a fault, triggering the acquisition module;
[0045] The acquisition module is used to acquire original data of each register related to the accelerator card based on the operating system band;
[0046] An analysis module, used for analyzing the raw data through a baseboard management controller to generate a first fault analysis result of the acceleration card;
[0047] A generating module, configured to obtain a fault diagnosis rule table, and generate a second fault analysis result of the accelerator card according to the fault diagnosis rule table and the original data; wherein the fault diagnosis rule table is a rule table provided by the manufacturer of the accelerator card for analyzing the health status and fault risk of the accelerator card;
[0048] A determination module is used to determine the fault cause of the acceleration card according to the first fault analysis result and the second fault analysis result.
[0049] In order to solve the above technical problems, the present invention further provides an accelerator card fault determination device, comprising:
[0050] Memory for storing computer programs;
[0051] The processor is used to implement the steps of the above-mentioned acceleration card failure determination method when executing the computer program.
[0052] In order to solve the above technical problem, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned acceleration card fault determination method are implemented.
[0053] The method for determining an accelerator card fault provided by the present invention monitors the operating data of the accelerator card, and determines whether the accelerator card has a fault based on the operating data of the accelerator card; if the accelerator card has a fault, the original data of each register related to the accelerator card is obtained based on the operating system band; the original data is parsed by a baseboard management controller to generate a first fault analysis result of the accelerator card; a fault diagnosis rule table is obtained, and a second fault analysis result of the accelerator card is generated based on the fault diagnosis rule table and the original data; wherein the fault diagnosis rule table is a rule table provided by the manufacturer of the accelerator card for analyzing the health status and fault risk of the accelerator card; and the cause of the fault of the accelerator card is determined based on the first fault analysis result and the second fault analysis result. The beneficial effects of the present invention are that whether the accelerator card has a fault is monitored through the operation data of the accelerator card; if the accelerator card has a fault, the original data of the relevant registers of the accelerator card is obtained in-band based on the operating system, and the baseboard management controller parses the original data to generate a first fault analysis result, thereby realizing indirect acquisition of the relevant register data of the accelerator card and preliminary fault analysis by the BMC in-band; further, based on the original data and a fault diagnosis rule table that can accurately analyze the health status and fault risk of the accelerator card, a second fault analysis result of the accelerator card is generated, the first fault analysis result and the second fault analysis result are compared, and finally the cause of the fault of the accelerator card is accurately determined, thereby solving the problem that the current BMC cannot collect and analyze various errors of the accelerator card of the server device in-band and cannot determine the cause of the fault, thereby reducing the probability of server downtime and improving the work efficiency of the server.
[0054] On the other hand, the present invention specifically implements the monitoring of the operation data of the accelerator card and the judgment of whether the accelerator card has a fault by executing the fault diagnosis script through the operating system of the server, so as to accurately determine the cause of the fault of the accelerator card in the future. Through the script collection module under the operating system, the state information and description information of the accelerator card in the enhanced error reporting register, the machine check architecture register, the kernel controller error log register, the kernel status register, the first accelerator card status register and the second accelerator card status register are obtained, so as to achieve the complete acquisition of the original data. Based on the preset communication protocol, the script sending module is called to transmit the original data to the baseboard management controller; the original data is parsed by the fault diagnosis module of the baseboard management controller to generate the first fault analysis result; the first fault analysis result and the original data are stored in the data log, so as to achieve the preliminary analysis of the original data and the storage of the first fault analysis result and the original data, so as to facilitate the subsequent analysis of the cause of the fault.
[0055] In addition, the present invention also provides an acceleration card fault determination device, equipment and medium, with the same effect as above. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0057] Figure 1 A flowchart of a method for determining an accelerator card fault provided by an embodiment of the present invention;
[0058] Figure 2 A schematic diagram of an accelerator card fault determination device provided by an embodiment of the present invention;
[0059] Figure 3 A schematic diagram of an accelerator card fault determination device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0061] The core of the present invention is to provide an accelerator card fault determination method, device, equipment and medium to solve the problem that the current BMC cannot collect and analyze multiple errors of the server device accelerator card in-band and cannot determine the cause of the fault.
[0062] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0063] Currently, BMC supports the use of MCTP over I2C protocol to obtain the health status information of internal devices in the server, and supports the acquisition of CPU register data out-of-band through PECI, and performs fault location and analysis according to the fault diagnosis rule table. However, BMC cannot collect and analyze UCE, CE and FAULT type faults of the server device accelerator card in-band, so it is impossible to monitor its health status according to the fault diagnosis rule table provided by the device manufacturer. This results in the inability of out-of-band fault diagnosis to cover all scenarios. In order to solve the above problems, the present invention provides an accelerator card fault determination method.
[0064] Figure 1 The flowchart of a method for determining an accelerator card fault provided by an embodiment of the present invention is as follows. Figure 1 As shown, the method includes:
[0065] S10: monitor the operation data of the accelerator card, and determine whether the accelerator card has a fault according to the operation data of the accelerator card; if the accelerator card has a fault, proceed to step S11.
[0066] The server acceleration card, also known as a server accelerator card or hardware acceleration card, is a hardware device used to improve server performance. It is usually inserted into the PCIe slot of the server motherboard and uses a dedicated hardware chip to accelerate the performance of the server when processing network traffic, data transmission, and computing tasks. In specific implementations, the server's operating system (OS) monitors the operating data of the acceleration card and determines whether the acceleration card is faulty based on the operating data of the acceleration card.
[0067] It should be noted that the operation data of the acceleration card may include UCE, CE, temperature, voltage, current and other data. In this embodiment, it can be determined whether the acceleration card has a fault through one or more of the data, which is not limited in this embodiment.
[0068] S11: Acquire original data of each register related to the accelerator card based on the operating system in-band.
[0069] When it is confirmed that there is no fault in the accelerator card, the current process is terminated. When it is confirmed that there is a fault in the accelerator card, the original data of each register related to the accelerator card is obtained based on the server's OS band. In this embodiment, there is no restriction on the specific type of each register related to the accelerator card, for example, it may include an Advanced Error Report (AER) register and a Machine Check Architecture (MCA) register; in addition, there is no restriction on the specific content of the original data, for example, it may include kernel operation data, memory operation data, etc.
[0070] S12: parsing the original data through the baseboard management controller to generate a first fault analysis result of the acceleration card.
[0071] Furthermore, the BMC obtains raw data, that is, indirectly obtains raw data of each register related to the accelerator card through the OS band, and preliminarily analyzes the extracted raw data strictly according to preset analysis rules to generate a first fault analysis result of the accelerator card.
[0072] It should be noted that the preset parsing rules are not limited in this embodiment and are determined according to specific implementation conditions. During the parsing process, the BMC can complete the preliminary parsing of the original data through the fault diagnosis function provided by the WEB page.
[0073] S13: Obtain a fault diagnosis rule table, and generate a second fault analysis result of the acceleration card according to the fault diagnosis rule table and the original data.
[0074] The fault diagnosis rule table is a rule table provided by the manufacturer of the accelerator card for analyzing the health status and fault risk of the accelerator card.
[0075] Further obtain a fault diagnosis rule table, and generate a second fault analysis result of the accelerator card based on the fault diagnosis rule table and the original data. It should be noted that the fault diagnosis rule table is a rule table provided by the manufacturer of the accelerator card for analyzing the health status and fault risk of the accelerator card, which is specifically designed and provided by the manufacturer's underlying driver development personnel, and can record the specific faults, CE, UCE, etc. of a certain part of the accelerator card to the specified register, and then form a specific diagnostic rule or process. In this embodiment, there is no restriction on the specific generation process of the second fault analysis result, which depends on the specific implementation situation.
[0076] S14: Determine the fault cause of the accelerator card according to the first fault analysis result and the second fault analysis result.
[0077] It is understandable that since the first fault analysis result is obtained by preliminary analysis of the BMC, it may not cover all diagnostic rules and is therefore only suitable for reference. In order to obtain the accurate cause of the fault, the first fault analysis result of the preliminary analysis of the BMC is specifically compared with the second fault analysis result obtained based on the fault diagnosis rule table, so as to accurately determine the cause of the accelerator card failure. In this embodiment, there is no restriction on the process of determining the cause of the accelerator card failure, which depends on the specific implementation situation.
[0078] In this embodiment, whether the accelerator card has a fault is monitored through the operation data of the accelerator card; if the accelerator card has a fault, the original data of the relevant registers of the accelerator card is obtained in-band based on the operating system, and the baseboard management controller parses the original data to generate a first fault analysis result, thereby realizing the indirect acquisition of the relevant register data of the accelerator card and the preliminary fault analysis by the BMC in-band; further, based on the original data and a fault diagnosis rule table that can accurately analyze the health status and fault risk of the accelerator card, a second fault analysis result of the accelerator card is generated, the first fault analysis result and the second fault analysis result are compared, and finally the cause of the failure of the accelerator card is accurately determined, thereby solving the problem that the current BMC cannot collect and analyze various errors of the accelerator card of the server device in-band and cannot determine the cause of the failure, thereby reducing the probability of server downtime and improving the work efficiency of the server.
[0079] Based on the above embodiments, in some embodiments, monitoring the operation data of the accelerator card and determining whether the accelerator card has a fault according to the operation data of the accelerator card include:
[0080] S101: executing a fault diagnosis script based on the operating system of the server;
[0081] S102: monitoring the operation data of the accelerator card through a fault diagnosis script;
[0082] The operation data at least includes uncorrectable errors, correctable errors, temperature, voltage and current;
[0083] S103: respectively determining whether the triggering times of the uncorrectable errors and the correctable errors of the accelerator card have reached corresponding thresholds; if the triggering times of the uncorrectable errors of the accelerator card have reached the corresponding threshold, or the triggering times of the correctable errors of the accelerator card have reached the corresponding threshold, proceeding to step S104;
[0084] S104: Confirm that the accelerator card is faulty, and proceed to step S11.
[0085] In the specific implementation, the fault diagnosis script is specifically executed in-band under the server OS. It should be noted that the relevant instructions executed by the fault diagnosis script are provided by the accelerator card manufacturer, and the user generates the fault diagnosis script according to the relevant instructions.
[0086] The fault diagnosis script monitors the operation data of the accelerator card in real time, including at least UCE, CE, temperature, voltage and current, and determines whether the trigger times of UCE and CE of the accelerator card reach the corresponding thresholds. It should be noted that the threshold of the trigger times of UCE and the threshold of the trigger times of CE in this embodiment can be the same threshold or different thresholds. In this embodiment, there is no restriction on the threshold value, which depends on the specific implementation situation.
[0087] If the number of triggering times of the UCE of the accelerator card reaches the corresponding threshold, or the number of triggering times of the CE of the accelerator card reaches the corresponding threshold, it is confirmed that there is a fault in the accelerator card, and the step of obtaining the original data of each register related to the accelerator card based on the operating system band is entered.
[0088] In this embodiment, the operating system of the server executes a fault diagnosis script to monitor the operation data of the accelerator card and determine whether the accelerator card has a fault, so as to accurately determine the cause of the fault of the accelerator card later.
[0089] Based on the above embodiments, in some embodiments, the original data of each register related to the accelerator card is obtained in the operating system band, including:
[0090] S111: Obtain the status information and description information of the accelerator card in the enhanced error reporting register, the machine check architecture register, the kernel controller error log register, the kernel status register, the first accelerator card status register and the second accelerator card status register through the script collection module under the operating system to obtain the original data.
[0091] In a specific implementation, when it is confirmed that the accelerator card has a fault, the original data of the accelerator card related registers are collected through the script collection module under the server OS. The related registers specifically include the AER register, the MCA register, the kernel controller error log register MCerrlogReg, the kernel status register MC0_Status, the first accelerator card status register MStateReg and the second accelerator card status register MState_Status.
[0092] At the same time, the original data specifically includes the status information and description information of the accelerator card. Specifically, the status information specifically includes the UCE count of the accelerator card's internal chip computing core, the first level (L1) cache and the second level (L2) cache, the FAULT type fault information of the logical operation unit (Arithmetic Logic Unit, ALU) and the floating point unit (Floating Point Unit, FPU), the UCE and CE information of the memory, etc. The description information specifically includes the accelerator card's in-place status, temperature, voltage, current and other information, which can describe whether the accelerator card has problems such as overheating and overcurrent.
[0093] In this embodiment, the script collection module under the operating system is used to obtain the status information and description information of the accelerator card in the enhanced error reporting register, the machine check architecture register, the kernel controller error log register, the kernel status register, the first accelerator card status register and the second accelerator card status register, thereby achieving complete acquisition of the original data.
[0094] Based on the above embodiments, in some embodiments, the baseboard management controller parses the original data to generate a first fault analysis result of the acceleration card, including:
[0095] S121: Based on a preset communication protocol, calling a script sending module to transmit the original data to a baseboard management controller;
[0096] S122: parsing the original data through a fault diagnosis module of the baseboard management controller to generate a first fault analysis result;
[0097] S123: Store the first fault analysis result and the original data in a data log.
[0098] In a specific implementation, after the script collection module completes the collection of raw data, the script sending module under the server OS will transmit the raw data to the BMC. It should be noted that the data exchange transmission between the script sending module and the BMC needs to strictly comply with the preset communication protocol. This embodiment does not limit the preset communication protocol. For example, the data exchange transmission between the script sending module and the BMC can be performed through the standard Intelligent Platform Management Interface (IPMI) command protocol. The raw data is further parsed by the fault diagnosis module of the BMC to generate a first fault analysis result, and the first fault analysis result and the raw data are stored in the data log.
[0099] In this embodiment, based on the preset communication protocol, the script sending module is called to transmit the original data to the baseboard management controller; the original data is parsed by the fault diagnosis module of the baseboard management controller to generate a first fault analysis result; the first fault analysis result and the original data are stored in the data log, thereby realizing the preliminary analysis of the original data and the storage of the first fault analysis result and the original data, so as to facilitate the subsequent analysis of the cause of the fault.
[0100] On the basis of the above embodiments, in some embodiments, obtaining a fault diagnosis rule table, and generating a second fault analysis result of the accelerator card according to the fault diagnosis rule table and the original data include:
[0101] S131: Determine the setting rule of the data bit of each register based on the fault diagnosis rule table;
[0102] S132: judging whether the eighth bit in the kernel controller error log register indicating the first fault source from the kernel is 1 according to the original data; if it is confirmed that the eighth bit in the kernel controller error log register indicating the first fault source from the kernel is 1, proceeding to step S133; if it is confirmed that the eighth bit in the kernel controller error log register indicating the first fault source from the kernel is not 1, proceeding to step S136;
[0103] S133: confirm that the microcontroller unit of the accelerator card fails, and determine whether the seventh bit in the kernel controller error log register indicating the second fault source from the kernel is 1 according to the original data; if it is confirmed that the seventh bit in the kernel controller error log register indicating the second fault source from the kernel is not 1, then proceed to step S134; if it is confirmed that the seventh bit in the kernel controller error log register indicating the second fault source from the kernel is 1, then proceed to step S135;
[0104] S134: confirming that the core of the microcontroller unit has no fault;
[0105] S135: confirming that the core of the micro control unit fails, and determining the cause of the core failure of the micro control unit according to the core status register and the core diagnosis process;
[0106] S136: confirm that the microcontroller unit of the accelerator card has no fault, and determine whether the sixth bit in the kernel controller error log register indicating the first fault source from the memory is 1 according to the original data; if it is confirmed that the sixth bit in the kernel controller error log register indicating the first fault source from the memory is not 1, then proceed to step S137; if it is confirmed that the sixth bit in the kernel controller error log register indicating the first fault source from the memory is 1, then proceed to step S138;
[0107] S137: confirm that the memory of the microcontroller unit is not faulty;
[0108] S138: Confirm that the memory of the microcontroller unit is faulty, and determine the cause of the memory fault of the microcontroller unit based on other register data bits and input / output diagnostic procedures.
[0109] In the specific implementation, since the fault diagnosis rule table is designed and provided by the manufacturer's underlying driver development personnel, it can record the specific faults, CE, UCE, etc. of a certain part of the accelerator card into the specified register, and then form specific diagnostic rules or processes. Therefore, in order to generate the second fault analysis result, it is first necessary to determine the setting rules of the data bits of each register based on the fault diagnosis rule table.
[0110] Further, it is determined whether the eighth bit (Bit8) of the kernel controller error log register MCerrlogReg indicating the first fault source (FirstMCerrSrcFromCore) from the kernel is 1 according to the original data; if it is confirmed that the eighth bit of the kernel controller error log register indicating the first fault source from the kernel is 1, it is confirmed that the microcontroller unit (MCU) of the acceleration card is faulty. To further determine the cause of the MCU fault, it is determined whether the seventh bit (Bit7) of the kernel controller error log register MCerrlogReg indicating the second fault source from the kernel is 1 according to the original data; if it is confirmed that the seventh bit of the kernel controller error log register MCerrlogReg indicating the second fault source from the kernel is not 1, it is confirmed that the core of the MCU is not faulty; if it is confirmed that the seventh bit of the kernel controller error log register MCerrlogReg indicating the second fault source from the kernel is 1, it is confirmed that the core of the MCU is faulty, and it is necessary to determine the cause of the core fault of the MCU according to the kernel status register MC0_Status and the kernel diagnosis process.
[0111] If it is confirmed that the eighth bit of the kernel controller error log register MCerrlogReg used to indicate the first fault source from the kernel is not 1, it is confirmed that the MCU of the acceleration card has not failed, and it can be further determined whether the memory of the MCU has failed. Specifically, according to the original data, it is determined whether the sixth bit (Bit6) of the kernel controller error log register MCerrlogReg used to indicate the first fault source from the memory (FirstMCerrSrcFromIMC) is 1; if it is confirmed that the sixth bit of the kernel controller error log register MCerrlogReg used to indicate the first fault source from the memory is not 1, it is confirmed that the memory of the MCU has not failed; if it is confirmed that the sixth bit of the kernel controller error log register MCerrlogReg used to indicate the first fault source from the memory is 1, it is confirmed that the memory of the MCU has failed, and it is necessary to determine the cause of the memory failure of the MCU according to other register data bits and the input and output (Integrated I / Omodule, IIO) diagnostic process. In this embodiment, there is no restriction on the specific types of other registers, which depends on the specific implementation situation.
[0112] In addition, in some embodiments, the overheating and overcurrent of the accelerator card can also be analyzed. Specifically, a fault diagnosis rule table is obtained, and a second fault analysis result of the accelerator card is generated according to the fault diagnosis rule table and the original data, including:
[0113] S139: judging whether the eighth bit in the first accelerator card status register indicating the first fault source from the non-core is 1 according to the original data; if it is confirmed that the eighth bit in the first accelerator card status register indicating the first fault source from the non-core is not 1, then proceeding to step S140; if it is confirmed that the eighth bit in the first accelerator card status register indicating the first fault source from the non-core is 1, then proceeding to step S141:
[0114] S140: Confirm that the accelerator card does not have an overheat or overcurrent fault;
[0115] S141: confirming that the accelerator card has an overheat and overcurrent fault, and determining the values of the seventh and eighth bits in the second accelerator card status register according to the original data;
[0116] S142: When the eighth bit in the second accelerator card status register is 1, it is confirmed that the accelerator card has an overheating fault;
[0117] S143: When the seventh bit in the second acceleration card status register is 1, it is confirmed that an overcurrent fault occurs in the acceleration card.
[0118] In a specific implementation, firstly, it is determined based on the original data whether the eighth bit of the first acceleration card status register MStateReg used to indicate the first fault source from the non-core (FirstMStateFromUncore) is 1; if it is confirmed that the eighth bit of the first acceleration card status register MStateReg used to indicate the first fault source from the non-core is not 1, it is confirmed that the acceleration card has not had an overheating or overcurrent fault; if it is confirmed that the eighth bit of the first acceleration card status register MStateReg used to indicate the first fault source from the non-core is 1, it is confirmed that the acceleration card has had an overheating or overcurrent fault.
[0119] In order to confirm whether the fault is an overheat fault or an overcurrent fault, the values of the seventh and eighth bits in the second accelerator card status register MState_Status are further determined based on the original data; when the eighth bit in the second accelerator card status register is 1, it is confirmed that the accelerator card has an overheat fault; when the seventh bit in the second accelerator card status register is 1, it is confirmed that the accelerator card has an overcurrent fault. In this way, the second fault analysis result of the accelerator card is obtained.
[0120] In addition, when the accelerator card has an overcurrent fault, you can check whether the power cord and connection of the accelerator card are normal to ensure that they are not loose or damaged; check whether the power supply of the accelerator card meets the requirements, including voltage, current and power. If the power supply of the accelerator card is normal, there may be a problem with the accelerator card itself, and you need to contact the manufacturer or professional technical support for repair or replacement. In this way, the overcurrent fault of the accelerator card is eliminated and the normal operation of the accelerator card is ensured. When the accelerator card has an overheating fault, you need to check whether the cooling system of the accelerator card is working properly, including fans, radiators, etc., to ensure that there is no blockage, damage or failure. Further clean the accelerator card's radiator and fan to remove dust and other impurities to improve the heat dissipation efficiency. If the cooling system of the accelerator card is normal, there may be a problem with the accelerator card itself, such as chip overheating or poor heat dissipation. You need to contact the manufacturer or professional technical support for repair or replacement. In this way, the overheating fault of the accelerator card is eliminated and the normal operation of the accelerator card is ensured.
[0121] Based on the above embodiments, in some embodiments, determining the fault cause of the accelerator card according to the first fault analysis result and the second fault analysis result includes:
[0122] S151: Determine whether the first fault analysis result and the second fault analysis result are the same; if it is confirmed that the first fault analysis result and the second fault analysis result are the same, proceed to step S152; if it is confirmed that the first fault analysis result and the second fault analysis result are not the same, proceed to step S153;
[0123] S152: using the first fault analysis result or the second fault analysis result as a final fault result of the accelerator card, so as to determine the fault cause of the accelerator card according to the final fault result;
[0124] S153: Using the second fault analysis result as the final fault result of the accelerator card;
[0125] S154: updating the parsing method of the baseboard management controller for the original data according to the final fault result.
[0126] In a specific implementation, in order to obtain an accurate cause of the accelerator card failure, after obtaining the first fault analysis result and the second fault analysis result, it is determined whether the first fault analysis result and the second fault analysis result are the same; if it is confirmed that the first fault analysis result and the second fault analysis result are the same, the first fault analysis result or the second fault analysis result is used as the final fault result of the accelerator card to determine the cause of the accelerator card failure according to the final fault result. If it is confirmed that the first fault analysis result and the second fault analysis result are not the same, it is considered that the first fault analysis result for reference obtained by the preliminary analysis of the BMC is not completely accurate, so the second fault analysis result is used as the final fault result of the accelerator card, thereby obtaining an accurate final fault result, so as to determine the cause of the accelerator card failure.
[0127] Furthermore, in order to improve the accuracy of the BMC's initial analysis of raw data, the BMC's analysis method for raw data may be updated according to the final fault result, so that the BMC can provide relatively accurate initial analysis results in the next acceleration card fault analysis process.
[0128] In the above embodiments, the method for determining an accelerator card fault is described in detail. The present invention also provides an embodiment corresponding to the device for determining an accelerator card fault.
[0129] Figure 2 Schematic diagram of an accelerator card fault determination device provided by an embodiment of the present invention. Figure 2 As shown, the device comprises:
[0130] The monitoring module 10 is used to monitor the operation data of the accelerator card and determine whether the accelerator card has a fault according to the operation data of the accelerator card; if the accelerator card has a fault, the acquisition module 11 is triggered.
[0131] The acquisition module 11 is used to acquire the original data of each register related to the accelerator card based on the operating system band.
[0132] The analysis module 12 is used to analyze the original data through the baseboard management controller to generate a first fault analysis result of the acceleration card.
[0133] The generation module 13 is used to obtain a fault diagnosis rule table and generate a second fault analysis result of the accelerator card according to the fault diagnosis rule table and the original data; wherein the fault diagnosis rule table is a rule table provided by the manufacturer of the accelerator card for analyzing the health status and fault risk of the accelerator card.
[0134] The determination module 14 is used to determine the fault cause of the acceleration card according to the first fault analysis result and the second fault analysis result.
[0135] In some embodiments, the monitoring module 10 includes:
[0136] A first execution submodule, used for executing a fault diagnosis script based on an operating system of the server;
[0137] A first monitoring submodule is used to monitor the operation data of the accelerator card through a fault diagnosis script; wherein the operation data at least includes uncorrectable errors, correctable errors, temperature, voltage and current;
[0138] A first judgment submodule is used to judge whether the triggering times of the uncorrectable errors and the correctable errors of the accelerator card have reached corresponding thresholds respectively; if the triggering times of the uncorrectable errors of the accelerator card have reached the corresponding threshold, or the triggering times of the correctable errors of the accelerator card have reached the corresponding threshold, the first confirmation submodule is triggered;
[0139] The first confirmation submodule is used to confirm that there is a fault in the acceleration card and trigger the acquisition module 11.
[0140] In some embodiments, the acquisition module 11 includes:
[0141] The first acquisition submodule is used to obtain the status information and description information of the accelerator card in the enhanced error reporting register, the machine check architecture register, the kernel controller error log register, the kernel status register, the first accelerator card status register and the second accelerator card status register through the script collection module under the operating system to obtain the original data.
[0142] In some embodiments, the parsing module 12 includes:
[0143] A first calling submodule is used to call the script sending module to transmit the original data to the baseboard management controller based on a preset communication protocol;
[0144] A first parsing submodule, configured to parse the original data through a fault diagnosis module of the baseboard management controller to generate a first fault analysis result;
[0145] The first storage submodule is used to store the first fault analysis result and the original data in the data log.
[0146] In some embodiments, the generating module 13 includes:
[0147] A first determination submodule, used to determine a setting rule of a data bit of each register based on a fault diagnosis rule table;
[0148] The second judgment submodule is used to judge whether the eighth bit in the kernel controller error log register indicating the first fault source from the kernel is 1 according to the original data; if it is confirmed that the eighth bit in the kernel controller error log register indicating the first fault source from the kernel is 1, the third judgment submodule is triggered; if it is confirmed that the eighth bit in the kernel controller error log register indicating the first fault source from the kernel is not 1, the fourth judgment submodule is triggered;
[0149] The third judgment submodule is used to confirm that the microcontroller unit of the acceleration card fails, and judge whether the seventh bit in the kernel controller error log register used to indicate the second fault source from the kernel is 1 according to the original data; if it is confirmed that the seventh bit in the kernel controller error log register used to indicate the second fault source from the kernel is not 1, the second confirmation submodule is triggered; if it is confirmed that the seventh bit in the kernel controller error log register used to indicate the second fault source from the kernel is 1, the second determination submodule is triggered;
[0150] A second confirmation submodule is used to confirm that the core of the microcontroller unit has no faults;
[0151] The second determination submodule is used to confirm that the core of the micro control unit fails, and determine the cause of the core failure of the micro control unit according to the core status register and the core diagnosis process;
[0152] The fourth judgment submodule is used to confirm that the microcontroller unit of the acceleration card has not failed, and to judge whether the sixth bit in the kernel controller error log register used to indicate the first fault source from the memory is 1 according to the original data; if it is confirmed that the sixth bit in the kernel controller error log register used to indicate the first fault source from the memory is not 1, the third confirmation submodule is triggered; if it is confirmed that the sixth bit in the kernel controller error log register used to indicate the first fault source from the memory is 1, the third determination submodule is triggered;
[0153] A third confirmation submodule is used to confirm that there is no fault in the memory of the micro control unit;
[0154] The third determination submodule is used to confirm that the memory of the micro control unit is faulty, and determine the cause of the memory fault of the micro control unit according to other register data bits and input and output diagnosis processes.
[0155] In some embodiments, the generating module 13 includes:
[0156] a fifth judgment submodule, configured to judge whether the eighth bit in the first accelerator card status register indicating the first fault source from the non-core is 1 according to the original data; if it is confirmed that the eighth bit in the first accelerator card status register indicating the first fault source from the non-core is not 1, triggering the fourth confirmation submodule; if it is confirmed that the eighth bit in the first accelerator card status register indicating the first fault source from the non-core is 1, triggering the fifth confirmation submodule;
[0157] The fourth confirmation submodule is used to confirm that the acceleration card has no overheating or overcurrent fault;
[0158] A fifth confirmation submodule, used to confirm that an overheating and overcurrent fault occurs in the acceleration card, and to determine the values of the seventh and eighth bits in the second acceleration card status register according to the original data;
[0159] a sixth confirmation submodule, configured to confirm that an overheating fault occurs in the accelerator card when the eighth bit in the second accelerator card status register is 1;
[0160] The seventh confirmation submodule is used to confirm that an overcurrent fault occurs in the acceleration card when the seventh bit in the second acceleration card status register is 1.
[0161] In some embodiments, the determination module 14 includes:
[0162] The sixth judgment submodule is used to judge whether the first fault analysis result and the second fault analysis result are the same; if it is confirmed that the first fault analysis result and the second fault analysis result are the same, the first processing submodule is triggered; if it is confirmed that the first fault analysis result and the second fault analysis result are not the same, the second processing submodule is triggered;
[0163] A first processing submodule, configured to use the first fault analysis result or the second fault analysis result as a final fault result of the accelerator card, so as to determine a fault cause of the accelerator card according to the final fault result;
[0164] A second processing submodule, used for using the second fault analysis result as the final fault result of the accelerator card;
[0165] The update submodule is used to update the parsing method of the baseboard management controller for the original data according to the final fault result.
[0166] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.
[0167] Figure 3 A schematic diagram of an accelerator card fault determination device provided by an embodiment of the present invention. Figure 3 As shown, the accelerator card fault determination device includes:
[0168] A memory 20, for storing computer programs;
[0169] The processor 21 is used to implement the steps of the method for determining an accelerator card failure as mentioned in the above embodiment when executing a computer program.
[0170] The accelerator card fault determination device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer, or a desktop computer.
[0171] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU; the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.
[0172] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the acceleration card fault determination method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. Data 203 may include but is not limited to data related to the acceleration card fault determination method.
[0173] In some embodiments, the acceleration card fault determination device may further include a display screen 22 , an input and output interface 23 , a communication interface 24 , a power supply 25 , and a communication bus 26 .
[0174] Those skilled in the art will understand that Figure 3 The structure shown in the figure does not constitute a limitation on the accelerator card fault determination device, and may include more or less components than those shown in the figure.
[0175] Finally, the present invention also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps recorded in the above method embodiment are implemented.
[0176] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.
[0177] The above is a detailed introduction to an acceleration card fault determination method, device, equipment and medium provided by the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referenced to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.
[0178] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.
Claims
1. A method for determining an accelerator card fault, characterized in that: include: Monitoring the operation data of the accelerator card, and determining whether the accelerator card has a fault according to the operation data of the accelerator card; If the accelerator card fails, obtaining original data of each register related to the accelerator card based on the operating system in-band; Parsing the raw data through a baseboard management controller to generate a first fault analysis result of the acceleration card; Obtain a fault diagnosis rule table, and generate a second fault analysis result of the accelerator card according to the fault diagnosis rule table and the original data; wherein the fault diagnosis rule table is a rule table provided by the manufacturer of the accelerator card for analyzing the health status and fault risk of the accelerator card; The fault cause of the accelerator card is determined according to the first fault analysis result and the second fault analysis result.
2. The method for determining an accelerator card failure according to claim 1, wherein: The monitoring the operation data of the accelerator card and determining whether the accelerator card has a fault according to the operation data of the accelerator card includes: Execute troubleshooting scripts based on the server's operating system; Monitoring the operation data of the accelerator card through the fault diagnosis script; wherein the operation data at least includes uncorrectable errors, correctable errors, temperature, voltage and current; Determine whether the number of times the uncorrectable errors are triggered and the number of times the correctable errors are triggered by the accelerator card reach corresponding thresholds; If the number of times the uncorrectable errors of the accelerator card are triggered reaches a corresponding threshold, or the number of times the correctable errors of the accelerator card are triggered reaches a corresponding threshold, it is confirmed that the accelerator card is faulty, and the step of obtaining the original data of each register related to the accelerator card based on the operating system band is entered.
3. The method for determining an accelerator card failure according to claim 1, wherein: The obtaining of original data of each register related to the accelerator card based on the operating system in-band includes: Through the script collection module under the operating system, the status information and description information of the acceleration card in the enhanced error reporting register, the machine check architecture register, the kernel controller error log register, the kernel status register, the first acceleration card status register and the second acceleration card status register are obtained to obtain the original data.
4. The method for determining an accelerator card failure according to claim 1, wherein: The parsing of the original data by a baseboard management controller to generate a first fault analysis result of the acceleration card includes: Based on a preset communication protocol, calling a script sending module to transmit the original data to the baseboard management controller; Parsing the original data through the fault diagnosis module of the baseboard management controller to generate the first fault analysis result; The first fault analysis result and the original data are stored in a data log.
5. The method for determining an accelerator card failure according to claim 3, characterized in that: The obtaining of the fault diagnosis rule table and generating the second fault analysis result of the acceleration card according to the fault diagnosis rule table and the original data includes: Determine a setting rule for the data bits of each register based on the fault diagnosis rule table; Determine, according to the original data, whether the eighth bit in the kernel controller error log register, which is used to indicate the first fault source from the kernel, is 1; If it is confirmed that the eighth bit in the kernel controller error log register for indicating the first fault source from the kernel is 1, it is confirmed that the micro control unit of the acceleration card fails, and it is determined according to the original data whether the seventh bit in the kernel controller error log register for indicating the second fault source from the kernel is 1; If it is confirmed that the seventh bit in the core controller error log register indicating the second fault source from the core is not 1, it is confirmed that the core of the micro control unit has no fault; If it is confirmed that the seventh bit in the kernel controller error log register indicating the second fault source from the kernel is 1, it is confirmed that the kernel of the micro control unit fails, and the cause of the kernel fault of the micro control unit is determined according to the kernel status register and the kernel diagnosis process; If it is confirmed that the eighth bit in the kernel controller error log register for indicating the first fault source from the kernel is not 1, it is confirmed that the micro control unit of the acceleration card has not failed, and it is determined according to the original data whether the sixth bit in the kernel controller error log register for indicating the first fault source from the memory is 1; If it is confirmed that the sixth bit in the kernel controller error log register indicating the first fault source from the memory is not 1, it is confirmed that the memory of the micro control unit has no fault; If it is confirmed that the sixth bit in the core controller error log register indicating the first fault source from the memory is 1, it is confirmed that the memory of the micro control unit has a fault, and the cause of the memory fault of the micro control unit is determined based on other register data bits and input and output diagnostic processes.
6. The method for determining an accelerator card fault according to claim 5, characterized in that: The obtaining of the fault diagnosis rule table and generating the second fault analysis result of the acceleration card according to the fault diagnosis rule table and the original data includes: Determine, according to the original data, whether the eighth bit in the first accelerator card status register, which is used to indicate a first fault source from a non-core, is 1; If it is confirmed that the eighth bit in the first accelerator card status register indicating the first fault source from the non-core is not 1, it is confirmed that the accelerator card does not have an overheating or overcurrent fault; If it is confirmed that the eighth bit in the first accelerator card status register indicating the first fault source from the non-core is 1, it is confirmed that the accelerator card has an overheating and overcurrent fault, and the values of the seventh and eighth bits in the second accelerator card status register are determined according to the original data; When the eighth bit in the second accelerator card status register is 1, it is confirmed that the accelerator card has an overheating fault; When the seventh bit in the second acceleration card status register is 1, it is confirmed that an overcurrent fault occurs in the acceleration card.
7. The method for determining an accelerator card fault according to any one of claims 1 to 6, characterized in that: The determining the fault cause of the accelerator card according to the first fault analysis result and the second fault analysis result includes: Determining whether the first fault analysis result and the second fault analysis result are the same; If it is confirmed that the first fault analysis result and the second fault analysis result are the same, the first fault analysis result or the second fault analysis result is used as the final fault result of the accelerator card, so as to determine the fault cause of the accelerator card according to the final fault result; If it is confirmed that the first fault analysis result and the second fault analysis result are different, taking the second fault analysis result as the final fault result of the accelerator card; The parsing method of the baseboard management controller for the original data is updated according to the final fault result.
8. An accelerator card fault determination device, characterized in that: include: A monitoring module, used to monitor the operation data of the accelerator card, and determine whether the accelerator card has a fault according to the operation data of the accelerator card; if the accelerator card has a fault, triggering the acquisition module; The acquisition module is used to acquire original data of each register related to the accelerator card based on the operating system band; An analysis module, configured to analyze the raw data through a baseboard management controller to generate a first fault analysis result of the acceleration card; A generating module, used to obtain a fault diagnosis rule table, and generate a second fault analysis result of the accelerator card according to the fault diagnosis rule table and the original data; wherein the fault diagnosis rule table is a rule table provided by the manufacturer of the accelerator card for analyzing the health status and fault risk of the accelerator card; A determination module is used to determine the fault cause of the acceleration card according to the first fault analysis result and the second fault analysis result.
9. An accelerator card fault determination device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the method for determining an accelerator card failure as described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for determining an accelerator card fault according to any one of claims 1 to 7 are implemented.
Citation Information
Cited By
Fault processing method and device for heterogeneous accelerator card, electronic equipment and storage medium
CN119883750A
Fault Handling Method, Device, Electronic Device and Storage Medium for Heterogeneous Acceleration Card
CN119883750B
