Correctable error storm suppression method and computing device
By obtaining the type and module information of CE signals in the BMC, reading error records and increasing the error count, and using suppression levels to suppress CE storms, the problem of incomplete information and limited suppression effect in existing technologies is solved, achieving comprehensive error collection and improved business smoothness.
Patent Information
- Application Number
- CN202510743592.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-04
- Publication Date
- 2025-11-07
AI Technical Summary
Existing technologies cannot comprehensively collect error information when dealing with CE storms, resulting in incomplete error information collected by the BMC and operating system, which affects subsequent error diagnosis and cannot effectively suppress CE storms other than memory types.
After receiving the CE signal, the BMC obtains the CE type and module information, reads the error record and increases the error count. When the threshold is reached, different suppression levels are used to suppress the CE storm, including stopping the sending of interrupt commands, logging, and fault diagnosis, to ensure that fault information is not lost.
It effectively suppresses various types of CE storms, ensuring that the comprehensive collection and analysis of error information are unaffected, reducing BMC performance and operating system resource consumption, and improving the smoothness of business execution.
Smart Images
Figure CN120909818A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical fields of the application and computing devices, and particularly relates to a correctable error storm suppression method and a computing device. BACKGROUND
[0002] The normal operation of a server platform depends on the broadcast SMI (System Manage Interrupt) for the processing of CE (Correctable Error). However, the BIOS may receive the error type SMI continuously for a certain period of time, which is defined as a CE storm.
[0003] In the related art, the CE storm can be controlled by performing storm suppression on all modules that generate CE and releasing the suppression after a certain period of time, or by counting the number of CEs of a certain storage unit within a time window by the BMC (Baseboard Management Controller), and limiting the triggering of reporting by closing the CE count register when the threshold is exceeded, thereby realizing CE storm suppression.
[0004] However, if all modules that generate CE are suppressed, the suppressed modules cannot report new CE, resulting in incomplete error information collected by the BMC and the operating system, affecting subsequent error diagnosis, and the BMC limits the reporting of storage unit CE by closing the CE count register, which cannot suppress CE storms other than memory types. SUMMARY
[0005] Embodiments of the present application provide a correctable error storm suppression method and a computing device. The CE storm can be suppressed in a targeted manner, making the collected error information more comprehensive, and the CE storm of multiple types can be suppressed.
[0006] In a first aspect, embodiments of the present application provide a correctable error storm suppression method, comprising: in response to receiving a correctable error signal, determining the error type and error module of the correctable error corresponding to the correctable error signal; obtaining an error record corresponding to the error type, wherein the error record includes one or more record items; if the target record item corresponding to the error module is included in the record item, increasing the error number corresponding to the target record item by n to obtain a new error number, wherein n is a positive integer; if the target record item is not included in the record item, adding the target record item to the error record and increasing the error number corresponding to the target record item by n to obtain a new error number; if the new error number is greater than a preset error number threshold, determining a suppression level; and performing correctable error storm suppression according to the suppression level.
[0007] The embodiment of the application determines the corresponding error type and error module information after receiving the CE signal, and obtains the error record corresponding to the error type, so that comprehensive processing of various types of errors is realized. After obtaining the error module information, the error number of the corresponding target record item is also increased. In the case where the new error number is greater than the preset error number threshold, the CE storm is suppressed by using the suppression mode corresponding to the suppression level, so that the hierarchical suppression of the CE storm is realized, thereby limiting the degree of CE storm suppression and providing protection for subsequent error diagnosis and other error processing processes. The scheme can improve the comprehensiveness of CE error processing and the smoothness of business execution.
[0008] In a possible implementation, according to the suppression level, the correctable error storm suppression is performed, including: if the suppression level includes a first suppression level, stopping sending a system management interrupt instruction to an operating system for a correctable error signal; if the suppression level includes a second suppression level, stopping log record storage for the correctable error signal; if the suppression level includes a third suppression level, stopping fault prediction, fault diagnosis and log record storage for the correctable error signal; and if the suppression level includes a fourth suppression level, stopping format conversion, fault prediction, fault diagnosis and log record storage for the correctable error signal.
[0009] The embodiment of the application reduces the influence on the performance, resource occupation and operating system business continuity of the BMC by suppressing the signal processing flow to different degrees under different suppression levels while ensuring that fault information is not lost.
[0010] In a possible implementation, according to the suppression level, the correctable error storm suppression is performed, including: if the suppression level is a fifth suppression level, performing format conversion on the correctable error signal to obtain a converted error signal; analyzing the converted error signal to obtain an analysis result; generating an error log according to the analysis result, persisting the error log, and sending error information to an operating system for the correctable error signal.
[0011] The embodiment of the application maintains normal signal processing, error analysis and log recording by performing all processing flows of the CE signal, so that the analysis and processing of errors are not affected.
[0012] In a possible implementation, if the target record item is not included in the record items, the target record item is added in the error record, and the error number corresponding to the target record item is increased by n, including: if the target record item corresponding to the error module is not included in the error record, and a blank record item is included, the information of the error module is written in the blank record item to obtain the target record item, and the error number corresponding to the target record item is increased by n; if the target record item corresponding to the error module is not included in the error record, and the historical record items are full, the error number and the reset number corresponding to each historical record item are obtained; the to-be-replaced item in the historical record item is determined according to the error number and the reset number; the information of the error module is used to cover the to-be-replaced item, and the error number corresponding to the to-be-replaced item is set to n.
[0013] The embodiment of the application directly writes the information of the error module into the blank record item in the case of having the blank record item, and in the case of not having the blank record item, calculates the total error number corresponding to each record item, determines the record item with the least total error number as the to-be-replaced item, and uses the information of the error module to cover the to-be-replaced item, thereby keeping limited record items, reducing the memory occupied by data storage, and effectively recording new error module information and eliminating the record item corresponding to the error module with less errors.
[0014] In a possible implementation, the to-be-replaced item in the historical record item is determined according to the error number and the reset number, including: multiplying the reset number of the target historical record item by the error number threshold to obtain the reset error number; adding the reset error number and the error number of the target historical record item to obtain the total error number; and determining the historical record item with the minimum total error number as the to-be-replaced item.
[0015] In a possible implementation, after obtaining the error record corresponding to the error type, the method further includes: obtaining the historical error time corresponding to the target record item in the error record, wherein the historical error time is the time of the latest error corresponding to the target record item; determining whether the time length from the historical error time to the current time is greater than a preset suppression time threshold; if the time length from the historical error time to the current time is not greater than the suppression time threshold, performing the step of increasing the error number corresponding to the target record item by n to obtain a new error number.
[0016] The embodiment of the application increases the error number normally in the case that the time length from the historical error time to the current time is less than or equal to the suppression time threshold, so that the correctable error signal that frequently occurs can normally accumulate the error number.
[0017] In a possible implementation, the method further includes: if the time length from the historical error time to the current time is greater than or equal to the preset suppression time threshold, resetting the error number corresponding to the error module.
[0018] In a possible implementation, after the correctable error storm suppression is performed according to the suppression level, the method further includes: increasing, by m, the suppression times corresponding to the target record, where m is a positive integer.
[0019] In a possible implementation, after the correctable error storm suppression is performed according to the suppression level, the method further includes: if a duration of the correctable error storm suppression is greater than a preset duration threshold, releasing the correctable error storm suppression; and resetting the error times and the suppression times of the record subjected to the correctable error storm suppression.
[0020] In a second aspect, an embodiment of the present application provides a computing device, which includes a memory and a processor; the memory and the processor are coupled; the memory is configured to store program instructions; and the processor is configured to invoke the program instructions to enable the computing device to perform the method in the first aspect and / or any embodiment of the first aspect.
[0021] In a third aspect, an embodiment of the present application provides a correctable error storm suppression apparatus, which includes: an information obtaining module, a record obtaining module, a first increasing module, a second increasing module, a level obtaining module, and a storm suppression module. The information obtaining module is configured to determine an error type and an error module of a correctable error corresponding to a correctable error signal, in response to receiving the correctable error signal; the record obtaining module is configured to obtain an error record corresponding to the error type, where the error record includes one or more record items; the first increasing module is configured to increase, by n, error times corresponding to a target record item of the error module, to obtain new error times, if the record item includes the target record item, where n is a positive integer; the second increasing module is configured to add the target record item in the error record, and increase, by n, the error times corresponding to the target record item, to obtain new error times, if the record item does not include the target record item; the level obtaining module is configured to determine a suppression level, if the new error times are greater than a preset error times threshold; and the storm suppression module is configured to perform correctable error storm suppression according to the suppression level.
[0022] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores computer program instructions; when the program instructions are executed by a computing device or a processor of the computing device, the computing device or the processor performs the method in the first aspect and / or any embodiment of the first aspect.
[0023] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes computer program instructions; when the computer program instructions are executed, the computing device performs the method in the first aspect and / or any embodiment of the first aspect.
[0024] The correctable error storm suppression method and the computing device provided by the embodiments of the present application can process various types of errors by determining the corresponding error type and error module after receiving the CE signal, obtaining the error record corresponding to the error type, increasing the error number of the corresponding target record item after obtaining the error module, and using the suppression method corresponding to the suppression level to suppress the CE storm when the new error number is greater than the preset error number threshold, thereby realizing the case-based CE storm suppression, controlling the degree of CE storm suppression, not sending the system management interrupt instruction to the operating system in some cases, but retaining the log record capability, and providing guarantee for the subsequent error diagnosis and other error processing processes. BRIEF DESCRIPTION OF DRAWINGS
[0025] Figure 1 The correctable error signal processing process diagram without suppression provided by the embodiments of the present application is shown in FIG. 4.
[0026] Figure 2 The correctable error signal processing flow diagram provided by the embodiments of the present application is shown in FIG. 5.
[0027] Figure 3 The error record diagram provided by the embodiments of the present application is shown in FIG. 6.
[0028] Figure 4 The suppression measure diagram corresponding to the suppression level provided by the embodiments of the present application is shown in FIG. 7.
[0029] Figure 5 The error number and suppression number diagram after the record item is reset provided by the embodiments of the present application is shown in FIG. 8.
[0030] Figure 6 The structure diagram of the correctable error storm suppression device provided by the embodiments of the present application is shown in FIG. 9.
[0031] Figure 7 The structure diagram of the computing device provided by the embodiments of the present application is shown in FIG. 10. DETAILED DESCRIPTION
[0032] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is with reference to the drawings, in which like numerals indicate like elements, unless otherwise described in the following description. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present application, as detailed in the appended claims.
[0033] The stable operation of the server platform highly depends on the broadcast SMI mechanism to handle the CE. However, in a specific scenario, the BIOS may continuously receive a large number of correctable error SMI signals in a short time, and this abnormal phenomenon is defined as a CE storm. The occurrence of the CE storm interrupts the processing flow, and frequent interruption of the processing flow occupies CPU resources, causing delay in normal business processing and a decrease in system throughput.
[0034] To prevent and control the CE storm, the related technologies provide two main strategies: first, a global storm suppression measure is adopted, that is, all modules triggering the CE are uniformly suppressed; second, the intelligent monitoring function of the BMC is used to count the frequency of the occurrence of the CE of a specific storage unit within a specific time window, and once the frequency exceeds a preset threshold, the error reporting trigger is limited by closing the CE count register, thereby effectively suppressing the further development of the CE storm. However, if all modules that have generated the CE are suppressed in the related technologies, the suppressed modules cannot report new CEs, resulting in incomplete error information collected by the BMC and the operating system, affecting subsequent error diagnosis, and the limitation of the type of CE storm that can be suppressed by the BMC by closing the CE count register is limited, and the CE storm of types other than the memory type cannot be suppressed.
[0035] To solve the above technical problems, the inventors propose the following technical concept: by obtaining the CE type and error module information corresponding to the CE signal after the BMC receives the CE signal, reading the error record corresponding to the CE type, and increasing the error times in the error record, the CE storm is suppressed by using the suppression method corresponding to the suppression level when the new error times reach the threshold.
[0036] The correctable error storm suppression method provided by the embodiments of the present application is described in detail below through specific implementation manners.
[0037] Figure 1 The correctable error signal processing process without suppression provided by the embodiments of the present application is shown in the following schematic diagram. Figure 1 The processing process in the correctable error signal processing process without suppression provided by the embodiments of the present application is for the CE signal of the transferable error type.
[0038] As shown in FIG. 1, the correctable error signal processing process without suppression provided by the embodiments of the present application includes the following steps. Figure 1As shown, the correctable error signal of the transferable error type is sent from the host side to the BMC side, received by the signal transceiver module in the BMC, and transmitted to the transfer module. The transfer module converts the correctable error signal to obtain a converted error signal and sends the converted error signal to the message processing module. The message processing module analyzes the converted error signal to obtain an analysis result and inputs the analysis result into the diagnostic prediction module. The diagnostic prediction module generates an error log using the analysis result and writes the error log into the storage unit. The information transceiver module also sends error information to the host for the correctable error signal.
[0039] It should be noted that the exemplary, Figure 1 The host in the above embodiment can be a central processing unit.
[0040] Figure 2 The correctable error signal processing flow provided by the embodiment of the present application is shown in the figure. The correctable error signal processing flow is applied to a baseboard management controller. As shown in Figure 2 The correctable error signal processing flow includes steps S201 to S206.
[0041] S201: In response to receiving a correctable error signal, determine the error type and error module corresponding to the correctable error signal.
[0042] In this step, the correctable error signal can be received from the pin of the processor, or the device connected to the processor pin can record the level value change of the processor pin to obtain the correctable error signal. The correctable error signal can be obtained by reading the data stored in the above-mentioned device connected to the processor pin. Determining the error type and module information corresponding to the correctable error signal can include querying the register to obtain the error type and error module, or it can include reading the state parameter in the preset register. If the state parameter of the target register corresponds to the processor pin with the level value change, the type corresponding to the target register is determined as the error type, and the module information corresponding to the target register is determined as the error module.
[0043] The error type can include MCA (Machine Check Architecture) type, PCIe (Peripheral Component Interconnect Express) type, and local type. The MCA type mainly records the correctable error reported by the MCA error framework, the PCIe type mainly records the correctable error of the device hanging below the root port of the processing unit, and the local type mainly records the correctable error inside the processing unit.
[0044] S202: Retrieve the error record corresponding to the error type, wherein the error record includes one or more record items.
[0045] In this step, error records can be stored in a table format, or in key-value pairs, files, or other formats.
[0046] Figure 3 This is a schematic diagram illustrating error logging provided for an embodiment of this application. For example... Figure 3 As shown, all three types of error records can use the same format. Some record items have already been written, such as "A1", "A2", "A3", "B1", "C1", etc. The boxes without written characters represent blank record items. Record items can correspond to the last error reporting time, error count, error count threshold, CE storm time, storm resolution time, suppression level, and suppression count (error count has been reset).
[0047] S203: If the record item includes the target record item corresponding to the error module, increase the error count corresponding to the target record item by n to obtain the new error count, where n is a positive integer.
[0048] In this step, n is, for example, 1, 2, 3, etc.
[0049] Continue to refer to Figure 3 For example, if the error module information is A3, the original error count of the target record corresponding to A3 is 5. The error count of the target record is increased by 1 (n is 1), resulting in a new error count of 6. As another example, if the error module information is B1, the original error count of the target record corresponding to B1 is 2. The error count of the target record is increased by 2 (n is 2), resulting in a new error count of 4.
[0050] S204: If the target record is not included in the record, add the target record to the error record and increase the error count corresponding to the target record by n to get the new error count.
[0051] In this step, continue to refer to Figure 3 For example, if the error module information is C5, and there is no record for C5, then add a record for C5 to the error record and increment the error count of record for C5 by 1 (n is 1). As another example, if the error module information is A5, and there is no record for A5, then add a record for A5 to the error record and increment the error count of record for C5 by 1 (n is 1).
[0052] S205: If the number of new errors exceeds the preset error threshold, then determine the suppression level.
[0053] In this step, the error number threshold and the suppression level can be pre-set by the staff according to experimental data or empirical parameters. The suppression level can correspond to the record item and can be pre-set or can be obtained first by acquiring the remaining resource amount of the server and then adopting the corresponding suppression level according to the amount of the remaining resource.
[0054] S206: According to the suppression level, the correctable error storm suppression is performed.
[0055] In this step, the processing steps of the correctable error signal are reduced by adopting the suppression strategy corresponding to the suppression level.
[0056] From the description of the above embodiments, it can be known that, by determining the corresponding error type and error module information after receiving the CE signal and obtaining the error record corresponding to the error type, the embodiments of the present application can process various types of errors. After obtaining the error module information, the error number of the corresponding target record item is also increased. In the case that the new error number is greater than the pre-set error number threshold, the CE storm suppression is performed by adopting the suppression mode corresponding to the suppression level, the CE storm suppression is implemented in different cases, and thus the degree of CE storm suppression is limited, which provides protection for the subsequent error diagnosis and other error processing processes. In some cases, the system management interrupt instruction is not sent to the operating system, but the log record capability is reserved.
[0057] In a possible implementation, in the step S206, according to the suppression level, the correctable error storm suppression is performed, including the steps S206A, S206B, S206C and S206D.
[0058] S206A: If the suppression level includes the first suppression level, the sending of the system management interrupt instruction to the operating system for the correctable error signal is stopped.
[0059] In this step, the suppression level can be pre-set. The first suppression level, the second suppression level, the third suppression level and the fourth suppression level in the steps S206B, S206C and S206D can be represented by the bit positions of a parameter, for example, 0001 represents the first suppression level, 0010 represents the second suppression level, 0100 represents the third suppression level, 1000 represents the fourth suppression level, and 1001 represents the first suppression level and the fourth suppression level at the same time. By reading the pre-set parameter and determining which bit positions in the parameter are 1, it can be determined which suppression level is currently adopted.
[0060] S206B: If the suppression level includes the second suppression level, the log record storage for the correctable error signal is stopped.
[0061] In this step, in the case of including the second suppression level, the log generation and storage of the correctable error signal are not performed.
[0062] S206C: If the suppression level includes the third suppression level, stop the fault prediction, fault diagnosis and log record storage for the correctable error signal.
[0063] In this step, the converted error signal can be input to the fault diagnosis module, so that the fault prediction, fault diagnosis and log generation are not performed.
[0064] S206D: If the suppression level includes the fourth suppression level, stop the format conversion, fault prediction, fault diagnosis and log record storage for the correctable error signal.
[0065] In this step, after receiving the correctable error signal, the correctable error signal is not processed.
[0066] Figure 4 The suppression measures corresponding to the suppression levels provided by the embodiments of the present application are shown in the schematic diagram. As shown in the schematic diagram, the signal processing routes crossed by the cross signs are the stopped routes, and the corresponding suppression levels are marked beside the cross signs. In the first suppression level, the signal transceiver module does not send error information to the system control interrupt processing module, in the second suppression level, the error log is not stored to the storage unit, in the third suppression level, the analysis result is not input to the diagnosis prediction module, and in the fourth suppression level, the correctable error signal is not input to the transfer module. Figure 4
[0067] From the description of the above embodiments, it can be known that the embodiments of the present application can reduce the influence on the performance, resource occupation and operating system business continuity of the BMC while ensuring that the fault information is not lost, by suppressing the signal processing flow to different degrees under different suppression levels.
[0068] In a possible implementation, in the step S206, according to the suppression level, the correctable error storm suppression is performed, including steps S206E1 to S206E3.
[0069] S206E1: If the suppression level is the fifth suppression level, the correctable error signal is converted to obtain a converted error signal.
[0070] In this step, the fifth suppression level can be that each bit is not set, and each processing flow in the step S206E1 is executed without suppression. Figure 4 The format conversion can include converting the correctable error signal into a preset format.
[0071] S206E2: Analyzing the converted error signal to obtain an analysis result.
[0072] In this step, the process of analyzing the converted error signal can include reading the content of the converted error signal to obtain an analysis result.
[0073] S206E3: generating an error log according to the analysis result, persisting the error log, and sending error information to the operating system for the correctable error signal.
[0074] In this step, generating an error log according to the analysis result can include writing the analysis result into a log template to obtain the error log. Persisting the error log can include storing the error log into a storage unit, such as a flash memory. The way of sending error information to the operating system for the correctable error signal can include sending error information in a preset format to the operating system.
[0075] As can be seen from the description of the above embodiments, the embodiments of the present application perform all processing procedures of the CE signal without suppression, maintain normal signal processing, error analysis and log recording, and make the analysis and processing of errors not affected.
[0076] In a possible implementation, the baseboard management controller includes an information transceiving module, a transfer module and a fault diagnosis module, wherein the fault diagnosis module includes a message processing module, a diagnosis prediction module and a log recording module.
[0077] In the step S206E, the correctable error signal is converted to obtain a converted error signal, the converted error signal is analyzed to obtain an analysis result, an error log is generated according to the analysis result, the error log is persisted, and error information is sent to the host for the correctable error signal, which includes steps S206E1 to S206E5.
[0078] S206E1: receiving the correctable error signal by the information transceiving module, and transmitting the correctable error signal to the transfer module.
[0079] In this step, the signal transceiving module obtains the CE signal by detecting the level value of the pin of the processing unit.
[0080] S206E2: converting the correctable error signal by the transfer module to obtain a converted error signal, and sending the converted error signal to the message processing module.
[0081] In this step, the transfer module can convert the correctable error signal in a format standardization, field mapping or the like. The converted error signal can be sent to the message processing module in the format of RPC (Remote Procedure Call).
[0082] S206E3: The message processing module parses the converted error signal to obtain a parsing result, and inputs the parsing result into the diagnosis prediction module.
[0083] In this step, the message processing module can read a specific field in the converted error signal to obtain a parsing result, or can obtain a parsing result corresponding to a preset condition according to whether the data in the converted error signal meets the preset condition.
[0084] S206E4: The diagnosis prediction module generates an error log by using the parsing result, and writes the error log into a storage unit.
[0085] In this step, the parsing result is written into a log template to obtain the error log.
[0086] S206E5: The information transceiver module sends error information to the host for a correctable error signal.
[0087] In this step, the error information can be sent to a system control interrupt processing module on the host side, and the system control interrupt processing module sends the error information to an operating system.
[0088] As can be seen from the description of the above embodiments, the information processing flow is constructed by the modules, so that the received CE signal can be processed through a series of processes such as format conversion, parsing, fault diagnosis analysis, and log recording, the log recording of the CE signal is completed, subsequent abnormal analysis is facilitated, and error information is also sent to the host, so that the operating system of the host can timely process the abnormality.
[0089] In a possible implementation, in step S204, if the target record item is not included in the record item, the target record item is added in the error record, and the error number corresponding to the target record item is increased by n, including:
[0090] S2041: If the target record item corresponding to the error module is not included in the error record, and a blank record item that has not been written is included, the information of the error module is written in the blank record item to obtain the target record item, and the error number corresponding to the target record item is increased by n.
[0091] In this step, the information of the error module can be a number or a name of the error module, etc. For example, the information of the current error module is A10, the error record does not contain the target record item corresponding to A10, and contains a blank record item without writing. A10 is written in any blank record item to obtain the target record item, and the initial error number of the target record item corresponding to A10 is recorded as 1 (n is 1). For example, the information of the current error module is D12, the error record does not contain the target record item corresponding to D12, and contains a blank record item without writing. D12 is written in any blank record item to obtain the target record item, and the initial error number of the target record item corresponding to D12 is recorded as 1 (n is 1).
[0092] S2042: If the error record does not include the target record item corresponding to the error module, and the history record item is full, the error number and the reset number corresponding to each history record item are obtained. According to the error number and the reset number, the replacement item in the history record item is determined. The information of the error module is used to cover the replacement item, and the error number corresponding to the replacement item is set to n.
[0093] In this step, the error number and the reset number are weighted and summed to obtain the total error number, and the history record item with the smallest total error number is determined as the replacement item.
[0094] For example, the error record contains 5 record items, which are A1, A2, A3, A4 and A5, the information of the error module is A10, the error number of A1 is 2, the reset number is 2, the error number of A2 is 0, the reset number is 1, the error number of A3 is 1, the reset number is 0, the error number of A4 is 2, the reset number is 0, the error number of A5 is 3, the reset number is 0, the weight of the reset number is 5, and the weight of the error number is 1. The total error number corresponding to A1 is the largest, which is 12, the total error number corresponding to A2 is 5, the total error number corresponding to A3 is 1, the total error number corresponding to A4 is 2, and the total error number corresponding to A5 is 3. A3 is the replacement item. A10 covers A3, and the error number is set to 1 (n is 1) and the reset number is set to 0.
[0095] From the description of the above embodiments, it can be known that the embodiments of the present application directly write the error module information into the blank record item in the case of having a blank record item, and in the case of not containing a blank record item, the total error number corresponding to each record item is calculated, the record item with the smallest total error number is determined as the replacement item, and the information of the error module is used to cover the replacement item, so as to keep limited record items, thereby reducing the memory occupied by data storage, and effectively recording new error module information and eliminating the record item corresponding to the error module with less errors.
[0096] In a possible implementation, in step S2042, the to-be-replaced record in the history record is determined according to the error number and the reset number, including steps S421 to S423.
[0097] S421: multiplying the reset number of the target history record by the error number threshold to obtain the reset error number.
[0098] In this step, the error number threshold, for example, 4, 5, 6, etc., can be pre-set by the staff according to experimental data or experience parameters.
[0099] S422: adding the reset error number to the error number of the target history record to obtain the total error number.
[0100] In this step, for example, the reset error number is 10, and the error number of the target history record is 3, and the total error number is 13. For another example, the reset error number is 15, and the error number of the target history record is 1, and the total error number is 16.
[0101] S423: determining the history record with the minimum total error number as the to-be-replaced record.
[0102] In this step, for example, the third history record in the seven history records has the minimum total error number, and the third history record is determined as the to-be-replaced record.
[0103] As can be seen from the description of the above embodiments, the reset number is multiplied by the error number threshold to obtain the reset error number corresponding to the target history record, that is, the reset error number. The reset error number and the error number are added to obtain the total error number. The history record with the minimum total error number is determined as the to-be-replaced record, the record with the minimum error number is replaced, the record is updated, and the role of the record is greater.
[0104] In a possible implementation, after the error record corresponding to the error type is obtained in step S202, steps S220 and S222 are further included.
[0105] S220: obtaining the history error time corresponding to the target record in the error record, wherein the history error time is the time of the latest error corresponding to the target record.
[0106] In this step, it can include querying the correspondence between the record and the history error time according to the target record to obtain the history error time corresponding to the target record. It can also include reading the history error time attribute of the target record to obtain the history error time.
[0107] The correspondence between the record item and the historical error time can be stored in a table, a key-value pair, a file, or the like.
[0108] S221: Determine whether the duration from the historical error time to the current time is greater than a preset suppression duration threshold.
[0109] In this step, the current time can be obtained by obtaining a timestamp. The suppression duration threshold can be preset by a worker according to experimental data or an empirical parameter.
[0110] S222: If the duration from the historical error time to the current time is not greater than the preset suppression duration threshold, perform a step of increasing the error number corresponding to the target record item by n to obtain a new error number.
[0111] In this step, the duration from the historical error time to the current time is obtained by subtracting the historical error time from the current time. The step of increasing the error number corresponding to the target record item by n to obtain a new error number is step S203 or step S204.
[0112] As can be known from the description of the above embodiments, the embodiments of the present application increase the error number normally in the case where the duration from the historical error time to the current time is less than or equal to the suppression duration threshold, so that the frequently occurring correctable error signal is normally accumulated with the error number.
[0113] In a possible implementation, after the error record corresponding to the error type is obtained in step S202, the method further includes step S223.
[0114] S223: If the duration from the historical error time to the current time is greater than or equal to the preset suppression duration threshold, reset the error number corresponding to the error module.
[0115] In this step, resetting the error number can include resetting the error number to zero.
[0116] As can be known from the description of the above embodiments, the embodiments of the present application reset the error number corresponding to the target record item in the case where the target record item corresponding CE signal is not received for a long time, so as to avoid the case where CE storm suppression is performed in the case where the CE signal frequency is low, reduce the frequency of CE storm suppression, and maintain the normal operation of the system.
[0117] In a possible implementation, after the correctable error storm suppression is performed according to the suppression level in step S205, the method further includes step S230.
[0118] S230: Increase the suppression number corresponding to the target record item by m, where m is a positive integer.
[0119] In the step, m, for example, 1, 2, 3, etc., can be set by the staff in advance.
[0120] From the description of the above embodiments, the number of times of CE information appearing in the record item and the suppression frequency are counted by increasing the number of times of suppression after CE storm suppression, so that relevant records are not ignored even if CE storm suppression is performed.
[0121] In a possible implementation, after the step S205 of performing correctable error storm suppression according to the suppression level, the method further includes steps S240 and S241.
[0122] S240: If the duration of performing the correctable error storm suppression is greater than a preset duration threshold, the correctable error storm suppression is released.
[0123] In the step, the duration threshold can be set by the staff according to experimental data or experience parameters. Releasing the correctable error storm suppression can include re-executing the processing procedure in the case of not performing suppression in the step S206E after receiving a new CE signal.
[0124] S241: Resetting the error number and the suppression number of the record item subjected to the correctable error storm suppression.
[0125] In the step, the error number and the suppression number of the record item subjected to the suppression are set to 0.
[0126] Figure 5 The error number and the suppression number of the record item after resetting provided by the embodiments of the present application are shown in the schematic diagram. Figure 5 As shown, the error number and the suppression number of the record items A1, A2 and A4 are reset, and the error number and the suppression number of A3 are not reset because A3 is not subjected to suppression.
[0127] From the description of the above embodiments, the error number and the suppression number of the record item subjected to the correctable error storm suppression are reset when the duration of the CE storm suppression exceeds the duration threshold, so that new CE errors are not ignored due to long-time CE suppression, and resetting the error number and the suppression number helps to more accurately reflect the actual error situation.
[0128] Figure 6 The structural schematic diagram of the correctable error storm suppression device provided by the embodiments of the present application is shown in the diagram. Figure 6 As shown, the correctable error storm suppression device 600 includes an information obtaining module 601, a record obtaining module 602, a first increasing module 603, a second increasing module 604, a level obtaining module 605 and a storm suppression module 606.
[0129] The information obtaining module 601 is configured to determine an error type and an error module of the correctable error corresponding to the correctable error signal in response to receiving the correctable error signal.
[0130] The record obtaining module 602 is configured to obtain an error record corresponding to the error type, wherein the error record comprises one or more record items.
[0131] The first increasing module 603 is configured to increase an error number corresponding to a target record item in the target record item by n to obtain a new error number, if the target record item corresponds to the error module, wherein n is a positive integer.
[0132] The second increasing module 604 is configured to add the target record item in the error record and increase an error number corresponding to the target record item by n to obtain a new error number, if the target record item is not included in the record item.
[0133] The level obtaining module 605 is configured to determine the suppression level if the new error number is greater than a preset error number threshold.
[0134] The storm suppression module 606 is configured to perform correctable error storm suppression according to the suppression level.
[0135] The apparatus provided in the embodiment can be used to execute the technical solutions of the above method embodiments, and has similar implementation principles and technical effects, which will not be described here in detail.
[0136] In a possible implementation, the storm suppression module 606 is configured to stop sending a system management interrupt instruction to an operating system for the correctable error signal if the suppression level comprises a first suppression level; stop log record storage for the correctable error signal if the suppression level comprises a second suppression level; stop fault prediction, fault diagnosis and log record storage for the correctable error signal if the suppression level comprises a third suppression level; and stop format conversion, fault prediction, fault diagnosis and log record storage for the correctable error signal if the suppression level comprises a fourth suppression level.
[0137] In a possible implementation, the storm suppression module 606 is configured to perform format conversion on the correctable error signal to obtain a converted error signal if the suppression level is a fifth suppression level; analyze the converted error signal to obtain an analysis result; generate an error log according to the analysis result, persist the error log, and send an error reporting message to the operating system for the correctable error signal.
[0138] In a possible implementation, the second increasing module 604 is configured to: if the error record does not include the target record item corresponding to the error module, and includes a blank record item that is not written, write information of the error module in the blank record item to obtain the target record item, and increase the error number corresponding to the target record item by n; if the error record does not include the target record item corresponding to the error module, and the historical record items are full, obtain the error number and the reset number corresponding to each historical record item; determine a to-be-replaced item in the historical record items according to the error number and the reset number; and cover the to-be-replaced item with the information of the error module, and set the error number corresponding to the to-be-replaced item as n.
[0139] In a possible implementation, the second increasing module 604 is configured to: multiply the reset number of the target historical record item by the error number threshold to obtain a reset error number; add the reset error number to the error number of the target historical record item to obtain a total error number; and determine a historical record item with the smallest total error number as the to-be-replaced item.
[0140] In a possible implementation, the correctable error storm suppression apparatus 600 further includes a number increasing module 607.
[0141] The number increasing module 607 is configured to: obtain a historical error time corresponding to a target record item in an error record, where the historical error time is a time of a most recent error corresponding to the target record item; determine whether a time length from the historical error time to a current time is greater than a preset suppression time threshold; and if the time length from the historical error time to the current time is not greater than the suppression time threshold, perform a step of increasing an error number corresponding to the target record item by n to obtain a new error number.
[0142] In a possible implementation, the correctable error storm suppression apparatus 600 further includes a number reset module 608.
[0143] The number reset module 608 is configured to: if the time length from the historical error time to the current time is greater than or equal to the preset suppression time threshold, reset the error number corresponding to the error module.
[0144] In a possible implementation, the number increasing module 607 is further configured to increase a suppression number corresponding to the target record item by m, where m is a positive integer.
[0145] In a possible implementation, the number reset module 608 is further configured to: if a time length of performing correctable error storm suppression is greater than a preset time threshold, cancel the correctable error storm suppression; and reset the error number and the suppression number of the record item subjected to the correctable error storm suppression.
[0146] The device provided by the embodiment can be used to execute the technical solutions of the method embodiments, and has similar implementation principles and technical effects, which will not be described here again.
[0147] Figure 7 A structural schematic diagram of a computing device is provided in the embodiment. The computing device can be a first computing device, and the computing device can also be a second computing device.
[0148] As shown in Figure 7 The computing device 700 can include a baseboard management controller 701 and a memory 702, where the baseboard management controller 701 and the memory 702 are coupled and can communicate. For example, the baseboard management controller 701 and the memory 702 communicate through a communication bus 703. The memory 702 is configured to store computer-executable instructions, and the baseboard management controller 701 is configured to invoke the computer-executable instructions in the memory to execute the correctable error storm suppression method shown in the method embodiments. The computing device 700 can be a server.
[0149] Optionally, the computing device 700 can further include a communication interface, which can include a transmitter and / or a receiver.
[0150] Optionally, the baseboard management controller can be a central processing unit (CPU), and can also be a graphics processing unit (GPU), another general-purpose processor, a digital signal processor (DSP), or an application specific integrated circuit (ASIC), etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment can be directly embodied by a hardware processor for execution, or executed by a combination of hardware and software modules in the processor.
[0151] The embodiment provides a computer readable storage medium, and the computer readable storage medium stores computer program instructions; the program instructions are executed by a computing device or the computing device, so that the computing device or the computing device executes the correctable error storm suppression method.
[0152] The embodiment provides a computer program product, and the computer program product includes computer program instructions, when the computer program instructions are executed, the computing device or the computing device executes the correctable error storm suppression method.
[0153] All or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The aforementioned program can be stored in a readable memory. The program, when executed, performs steps including the above-mentioned method embodiments; and the aforementioned memory (storage medium) includes: a read-only memory (English: read-only memory, abbreviation: ROM), a RAM, a flash memory, a hard disk, a solid state disk, a magnetic tape, a floppy disk (English: floppy disk), an optical disc (English: optical disc), and any combination thereof.
[0154] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable terminal equipment processing unit to produce a machine, so that the instructions executed by the computer or other programmable terminal equipment processing unit produce a device implemented in the flowcharts and / or block diagrams. Figure 1 The device that implements the function specified in one flow or multiple flows and / or blocks. Figure 1 The device that implements the function specified in one flow or multiple flows and / or blocks.
[0155] These computer program instructions can also be stored in a computer readable memory that can guide the computer or other programmable terminal equipment to work in a specific way, so that the instructions stored in the computer readable memory produce a product including instruction devices that implement the flowcharts and / or block diagrams. Figure 1 The device that implements the function specified in one flow or multiple flows and / or blocks. Figure 1 The device that implements the function specified in one flow or multiple flows and / or blocks.
[0156] These computer program instructions can also be loaded into a computer or other programmable terminal equipment, so that a series of operation steps are performed on the computer or other programmable equipment to produce a computer-implemented process, so that the instructions executed on the computer or other programmable equipment provide a product for implementing the flowcharts and / or block diagrams. Figure 1 The device that implements the function specified in one flow or multiple flows and / or blocks. Figure 1 Figure 1 The device that implements the function specified in one flow or multiple flows and / or blocks.
[0157] Obviously, those skilled in the art can make various modifications and variations to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalent technologies, the embodiments of the present application also intend to include these modifications and variations.
[0158] In the embodiments of the present application, the terms "comprising", "containing" or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article, or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed, or also include elements inherent in such process, method, article, or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element. The term "or" and its variants can refer to "and / or". In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. In the embodiments of the present application, "a plurality of" means two or more. "And / or" describes the relationship between the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A alone, A and B together, and B alone. The character " / " generally represents an "or" relationship between the associated objects.
[0159] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the embodiments of the present application, and not to limit them; although the embodiments of the present application have been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A method of error storm mitigation that can be corrected, characterized by, The method comprises: in response to receiving a correctable error signal, determining an error type and an error module of a correctable error corresponding to the correctable error signal; obtaining an error record corresponding to the error type, wherein the error record comprises one or more record items; if the target record item corresponding to the error module is included in the record item, increasing an error number corresponding to the target record item by n to obtain a new error number, wherein n is a positive integer; if the target record item is not included in the record item, adding the target record item in the error record and increasing the error number corresponding to the target record item by n to obtain the new error number; if the new error number is greater than a preset error number threshold, determining a suppression level; performing correctable error storm suppression according to the suppression level.
2. The method of claim 1, wherein, The performing correctable error storm suppression according to the suppression level comprises: if the suppression level comprises a first suppression level, stopping sending a system management interrupt instruction to an operating system for the correctable error signal; if the suppression level comprises a second suppression level, stopping log record storage for the correctable error signal; if the suppression level comprises a third suppression level, stopping fault prediction, fault diagnosis and log record storage for the correctable error signal; if the suppression level comprises a fourth suppression level, stopping format conversion, fault prediction, fault diagnosis and log record storage for the correctable error signal.
3. The method of claim 1, wherein, The performing correctable error storm suppression according to the suppression level comprises: if the suppression level is a fifth suppression level, performing format conversion on the correctable error signal to obtain a converted error signal; parsing the converted error signal to obtain a parsing result; generating an error log according to the parsing result, persisting the error log, and sending error information to an operating system for the correctable error signal.
4. The method of claim 1, wherein, If the target record item is not included in the record item, the target record item is added in the error record, and the error number corresponding to the target record item is increased by n, which comprises: if the target record item corresponding to the error module is not included in the error record and a blank record item is included, writing information of the error module in the blank record item to obtain the target record item and increase the error number corresponding to the target record item by n; if the target record item corresponding to the error module is not included in the error record and historical record items have been written full, obtaining error numbers and reset numbers corresponding to the historical record items; determining a to-be-replaced item in the historical record items according to the error numbers and the reset numbers; covering the to-be-replaced item with the information of the error module and setting the error number corresponding to the to-be-replaced item to n.
5. The method of claim 4, wherein, The determining a to-be-replaced item in the historical record items according to the error numbers and the reset numbers comprises: multiplying the reset number of a target historical record item by the error number threshold to obtain a reset error number; adding the reset error number and the error number of the target historical record item to obtain a total error number; The history record item with the minimum total error number is determined as the replacement item.
6. The method according to any one of claims 1 to 5, characterized in that, After the error record corresponding to the error type is obtained, the method further includes: obtaining a history error reporting time corresponding to the target record item in the error record, wherein the history error reporting time is a time of a latest error reporting corresponding to the target record item; determining whether a time length from the history error reporting time to a current time is greater than a preset suppression time threshold; if the time length from the history error reporting time to the current time is not greater than the suppression time threshold, increasing the error number corresponding to the target record item by n to obtain a new error number.
7. The method of claim 6, wherein, The method further includes: if the time length from the history error reporting time to the current time is greater than or equal to the preset suppression time threshold, resetting the error number corresponding to the error module.
8. The method according to any one of claims 1 to 7, characterized in that, After the correctable error storm suppression is performed according to the suppression level, the method further includes: increasing a suppression number corresponding to the target record item by m, wherein m is a positive integer.
9. The method according to any one of claims 1 to 8, characterized in that, After the correctable error storm suppression is performed according to the suppression level, the method further includes: if a time length of the correctable error storm suppression is greater than a preset time threshold, releasing the correctable error storm suppression; resetting the error number and the suppression number of the record item subjected to the correctable error storm suppression.
10. A computing device, comprising: The computing device includes a memory and a baseboard management controller; the memory and the baseboard management controller are coupled; the memory is configured to store program instructions; the baseboard management controller is configured to invoke the program instructions to enable the computing device to perform the method according to any one of claims 1 to 9.