Fault memory processing method and device, storage medium and server

Abstract

The invention discloses a fault memory processing method and device, a storage medium and a server, and relates to the field of server security, fault information is rapidly collected and a fault type is determined when an interrupt signal of a fault memory is received, so that correctable errors and uncorrectable errors can be effectively distinguished, and the correctable errors are more accurate. Especially for more complex downtime errors in uncorrectable errors, downtime restart of the whole system is avoided by directly controlling the fault memory to be powered off. By means of the processing mode, the influence on the task in progress is remarkably reduced, the bad experience of clients caused by server downtime is reduced, meanwhile, through real-time monitoring and quick response, the method can improve the stability and reliability of the system, and therefore challenges brought by memory faults can be more effectively handled.

Description

Technical Field

[0001] The present invention relates to the field of server security, and in particular to a method, device, storage medium and server for processing faulty memory. Background Art

[0002] Memory, also known as main memory, is a storage space that can be directly accessed by the CPU (Central Processing Unit), and is composed of semiconductor devices. Its main feature is the fast data access speed. Memory is one of the core components in a computer, as opposed to external memory. Usually, a large amount of data that needs to be permanently saved is stored in external memory, while temporary or small amounts of data and programs are stored in memory.

[0003] Memory failures are mainly divided into correctable errors and uncorrectable errors. Correctable errors will not cause the machine to crash or the system to hang, the process will not end, and data will not be lost. Uncorrectable errors are more complicated. Some error types will not cause the system to hang, while others will cause the system to hang, ending the current process and losing valid data. In particular, some uncorrectable errors are considered fatal errors, which will cause the system to hang, resulting in the end of the process and loss of data.

[0004] The current solution for downtime failures that cause system hangs is to restart the entire server, but restarting the server machine will interrupt ongoing tasks and bring a bad experience to customers. Summary of the invention

[0005] The purpose of the present invention is to provide a method, device, storage medium and server for processing faulty memory. For more complex downtime errors among uncorrectable errors, the faulty memory is directly controlled to power off, thereby avoiding downtime and restart of the entire system. This processing method significantly reduces the impact on ongoing tasks and reduces the negative experience of customers caused by server downtime. At the same time, through real-time monitoring and rapid response, the method can improve the stability and reliability of the system, thereby more effectively responding to the challenges brought by memory failures.

[0006] On the one hand, the present application provides a method for processing faulty memory, comprising:

[0007] When receiving an interrupt signal corresponding to a fault memory, collecting fault information of the fault memory, the fault information including location information, register information and fault cause;

[0008] Determine the fault type of the fault memory according to the fault information, where the fault type includes a correctable error and an uncorrectable error;

[0009] When it is determined that the fault type of the faulty memory is the uncorrectable error, determining an error type of the faulty memory, the error type including a downtime error and a non-downtime error;

[0010] When it is determined that the error type of the faulty memory is the downtime error, the faulty memory is controlled to be powered off.

[0011] Wherein, when it is determined that the error type of the faulty memory is the downtime error, controlling the faulty memory to power off includes:

[0012] When it is determined that the error type of the faulty memory is the downtime error, setting a preset flag bit to a first state, the flag bit being set in a structure of a message for reporting to a baseboard management controller, and the flag bit being in a second state when the error type of the faulty memory is the non-downtime error;

[0013] triggering the baseboard management controller to report the fault information of the fault memory to the complex programmable logic device and control the fault memory to be powered off when detecting that the state of the flag bit is the first state;

[0014] triggering the baseboard management controller to report the fault information of the fault memory to the complex programmable logic device when detecting that the state of the flag bit is the second state;

[0015] The complex programmable logic device is triggered to control the fault memory to power off.

[0016] The complex programmable logic device is connected to the power-on and power-off control pins of the power management integrated circuit through a preset output pin, and the power management integrated circuit is connected to the power supply terminal of the fault memory through the power-on and power-off control pins to supply power to the fault memory;

[0017] Triggering the complex programmable logic device to control the fault memory to power off, including:

[0018] The complex programmable logic device is triggered to set the power-on and power-off control pins of the power management integrated circuit to a power-off state through the preset output pin, so that the power management integrated circuit stops supplying power to the faulty memory.

[0019] Wherein, the fault memory management method further includes:

[0020] When receiving interrupt signals corresponding to more than one faulty memory, storing the interrupt signals in an interrupt queue;

[0021] Recording a timestamp of receiving each interrupt signal, and sorting all the interrupt signals according to the timestamp;

[0022] For each of the interrupt signals in sequence, the step of collecting the fault information of the fault memory is performed.

[0023] The uncorrectable errors include catastrophic errors, fatal errors and recoverable errors, and the recoverable errors include uncorrectable non-fatal array errors, single-bit read-after-write errors, and single-bit read-after-read errors;

[0024] The downtime errors include the single-bit read-after-read errors, the catastrophic errors and the fatal errors; the non-downtime errors include uncorrectable non-fatal array errors and single-bit read-after-write errors.

[0025] Among them, it also includes:

[0026] Acquire the current status indicators of the server in real time at a preset frequency, wherein the current status indicators include the CPU load indicator, memory usage indicator, disk input and output activity indicator, and network traffic indicator;

[0027] Store the current status indicators within a preset time length into a temporary buffer;

[0028] Upon receiving an interrupt signal corresponding to the fault memory, obtaining a current state indicator of a preset time length before the current moment stored in the temporary cache to obtain a system state before the fault occurs;

[0029] Integrate the current status indicators of the preset time length and mark them with timestamps;

[0030] Analyze the integrated current status indicators of the preset time length to determine the probability that each of the status indicators causes the faulty memory to occur;

[0031] Constructing a prediction model based on the probability and the current state indicator of the preset time length;

[0032] Using the prediction model to predict the probability of each memory in the server failing and the type of error that occurs;

[0033] For a memory predicted to fail, if the error type predicted to fail is a downtime error among the uncorrectable errors and the probability of failure is greater than a preset threshold, the memory predicted to fail is powered off or warning information is reported.

[0034] The integrated current status indicators of the preset time length are analyzed to determine the probability that each of the status indicators causes the faulty memory to occur, including:

[0035] Standardizing the integrated current status indicator of the preset time length to obtain a current standard status indicator;

[0036] Select key features related to fault memory from the current standard state indicators, calculate the statistics of the key features and extract the time series features of the key features; establish and optimize a learning model based on the statistics and the time series features, the learning model characterizes the corresponding relationship between the current standard state indicators and the probability of fault memory occurrence;

[0037] The probability of each of the state indicators causing the faulty memory to occur is determined according to the learning model.

[0038] On the other hand, the present application provides a device for processing a faulty memory, comprising:

[0039] Memory for storing computer programs;

[0040] The processor is used to implement the steps of the method for processing faulty memory described above when executing a computer program.

[0041] On the other hand, the present application provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for processing faulty memory described above are implemented.

[0042] On the other hand, the present application provides a server, including the above-mentioned fault memory processing device.

[0043] The present invention discloses a method, device, storage medium and server for processing faulty memory, which relates to the field of server security. By quickly collecting fault information and determining the fault type when receiving an interrupt signal from the faulty memory, it is possible to effectively distinguish between correctable errors and uncorrectable errors, especially for more complex downtime errors among uncorrectable errors, by directly controlling the power-off of the faulty memory, the downtime and restart of the entire system is avoided. This processing method significantly reduces the impact on ongoing tasks and reduces the negative experience of customers caused by server downtime. At the same time, through real-time monitoring and rapid response, the method can improve the stability and reliability of the system, thereby more effectively coping with the challenges brought by memory failures. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A flowchart of a method for processing faulty memory provided by the present invention;

[0046] Figure 2 A schematic diagram of an error type of a fault memory provided by the present invention;

[0047] Figure 3 A schematic diagram of a fault memory processing device provided by the present invention;

[0048] Figure 4 A schematic diagram of a computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0049] The core of the present invention is to provide a method, device, storage medium and server for processing faulty memory. For more complex downtime errors among uncorrectable errors, the faulty memory is directly controlled to power off, thereby avoiding downtime and restart of the entire system. This processing method significantly reduces the impact on ongoing tasks and reduces the negative experience of customers caused by server downtime. At the same time, through real-time monitoring and rapid response, the method can improve the stability and reliability of the system, thereby more effectively coping with the challenges brought by memory failures.

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] On the one hand, if Figure 1 As shown, the present application provides a method for processing faulty memory, including:

[0052] S11: upon receiving an interrupt signal corresponding to the faulty memory, collecting fault information of the faulty memory, the fault information including location information, register information and fault cause;

[0053] In this step, when the interrupt signal corresponding to the faulty memory is received, a fault information collection mechanism will be triggered first. This mechanism will quickly record the specific location information of the faulty memory for subsequent positioning and diagnosis. At the same time, register information will be extracted, including the current register status and operation code. This information can reflect the specific execution of the system when the fault occurs, thereby helping to determine the cause of the fault. In addition, the cause of the fault will be analyzed, which may include hardware damage, memory unit failure, data transmission errors, etc. The purpose of this process is to quickly obtain comprehensive fault information, provide sufficient data support for subsequent fault type judgment and processing decisions, and ensure timely and accurate response and repair when a fault occurs, reduce system downtime and improve overall stability.

[0054] S12: determining a fault type of the faulty memory according to the fault information, where the fault type includes a correctable error and an uncorrectable error;

[0055] In this step, the collected fault information will be analyzed in depth to determine the fault type of the faulty memory. First, the fault information will be compared and judged to identify whether the nature of the fault is a correctable error or an uncorrectable error. Correctable errors usually refer to those problems that can be fixed by retry or correction mechanisms, such as temporary errors in certain data bits, which will not cause system crashes or data loss. Relatively speaking, uncorrectable errors are more serious, and such errors may cause system downtime or data loss. Therefore, the memory behavior, error patterns, and historical fault records reflected in the fault information will be analyzed, and combined with the current system status, specific algorithms or rules will be used to determine the fault type. This process is crucial because different fault types determine the subsequent processing strategy, ensuring that the system can take targeted measures to maintain the stability and reliability of operation.

[0056] S13: When it is determined that the fault type of the faulty memory is an uncorrectable error, determining an error type of the faulty memory, where the error type includes a downtime error and a non-downtime error;

[0057] In this step, after confirming that the fault type of the faulty memory is an uncorrectable error, the specific nature of the error will be further analyzed to identify its error type, which mainly includes downtime errors and non-downtime errors. Downtime errors refer to those errors that may cause the system to crash or stop working completely. The occurrence of such errors usually means that the system cannot continue to perform the current task, which may cause serious data loss or service interruption. Relatively speaking, non-downtime errors will not immediately cause the system to hang. Although these errors may also affect the stability and data integrity of the system, their impact is relatively small, and the system is usually allowed to continue to run. Therefore, these error types will be classified by analyzing the specific error behavior recorded in the fault information, the response mode of the memory, and the historical fault data, using preset logical judgment rules or algorithms. Through this refinement process, the severity of the fault can be accurately identified, so that corresponding response strategies can be adopted in subsequent processing to ensure the stability and availability of the system.

[0058] S14: When it is determined that the error type of the faulty memory is a downtime error, the faulty memory is controlled to be powered off.

[0059] In this step, when it is determined that the error type of the faulty memory is a downtime error, the process of controlling the power off of the faulty memory will be started. This process is based on the severity assessment of the fault type, because downtime errors mean that the memory may cause the system to crash or completely lose response, thereby affecting the stability of the entire computing environment and data security. Therefore, first of all, by monitoring the relevant status information and fault reports, it is confirmed that the status of the memory module is no longer reliable, and then it is decided to immediately cut off its power supply to prevent possible further damage. This control is usually implemented by the baseboard management controller (BMC) or other management modules, which instructs the power management integrated circuit (PMIC) to turn off the power supply to the faulty memory by issuing a power control signal. The power-off operation not only helps to protect other normally functioning memory modules and system components from potential cascading failures, but also provides conditions for subsequent troubleshooting and maintenance, ensuring that the system can quickly recover to a safe state after a fault occurs, avoiding greater system risks caused by continuous power supply.

[0060] In summary, the method for processing faulty memory provided by the present invention can effectively distinguish correctable errors from uncorrectable errors by quickly collecting fault information and determining the fault type when receiving the interrupt signal of the faulty memory. In particular, for more complex downtime errors among uncorrectable errors, the faulty memory can be directly powered off to avoid downtime and restart of the entire system. This processing method significantly reduces the impact on ongoing tasks and reduces the negative experience of customers caused by server downtime. At the same time, through real-time monitoring and rapid response, the method can improve the stability and reliability of the system, thereby more effectively coping with the challenges brought by memory failures.

[0061] Based on the above embodiments:

[0062] As a preferred embodiment, when it is determined that the error type of the faulty memory is a downtime error, controlling the faulty memory to power off includes:

[0063] When it is determined that the error type of the faulty memory is a downtime error, a preset flag bit is set to a first state, the flag bit is set in a structure of a message for reporting to the baseboard management controller, and the flag bit is in a second state when the error type of the faulty memory is a non-downtime error;

[0064] The baseboard management controller is triggered to report the fault information of the faulty memory to the complex programmable logic device and control the power-off of the faulty memory when the state of the flag bit is detected to be the first state;

[0065] The baseboard management controller is triggered to report the fault information of the fault memory to the complex programmable logic device when detecting that the state of the flag bit is the second state.

[0066] In a preferred embodiment, when it is determined that the error type of the faulty memory is a downtime error, a preset flag bit is first set to a first state, and this state change is incorporated into the message structure for reporting to the baseboard management controller (BMC). This flag bit setting mechanism allows the system to respond in a timely manner when a downtime error is detected, indicating that the state of the memory module is no longer reliable, and more stringent processing measures need to be taken.

[0067] When the BMC detects that the flag is in the first state, it will immediately report the relevant fault information of the faulty memory to the complex programmable logic device (CPLD). At this time, the CPLD can not only record the fault information, but also execute the control signal to quickly cut off the power supply of the faulty memory to prevent further damage. This process ensures the efficiency and reliability when handling downtime errors, aiming to eliminate the hidden dangers that may cause system crashes as soon as possible. When the flag is in the second state, it indicates that the faulty memory is a non-downtime error, then only the relevant fault information needs to be reported without the need to immediately power off the power.

[0068] This design effectively achieves flexible response to different error types, ensuring stable operation of the system and data security.

[0069] As a preferred embodiment, reporting the fault information of the faulty memory to a complex programmable logic device, and controlling the power-off of the faulty memory, includes: reporting the fault information of the faulty memory to the complex programmable logic device; triggering the complex programmable logic device to control the power-off of the faulty memory. As a preferred embodiment, the complex programmable logic device is connected to the power-on and power-off control pins of the power management integrated circuit through a preset output pin, and the power management integrated circuit is connected to the power supply end of the faulty memory through the power-on and power-off control pins to supply power to the faulty memory; triggering the complex programmable logic device to control the power-off of the faulty memory includes: triggering the complex programmable logic device to set the power-on and power-off control pins of the power management integrated circuit to the power-off state through the preset output pin, so that the power management integrated circuit stops supplying power to the faulty memory.

[0070] In a preferred embodiment, when an error occurs in a faulty memory, its fault information is reported to a complex programmable logic device (CPLD) for centralized processing. As a flexible control unit, the CPLD is capable of receiving various fault information from the system and making corresponding decisions based on the information. After receiving the fault information, the CPLD will send a control signal to trigger the power management integrated circuit (PMIC) connected to it to perform a power-off operation. Specifically, the CPLD is connected to the power-on and power-off control pins of the PMIC through a preset output pin, forming a power management link. The key to this link is that the PMIC is directly connected to the power supply end of the faulty memory through the power-on and power-off control pins to provide the necessary power supply for the faulty memory. When the CPLD detects that the state of the faulty memory needs to be powered off, it will set the signal of the preset output pin to the power-off state, instructing the PMIC to stop supplying power to the faulty memory.

[0071] In a specific embodiment, the operation of the power-on and power-off control pins is to control the power state of the memory by pulling down and pulling up the signal. Specifically, when the faulty memory detects an error, the complex programmable logic device (CPLD) pulls down the power-on and power-off control pins, and this operation instructs the power management integrated circuit (PMIC) to stop supplying power to the faulty memory, thereby realizing the power-off state of the memory. In this state, the memory is powered off, effectively preventing data corruption and system instability. On the contrary, when the system needs to restart or verify that the fault is repaired, the CPLD will pull up the power-on and power-off control pins to instruct the PMIC to re-supply power to the faulty memory. This pull-up signal restores power supply to the memory, allowing the system to re-detect the state of the memory and operate normally. Through this simple and effective control mechanism, the system can flexibly respond to memory failures and improve overall reliability and stability.

[0072] In summary, the design of this mechanism ensures that when a fault occurs, the power supply of the faulty memory can be quickly cut off, thereby preventing potential data corruption or system crashes, thereby improving the security and stability of the system. Through this process, not only can faults be responded to in a timely manner, but also multiple fault handling strategies can be implemented through the flexibility of CPLD, improving the overall fault management capability.

[0073] As a preferred embodiment, the fault memory management method further includes:

[0074] When receiving interrupt signals corresponding to more than one faulty memory, storing the interrupt signals in an interrupt queue;

[0075] Record the timestamp of each interrupt signal received, and sort all interrupt signals according to the timestamp;

[0076] For each interrupt signal in sequence, the step of collecting fault information of the fault memory is entered.

[0077] In a preferred embodiment, the fault memory management method ensures accurate and timely response to multiple fault memories through an efficient interrupt signal processing mechanism. When more than one interrupt signal corresponding to a fault memory is received, these signals will be stored in an interrupt queue for subsequent processing. During the storage process, a timestamp will be recorded for each received interrupt signal, which reflects the specific time when the signal occurred. This step is critical because it allows the system to sort all interrupt signals and ensure that they are processed in the order in which the events occurred, thereby avoiding confusion or information loss caused by multiple faults occurring at the same time.

[0078] After the interrupt signals are sorted, each signal in the queue will be processed one by one, and the step of collecting fault information in the fault memory will be entered. This sequential processing method not only ensures the integrity of all fault information, but also improves the response speed and accuracy of the system, thereby more effectively identifying and classifying fault types.

[0079] Through this mechanism, corresponding measures can be taken in a timely manner to reduce the impact of failures on overall operations and improve the reliability and stability of the system.

[0080] like Figure 2 As shown, as a preferred embodiment, the uncorrectable errors include catastrophic errors, fatal errors and recoverable errors, and the recoverable errors include uncorrectable non-fatal array errors, single-bit read-after-write errors, and single-bit read-after-read errors;

[0081] Among them, downtime errors include single-bit read-after-read errors, catastrophic errors, and fatal errors; non-downtime errors include uncorrectable non-fatal array errors and single-bit read-after-write errors.

[0082] In this preferred embodiment, the classification of uncorrectable errors helps to accurately identify and handle different types of memory failures, thereby improving the stability and reliability of the system. Uncorrectable errors are mainly divided into catastrophic errors, fatal errors, and recoverable errors. Catastrophic errors usually refer to those failures that cause system crashes or unrecoverable states, while fatal errors are errors that seriously affect system functions, resulting in data loss and service interruptions. Recoverable errors refer to errors that can be handled or bypassed under certain conditions, but they still cannot be automatically corrected by the system. For example, uncorrectable non-fatal array errors, single-bit read-after-write errors, and single-bit read-after-read errors are all specific instances of recoverable errors. Although these errors will not immediately cause system downtime, they may affect the consistency or integrity of the data.

[0083] When further subdividing the error types, the definition of downtime errors is particularly important. Downtime errors include single-bit read-after-read errors, catastrophic errors, and fatal errors, which directly cause the system to hang or restart, seriously affecting the continuity and stability of operation. Non-downtime errors include uncorrectable non-fatal array errors and single-bit read-after-write errors. Although these errors do not immediately cause system crashes, they still need to be taken seriously so that appropriate repair measures can be taken in subsequent processing.

[0084] Through this classification system, the system can more effectively assess the severity of the failure and formulate corresponding response strategies to ensure the smoothness of the overall operation and the security of the data.

[0085] As a preferred embodiment, it also includes:

[0086] Obtain the current status indicators of the server in real time at a preset frequency. The current status indicators include CPU load indicators, memory usage indicators, disk input and output activity indicators, and network traffic indicators.

[0087] Store the current status indicators within a preset time length into a temporary buffer;

[0088] Upon receiving an interrupt signal corresponding to the fault memory, obtaining a current state indicator of a preset time length before the current moment stored in the temporary cache to obtain a system state before the fault occurs;

[0089] Integrate the current status indicators of a preset time length and mark them with timestamps;

[0090] Analyze the current status indicators of the preset time length after integration to determine the probability of each status indicator causing the faulty memory to occur;

[0091] Build a prediction model based on the current state indicators of probability and preset time length;

[0092] Use the prediction model to predict the probability of each memory failure in the server and the type of error that occurs;

[0093] For a memory predicted to fail, if the error type predicted to fail is a downtime error among uncorrectable errors and the probability of failure is greater than a preset threshold, the memory predicted to fail is powered off or warning information is reported.

[0094] In this preferred embodiment, by acquiring various current status indicators of the server (such as CPU load, memory usage, disk input / output activity, and network traffic, etc.) in real time at a preset frequency, the operating status can be fully monitored so that various performance indicators can be grasped in time before a failure occurs. These indicators will be stored in a temporary cache for a preset time length to ensure that when an interrupt signal corresponding to the faulty memory is received, the system status information before the failure can be quickly accessed. By integrating these indicators and marking them with timestamps, a time series can be established for subsequent analysis.

[0095] When analyzing the integrated status indicators, by determining the probability of each status indicator causing a faulty memory, it is possible to identify which indicators may be the cause of the failure. The prediction model built based on these probabilities and the current status indicators provides a basis for future failures and can predict the probability of failure of each memory in the server and its error type. This prediction not only helps to identify potential risks in advance, but also guides the operation and maintenance team to take appropriate preventive measures.

[0096] In particular, for memory predicted to fail, if its failure type is assessed as a downtime error among uncorrectable errors, and its probability of occurrence exceeds the set threshold, proactive measures will be taken, such as powering off the memory to prevent system crashes, or reporting warning information to notify the operation and maintenance team.

[0097] Such a mechanism not only improves the reliability of the system, but also significantly reduces the risk of service interruption and data loss due to failures, thereby ensuring the smoothness and stability of the overall operation.

[0098] As a preferred embodiment, analyzing the integrated current status indicators of the preset time length to determine the probability of each status indicator causing the fault memory to occur includes:

[0099] Standardizing the integrated current status indicators of the preset time length to obtain current standard status indicators;

[0100] Select key features related to fault memory from the current standard state indicators, calculate the statistics of the key features, and extract the time series features of the key features; establish and optimize a learning model based on the statistics and time series features, and the learning model represents the corresponding relationship between the current standard state indicators and the probability of fault memory occurrence;

[0101] The probability of each state indicator causing the faulty memory to occur is determined according to the learning model.

[0102] In this preferred embodiment, first of all, it is key to standardize the integrated current state indicators. This step can eliminate the impact caused by dimensional differences between different indicators, so that subsequent analysis is more accurate. Through standardization, the current standard state indicators obtained can more clearly reflect the relative changes of each indicator. Next, key features related to fault memory are selected from these standard state indicators. This usually requires the application of statistical analysis techniques, such as correlation analysis, to identify which indicators show significant correlations in past fault cases. Subsequently, statistics of these key features, such as mean, variance, and extreme values, are calculated. These statistics help to understand the basic distribution and fluctuation range of the features. In addition, extracting time series features further captures the patterns of changes in indicators over time, such as trends and seasonality, which is crucial for fault prediction.

[0103] Based on the above statistics and time series characteristics, a learning model can be constructed and optimized, which aims to characterize the corresponding relationship between the current standard state indicators and the probability of fault memory occurrence. Select a suitable learning model (such as logistic regression, decision tree or random forest) and train it with historical data so that the model can accurately capture the complex relationship between the indicator and the occurrence of faults. In the process of model training, methods such as cross-validation are used to optimize model parameters to improve its prediction performance and generalization ability. Finally, with the help of the trained learning model, the probability of each state indicator causing the occurrence of fault memory can be calculated. The output of the model not only provides the possibility of fault occurrence, but also can further analyze the relative contribution of each state indicator to the fault risk, which provides a scientific basis and guidance for subsequent fault warning and system maintenance.

[0104] Through this series of systematic analysis and modeling processes, dynamic monitoring of system status and effective prediction of failure risks can be achieved, thereby improving the reliability and safety of the system.

[0105] As a preferred embodiment, the process of powering off the memory predicted to fail or reporting warning information may also include: dynamically adjusting the failure warning threshold mechanism.

[0106] Specifically, in order to more accurately respond to system performance under different loads, the fault warning threshold can be dynamically adjusted based on real-time data. This mechanism can improve the system's responsiveness to potential faults, especially when the load fluctuates greatly.

[0107] Specifically, the current status indicators of the server are continuously collected, including CPU load, memory usage, disk I / O activity, and network traffic. The historical data of these indicators are recorded for analysis. The fluctuations of each indicator are analyzed based on the historical data to identify the normal range under different load conditions. The dynamic threshold of each indicator is calculated using statistical methods (such as standard deviation, mean, etc.) to adapt to different load conditions. When the system is running, the current load is monitored and the warning threshold is adjusted dynamically. For example, the threshold is increased when the load is high and the threshold is lowered when the load is low to reduce false alarms. The timestamp of each adjustment and the corresponding threshold are recorded for subsequent analysis. Based on the dynamically adjusted threshold, the current status indicator is re-evaluated to determine whether the warning mechanism is triggered. If a status indicator exceeds the dynamic threshold, the fault prediction process is started to determine the fault type and its severity. If the memory error type predicted to be a downtime error among uncorrectable errors and the probability of the fault occurring is greater than the dynamic threshold, the power-off operation is performed or the warning information is reported. At the same time, the status indicator that triggers the warning and its corresponding dynamic threshold are recorded for subsequent analysis and optimization. Collect feedback information after a fault occurs, analyze whether there is any unreasonable threshold setting, and further optimize the dynamic adjustment algorithm accordingly.

[0108] By introducing a dynamic threshold adjustment mechanism, the system can respond to different operating conditions more flexibly, improving the accuracy of fault prediction and the overall stability of the system.

[0109] On the other hand, the present application provides a system for processing a faulty memory, comprising:

[0110] A collecting unit, configured to collect fault information of the fault memory upon receiving an interrupt signal corresponding to the fault memory, wherein the fault information includes location information, register information and a fault cause;

[0111] A first determining unit, configured to determine a fault type of the fault memory according to the fault information, wherein the fault type includes a correctable error and an uncorrectable error;

[0112] A second determining unit is used to determine an error type of the faulty memory when it is determined that the fault type of the faulty memory is the uncorrectable error, wherein the error type includes a downtime error and a non-downtime error;

[0113] The power-off unit is used to control the faulty memory to power off when it is determined that the error type of the faulty memory is the downtime error.

[0114] As a preferred embodiment, the power-off unit is specifically used for:

[0115] When it is determined that the error type of the faulty memory is the downtime error, setting a preset flag bit to a first state, the flag bit being set in a structure of a message for reporting to a baseboard management controller, and the flag bit being in a second state when the error type of the faulty memory is the non-downtime error;

[0116] triggering the baseboard management controller to report the fault information of the fault memory to the complex programmable logic device and control the fault memory to be powered off when detecting that the state of the flag bit is the first state;

[0117] The baseboard management controller is triggered to report the fault information of the fault memory to the complex programmable logic device when it is detected that the state of the flag bit is the second state.

[0118] As a preferred embodiment, reporting the fault information of the fault memory to a complex programmable logic device, and controlling the fault memory to be powered off, includes:

[0119] reporting the fault information of the fault memory to the complex programmable logic device;

[0120] The complex programmable logic device is triggered to control the fault memory to power off.

[0121] As a preferred embodiment, the complex programmable logic device is connected to the power-on and power-off control pins of the power management integrated circuit through a preset output pin, and the power management integrated circuit is connected to the power supply end of the fault memory through the power-on and power-off control pins to supply power to the fault memory;

[0122] Triggering the complex programmable logic device to control the fault memory to power off, including:

[0123] The complex programmable logic device is triggered to set the power-on and power-off control pins of the power management integrated circuit to a power-off state through the preset output pin, so that the power management integrated circuit stops supplying power to the faulty memory.

[0124] As a preferred embodiment, the fault memory management method further includes:

[0125] When receiving interrupt signals corresponding to more than one faulty memory, storing the interrupt signals in an interrupt queue;

[0126] Recording a timestamp of receiving each interrupt signal, and sorting all the interrupt signals according to the timestamp;

[0127] For each of the interrupt signals in sequence, the step of collecting the fault information of the fault memory is performed.

[0128] As a preferred embodiment, the uncorrectable errors include catastrophic errors, fatal errors and recoverable errors, and the recoverable errors include uncorrectable non-fatal array errors, single-bit read-after-write errors, and single-bit read-after-read errors;

[0129] The downtime errors include the single-bit read-after-read errors, the catastrophic errors and the fatal errors; the non-downtime errors include uncorrectable non-fatal array errors and single-bit read-after-write errors.

[0130] As a preferred embodiment, it also includes:

[0131] Acquire the current status indicators of the server in real time at a preset frequency, wherein the current status indicators include the CPU load indicator, memory usage indicator, disk input and output activity indicator, and network traffic indicator;

[0132] Store the current status indicators within a preset time length into a temporary buffer;

[0133] Upon receiving an interrupt signal corresponding to the fault memory, obtaining a current state indicator of a preset time length before the current moment stored in the temporary cache to obtain a system state before the fault occurs;

[0134] Integrate the current status indicators of the preset time length and mark them with timestamps;

[0135] Analyze the integrated current status indicators of the preset time length to determine the probability that each of the status indicators causes the faulty memory to occur;

[0136] Constructing a prediction model based on the probability and the current state indicator of the preset time length;

[0137] Using the prediction model to predict the probability of each memory in the server failing and the type of error that occurs;

[0138] For a memory predicted to fail, if the error type predicted to fail is a downtime error among the uncorrectable errors and the probability of failure is greater than a preset threshold, the memory predicted to fail is powered off or warning information is reported.

[0139] For further introduction to the system for processing faulty memory, please refer to the above embodiments, and this application will not go into details here.

[0140] On the other hand, Figure 3 As shown, the present application provides a device for processing a faulty memory, comprising:

[0141] A memory 31, used for storing computer programs;

[0142] The processor 32 is used to implement the steps of the above-mentioned method for processing faulty memory when executing the computer program. For other introductions to the device for processing faulty memory, please refer to the above-mentioned embodiments, which will not be described in detail in this application.

[0143] On the other hand, Figure 4 As shown, the present application provides a non-volatile storage medium 41, on which a computer program 42 is stored, and when the computer program 42 is executed by a processor, the steps of the above-mentioned method for processing faulty memory are implemented. For other introductions to the non-volatile storage medium 41, please refer to the above-mentioned embodiments, and the present application will not repeat them here.

[0144] On the other hand, the present application provides a server, including the above-mentioned fault memory processing device.

[0145] For other introductions to the server, please refer to the above embodiments, and this application will not go into details here.

[0146] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0147] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing faulty memory, characterized in that: include: When receiving an interrupt signal corresponding to a fault memory, collecting fault information of the fault memory, the fault information including location information, register information and fault cause; Determine the fault type of the fault memory according to the fault information, where the fault type includes a correctable error and an uncorrectable error; When it is determined that the fault type of the faulty memory is the uncorrectable error, determining an error type of the faulty memory, the error type including a downtime error and a non-downtime error; When it is determined that the error type of the faulty memory is the downtime error, the faulty memory is controlled to be powered off.

2. The method for processing faulty memory according to claim 1, characterized in that: When it is determined that the error type of the faulty memory is the downtime error, controlling the faulty memory to power off includes: When it is determined that the error type of the faulty memory is the downtime error, setting a preset flag bit to a first state, the flag bit being set in a structure of a message for reporting to a baseboard management controller, and the flag bit being in a second state when the error type of the faulty memory is the non-downtime error; triggering the baseboard management controller to report the fault information of the fault memory to the complex programmable logic device and control the fault memory to be powered off when detecting that the state of the flag bit is the first state; triggering the baseboard management controller to report the fault information of the fault memory to the complex programmable logic device when detecting that the state of the flag bit is the second state; The complex programmable logic device is triggered to control the fault memory to power off.

3. The method for processing faulty memory according to claim 2, characterized in that: The complex programmable logic device is connected to the power-on and power-off control pins of the power management integrated circuit through a preset output pin, and the power management integrated circuit is connected to the power supply end of the fault memory through the power-on and power-off control pins to supply power to the fault memory; Triggering the complex programmable logic device to control the fault memory to power off, including: The complex programmable logic device is triggered to set the power-on and power-off control pins of the power management integrated circuit to a power-off state through the preset output pin, so that the power management integrated circuit stops supplying power to the faulty memory.

4. The method for processing faulty memory according to claim 1, characterized in that: The fault memory management method further includes: When receiving interrupt signals corresponding to more than one faulty memory, storing the interrupt signals in an interrupt queue; Recording a timestamp of receiving each interrupt signal, and sorting all the interrupt signals according to the timestamp; For each of the interrupt signals in sequence, the step of collecting the fault information of the fault memory is performed.

5. The method for processing faulty memory according to claim 1, characterized in that: The uncorrectable errors include catastrophic errors, fatal errors and recoverable errors, and the recoverable errors include uncorrectable non-fatal array errors, single-bit read-after-write errors, and single-bit read-after-read errors; The downtime errors include the single-bit read-after-read errors, the catastrophic errors and the fatal errors; the non-downtime errors include uncorrectable non-fatal array errors and single-bit read-after-write errors.

6. The method for processing faulty memory according to any one of claims 1 to 5, characterized in that: Also includes: Acquire the current status indicators of the server in real time at a preset frequency, wherein the current status indicators include the CPU load indicator, memory usage indicator, disk input and output activity indicator, and network traffic indicator; Store the current status indicators within a preset time length into a temporary buffer; Upon receiving an interrupt signal corresponding to the fault memory, obtaining a current state indicator of a preset time length before the current moment stored in the temporary cache to obtain a system state before the fault occurs; Integrate the current status indicators of the preset time length and mark them with timestamps; Analyze the integrated current status indicators of the preset time length to determine the probability that each of the status indicators causes the faulty memory to occur; Constructing a prediction model based on the probability and the current state indicator of the preset time length; Using the prediction model to predict the probability of each memory in the server failing and the type of error that occurs; For a memory predicted to fail, if the error type predicted to fail is a downtime error among the uncorrectable errors and the probability of failure is greater than a preset threshold, the memory predicted to fail is powered off or warning information is reported.

7. The method for processing faulty memory according to claim 6, characterized in that: Analyzing the integrated current status indicators of the preset time length to determine the probability of each of the status indicators causing the faulty memory to occur, including: Standardizing the integrated current status indicator of the preset time length to obtain a current standard status indicator; Select key features related to fault memory from the current standard state indicators, calculate the statistics of the key features and extract the time series features of the key features; establish and optimize a learning model based on the statistics and the time series features, the learning model characterizes the corresponding relationship between the current standard state indicators and the probability of fault memory occurrence; The probability of each of the state indicators causing the faulty memory to occur is determined according to the learning model.

8. A device for processing faulty memory, characterized in that: include: Memory for storing computer programs; A processor, used to implement the steps of the method for processing faulty memory as described in any one of claims 1 to 7 when executing a computer program.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for processing a faulty memory as described in any one of claims 1 to 7 are implemented.

10. A server, characterized in that: A processing device comprising a fault memory as claimed in claim 8.

Images