Method, device and medium for capturing abnormal information when a system crashes

Abstract

The present application discloses a method, device, and medium for capturing abnormal information during system downtime, which relates to the field of computer information technology. The method includes: determining system downtime, triggering a kernel crash capture mechanism, automatically generating a kernel dump file through the kernel crash capture mechanism, obtaining the kernel dump file, determining whether the kernel crash capture mechanism has captured successfully based on the kernel dump file. If not, execute a preset capture script, retrieve the kernel crash log, retrieve the kernel stack information in the kernel crash log, and send the retrieved kernel stack information to the baseboard management controller through the command line interface. Through the baseboard management controller, record the system downtime event in the log according to the kernel stack information, and display it through the baseboard management controller interface. When the kernel stack is not captured, capture it again, thereby improving the success rate of capturing the kernel stack again and providing comprehensive and accurate data support for subsequent problem analysis.

Description

Technical Field

[0001] This application relates to the field of computer information technology, and particularly to a method, device, and medium for capturing abnormal information during system downtime. Background Art

[0002] With the rapid development of computer applications, Linux, as an open-source operating system, has continuously expanded its scope of use and application scenarios and is deeply favored by developers, enterprises, and individual users. The stability and fault tolerance of the Linux kernel generally prevent the system from experiencing irreparable failures, but the problem of Linux system crashes resulting in system downtime cannot be completely avoided.

[0003] The Baseboard Management Controller (BMC) is an independent processor and software component, usually embedded in server hardware, independent of the main processor and operating system, monitors the hardware status of the operating system (such as temperature, voltage, fan speed) through the IPMI protocol, and can also collect event logs, perform remote restarts, remote power-on and power-off, etc. However, when the system is down, the BMC and the server (OS) cannot interact.

[0004] When the Linux system crashes, the Linux system usually provides a kernel crash capture mechanism (such as the kdump mechanism) to discover and collect system crash problems tools to capture and dump kernel crash information. The BMC can understand the cause of system downtime based on the information captured by the kernel crash capture mechanism, and thus take targeted recovery measures for the Linux system. However, the kernel crash capture mechanism requires sufficient free memory to save the dump information during kernel dumping. If the system memory is insufficient, especially when the system is running at high load, it may not be able to reserve enough memory space for the kernel crash capture mechanism, resulting in capture failure and unclear system downtime reasons. Summary of the Invention

[0005] To solve the above problems, this application proposes a method for capturing abnormal information during system downtime, including:

[0006] Determine that the system is down, trigger the kernel crash capture mechanism, and automatically generate a kernel dump file through the kernel crash capture mechanism;

[0007] Obtain the kernel dump file, and determine whether the kernel crash capture mechanism captures successfully according to the kernel dump file;

[0008] If not, execute a preset capture script, retrieve the kernel crash log, retrieve the kernel stack information in the kernel crash log, and send the retrieved kernel stack information to the Baseboard Management Controller through the command line interface;

[0009] Through the baseboard management controller, according to the kernel stack information, record the system crash event in the log and display it through the baseboard management controller interface.

[0010] On the other hand, the present application also proposes a method and device for capturing abnormal information during system crashes, including:

[0011] At least one processor; and,

[0012] A memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute a method for capturing abnormal information during system crashes as described in the above example.

[0014] On the other hand, the present application also proposes a non-volatile computer storage medium storing computer-executable instructions, and the computer-executable instructions are set to: a method for capturing abnormal information during system crashes as described in the above example.

[0015] The method for capturing abnormal information during system crashes proposed by the present application can bring the following beneficial effects:

[0016] By capturing abnormal information, the specific cause leading to the system crash and the corresponding error code can be quickly located, greatly shortening the time to solve the problem, quickly responding to and solving the system crash problem, reducing the service interruption time, and reducing the impact on users.

[0017] When the kernel stack is not captured, execute a preset script and capture it again, thereby increasing the success rate of capturing the kernel stack again, ensuring that as much important information as possible is retained, and providing comprehensive and accurate data support for subsequent problem analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application, form a part of the present application, and the illustrative embodiments and descriptions thereof of the present application are used to explain the present application, and do not constitute an improper limitation to the present application. In the drawings:

[0019] Figure 1 is a schematic flow chart of a method for capturing abnormal information during system crashes in an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of a device for capturing abnormal information during system crashes in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following will clearly and completely describe the technical solutions of this application in combination with specific embodiments of this application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts belong to the scope of protection of this application.

[0022] The following will detail the technical solutions provided by each embodiment of this application in combination with the drawings.

[0023] As Figure 1 shown, an embodiment of this application provides a method for capturing abnormal information when the system crashes, including:

[0024] S101: Determine that the system has crashed, trigger the kernel crash capture mechanism, and automatically generate a kernel dump file through the kernel crash capture mechanism.

[0025] Specifically, when the computer system suddenly stops running due to some reason and is unable to continue executing normal tasks or respond to external requests, it is determined that the system has crashed, the kernel crash capture mechanism (such as the kdump mechanism) is triggered, the second kernel of the kernel crash capture mechanism is started, and the critical data in the current memory and the kernel stack information are captured through the kernel crash capture mechanism.

[0026] Furthermore, according to the configuration file of the kernel crash capture mechanism, the critical data and the kernel stack information are used to generate a kernel dump file (such as a vmcore file), the storage path is determined, and the kernel dump file is compressed and stored in the preset storage path.

[0027] Among them, the kernel crash capture mechanism is a security mechanism designed in the operating system to be automatically activated when the system kernel encounters an unrecoverable error. Through the kernel crash capture mechanism, the critical data in the current system memory is saved to the kernel dump file. The kernel dump file contains a memory snapshot of the system crash, including register status, stack trace, process information, kernel data structures, etc. The crash through the kernel dump file provides the system administrator and developers with preliminary clues about the cause of the crash, which helps to quickly locate the problem.

[0028] Among them, the kernel stack information refers to the information recorded and managed about function calls, variable states, and execution locations, which is used for debugging and analyzing the situation of the operating system or application when an exception or error occurs, including the function call chain, register and variable states, exception and interrupt handling information, stack pointer, etc.

[0029] Further, before determining that the system has crashed and triggering the kernel crash capture mechanism, obtain the configuration file of the kernel crash capture mechanism, and deploy a preset capture script in the configuration file through configuration parameters.

[0030] It should be noted that system crash problems are called crashes. The kdump mechanism is a tool provided by Linux to discover and collect crash dump files. By analyzing it, the root cause of the problem can be found and a method to solve critical errors can be sought. kdump utilizes a reliable kernel crash dump mechanism implemented by kexec. When the system crashes, it uses kexec to restart the second kernel to capture and dump the kernel crash information, collect the memory before the kernel crash, and generate a dump vmcore file, which is equivalent to a mirror of the physical memory. Therefore, it contains comprehensive and complete information, which is extremely helpful for analyzing and locating various difficult problems. By analyzing the vmcore file, the cause of the kernel crash can be diagnosed. Since the vmcore is the data in the memory at the time of the crashed system, the generated vmcore file is relatively large, and its size is related to the size of the system memory. The larger the memory, the larger the generated vmcore file. In the current mainstream server memory configurations, even after the vmcore file is compressed, it requires at least several GB in size, or even dozens or hundreds of GB.

[0031] It should be noted that although the system has been triggered to start and enter the kdump kernel, the configured dump path or file system may have insufficient space, or other abnormal situations may occur, resulting in the vmcore file not being able to be saved normally. That is, although kdump is triggered, the process of saving the dump vmcore file fails, and the kdump process will still cause the system to restart automatically. In this way, after the restart, the user may not even know whether the system has crashed.

[0032] S102: Obtain the kernel dump file, and determine whether the kernel crash capture mechanism has captured successfully according to the kernel dump file.

[0033] Specifically, call the preset storage path in the configuration file to obtain the kernel dump file therein, analyze the kernel dump file, and determine whether the kernel crash capture mechanism has captured the kernel stack information according to the analysis result.

[0034] Further, analyze the kernel dump file through a parsing tool (such as the crash tool), obtain the information stored in the kernel file, and query whether it contains the kernel stack information to determine whether the kernel crash capture mechanism has captured successfully.

[0035] It should be noted that in the embodiments of the present application, to determine whether the kernel crash capture mechanism captures successfully, it can also be determined by whether the kernel dump file exists in the preset storage path, or the integrity and size of the kernel dump file can be checked, or the system log or kdump log can be viewed to obtain information about the capture process.

[0036] S103: If not, execute a preset capture script to retrieve the kernel crash log, retrieve the kernel stack information from the kernel crash log, and send the retrieved kernel stack information to the baseboard management controller through the command line interface.

[0037] Specifically, if the kernel crash capture mechanism fails to capture, execute a preset capture script to collect information such as system logs, process information during runtime, hardware status, and kernel stack information, retrieve the kernel stack information from these information in the kernel crash log, and send the retrieved kernel stack information to the baseboard management controller through the command line interface.

[0038] More specifically, when it is determined that the kernel crash capture mechanism fails to capture and the system has not restarted, call the configuration file, execute a preset capture script, and retrieve the kernel stack information and register information from the kernel crash log.

[0039] It should be noted that the baseboard management controller (BMC) is a controller that provides intelligent management for the server motherboard based on the IPMI and Redfish technical specifications, allowing remote monitoring, management, and recovery of the server hardware. The command line interfaces are Ipmitool and Redfish commands. Ipmitool is a command line interface provided under the system, through which it can interact with the BMC or execute relevant setting instructions, such as reading the sensor data repository (SDR) to print sensor values, displaying the content in the system event log (SEL), printing the inventory information of the field replaceable unit (FRU), reading and setting LAN configuration parameters, etc., to achieve control of the remote server. RedFish is a replacement for IPMI, which updates the unified standard and has more powerful and secure functions than IPMI. Currently, the functions of RedFish and IPMI do not conflict, and many existing BMCs support both IPMI and RedFish and can be used together.

[0040] For example, if using the IPMI command line, the IPMI command to retrieve the kernel stack information and register information from the kernel crash log (such as the vmcore - dmesg.txt file) is: "ipmitool raw 0x06 0x40 0x00 0x00 0x01 0x01 \"BUG:kernel NULL pointer dereference\"".

[0041] S104: Through the baseboard management controller, record the system crash event in the log according to the kernel stack information, and display it through the baseboard management controller interface.

[0042] Specifically, use the data such as the program execution status, function call chain, register values, etc. saved in the kernel stack information as the key basis for recording the system crash event. The baseboard management controller records the relevant details of the system crash event in the log according to the kernel stack information, including the time, type, severity, identifiers of relevant hardware or software components, etc. of the event occurrence, and displays it through the baseboard management controller interface.

[0043] More specifically, according to the kernel stack information, determine multiple error codes when the system crashes and the execution logs corresponding to the error codes, and determine the execution time of the multiple error codes. Store the multiple error codes, the execution logs corresponding to the error codes, and the execution time of the multiple error codes in the log of the baseboard management controller item by item.

[0044] Furthermore, according to the preset exception level standard, determine the exception level of the system, generate an alarm message based on the exception level, and display the received kernel stack information, register information, and alarm message through the system log interface of the baseboard management controller. After sending the kernel stack information to the baseboard management controller, execute the automatic restart command to restart the system, determine the restart time of the system, and record the restart time in the baseboard management controller.

[0045] It should be noted that the baseboard management controller usually provides a user interface (UI), allowing system administrators or remote management tools to access and view system status, event logs, and configuration information through a Web browser, command-line interface (CLI), or other means. Displaying through the baseboard management controller interface enables system administrators or relevant users to promptly understand the situation of the system crash and take corresponding measures to solve the problem according to the information in the log.

[0046] By capturing exception information, the specific cause leading to the system crash and the corresponding error code can be quickly located, greatly shortening the time to solve the problem, quickly responding to and solving the system crash problem, reducing the service interruption time, and reducing the impact on users.

[0047] When the kernel stack is not captured, execute a preset script to capture it again, thereby increasing the success rate of capturing the kernel stack again, ensuring that as much important information as possible is retained, and providing comprehensive and accurate data support for subsequent problem analysis.

[0048] Such as Figure 2As shown, an embodiment of the present application also proposes a method and device for capturing abnormal information during system downtime, including:

[0049] At least one processor; and,

[0050] A memory communicatively connected to the at least one processor; wherein,

[0051] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a method for capturing abnormal information during system downtime as described in any of the above embodiments.

[0052] An embodiment of the present application also provides a non-volatile computer storage medium storing computer-executable instructions, and the computer-executable instructions are configured as: a method for capturing abnormal information during system downtime as described in any of the above embodiments.

[0053] Each embodiment in the present application is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device and the medium, since they are basically similar to the method embodiments, the description is relatively simple, and reference can be made to the corresponding parts of the method embodiments for the relevant parts.

[0054] The device and the medium provided by the embodiments of the present application correspond one-to-one with the method. Therefore, the device and the medium also have beneficial technical effects similar to those of the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device and the medium will not be elaborated here.

[0055] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code therein.

[0056] The present invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each flow and / or block in the flowchart illustrations and / or block diagrams, and combinations of flows and / or blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processors of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processors of the computer or other programmable data processing device create means for implementing the functions specified in the flow Figure 1 one or more flows and / or blocks Figure 1 or means for implementing the functions specified in a block or blocks.

[0057] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means that implement the functions specified in the flow Figure 1 one or more flows and / or blocks Figure 1 or means for implementing the functions specified in a block or blocks.

[0058] These computer program instructions may also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are performed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flow Figure 1 one or more flows and / or blocks Figure 1 or means for implementing the functions specified in a block or blocks.

[0059] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0060] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash memory. The memory is an example of computer-readable media.

[0061] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0062] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0063] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for capturing abnormal information when a system crashes, characterized in that, including: Determine that the system crashes, trigger the kernel crash capture mechanism, and automatically generate a kernel dump file through the kernel crash capture mechanism; Obtain the kernel dump file, and determine whether the kernel crash capture mechanism captures successfully according to the kernel dump file; If not, execute a preset scraping script, retrieve the kernel crash log, retrieve the kernel stack information in the kernel crash log, and send the retrieved kernel stack information to the baseboard management controller through the command line interface; Through the baseboard management controller, record the system crash event in the log according to the kernel stack information, and display it through the baseboard management controller interface.

2. The method for capturing abnormal information when the system crashes according to claim 1, characterized in that The automatically generating a kernel dump file through the kernel crash capture mechanism specifically includes: Start the second kernel of the kernel crash capture mechanism to automatically generate a kernel dump file; Compress the kernel dump file according to the configuration file of the kernel crash capture mechanism, and store it in a preset storage path.

3. A method for capturing abnormal information when a system crashes, as described in claim 1, wherein The determining whether the kernel crash capture mechanism captures successfully according to the kernel dump file specifically includes: Analyze the kernel dump file through a parsing tool, and determine whether the kernel dump file contains kernel stack information to determine whether the kernel crash capture mechanism captures successfully.

4. A method for capturing abnormal information when a system crashes, as described in claim 1, characterized in that, Before the determining that the system crashes and triggering the kernel crash capture mechanism, the method further includes: Obtain the configuration file of the kernel crash capture mechanism; Deploy a preset scraping script in the configuration file through configuration parameters.

5. A method for capturing abnormal information when a system crashes, as described in claim 4, wherein The executing the preset scraping script, retrieving the kernel crash log, and retrieving the kernel stack information in the kernel crash log specifically includes: Determine that the kernel crash capture mechanism fails to capture and the system has not restarted; Call the configuration file, execute the preset scraping script, and retrieve the kernel stack information and register information in the kernel crash log.

6. A method for capturing abnormal information when a system crashes, as described in claim 5, characterized in that The displaying through the baseboard management controller interface specifically includes: Determine the exception level of the system according to a preset exception level standard, and generate an alarm message based on the exception level; Display the received kernel stack information, register information, and alarm message through the system log interface of the baseboard management controller.

7. A method for capturing abnormal information when a system crashes, as claimed in claim 6, wherein The displaying the received kernel stack information, register information, and alarm message specifically includes: Execute an automatic restart command to restart the system; Determine the restart time of the system, and record the restart time in the baseboard management controller.

8. A method for capturing abnormal information during system downtime according to claim 1, characterized in that The recording the system crash event in the log according to the kernel stack information specifically includes: According to the kernel stack information, determine multiple error codes when the system crashes and the execution logs corresponding to the error codes, and determine the execution times of the multiple error codes; Store the multiple error codes, the execution logs corresponding to the error codes, and the execution times of the multiple error codes in the log of the baseboard management controller one by one.

9. A method and device for capturing abnormal information when a system crashes, characterized in that, including: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute, such as: Determine a system downtime, trigger a kernel crash capture mechanism, and automatically generate a kernel dump file through the kernel crash capture mechanism; Obtain the kernel dump file, and determine whether the kernel crash capture mechanism captures successfully according to the kernel dump file; If not, execute a preset scraping script, retrieve the kernel crash log, retrieve kernel stack information in the kernel crash log, and send the retrieved kernel stack information to the baseboard management controller through a command-line interface; Through the baseboard management controller, record the system downtime event in a log according to the kernel stack information, and display it through the baseboard management controller interface.

10. A non-volatile computer storage medium stores computer-executable instructions, characterized in that, The computer-executable instructions are set to: Determine a system downtime, trigger a kernel crash capture mechanism, and automatically generate a kernel dump file through the kernel crash capture mechanism; Obtain the kernel dump file, and determine whether the kernel crash capture mechanism captures successfully according to the kernel dump file; If not, execute a preset scraping script, retrieve the kernel crash log, retrieve kernel stack information in the kernel crash log, and send the retrieved kernel stack information to the baseboard management controller through a command-line interface; Through the baseboard management controller, record the system downtime event in a log according to the kernel stack information, and display it through the baseboard management controller interface.

Images