Server downtime automatic processing method and device, data processing unit and medium
By installing a data processing unit on the server, automatically identifying and handling downtime, the test progress and efficiency problems caused by the probabilistic downtime of the server are solved, unattended automated processing is achieved, and labor costs and project risks are reduced.
Patent Information
- Application Number
- CN202411045055.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-29
AI Technical Summary
The server may experience a probabilistic downtime during the testing process, resulting in the termination of the test task, affecting the progress and efficiency, and cannot be processed in time without being on duty, increasing labor costs and the risks of project development.
By installing a data processing unit on the server, receiving the downtime indication signal sent by the cloud server, identifying the current status and collecting fault log information, and automatically transmitting it to the cloud server, realizing automatic processing without participation.
Even when unattended, the downtime problem can be dealt with in a timely manner, avoiding the impact of test progress and efficiency, reducing labor costs, reducing test interruptions and project development delays caused by downtime, especially in multiple projects and multiple units. Significantly reduce losses.
Smart Images

Figure CN119046038A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method, device, data processing unit and medium for automatically processing server downtime. Background Art
[0002] With the development of automated testing technology, automated testing is becoming an indispensable tool in the server development and testing process. However, due to some unknown design defects in the server in the early stage of research and development, the test prototype may experience probabilistic downtime during stability or long-term stress testing, resulting in the termination of the automated testing task of the test prototype. If there is no on-duty personnel to handle it in time, it may greatly affect the test progress and efficiency.
[0003] At present, a certain amount of manpower is needed to deal with probabilistic downtime. If it involves reproduction, it is necessary to pay attention to the operating status of the machine many times, which greatly increases the manpower cost of testing. In addition, since automated testing is performed 24 hours a day, if the problem occurs at night or on non-working days, engineers cannot discover and handle it on site in time, and the prototype will be idle, resulting in the risk of obstruction and delay in problem solving and project development progress. Especially when problems occur on multiple test prototypes in multiple projects, the cumulative losses are also very considerable. Summary of the invention
[0004] In view of this, the present invention provides a method, device, data processing unit and medium for automatically processing server downtime to solve the problem of probabilistic server downtime, which requires a large amount of manpower to handle and has the risks of problem solving timeliness and project development progress being hindered and delayed.
[0005] In a first aspect, the present invention provides a method for automatically processing a server downtime, the method being executed by a data processing unit installed on a server to be tested, the server to be tested having a power supply function in a soft shutdown state, the method comprising:
[0006] Receiving an indication signal of a server crash to be tested sent by a cloud server;
[0007] According to the indication signal, identify the current state of the server to be tested;
[0008] According to the current state of the server to be tested, the fault log information corresponding to the current state is collected;
[0009] Transmit fault log information to the cloud server.
[0010] The present invention provides a method for automatically processing server downtime, which has the following advantages:
[0011] After receiving the signal indicating the downtime of the server to be tested sent by the cloud server, the data processing unit will identify the current state of the server to be tested according to the indication, and then collect the fault log information corresponding to the current state of the server to be tested according to the current state of the server to be tested, and then send the fault log information to the cloud server. The entire process is performed by the data processing unit on its own without human intervention. Therefore, even in the absence of on-duty personnel, it will not affect the progress and efficiency of the test. It will not increase labor costs. Even if the automated test is performed uninterruptedly for 24 hours, if the server to be tested is down when the staff is absent, the data processing unit can also handle it in time, thereby avoiding the idleness of the prototype, the timeliness of problem solving and the development progress of the project being blocked and delayed. The effect of this method is particularly obvious when problems occur in multiple test prototypes of multiple projects, and it can avoid the loss caused by the failure to solve the problem in time as much as possible.
[0012] In an optional implementation manner, identifying the current state of the server to be tested according to the indication signal specifically includes:
[0013] Obtain log information corresponding to the baseboard management controller and a downtime screenshot corresponding to the downtime moment of the server to be tested through the internal serial port of the data processing unit;
[0014] Determine the current status of the server based on the log information and downtime screenshots corresponding to the baseboard management controller.
[0015] Specifically, by obtaining the log information and downtime screenshots of the baseboard management controller through the internal serial port of the data processing unit, the current state of the server can be quickly determined, reducing the time and workload of manual troubleshooting, improving test efficiency, and avoiding the termination of test tasks and progress delays caused by the downtime of the test prototype. The method can automatically obtain relevant information and determine the server status, reduce dependence on manpower, and reduce the manpower cost of testing, especially when problems occur in multiple test prototypes of multiple projects, which can significantly reduce manpower investment. Even at night or on non-working days, the method can automatically monitor the server status and discover problems in time, avoiding the idleness of prototypes and delays in problem solving due to the inability of engineers to handle them on site in time, and reducing the risk of obstruction and delay in project development progress. The obtained log information and downtime screenshots provide a detailed basis for determining the server status, which helps to more accurately analyze the cause of the problem, improve the ability and efficiency of problem solving, and thus promote the smooth progress of the server development and testing process.
[0016] In an optional implementation manner, the current state of the server includes a state of staying in an operating system interface, a state of staying in a power-on self-test interface, or a black screen state.
[0017] In an optional implementation, when the current state of the server to be tested is a state of staying in the operating system interface, according to the current state of the server to be tested, the fault log information corresponding to the current state is collected, including:
[0018] The system log information, downtime log information and component log information of the operating system of the server to be tested are collected, wherein the system log information, downtime log information and component log information constitute the fault log information corresponding to the current state.
[0019] Specifically, when the server to be tested is in the state of staying in the operating system interface, the system log information, downtime log information and component log information of the operating system can be collected. The comprehensive collection of this information can understand the operating status of the server before the downtime, the error information that occurred, etc., which helps to find out the cause of the downtime. More targeted fault diagnosis can be carried out to improve the efficiency and accuracy of troubleshooting. By analyzing the cause of the downtime, potential problems in the system can be discovered, and corresponding measures can be taken to improve the stability and reliability of the system and avoid the recurrence of similar problems. Effective fault diagnosis and resolution can also reduce the idle time of the server, improve the utilization rate of server resources, and avoid resource waste caused by faults. It also helps to meet compliance requirements and improve the transparency of system management.
[0020] In an optional implementation, when the current state of the server to be tested is a state of staying in the power-on self-test interface or a black screen state, according to the current state of the server to be tested, the fault log information corresponding to the current state is collected, including:
[0021] Control the power off of the server to be tested, and automatically switch to the super capacitor for power supply, completing the power off and restart operation of the server to be tested;
[0022] After the server to be tested restarts, enter the basic input and output system operation interface and open the basic input and output system serial port;
[0023] Adjust the serial port log mode of the basic input and output system to normal mode;
[0024] Open the serial port and transmit the serial port through the network, and run the test item program until the test item program reproduces the situation where the server to be tested crashes again;
[0025] Collect log information through the basic input and output system serial port and the serial port through the network transmission serial port respectively;
[0026] The collected log information of the basic input / output system serial port and the serial port transmitted through the network are respectively transmitted to the cloud server, wherein the log information respectively collected through the basic input / output system serial port and the serial port transmitted through the network constitutes the fault log information corresponding to the current state.
[0027] Specifically, when the server is in the state of the power-on self-test interface or the black screen state, through a series of automated operations, including powering off the cloud server, switching to supercapacitor power supply, restarting the server, adjusting the serial port log mode, running the test item program, etc., the downtime situation can be quickly reproduced and relevant log information can be collected. It helps to diagnose and solve problems more efficiently and reduce the time and cost of troubleshooting the cause of downtime. Collecting the log information of the basic input and output system serial port and the serial port transmitted through the network can provide more comprehensive fault information, help to more accurately analyze the cause of the server problem and improve the accuracy of fault diagnosis. The whole process realizes automated operation, reduces the dependence on manual intervention, and reduces the demand for manpower. Especially in some non-working hours or complex fault situations, it can avoid delays in problem solving caused by insufficient manpower or inability to respond in time. By quickly solving the fault of the server in a special state, the continuity of the test can be guaranteed, the test interruption and delay caused by the fault can be avoided, and the test task can be completed on time. According to the collected fault log information, the server can be optimized and improved in a targeted manner to improve the performance and stability of the server and reduce the recurrence of similar problems. Timely discover and resolve server failures in various states, enhance the reliability of the entire system, and improve the stability and reliability of the server in actual applications.
[0028] In an optional implementation, after the fault log information is transmitted to the cloud server, the method further includes:
[0029] Reset the server to be tested and proceed to the next test item.
[0030] Specifically, after the fault log information is transmitted to the cloud server, the server to be tested is immediately reset and the next test item is entered, which can minimize the test interruption time caused by the downtime of the server to be tested, ensure the continuity of the test work, and improve the test efficiency. This method of quickly handling faults and continuing testing can make the test process more compact and efficient, reduce unnecessary waiting and delays, and improve the fluency of the entire test process.
[0031] In an optional implementation, when running the test item program to reproduce the situation where the server to be tested crashes fails, the server to be tested is reset and the next test item is entered.
[0032] Specifically, when the attempt to reproduce the downtime situation fails, it means that the problem cannot be reproduced. In this case, the problem is defined as a problem with a low probability. Instead of making too many useless attempts, the server is reset in time and the next test item is entered, which avoids wasting time and improves the overall test efficiency. By entering the next test item in time, the various functions and performance of the server can be tested more comprehensively, reducing the possibility of missing important test content. Excessive consumption of resources on test items that cannot reproduce the downtime situation is avoided, so that resources can be more reasonably allocated to other test items, improving the efficiency of resource utilization. After entering the next test item, it is possible to discover problems in other aspects of the server, providing more information and basis for the overall optimization and improvement of the server.
[0033] In an optional implementation, the log information corresponding to the baseboard management controller includes: a system operation status record and a detection result of the server, the system operation status record includes one or more of power supply status, temperature monitoring data, fan speed, unified extensible interface boot status, and traditional BIOS boot status, and the detection result of the server includes one or more of data indicating the health of the memory, a hard disk self-detection analysis report, sensor data inside the baseboard management controller, and power management change data; the downtime screenshot is used to present one or more of error information, warning prompt information, mark information indicating image abnormality, and freeze state characteristics of the system interface on the server screen;
[0034] Determine the current status of the server based on the log information and downtime screenshots corresponding to the baseboard management controller, including:
[0035] Identify preset information features and / or preset behavior patterns of the server based on the log information;
[0036] Determine the cause of server downtime based on preset information features and downtime screenshots;
[0037] and / or, determine the cause of the server downtime based on the preset behavior patterns and downtime screenshots;
[0038] Determine the current status of the server based on the reason for the server downtime.
[0039] Specifically, through log information containing system operation status records and server detection results, it is possible to understand the operation of the server from multiple aspects, assist in accurately locating possible hardware or software problems with the server, thereby more comprehensively diagnosing possible problems with the server and improving the accuracy of troubleshooting. The downtime screenshot can present a variety of error messages on the server screen to help quickly identify the cause of the server downtime. Combined with the preset information features and preset behavior patterns in the log information, the current state of the server can be determined more accurately, thereby improving the efficiency of troubleshooting, reducing server downtime, and reducing the impact on the business.
[0040] In a second aspect, the present invention provides a device for automatically processing server downtime, the device comprising:
[0041] A receiving module, used for receiving an indication signal of a server crash to be tested, sent by a cloud server;
[0042] An identification module, used to identify the current state of the server to be tested according to the indication signal;
[0043] The acquisition module is used to acquire the fault log information corresponding to the current state of the server to be tested;
[0044] The sending module is used to transmit the fault log information to the cloud server.
[0045] The present invention provides an automatic server downtime processing device, which has the following advantages:
[0046] After receiving the signal indicating the downtime of the server to be tested sent by the cloud server, the data processing unit will identify the current state of the server to be tested according to the indication, and then collect the fault log information corresponding to the current state of the server to be tested according to the current state of the server to be tested, and then send the fault log information to the cloud server. The entire process is performed by the data processing unit on its own without human intervention. Therefore, even in the absence of on-duty personnel, it will not affect the progress and efficiency of the test. It will not increase labor costs. Even if the automated test is performed uninterruptedly for 24 hours, if the server to be tested is down when the staff is absent, the data processing unit can also handle it in time, thereby avoiding the idleness of the prototype, the timeliness of problem solving and the development progress of the project being blocked and delayed. The effect of this method is particularly obvious when problems occur in multiple test prototypes of multiple projects, and it can avoid the loss caused by the failure to solve the problem in time as much as possible.
[0047] In a third aspect, the present invention provides a data processing unit, which is installed on a server. The data processing unit includes: a memory and a processor, which are communicatively connected to each other, and the memory stores computer instructions. The processor executes the server downtime automatic processing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.
[0048] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a data processing unit to execute the method for automatically handling server downtime according to the first aspect or any corresponding embodiment thereof.
[0049] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a data processing unit to execute the method for automatically handling server downtime according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0051] Figure 1 It is a flow chart of a method for automatically handling server downtime provided by an embodiment of the present invention;
[0052] Figure 2 It is a flow chart of another method for automatically handling server downtime provided by an embodiment of the present invention;
[0053] Figure 3 ] is a structural relationship diagram between the cloud server, DPU and the server to be tested provided by the present invention;
[0054] Figure 4 It is a schematic diagram of the structure of the DPU+supercapacitor provided by the present invention;
[0055] Figure 5 This is a simplified flowchart of the DPU processing downtime problem provided by the present invention;
[0056] Figure 6 This is a structural block diagram of a server downtime automatic processing device provided by an embodiment of the present invention;
[0057] Figure 7 It is a hardware structure diagram of a data processing unit provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0059] With the development of automated testing technology, automated testing is becoming an indispensable tool in the server development and testing process.
[0060] However, due to some unknown design defects in the server in the early stage of research and development, the test prototype may experience probabilistic downtime during stability or long-term stress testing, resulting in the termination of the test prototype's automated testing task. If there is no on-duty personnel to handle it in a timely manner, it may greatly affect the test progress and efficiency, and the advantages of automation cannot be fully utilized.
[0061] The current automation technology is basically based on testing under normal server conditions. If a downtime problem occurs, the remote execution machine is helpless and can only terminate the task forcibly. The test engineer has to handle it on-site and manually collect the logs at that time. Since the server downtime has already occurred, most of the collected logs have no reference value. It is necessary to reproduce the problem to see where the error occurs on the server, so as to locate the cause and formulate relevant solutions.
[0062] At present, the problem reproduction method requires a certain amount of manpower to handle. If reproduction is involved, it is necessary to pay attention to the running status of the machine many times, which greatly increases the manpower cost of testing. In addition, since automated testing is performed 24 hours a day, if the problem occurs at night or on non-working days, engineers cannot discover and handle it on site in time, and the prototype will be idle, resulting in the risk of blocking and delaying the problem solving timeliness and project development progress. Especially when problems occur in multiple test prototypes of multiple projects, the cumulative losses are also very considerable.
[0063] To solve the above problems, an embodiment of the present invention provides an embodiment of automatic processing of server crashes. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system (data processing unit) including a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0064] In this embodiment, a method for automatically handling server downtime is provided, which can be used for the above-mentioned terminal devices, such as mobile phones, tablet computers, etc. Figure 1 FIG. 1 is a flow chart of a method for automatically processing a server downtime provided by an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:
[0065] Step S101, receiving an indication signal of a server crash to be tested sent by a cloud server.
[0066] Specifically, the server to be tested has a soft off (Soft Off, S5 for short). The S5 state is a power state. When the server is in the S5 state, the system is in a shutdown state, but there is still some power supply to maintain some basic functions, such as the remote wake-up (Wake-on-LAN) function, the preservation of the basic input / output system (BIOS for short) settings, etc. The S5 power supply function is mainly to be able to respond to some specific operations when the server is completely shut down, such as remotely waking up the server through the network, or keeping some basic configuration information of the server from being lost. In the S5 state, most of the server's hardware devices have stopped working to save energy, but a small amount of power is still supplied to necessary components.
[0067] In the embodiment of the present application, the S5 power supply function of the server is utilized, and the data processing unit (DPU) can operate normally except when the server is powered off; since the DPU has independent computing and storage modules, it can be used as a micro server.
[0068] Therefore, when the server to be tested is in a soft shutdown state, the cloud server will send a downtime indication signal to the DPU to instruct the DPU to perform corresponding work according to the indication signal.
[0069] In an optional example, the DPU is installed on a PCIe slot of a server that supports the S5 power supply function, and the network port of the DPU and the management port of the server to be tested are connected to the same network. The DPU communicates with a cloud server, and an automated execution script is configured in the cloud server to complete the test of the server to be tested.
[0070] The cloud server remotely controls the machine under test through a network connection, sends test instructions and receives test results. The automated execution machine monitors the operating status of the machine under test. When certain parameters or behaviors of the machine under test do not meet normal expectations, the cloud server can detect and identify this abnormality. For example, when a server goes down, the server's management port may also be inaccessible, and the cloud service desk cannot further analyze or control the server status.
[0071] Once an exception is found, in order to avoid problems or errors caused by the continued execution of the automation script, the cloud server will pause the running automation script process and wait until the problem is solved before continuing to execute it. At the same time, the DPU will be awakened to start the downtime problem handling program.
[0072] Step S102: identifying the current state of the server to be tested according to the indication signal.
[0073] Specifically, when the DPU receives a downtime indication sent by the cloud server, it will preferentially communicate with the server to be tested through the DPU internal interface to determine the current status of the server to be tested.
[0074] In an optional example, the current state of the server includes a state of staying in an operating system (Operating System, referred to as OS) interface, a state of staying in a power-on self-test (Power-On Self-Test, referred to as POST) interface, or a black screen state.
[0075] Step S103: According to the current state of the server to be tested, the fault log information corresponding to the current state is collected.
[0076] Specifically, the OS interface refers to the user interface of the operating system. When the server is stuck on the OS interface, it means that the server's operating system has been started and is running, and users can perform various operations on this interface, but the server is down at this time. At this time, some fault log information related to OS operations needs to be collected.
[0077] Alternatively, if the server is currently in the POST interface state or a black screen state, it means that the server has not yet completed the corresponding operation of the startup phase. Therefore, it is necessary to collect some fault log information corresponding to the startup phase.
[0078] Step S104, transmitting the fault log information to the cloud server.
[0079] Specifically, the fault log information is transmitted to the cloud server so that the cloud server can obtain the log information before and after the downtime in a timely manner and analyze the cause of the downtime in a timely manner according to the log information.
[0080] The automatic processing method for server downtime provided in this embodiment is that after receiving the signal indicating the downtime of the server to be tested sent by the cloud server, the data processing unit will identify the current state of the server to be tested according to the indication, and then collect the fault log information corresponding to the current state according to the current state of the server to be tested, and then send the fault log information to the cloud server. The whole process is performed by the data processing unit by itself without human participation. Therefore, even in the absence of on-duty personnel, it will not affect the progress and efficiency of the test. It will not increase the labor cost. Even if the automated test is performed uninterruptedly within 24 hours, if the server to be tested is down when the staff is not present, the data processing unit can also handle it in time, thereby avoiding the idleness of the prototype, the timeliness of problem solving and the development progress of the project being blocked and delayed. The effect of this method is particularly obvious when problems occur in multiple test prototypes of multiple projects, and the loss caused by the failure to solve the problem in time can be avoided as much as possible.
[0081] In this embodiment, a method for automatically handling server downtime is provided, which can be used in the above-mentioned mobile terminals, such as mobile phones, tablet computers, etc. Figure 2 FIG. 1 is a flow chart of another method for automatically processing server downtime provided by an embodiment of the present invention. Figure 2 As shown, according to the indication signal, identifying the current state of the server to be tested includes the following steps:
[0082] Step S201 , obtaining log information corresponding to a baseboard management controller and a downtime screenshot corresponding to the downtime moment of the server to be tested through an internal serial port of a data processing unit.
[0083] Step S202: determining the current state of the server according to the log information and the downtime screenshot corresponding to the baseboard management controller.
[0084] Specifically, by analyzing the BMC log information and the crash screenshot, we can understand the server's operating status before the crash, and based on the crash screenshot, we can determine the situation at the time of the crash. Based on the server's operating status before the crash and the specific situation at the time of the crash, we can analyze the error information that appears and find out the cause of the crash.
[0085] The log information corresponding to the baseboard management controller includes: system operation status records and server detection results, the system operation status records include one or more of power supply status, temperature monitoring data, fan speed, unified extensible interface boot status, and traditional BIOS boot status, and the server detection results include one or more of data indicating the health of memory, hard disk self-detection analysis report, sensor data inside the baseboard management controller, and power management change data; downtime screenshots are used to present one or more of error information, warning prompt information, mark information indicating image abnormality, and freeze state characteristics of the system interface on the server screen;
[0086] Determine the current state of the server according to the log information and downtime screenshots corresponding to the baseboard management controller, including the following method steps:
[0087] Step a1: Identify preset information features and / or preset behavior patterns of the server based on log information.
[0088] Step a2: Determine the cause of the server downtime based on the preset information features and downtime screenshots;
[0089] And / or, determine the cause of the server downtime based on preset behavior patterns and downtime screenshots.
[0090] Step a3: Determine the current state of the server based on the reason for the server downtime.
[0091] Specifically, when obtaining the log information corresponding to the baseboard management controller, the data processing unit will use the protocol parsing technology to deeply analyze the data transmitted by the serial port. These log information not only contains the BMC's regular system operation status records, such as power status, temperature monitoring, fan speed, unified extensible firmware interface boot status (UEFI), traditional BIOS (Legacy BIOS) boot status and other key parameters of real-time data, but also covers the BMC's detailed detection results of the server hardware, such as the health of the memory, the SMART information of the hard disk, the sensor data inside the BMC (such as the precise value of the sensor data), power management change data (such as microsecond changes in power management), etc. Among them, SMART (Self-Monitoring, Analysis and Reporting Technology) is a hard disk self-monitoring, analysis and reporting technology. SMART information contains various operating parameters and status information of the hard disk, such as:
[0092] 1. Read and write error rate: records the error rate when the hard disk reads and writes data.
[0093] 2. Seek error rate: reflects the situation where errors occur when the hard disk seeks tracks.
[0094] 3. Start / Stop Count: The number of times the hard disk is started and stopped.
[0095] 4. Power-on time: The accumulated power-on operation time of the hard disk.
[0096] 5. Temperature: The working temperature of the hard disk.
[0097] 6. Remaining life prediction: Estimate the remaining service life of the hard disk based on the usage and various parameters of the hard disk.
[0098] By analyzing this rich information, we can fully understand the hardware operation status of the server, providing a solid foundation for accurately judging the current status of the server.
[0099] When obtaining the downtime screenshots corresponding to the time when the server to be tested crashed, the DPU will enable high-resolution image capture technology. This technology can quickly capture the image information on the screen at the moment of the server crash and store it in a lossless format. These downtime screenshots are used to present one or more of the error messages, warning prompts, mark information indicating image abnormalities, and frozen state characteristics of the system interface on the server screen. Through careful analysis of these downtime screenshots, you can intuitively understand the specific performance of the server at the time of the crash, providing valuable visual clues for further exploring the cause of the crash.
[0100] Subsequently, statistical analysis is performed on the data in the log information to identify possible abnormal trends and patterns. These are the preset information features and / or preset behavior patterns of the server mentioned above. At the same time, combined with the visual information in the downtime screenshot, such as the error message, warning message, mark information indicating image abnormality, and one or more of the frozen state features of the system interface presented on the server screen, the possible cause of the downtime is further inferred. For example, if the log information shows that the temperature of a hardware component continues to rise, and an error message related to the hardware appears in the downtime screenshot, then it can be preliminarily determined that overheating of the hardware component may be one of the causes of the downtime.
[0101] Finally, the current state of the server is matched according to the reason for the server downtime. The log information and downtime screenshots of the baseboard management controller are obtained through the internal serial port of the data processing unit, which can quickly determine the current state of the server, reduce the time and workload of manual troubleshooting, improve test efficiency, and avoid the termination of test tasks and progress delays caused by the downtime of the test prototype. The method can automatically obtain relevant information and determine the server status, reduce the dependence on manpower, and reduce the test manpower cost, especially when problems occur in multiple test prototypes of multiple projects, which can significantly reduce manpower investment. Even at night or on non-working days, the method can automatically monitor the server status and find problems in time, avoiding the idleness of prototypes and the delay of problem solving time due to the inability of engineers to handle on-site in time, and reducing the risk of obstruction and delay in project development progress. The obtained log information and downtime screenshots provide a detailed basis for determining the server status, which helps to analyze the cause of the problem more accurately, improve the ability and efficiency of problem solving, and thus promote the smooth progress of the server development and testing process.
[0102] In an optional example, when the current state of the server to be tested is a state of staying in the operating system interface, according to the current state of the server to be tested, collecting fault log information corresponding to the current state includes the following method steps:
[0103] The system log information, downtime log information and component log information of the operating system of the server to be tested are collected, wherein the system log information, downtime log information and component log information constitute the fault log information corresponding to the current state.
[0104] In a specific example, the system log under the OS usually includes the following:
[0105] 1. System event log: records system startup, shutdown, errors, warnings and other event information.
[0106] 2. Application log: records the application's running status, error information, etc.
[0107] 3. Security log: records events related to system security, such as login attempts, permission changes, etc.
[0108] 4. Kernel log: records the operation information and errors of the operating system kernel.
[0109] By analyzing the system logs, downtime log information, and component logs under the OS, we can understand the server's operating status before the downtime, the error information that occurred, etc., which helps to find out the cause of the downtime. These log information can provide detailed clues for technicians to help them develop effective solutions to restore the normal operation of the server. By analyzing the cause of the downtime, we can find potential problems in the system and take corresponding measures to improve the stability and reliability of the system to avoid similar problems from happening again. When a server fails, collecting relevant logs is an important step in troubleshooting and auditing, which helps meet compliance requirements and improve the transparency of system management.
[0110] In another optional example, when the current state of the server to be tested is a state of staying in the power-on self-test interface or a black screen state, according to the current state of the server to be tested, collecting fault log information corresponding to the current state includes the following steps:
[0111] Step b1, controlling the server to be tested to power off, and automatically switching to the super capacitor for power supply, to complete the power-off and restart operation of the server to be tested.
[0112] Step b2: After the server to be tested is restarted, the basic input and output system operation interface is entered, and the basic input and output system serial port is opened.
[0113] Step b3, adjust the serial port log mode of the basic input and output system to the normal mode.
[0114] Step b4, open the serial port and transmit the serial port through the network, and run the test item program until the test item program reproduces the situation that the server to be tested crashes again.
[0115] Step b5, collecting log information through the basic input and output system serial port and the serial port through the network transmission serial port respectively.
[0116] Step b6, transmitting the collected log information of the basic input and output system serial port and the serial port to the cloud server through the network transmission serial port.
[0117] The log information collected through the basic input and output system serial port and the serial port through the network transmission serial port constitutes the fault log information corresponding to the current state.
[0118] Specifically, the POST interface is the stage of hardware self-check when the computer starts. If the server stays on this interface or is in a black screen state, it means that the system may have problems in the hardware self-check or earlier stages. At this time, it is not possible to collect related BIOS logs as conveniently as in the operating system interface because the system has not yet fully started and entered a normal operating state.
[0119] Therefore, the server can be restarted by controlling the power off of the server through the DPU and using the supercapacitor to ensure the normal operation of some key components. In this way, you can try to restart the system to see whether it can enter the normal working state or the downtime problem occurs again.
[0120] After the server restarts, enter the BIOS interface. Open the BIOS serial port and adjust the serial port log to normal mode, and open the serial port over the network (Serial Over LAN, SOL) serial port. The purpose of these settings is to be able to collect relevant information during the system startup process through the serial port, so as to obtain more diagnostic data when a downtime problem occurs.
[0121] Execute test items to simulate operations or scenarios that may cause downtime, in an attempt to reproduce the downtime problem.
[0122] During the test, serial port logs are collected through the previously set serial ports and SOL serial ports. These logs can record various information during the system startup process, including hardware self-test, initialization and other stages. When the downtime occurs again, the serial port log can provide detailed information about the system status when the downtime occurred, which helps to analyze the cause of the downtime.
[0123] The collected serial port logs are sent back to the cloud server for further analysis and diagnosis.
[0124] In the above method, when the server is in the state of the power-on self-test interface or the black screen state, through a series of automated operations, including powering off the cloud server, switching the supercapacitor power supply, restarting the server, adjusting the serial port log mode, running the test item program, etc., the downtime situation can be quickly reproduced and relevant log information can be collected. It helps to diagnose and solve problems more efficiently and reduce the time and cost of troubleshooting the cause of downtime. Collecting the log information of the basic input and output system serial port and the serial port transmitted through the network can provide more comprehensive fault information, help to more accurately analyze the cause of the server problem, and improve the accuracy of fault diagnosis. The whole process realizes automated operation, reduces the dependence on manual intervention, and reduces manpower requirements. Especially in some non-working hours or complex fault situations, it can avoid delays in problem solving caused by insufficient manpower or inability to respond in time. By quickly solving the fault of the server in a special state, the continuity of the test can be guaranteed, the test interruption and delay caused by the fault can be avoided, and the test task can be completed on time. According to the collected fault log information, the server can be optimized and improved in a targeted manner to improve the performance and stability of the server and reduce the recurrence of similar problems. Timely discover and resolve server failures in various states, enhance the reliability of the entire system, and improve the stability and reliability of the server in actual applications.
[0125] In an optional example, in the aforementioned method steps, the collected fault log information is transmitted to the cloud server for detection and analysis. In fact, for some simple or common fault problems, automatic repair attempts can also be completed locally. For example, the fault log information is analyzed locally to obtain the analysis results. And some common fault repair operations are performed based on the analysis results. For example, reconfigure certain settings, update drivers, or perform repair operations on system files. If the automatic repair is successful, the need for manual intervention and server downtime can be reduced. Furthermore, the lessons learned from fault diagnosis and repair can be organized into a knowledge base so that they can be quickly queried and referenced when encountering similar problems, thereby improving the speed and accuracy of problem solving. Furthermore, during the process of troubleshooting and repairing, ensure that important data in the server is properly backed up. At the same time, regularly verify the integrity and recoverability of data backups to prevent data loss when a serious failure occurs on the server.
[0126] In an optional example, after transmitting the fault log information to the cloud server, the method further includes:
[0127] Reset the server to be tested and proceed to the next test item.
[0128] Specifically, after the fault log information is transmitted to the cloud server, the server to be tested is immediately reset and the next test item is entered, which can minimize the test interruption time caused by the downtime of the server to be tested, ensure the continuity of the test work, and improve the test efficiency. This method of quickly handling faults and continuing testing can make the test process more compact and efficient, reduce unnecessary waiting and delays, and improve the fluency of the entire test process.
[0129] In an optional example, when running the test item program to reproduce the situation where the server to be tested crashes fails, the server to be tested is reset and the next test item is entered.
[0130] Specifically, when the attempt to reproduce the downtime situation fails, it means that the problem cannot be reproduced. In this case, the problem is defined as a problem with a low probability. Instead of making too many useless attempts, the server is reset in time and the next test item is entered, which avoids wasting time and improves the overall test efficiency. By entering the next test item in time, the various functions and performance of the server can be tested more comprehensively, reducing the possibility of missing important test content. Excessive consumption of resources on test items that cannot reproduce the downtime situation is avoided, so that resources can be more reasonably allocated to other test items, improving the efficiency of resource utilization. After entering the next test item, it is possible to discover problems in other aspects of the server, providing more information and basis for the overall optimization and improvement of the server.
[0131] Figure 3 : is a structural relationship diagram between the cloud server, DPU and the server to be tested provided by the present invention. When the cloud server (which can also be understood as a test platform) detects that the server to be tested is down, it sends a downtime processing command to the DPU, which is the indication information mentioned in the previous text. The DPU starts the downtime processing steps, such as the process handled by the DPU introduced in the previous text. For example, downtime-related logs are collected from the server to be tested. The DPU will also send back the progress and results of the downtime processing to the cloud server. The specific working principles between the three have been introduced in detail in the previous text, so I will not go into details here.
[0132] Figure 4 It is a schematic diagram of the structure of the DPU+supercapacitor provided by the present invention.
[0133] Specifically, the server under test supplies power to the DPU through the S5 power supply function, and a communication connection is established between the DPU and the server under test through the host interface. The supercapacitor is used to charge and power the DPU. The DPU includes a Field Programmable Gate Array (FPGA) and a System on Chip (SOC). The DPU establishes a communication connection with the cloud server through the communication and management network ports.
[0134] Figure 5 The figure shows a simple process of DPU handling downtime problem.
[0135] Figure 5 The overall workflow has been introduced in detail in the previous article, so I will not go into details here.
[0136] In this embodiment, a server downtime automatic processing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0137] This embodiment provides a server downtime automatic processing device, such as Figure 6 As shown, it includes: a receiving module 601, an identification module 602, a collection module 603, and a sending module 604.
[0138] The receiving module 601 is used to receive an indication signal of a server crash to be tested sent by a cloud server;
[0139] An identification module 602 is used to identify the current state of the server to be tested according to the indication signal;
[0140] The collection module 603 is used to collect the fault log information corresponding to the current state of the server to be tested;
[0141] The sending module 604 is used to transmit the fault log information to the cloud server.
[0142] In an optional example, the identification module 602 is specifically configured to:
[0143] Obtain log information corresponding to the baseboard management controller and a downtime screenshot corresponding to the downtime moment of the server to be tested through the internal serial port of the data processing unit;
[0144] Determine the current status of the server based on the log information and downtime screenshots corresponding to the baseboard management controller.
[0145] In an optional example, the current state of the server includes a state of staying in an operating system interface, a state of staying in a power-on self-test interface, or a black screen state.
[0146] In an optional example, when the current state of the server to be tested is a state of staying in the operating system interface, the acquisition module 603 is specifically used to:
[0147] The system log information, downtime log information and component log information of the operating system of the server to be tested are collected, wherein the system log information, downtime log information and component log information constitute the fault log information corresponding to the current state.
[0148] In an optional example, when the server to be tested is currently in a state of staying in a power-on self-test interface or a black screen state, the acquisition module 603 is specifically used to:
[0149] Control the power off of the server to be tested, and automatically switch to the super capacitor for power supply, completing the power off and restart operation of the server to be tested;
[0150] After the server to be tested restarts, enter the basic input and output system operation interface and open the basic input and output system serial port;
[0151] Adjust the serial port log mode of the basic input and output system to normal mode;
[0152] Open the serial port and transmit the serial port through the network, and run the test item program until the test item program reproduces the situation where the server to be tested crashes again;
[0153] Collect log information through the basic input and output system serial port and the serial port through the network transmission serial port respectively;
[0154] The collected log information of the basic input / output system serial port and the serial port transmitted through the network are respectively transmitted to the cloud server, wherein the log information respectively collected through the basic input / output system serial port and the serial port transmitted through the network constitutes the fault log information corresponding to the current state.
[0155] In an optional example, the device further includes: a reset module 605;
[0156] The reset module 605 is used to reset the server to be tested and enter the next test item.
[0157] In an optional example, the reset module 605 is further used to reset the server to be tested and enter the next test item when running the test item program to reproduce the situation that the server to be tested crashes fails.
[0158] In an optional example, the log information corresponding to the baseboard management controller includes: system operation status records and server detection results, the system operation status records include one or more of power supply status, temperature monitoring data, fan speed, unified extensible interface boot status, and traditional BIOS boot status, the server detection results include one or more of data indicating the health of memory, hard disk self-detection analysis report, sensor data inside the baseboard management controller, and power management change data; the downtime screenshot is used to present one or more of error information, warning prompt information, mark information indicating image abnormality, and freeze state characteristics of the system interface on the server screen;
[0159] The identification module 602 is specifically used to: identify the preset information characteristics and / or preset behavior patterns of the server according to the log information;
[0160] Determine the cause of server downtime based on preset information features and downtime screenshots;
[0161] and / or, determine the cause of the server downtime based on the preset behavior patterns and downtime screenshots;
[0162] Determine the current status of the server based on the reason for the server downtime.
[0163] The server downtime automatic processing device in this embodiment is presented in the form of a functional module, where the module refers to an application specific integrated circuit (ASIC), a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.
[0164] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.
[0165] An automatic server downtime processing device provided by an embodiment of the present invention, after receiving a signal indicating downtime of a server to be tested sent by a cloud server, a data processing unit will identify the current state of the server to be tested according to the indication, and then collect fault log information corresponding to the current state according to the current state of the server to be tested, and then send the fault log information to the cloud server. The entire process is performed by the data processing unit on its own without human participation. Therefore, even in the absence of on-duty personnel, the test progress and efficiency will not be affected. It will not increase labor costs. Even if the automated test is performed uninterruptedly within 24 hours, if the server to be tested is down when the staff is not present, the data processing unit can also handle it in time, thereby avoiding the idleness of the prototype, the problem solving timeliness and the development progress of the project being blocked and delayed. The effect of this method is particularly obvious when problems occur in multiple test prototypes of multiple projects, and the loss caused by the failure to solve the problem in time can be avoided as much as possible.
[0166] The embodiment of the present invention further provides a data processing unit having the above Figure 6 The server failure automatic processing device shown.
[0167] See also Figure 7 , Figure 7 is a structural diagram of a data processing unit provided by an optional embodiment of the present invention, the data processing unit is installed on a server, such as Figure 7 As shown, the data processing unit includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the data processing unit, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple data processing units can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.
[0168] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.
[0169] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.
[0170] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a data processing unit according to the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the data processing unit via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0171] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.
[0172] The data processing unit further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.
[0173] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the data processing unit, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.
[0174] The embodiment of the present invention also provides a computer-readable storage medium. The method provided in the above embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or is implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.
[0175] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.
[0176] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A method for automatically handling server downtime, characterized in that: The method is executed by a data processing unit installed on a server to be tested, wherein the server to be tested has a power supply function in a soft shutdown state, and the method includes: Receiving an indication signal of a server crash to be tested sent by a cloud server; According to the indication signal, identifying the current state of the server to be tested; According to the current state of the server to be tested, collecting fault log information corresponding to the current state; The fault log information is transmitted to the cloud server.
2. The method according to claim 1, characterized in that The step of identifying the current state of the server to be tested according to the indication signal specifically includes: Acquire log information corresponding to the baseboard management controller and a downtime screenshot corresponding to the downtime moment of the server to be tested through the internal serial port of the data processing unit; The current state of the server is determined according to the log information corresponding to the baseboard management controller and the downtime screenshot.
3. The method according to claim 1 or 2, characterized in that: The current state of the server includes a state of staying in an operating system interface, a state of staying in a power-on self-test interface, or a black screen state.
4. The method according to claim 3, characterized in that When the current state of the server to be tested is a state of staying in the operating system interface, collecting fault log information corresponding to the current state according to the current state of the server to be tested includes: Collect system log information, downtime log information and component log information of the operating system of the server to be tested, wherein the system log information, the downtime log information and the component log information constitute fault log information corresponding to the current state.
5. The method according to claim 3, characterized in that: When the current state of the server to be tested is a state of staying in a power-on self-test interface or a black screen state, collecting fault log information corresponding to the current state according to the current state of the server to be tested includes: Controlling the server to be tested to power off, and automatically switching the super capacitor to supply power, to complete the power-off and restart operation of the server to be tested; After the server to be tested is restarted, the basic input and output system operation interface is entered, and the basic input and output system serial port is opened; Adjusting the serial port log mode of the basic input and output system to a normal mode; Open the serial port and transmit the serial port through the network, and run the test item program until the test item program reproduces the situation that the server to be tested crashes again; Collect log information through the basic input and output system serial port and the serial port through the network transmission serial port respectively; The log information collected by the basic input / output system serial port and the serial port through the network transmission serial port are respectively transmitted to the cloud server, wherein the log information collected by the basic input / output system serial port and the serial port through the network transmission serial port respectively constitutes fault log information corresponding to the current state.
6. The method according to any one of claims 2, 4 or 5, characterized in that: After transmitting the fault log information to the cloud server, the method further includes: Reset the server to be tested and proceed to the next test item.
7. The method according to claim 2, characterized in that The log information corresponding to the baseboard management controller includes: system operation status records and server detection results, the system operation status records include one or more of power supply status, temperature monitoring data, fan speed, unified extensible interface boot status, and traditional BIOS boot status, and the server detection results include one or more of data indicating the health of memory, hard disk self-detection analysis report, sensor data inside the baseboard management controller, and power management change data; the downtime screenshot is used to present one or more of error information, warning prompt information, mark information indicating image abnormality, and freeze state characteristics of the system interface on the server screen; Determining the current state of the server according to the log information corresponding to the baseboard management controller and the downtime screenshot includes: According to the log information, identifying preset information features and / or preset behavior patterns of the server; Determine the cause of the server downtime according to the preset information features and the downtime screenshot; and / or, determining the cause of the server downtime according to the preset behavior pattern and the downtime screenshot; According to the reason why the server is down, the current state of the server is determined.
8. A server downtime automatic processing device, characterized in that: The device comprises: A receiving module, used for receiving an indication signal of a server crash to be tested, sent by a cloud server; An identification module, used to identify the current state of the server to be tested according to the indication signal; A collection module, used for collecting fault log information corresponding to the current state of the server to be tested according to the current state of the server to be tested; The sending module is used to transmit the fault log information to the cloud server.
9. A data processing unit, characterized in that: The data processing unit is installed on the server, and the data processing unit includes: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the server downtime automatic processing method according to any one of claims 1 to 7 by executing the computer instructions.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a data processing unit to execute the automatic server downtime processing method according to any one of claims 1 to 7.
Citation Information
Cited By
Downtime analysis method of server
CN120256186A
Server test method and device, storage medium and electronic device
CN120523672A
Data acquisition system and electronic equipment
CN121050925A
