Log collection method and device, equipment and storage medium

Abstract

The invention relates to the technical field of servers, and discloses a log collection method and device, equipment and a storage medium, and the method comprises the steps: collecting the index data of a server, and predicting a first target fault of the server in the future according to the index data; in other server potential faults except the first target fault, searching for a second target fault which meets a correlation degree requirement with the first target fault; and for one or more target faults in the first target fault and the second target fault, determining a log association degree between the target fault and each log of the server, and according to the log association degree, collecting the log related to the target fault in the server. The data processing efficiency can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of server technology, and in particular to a log collection method, device, equipment and storage medium. Background Art

[0002] With the development of technology, servers have become an important basic equipment. When the operation status of the server is abnormal (such as downtime, alarm), the server fault can be located by analyzing the logs in the server.

[0003] At present, in some technologies, after a certain component of a server fails, the server logs will be fully collected, and the cause of the component failure will be analyzed based on the collected logs. The problem with these technologies is that triggering full log collection after a server failure will be time-consuming, and logs cannot be obtained in time, which affects the efficiency of fault handling. Summary of the invention

[0004] In view of this, the present disclosure provides a log collection method, a log collection device, an electronic device and a computer-readable storage medium, which can collect logs in advance when a server failure occurs, thereby improving the efficiency of fault handling.

[0005] In a first aspect, the present disclosure provides a log collection method, the method comprising:

[0006] Collecting indicator data of the server, and predicting a first target failure that will occur in the server in the future based on the indicator data;

[0007] Searching for a second target fault that meets the correlation requirement with the first target fault among other potential server faults except the first target fault;

[0008] For one or more target faults among the first target fault and the second target fault, determine the log correlation between the target fault and each log of the server, and collect logs related to the target fault in the server based on the log correlation.

[0009] In a second aspect, the present disclosure provides a log collection device, the device comprising:

[0010] A data collection module, used to collect the index data of the server, and predict the first target failure that will occur in the server in the future based on the index data;

[0011] A fault finding module, used to find a second target fault that meets the correlation requirement with the first target fault among other potential server faults except the first target fault;

[0012] A log collection module is used to determine the log correlation between one or more target faults among the first target fault and the second target fault and each log of the server, and collect logs related to the target fault in the server based on the log correlation.

[0013] In a third aspect, the present disclosure provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the above method by executing the computer instructions.

[0014] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the above method.

[0015] In the technical solutions of some embodiments of the present disclosure, based on the indicator data of the server, the first target fault that may occur in the server in the future can be obtained by prediction. Based on the correlation between the faults, it can be determined that when the first target fault occurs, the second target fault that may occur can be associated. Therefore, the first target fault and the second target fault can be used as a set of target faults that may occur in the server in the future. Furthermore, according to the log correlation between each target fault and each log, the logs related to the target fault can be collected in advance, so that after the target fault occurs, the fault location can be directly based on the logs that have been collected, thereby improving the efficiency of fault handling. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 is a flowchart of a log collection method provided by some embodiments of the present disclosure;

[0018] Figure 2 is a flow chart of device log information collection provided by an embodiment of the present disclosure;

[0019] Figure 3 is a module diagram of a log collection device provided by an embodiment of the present disclosure;

[0020] Figure 4 It is a schematic diagram of the structure of an electronic device provided by some embodiments of the present disclosure. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0022] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0023] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "some embodiments" or "the embodiment" should be understood as "at least some embodiments". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0024] Herein, unless explicitly stated, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0025] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0026] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the types, scopes of use, usage scenarios, etc. of the information involved in the present disclosure should be informed to relevant users and their authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. The relevant users may include any type of right holders, such as individuals, enterprises, and groups.

[0027] For example, in response to receiving an active request from a user, a prompt message is sent to the relevant user to clearly prompt the relevant user that the operation requested to be performed will require obtaining and using the information of the relevant user, so that the relevant user can independently choose whether to provide information to software or hardware such as an electronic device, application, server or storage medium that executes the operation of the technical solution of the present disclosure based on the prompt message.

[0028] As an optional but non-limiting implementation, in response to receiving an active request from a relevant user, a prompt message is sent to the relevant user, for example, in the form of a pop-up window, in which the prompt message may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide information to the electronic device.

[0029] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0030] Faults in servers can be divided by components, such as hard disk failure, IO failure, motherboard failure, fan failure, central processing unit (CPU) failure, baseboard management controller (BMC) failure, etc. During the operation of a component, one or more faults may occur. For example, the fan speed could not be controlled on October 9, and the noise was too loud on October 20. For another example, the baseboard management controller had a network connection interruption with other devices on November 1, a data backup failure on December 15, and a server failure that could not be remotely managed on December 19.

[0031] When a component fails, information related to the failure will be recorded in one or more logs of the server. For example, when the baseboard management controller and other devices have a network connection interruption, the fault information of the failure (such as failure time, failure level, etc.) will be recorded in the log of the baseboard management controller. At the same time, the failure of the baseboard management controller may also be recorded in the logs of other devices that fail to communicate with the baseboard management controller. For example, when the baseboard management controller and the central processing unit have a network connection interruption, the failure information may also be recorded in the log of the central processing unit.

[0032] Based on the above reasons, currently, in some technologies, after a component of a server fails, all logs in the server are usually collected in full, so as to prevent the problem of missing log collection. However, there are many logs in the server. On the one hand, triggering full collection after a component of the server fails will be time-consuming, and the fault location processing cannot be carried out in time, which greatly reduces the efficiency of fault processing. On the other hand, the collected logs will take up a relatively large storage space and the cost is high.

[0033] In view of this, the present disclosure provides a log collection method, which can collect logs related to the fault in advance before the server fails. In this way, after the fault occurs, the collected logs can be directly used to locate the fault, thereby improving the efficiency of fault handling. At the same time, since only logs related to the fault are collected, the consumption of storage space can be greatly reduced, thereby reducing costs. The log collection method can be applied to a log collection device. The log collection device can be a tablet computer, a laptop computer, a desktop computer, etc. that is connected to the server for communication, or it can be a controller located in the server, etc. Figure 1 , which is a flow chart of a log collection method provided in some embodiments of the present disclosure. Figure 1 In the example, the log collection method includes the following steps:

[0034] Step S101, collecting indicator data of the server, and predicting the first target failure that will occur in the server in the future based on the indicator data.

[0035] Specifically, the index data can be divided by components, such as the index data of the central processing unit, the index data of the memory, the index data of the hard disk, etc. The index data of different components may not be exactly the same. For example, the index data of the central processing unit may include the number of cores, the usage rate, the cache size, etc. The index data of the memory may include CE (Correctable Error) and UCE (Uncorrectable Error), memory type, memory occupancy rate, etc. The index data of the hard disk may include the Smart information of the hard disk, the theoretical usable time, the used time, the read and write speed, etc. Among these index data, some of the index data may be burned into the component by the component manufacturer when the component leaves the factory, such as the theoretical usable time of the hard disk, the memory type, the number of cores of the central processing unit, etc. Some of the index data may be generated during the operation of the component. For example, the read and write speed of the hard disk, the usage rate of the central processing unit, etc. Based on technologies such as sensors and calling component interfaces, the index data of the server can be collected.

[0036] In actual applications, the indicator data that needs to be collected can be pre-configured. These indicator data can be used to predict whether various components in the server are about to fail (that is, the components have not yet failed). For example, based on the theoretical usage time, usage time, working temperature, etc. of the hard disk, the remaining usage time of the hard disk can be predicted. Based on the remaining usage time, the time point when the hard disk will fail to read or write data in the future can be predicted.

[0037] In this embodiment, the collected indicator data can be input into the fault prediction model, and the fault prediction model outputs the first target fault that will occur in the future of the server. Specifically, the fault prediction model can be obtained by fitting based on the indicator data collected when each component fails or is about to fail. The fitted fault prediction model can establish a corresponding relationship between the indicator data and the faults of each component. In this way, after the currently collected indicator data is input into the fault prediction model, the fault prediction model can determine whether each component is about to fail based on the corresponding relationship.

[0038] In some other embodiments, the fault prediction model can also be trained based on the indicator data of each component. Specifically, the indicator data of each component in the normal operation, impending failure, and failure states can be collected, and after the indicator data is labeled, the fault prediction model is trained using the labeled indicator data. In this way, the trained fault prediction model can predict whether each component is about to fail based on the indicator data of each component currently collected.

[0039] Step S102: searching for a second target fault that meets the correlation requirement with the first target fault among other potential server faults except the first target fault.

[0040] Potential server failures refer to all possible failures that may occur in the server in the future. In the daily maintenance of the server, historical failures that have occurred in multiple servers can be collected and these historical failures can be used as potential server failures.

[0041] There may be correlations between server failures, that is, when a failure A occurs, failure B may be triggered. For example, when the fan in the server fails to rotate, the heat dissipation problem in the server will cause the temperature of the central processing unit, baseboard management controller and other components to rise rapidly, which may cause hardware damage, data loss and other failures of the central processing unit, baseboard management controller and other components. For another example, when the baseboard management controller has a network connection failure, since the baseboard management controller cannot control the fan speed, it may cause the fan speed to be uncontrollable.

[0042] Based on the above description, the above-mentioned search for the second target fault that meets the correlation requirement with the first target fault can actually be to find a fault that has a higher fault correlation with the first target fault. Among them, the fault correlation refers to the probability of other faults occurring when the first target fault occurs. For example, assuming that the first target fault is fault A, other potential server faults other than the first target fault include fault B, fault C and fault D. Assuming that the correlation between fault A and fault B is 0.8, the correlation with fault C is 0.2, and the correlation with fault D is 0.1, it means that when fault A occurs, the probability of fault B occurring is 80%, the probability of fault C occurring is 20%, and the probability of fault D occurring is 10%. Therefore, it can be considered that fault B is the second target fault with a higher fault correlation with fault A.

[0043] In this embodiment, historical fault data of one or more servers may be analyzed to predetermine the fault correlation between any two faults. Thus, after obtaining the first target fault, a second target fault having a higher fault correlation with the first target fault may be found based on the predetermined fault correlation between any two faults.

[0044] Step S103 , for one or more target faults among the first target fault and the second target fault, determine the log correlation between the target fault and each log of the server, and collect logs related to the target fault in the server according to the log correlation.

[0045] Specifically, as mentioned above, since the fault information of the same fault may be distributed in multiple different logs, the log correlation can represent the probability of extracting the target fault information from each log. For example, assuming that the server includes logs L1, L2, and L3, and the log correlation of the target fault with log L1 is 0.1, the log correlation with log L2 is 0.8, and the correlation with log L3 is 0.5, it means that the probability of extracting the target fault information from log L1 is 10%, the probability of extracting the target fault information from log L2 is 80%, and the probability of extracting the target fault information from log L3 is 50%.

[0046] The above-mentioned collecting logs related to the target fault in the server according to the log correlation may refer to collecting logs with a higher log correlation with the target fault in the server. For example, among the above-mentioned logs L1, L2, and L3, logs L2 and L3 may be regarded as logs related to the target fault, and log L1 may be regarded as a log not related to the target fault.

[0047] In this embodiment, based on the log correlation, the logs related to the target fault are collected in the server, which may include:

[0048] Get the preset log correlation range;

[0049] For any server log, if the log correlation between the server log and the target fault is within the log correlation range, the server log is collected.

[0050] The range of log correlation can be configured according to actual needs, such as 0.3 to 1, and the present disclosure does not limit this.

[0051] In summary, in the technical solutions of some embodiments of the present disclosure, based on the indicator data of the server, the first target fault that may occur in the server in the future can be obtained by prediction. Based on the correlation between the faults, it can be determined that when the first target fault occurs, the second target fault that may occur can be associated. Therefore, the first target fault and the second target fault can be used as a set of target faults that may occur in the server in the future. Furthermore, according to the log correlation between each target fault and each log, the logs related to the target fault can be collected in advance, so that after the target fault occurs, the fault location can be directly based on the logs that have been collected, thereby improving the efficiency of fault handling.

[0052] In addition, compared with some technologies that collect all logs, the present disclosure only collects logs related to the target fault, which can greatly reduce the consumption of storage space and reduce costs.

[0053] In some embodiments, the fault prediction model of the present disclosure can also output a future time point when the first target fault occurs. Based on the future time point, the time point for collecting logs can be controlled. For example, assuming that the current time is October 6, 2024, and the future time point is October 9, 2024, then at 00:00 on October 9, the collection of logs related to the target fault can be triggered in the server. In this way, the waste of storage space caused by starting log collection too early can be further avoided, and the amount of data processing during fault location in the later stage can be greatly reduced.

[0054] In some embodiments, searching for a second target fault that meets the correlation requirement with the first target fault among other potential server faults except the first target fault in step S102 may include:

[0055] Inputting the first target fault into the fault correlation model, and outputting the fault correlation between the first target fault and the potential faults of each server from the fault correlation model;

[0056] According to the fault correlation, the second target fault is found from the potential faults.

[0057] Specifically, the fault correlation model may be obtained by fitting the historical fault data of one or more servers. The historical fault data may include the historical fault information of each component of the server. The historical fault information may include but is not limited to the historical fault description, the time when the historical fault occurred (i.e., the fault time), etc.

[0058] In this embodiment, historical fault data can be counted according to the dimensions of component + historical fault + occurrence time. The same component can have different faults at different time points, that is, the same component can correspond to one or more historical faults. For example, Table 1 exemplifies the storage structure of historical fault data:

[0059] Table 1 Historical fault data

[0060] Historical failures part Failure time Historical Fault A1 Component 1 2024.01.03 10:30 Historical Fault A2 Component 1 2024.03.08 11:20 Historical Fault A1 Component 1 2024.06.02 09:00 Historical Fault A1 Component 1 2024.09.01 08:02 Historical fault B1 Component 2 2024.01.03 10:32 Historical Fault B2 Component 2 2024.01.03 10:33 Historical Fault B2 Component 2 2024.03.01 21:25

[0061] Refer to Table 1. Based on the historical fault data, the fault correlation between the first historical fault occurring in the first component and the second historical fault occurring in the second component can be determined, and then the fault correlation model can be fitted based on the determined fault correlation. Specifically, the fault correlation between the first historical fault and the second historical fault can be determined based on the fault time of the first historical fault and the second historical fault in the historical fault data. If the second historical fault occurs within a period of time after the first historical fault occurs, it can be considered that the first historical fault has a higher fault correlation with the second historical fault; if the second historical fault does not occur within a period of time after the first historical fault occurs, it can be considered that the fault correlation between the first historical fault and the second historical fault is lower.

[0062] In actual historical fault data, there may be the following situations between the first historical fault and the second historical fault:

[0063] 11) After each occurrence of the first historical fault, a second historical fault will occur;

[0064] 12) After the first historical fault occurs several times, the second historical fault will occur, and after the first historical fault occurs several other times, the second historical fault will not occur;

[0065] 13) After each occurrence of a first historical fault, a second historical fault will not occur.

[0066] For the above situation 11), it means that the fault correlation between the first historical fault and the second historical fault is high; for the above situation 13), it means that the fault correlation between the first historical fault and the second historical fault is low; for the above situation 12), it means that the fault correlation between the first historical fault and the second historical fault is between situation 11) and situation 12).

[0067] According to this principle, based on historical fault data, the correlation law between any two historical faults is analyzed, and the model is fitted according to the analyzed law to obtain the fault correlation model.

[0068] After fitting the fault correlation model, any first target fault can be directly input into the fault correlation model. The fault correlation model can output the fault correlation between the first target fault and each server potential fault. For example, assuming that the first target fault is fault A1, and the server potential faults include fault B2, fault B3, and fault B4, then the output result of the fault correlation model can be as follows:

[0069] Fault A1<>Fault B2: 0.8

[0070] Fault A1<>Fault B3: 0.3

[0071] Fault A1<>Fault B4: 0.5

[0072] In the above embodiment, by constructing a fault correlation model and searching for a second target fault that meets the correlation requirement with the first target fault based on the fault correlation model, the search time and search workload can be greatly reduced.

[0073] In some embodiments, determining the fault correlation between a first historical fault occurring in a first component and a second historical fault occurring in a second component based on historical fault data may specifically include the following steps 21) to 25):

[0074] 21) Determine whether the second component fails when the first component fails, and determine a first fault correlation coefficient between the first component and the second component based on the determination result.

[0075] Specifically, the first fault correlation coefficient may have an initial value, such as 0.5.

[0076] Step 21) can be an iterative process. In simple terms, each record of the first component in the historical fault data is compared with each record of the second component to determine whether the second component fails within a period of time after the first component fails. Among them, if the second component fails within a period of time after the first component fails, the value of the first fault correlation coefficient can be increased by a preset value (such as 0.01); if the second component does not fail, the value of the first correlation coefficient can be reduced by a preset value (such as 0.01). The following is explained by example.

[0077] Refer to Table 1. Assume that the first component is component 1 and the second component is component 2. Take the first record of component 1, "Historical fault A1, 2024-01-03 10:30" as an example:

[0078] Comparing the first record of component 1 with the first record of component 2, "historical fault B1, 2024-01-03 10:32", it can be found that historical fault B1 occurred some time after historical fault A1, so the first fault correlation coefficient increases by a preset value of 0.01 to 0.51;

[0079] Comparing the first record of component 1 with the second record of component 2, "historical fault B2, 2024-01-03 10:33", it can be found that historical fault B2 occurred some time after historical fault A1, so the first fault correlation coefficient increases by a preset value of 0.01 to 0.52;

[0080] By comparing the first record of component 1 with the third record of component 2 "historical fault B2, 21:25 on March 1, 2024", it can be found that historical fault B2 did not occur some time after historical fault A1, so the first fault correlation coefficient is reduced by the preset value of 0.01 to 0.51.

[0081] By analogy, the value of the first fault correlation coefficient finally obtained is the first fault correlation coefficient between the first component and the second component.

[0082] 22) Determine whether a second historical fault occurs in the second component when a first historical fault occurs in the first component, and determine a second fault correlation coefficient between the first historical fault and the second historical fault based on the determination result.

[0083] Specifically, the second fault correlation coefficient may have an initial value, such as 0.6.

[0084] Step 22) can be an iterative process. Simply put, each record of the first component + the first historical fault in the historical fault data is compared with each record of the second component to determine whether the second fault occurs in the second component within a period of time after the first historical fault occurs in the first component. Among them, if the second historical fault occurs in the second component a period of time after the first historical fault occurs in the first component, the second fault correlation coefficient increases by a preset value (for example, 0.01); if the second historical fault does not occur in the second component, the second fault correlation coefficient decreases by a preset value (for example, 0.01). The following is explained by example.

[0085] Refer to Table 1. Assume that the first component is component 1, the second component is component 2, the first historical fault is historical fault A1, the second historical fault is historical fault B1, and the initial value of the second fault correlation coefficient is 0.6. From Table 1, it can be seen that there are 3 records of "component 1+first historical fault is A1". Take the first record "historical fault A1, component 1, 2024-01-03 10:30" as an example:

[0086] Comparing the first record with the first record of component 2, "historical fault B1, component 2, 2024-01-03 10:32", it can be found that a period of time after component 1 had historical fault A1, component 2 had historical fault B1, so the second fault correlation coefficient increased by the preset value of 0.01 to 0.51;

[0087] Comparing the first record with the second record of component 2, "historical fault B2, component 2, 2024-01-03 10:33", it can be found that although component 2 failed after component 1 had historical fault A1, the fault was historical fault B2, not historical fault B1 (i.e., the second historical fault), so the second fault correlation coefficient is reduced by the preset value of 0.01 to 0.5;

[0088] By comparing the first record with the second record of component 2, "historical fault B2, component 2, 21:25 on March 1, 2024", it can be found that historical fault B2 did not occur some time after historical fault A1 occurred, so the second fault correlation coefficient is reduced by the preset value of 0.01 to 0.49.

[0089] By analogy, the value of the second fault correlation coefficient finally obtained is the second fault correlation coefficient between the first historical fault and the second historical fault.

[0090] 23) Counting the first correlation times between the first historical fault and the second historical fault, the first correlation times representing the number of times the second historical fault occurs in the second component when the first historical fault of the first number occurs in the first component.

[0091] For example, refer to Table 1. Assume that the first component is component 1, the second component is component 2, the first historical fault is historical fault A1, and the second historical fault is historical fault B1. After component 1 has three historical faults A1, component 2 has one historical fault B1, so the first association number is 1.

[0092] 24) Determine the fault correlation between a first historical fault occurring in a first component and a second historical fault occurring in a second component according to the first correlation number, the first fault correlation coefficient and the second fault correlation coefficient.

[0093] In some embodiments, the fault correlation can be determined based on expression (1):

[0094]

[0095] Wherein, Pmn represents the fault correlation, k represents the first correlation number between the first historical fault and the second historical fault, M represents the number of iterations in step 22), Ri represents the first fault correlation coefficient, and O represents the second fault correlation coefficient.

[0096] In expression (1), the first correlation number, the first fault correlation coefficient and the second fault correlation coefficient are integrated and calculated, and the obtained correlation can be more accurate.

[0097] In some embodiments, determining the log correlation between the target fault and each log of the server in step S103 may include:

[0098] The target fault is input into the log correlation model, and the log correlation model outputs the log correlation between the target fault and each log of the server. The log correlation model is trained based on the following method:

[0099] Obtaining historical log data, the historical log data including historical logs collected from the server when a historical failure occurs on the server;

[0100] According to the historical log data, the log correlation between each historical fault and each historical log is determined respectively;

[0101] According to the determined log correlation, a log correlation model is fitted.

[0102] The principle of the log correlation model is similar to that of the fault correlation model, and will not be elaborated here.

[0103] Specifically, the historical log data may include historical faults and historical logs corresponding to the historical faults, wherein the historical logs corresponding to the historical faults refer to the historical logs collected from the server when the historical faults occur. Therefore, the historical log data may be similar to that shown in Table 2:

[0104] Table 2 Historical log data

[0105]

[0106]

[0107] In Table 2, although the name of the historical log corresponding to each historical fault is the same, the content included in the historical log may be different (ie, the content of the log file is different).

[0108] The above-mentioned determining the correlation between each historical fault and each historical log according to the historical log data may include the following steps 31) to 33):

[0109] 31) For any target historical fault and any target historical log in the historical log, it can be determined whether the target historical log includes fault information of the target historical fault, and based on the determination result, the log correlation coefficient between the target historical fault and the target historical log is determined.

[0110] Specifically, this process can be an iterative process. That is, in each record of the target historical fault, it is determined whether the target historical log includes the fault information of the target historical fault. The log correlation coefficient can have an initial value, such as 0.5. If the target historical log includes the fault information of the target historical fault, the log correlation coefficient between the target historical log and the target historical fault increases by a preset value (such as 0.01). If the target historical log does not include the fault information of the target historical fault, the log correlation coefficient between the target historical log and the target historical fault decreases by a preset value (such as 0.01).

[0111] The following is an example of Table 2. Assume that the target historical fault is historical fault A1, and the target historical log is log 1. In Table 2:

[0112] In the record of the first historical fault A1, if the fault information of historical fault A1 is found in log 1, the log correlation coefficient between historical fault A1 and log 1 increases by 0.01 and becomes 0.51;

[0113] In the second record of historical fault A1, if the fault information of historical fault A1 is still found in log 1, the log correlation coefficient between historical fault A1 and log 1 increases by 0.01 and becomes 0.52;

[0114] In the third record of historical fault A1, if the fault information of historical fault A1 is not found in log 1, the log correlation coefficient between historical fault A1 and log 1 decreases by 0.01 and becomes 0.51.

[0115] The same goes for the rest. The value of the log correlation coefficient finally obtained is the log correlation coefficient between the historical fault A1 and log 1.

[0116] 32) Counting a second number of associations between the target historical faults and the target historical log, the second number of associations representing the actual number of times the target historical fault information is included in the target historical log when a specified number of target historical faults occur.

[0117] For example, take the historical fault A1 and log 1 in Table 2 as an example. When the historical fault A1 occurs three times, the actual number of times that log 1 includes the target historical fault information is 2 times, so it is the second association number.

[0118] 33) Based on the log correlation coefficient and the second correlation number, determine the correlation between the target historical fault and the target historical log.

[0119] Specifically, according to the principle of step 24), the log correlation coefficient, the second correlation number, and the iteration number can be integrated to obtain the correlation between the target historical fault and the target historical log.

[0120] Combined with reference Figure 2 , which is a flowchart of device log information collection provided by an embodiment of the present disclosure.

[0121] Specifically, in some embodiments of the present disclosure, when collecting device log information, the following steps may be included:

[0122] 41) Obtain the server information of the target server for which log collection is required. The server information may include but is not limited to the server address, user name, and password of the server.

[0123] 42) Obtaining policy configuration information. The policy configuration information is used to control the log collection process, such as the log relevance range described in step S101.

[0124] 43) Collect asset allocation information.

[0125] In this step, the main task is to collect server asset configuration information for log collection and analysis, such as server model, server platform, processor model, memory manufacturer, etc.

[0126] 44) Collect the indicator data of the server.

[0127] 45) Analyze and warn the collected indicator data. Specifically, when the server indicator is about to reach the alarm threshold or when the server is predicted to be abnormal, an early warning can be issued and the user can be notified, and all fault logs related to the predicted fault can be collected. Alternatively, when the server status is abnormal in real-time monitoring, the fault and the correlation of the fault are automatically analyzed, and the related logs of the server are automatically collected.

[0128] 46) Collect logs and compress the collected log packages.

[0129] 47) The collected log compression files are stored in the repository. At the same time, the data in the repository is used as historical data to train the fault correlation model and the log correlation model.

[0130] This completes the entire description of the disclosed method.

[0131] Combined with reference Figure 3 , which is a module diagram of a log collection device provided by an embodiment of the present disclosure. Figure 3 In the example, the log collection device includes:

[0132] The data collection module 301 is used to collect the index data of the server and predict the first target failure that will occur in the server in the future based on the index data;

[0133] A fault finding module 302 is used to find a second target fault that meets the correlation requirement with the first target fault among other potential server faults except the first target fault;

[0134] The log collection module 303 is used to determine the log correlation between the target fault and each log of the server for one or more target faults of the first target fault and the second target fault, and collect logs related to the target fault in the server according to the log correlation.

[0135] In some embodiments, the fault finding module 302 is specifically used to:

[0136] Inputting the first target fault into the fault correlation model, and outputting the fault correlation between the first target fault and the potential faults of each server from the fault correlation model;

[0137] According to the fault correlation, the second target fault is found from the potential faults.

[0138] In some embodiments, the fault finding module 302 is specifically used to fit and obtain a fault correlation model based on the following method:

[0139] Obtain historical fault data of the server, where the historical fault data includes historical fault information of each component of the server;

[0140] Determining, based on the historical fault data, a fault correlation between a first historical fault occurring in a first component and a second historical fault occurring in a second component;

[0141] Based on the determined fault correlation, a fault correlation model is obtained by fitting.

[0142] In some embodiments, the fault finding module 302 is specifically used to:

[0143] Determine whether the second component fails when the first component fails, and determine a first fault correlation coefficient between the first component and the second component according to the determination result;

[0144] Determine whether a second historical fault occurs in the second component when a first historical fault occurs in the first component, and determine a second fault correlation coefficient between the first historical fault and the second historical fault according to the determination result;

[0145] Counting the first correlation number between the first historical fault and the second historical fault, the first correlation number representing the number of times the second historical fault occurs in the second component when the first historical fault of the first number occurs in the first component;

[0146] A fault correlation between a first historical fault occurring in the first component and a second historical fault occurring in the second component is determined according to the first correlation number, the first fault correlation coefficient, and the second fault correlation coefficient.

[0147] In some embodiments, the log collection module 303 is specifically used to:

[0148] The target fault is input into the log correlation model, and the log correlation model outputs the log correlation between the target fault and each log of the server. The log correlation model is trained based on the following method:

[0149] Obtaining historical log data, the historical log data including historical logs collected from the server when a historical failure occurs on the server;

[0150] According to the historical log data, determine the log correlation between each historical fault and each historical log;

[0151] According to the determined log correlation, a log correlation model is fitted.

[0152] In some embodiments, the log collection module 303 is specifically used to:

[0153] For any target historical fault and any target historical log in the historical logs, determine whether the target historical log includes fault information of the target historical fault, and determine a log correlation coefficient between the target historical fault and the target historical log according to the determination result;

[0154] Counting a second number of associations between the target historical fault and the target historical log, the number of associations representing an actual number of times the target historical fault information is included in the target historical log when the target historical fault occurs a specified number of times;

[0155] Based on the log correlation coefficient and the second correlation number, the correlation between the target historical fault and the target historical log is determined.

[0156] In some embodiments, the log collection module 303 is specifically used to:

[0157] Get the preset log correlation range;

[0158] For any server log, if the log correlation between the server log and the target fault is within the log correlation range, the server log is collected.

[0159] The log collection device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0160] In the technical solutions of some embodiments of the present disclosure, by obtaining the command writing speed of the host in the submission queue and the command reading speed of the input / output card in the submission queue, and obtaining the result writing speed of the input / output card in the completion queue and the result reading speed of the host in the completion queue, the performance monitoring result of the input / output card can be obtained based on the first speed difference between the command writing speed and the command reading speed, and the second speed difference between the result writing speed and the result reading speed. Since the implementation of performance monitoring does not modify the driver of the input / output card, it can prevent the problem of substitution errors due to modification of the driver, thereby achieving the purpose of reducing error introduction and improving the reliability of performance monitoring.

[0161] The present disclosure also provides an electronic device having the above Figure 3 The log collection device shown.

[0162] Combined with reference Figure 4 , is a schematic diagram of the structure of an electronic device provided by some embodiments of the present disclosure. Figure 4 As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0163] The processor 10 may be a first PCIe device, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0164] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0165] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0166] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0167] The electronic device further comprises a communication interface 30 for the electronic device to communicate with other devices or a communication network.

[0168] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0169] A part of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present disclosure through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0170] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A log collection method, characterized in that: The method comprises: Collecting indicator data of the server, and predicting a first target failure that will occur in the server in the future based on the indicator data; Searching for a second target fault that meets the correlation requirement with the first target fault among other potential server faults except the first target fault; For one or more target faults among the first target fault and the second target fault, determine the log correlation between the target fault and each log of the server, and collect logs related to the target fault in the server based on the log correlation.

2. The method according to claim 1, characterized in that The step of searching, among other potential server failures except the first target failure, a second target failure that meets a correlation requirement with the first target failure includes: Inputting the first target fault into a fault correlation model, and outputting the fault correlation between the first target fault and each of the potential faults of the server by the fault correlation model; The second target fault is searched from the potential faults according to the fault correlation.

3. The method according to claim 2, characterized in that The fault correlation model is fitted based on the following method: Acquire historical fault data of the server, wherein the historical fault data includes historical fault information of each component of the server; Based on the historical fault data, determining a fault correlation between a first historical fault occurring in a first component and a second historical fault occurring in a second component; Based on the determined fault correlation, the fault correlation model is obtained by fitting.

4. The method according to claim 3, characterized in that The determining, based on the historical fault data, a fault correlation between a first historical fault occurring in a first component and a second historical fault occurring in a second component comprises: determining whether the second component fails when the first component fails, and determining a first fault correlation coefficient between the first component and the second component according to the determination result; determining whether the second component has the second historical fault when the first component has the first historical fault, and determining a second fault correlation coefficient between the first historical fault and the second historical fault according to the determination result; Counting a first correlation number between the first historical fault and the second historical fault, where the first correlation number represents the number of times the second historical fault occurs in the second component when the first historical fault occurs for the first time in the first component; A fault correlation between a first historical fault occurring in a first component and a second historical fault occurring in a second component is determined according to the first correlation number, the first fault correlation coefficient, and the second fault correlation coefficient.

5. The method according to claim 1, characterized in that The determining the log correlation between the target fault and each log of the server includes: The target fault is input into a log correlation model, and the log correlation model outputs the log correlation between the target fault and each log of the server, wherein the log correlation model is trained based on the following method: Acquiring historical log data, the historical log data including historical logs collected from the server when a historical failure occurs on the server; According to the historical log data, respectively determine the log correlation between each of the historical faults and each of the historical logs; The log correlation model is obtained by fitting according to the determined log correlation.

6. The method according to claim 5, characterized in that Determining the correlation between each of the historical faults and each of the historical logs according to the historical log data includes: For any target historical fault and any target historical log in the historical logs, determine whether the target historical log includes fault information of the target historical fault, and determine a log correlation coefficient between the target historical fault and the target historical log according to the determination result; Counting a second number of associations between the target historical fault and the target historical log, the second number of associations representing an actual number of times the target historical fault information is included in the target historical log when the target historical fault occurs a specified number of times; Based on the log association coefficient and the second association number, a degree of association between the target historical fault and the target historical log is determined.

7. The method according to claim 1, characterized in that The collecting logs related to the target fault in the server according to the log correlation degree includes: Get the preset log correlation range; For any server log, if the log correlation degree between the server log and the target fault is within the log correlation degree range, the server log is collected.

8. A log collection device, characterized in that: The device comprises: A data collection module, used to collect the index data of the server, and predict the first target failure that will occur in the server in the future based on the index data; A fault finding module, used to find a second target fault that meets the correlation requirement with the first target fault among other potential server faults except the first target fault; A log collection module is used to determine the log correlation between one or more target faults among the first target fault and the second target fault and each log of the server, and collect logs related to the target fault in the server based on the log correlation.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the log collection method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the log collection method according to any one of claims 1 to 7.

Images