A method for log automated analysis for operating system repair

Abstract

The present application relates to the technical field of operating system repair, and particularly relates to a log automatic analysis method for operating system repair. The present application clusters fault log data to obtain log clusters; obtains abnormal event sequences of the fault log data and service components thereof; obtains comprehensive abnormality degrees according to fluctuation degrees and change trends of resource load indexes of each service component of the abnormal event sequences of the fault log data within a target analysis period; obtains fault decision degrees according to comprehensive abnormality degrees and abnormal event sequence numbers of corresponding abnormal event sequences of the fault log data within the analysis components of the log clusters, arranges the analysis components by using the fault decision degrees, and obtains a fault repair sequence; selects a reference cluster according to fault similarity between current log data and the fault log data within the log clusters, and performs operating system repair based on the fault repair sequence of the reference cluster. The present application converts complex fault logs into a repair list, so that the system can accurately and efficiently locate fault causes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of operating system repair, in particular to a log automatic analysis method for operating system repair. BACKGROUND

[0002] With the rapid development of information technology, various operating systems play a key role in complex network and application environments. A large amount of log data generated during the operation of the operating system records the hardware state, software behavior and system events, which is an important information source for system operation and maintenance state monitoring and fault diagnosis. Traditional log analysis mainly relies on the experience of operation and maintenance personnel for manual investigation. However, in the face of massive log data, manual analysis method is inefficient and prone to omissions. With the development of artificial intelligence, machine learning and other technologies, log automatic analysis method has been widely used, which can quickly and accurately process massive log data, automatically identify abnormal situations and locate fault sources.

[0003] At present, in the implementation of automatic repair link, the existing technology mainly adopts the method based on pre-defined rules. This method depends on the domain experts to summarize the fault mode in advance and formulate the corresponding rule template. When the system detects the log event that meets the specific rule, the corresponding repair script is automatically triggered. However, due to the complex internal process relationship of the operating system and the changeable fault causes, there is a deep and dynamic correlation between the abnormal codes and events represented in the log. The rule-based method lacks flexibility and adaptability, and it is difficult to accurately locate the service component that triggers the fault, resulting in long repair process or inaccurate repair action, even misrepair, which affects the overall efficiency of automatic operation and maintenance and the system reliability. SUMMARY

[0004] In order to solve the technical problem that the log analysis method based on pre-defined rules is difficult to adapt to the complexity and variability of operating system faults, and the fault root cause service component is not accurately located, the purpose of the present application is to provide a log automatic analysis method for operating system repair, and the technical scheme adopted is as follows:

[0005] The present application provides a log automatic analysis method for operating system repair, which comprises:

[0006] Obtaining the current log data of the operating system and the fault log data and its fault set in the historical analysis period, the fault set is composed of fault codes;

[0007] According to the similarity and correlation degree of the fault codes in the fault set of any two fault log data, the fault similarity of the corresponding two fault log data is obtained, and all fault log data is clustered to obtain log clusters;

[0008] According to the abnormal situation of the structured event corresponding to the log record parsed from the fault log data, each abnormal event sequence of the fault log data and a service component thereof are obtained; according to the fluctuation degree and the change trend of each type of resource load index obtained by each service component of each abnormal event sequence of the fault log data within a target analysis period, a comprehensive abnormality degree is obtained;

[0009] The service components of each abnormal event sequence of all fault log data in the log cluster are recorded as analysis components thereof; according to the comprehensive abnormality degree of the corresponding abnormal event sequence of the fault log data and the number of abnormal event sequences of the fault log data within each analysis component of the log cluster, a fault decision degree of each analysis component of the log cluster is obtained; all analysis components are sequentially arranged based on the fault decision degree, and a fault repair sequence of the log cluster is obtained;

[0010] According to the fault similarity of the current log data and all fault log data in the log cluster, a reference cluster of the current log data is selected from the log cluster, and an operating system repair is performed based on the fault repair sequence of the reference cluster.

[0011] Further, the fault similarity obtaining method comprises:

[0012] A predefined association set is obtained, each unordered pair in the association set being composed of two fault codes;

[0013] For any two fault log data, a union set of the fault sets of the two fault log data is obtained, the number of unordered pairs in the association set in which the fault codes belong to the union set is counted, and is recorded as an association number; the ratio of the association number to the number of fault codes in the union set is taken as the fault association degree of the two fault log data;

[0014] The Jaccard coefficient of the fault sets of the two fault log data is obtained, and is recorded as a fault set similarity;

[0015] According to the fault association degree and the fault set similarity, the fault similarity of the two fault log data is obtained.

[0016] Further, the method for obtaining each abnormal event sequence of the fault log data and a service component thereof comprises:

[0017] All log records corresponding to the fault log data are parsed into structured events, the attributes of the structured events including event timestamp, event type and service component; the structured events of all log records corresponding to the fault log data are arranged in order of event timestamp, and a structured event sequence is obtained;

[0018] a set of predefined abnormal event types is obtained; each abnormal event sequence of the fault log data is a subsequence of continuous structured events in the structured event sequence, whose event types belong to the set of abnormal event types;

[0019] the service component of each abnormal event sequence is the service component of all structured events in the abnormal event sequence.

[0020] Further, the comprehensive abnormality degree is obtained by:

[0021] an arbitrary service component of an arbitrary abnormal event sequence of the fault log data is recorded as an example component;

[0022] The attributes of the log records include generation timestamps; a time period between the generation timestamps of the log records corresponding to the two endpoint elements in the abnormal event sequence of the example component is recorded as a target analysis time period; different types of resource load indicators at each time point in the target analysis time period are obtained for the example component.

[0023] An arithmetic mean difference of the same type of resource load indicator at all time points in the target analysis time period is calculated as a data fluctuation degree of each type of resource load indicator of the example component.

[0024] A straight line is fitted to the same type of resource load indicator at all time points in the target analysis time period, and a slope of the obtained fitting straight line is recorded as a load trend value of each type of resource load indicator of the example component.

[0025] According to the data fluctuation degree and the load trend value, a local abnormality degree of each type of resource load indicator of the example component is obtained; and the local abnormality degrees of all types of resource load indicators are averaged to obtain a comprehensive abnormality degree of the example component.

[0026] Further, the fault decision degree of each analysis component of the log cluster is obtained by:

[0027] A ratio of the number of abnormal event sequences of each fault log data in the log cluster to the total number of abnormal event sequences of all fault log data is recorded as a fault degree of each fault log data in the log cluster.

[0028] An arbitrary analysis component of the log cluster is recorded as a target component; target data of the target component is selected from the fault log data in the log cluster, and the target data at least exists in the service component in which the target component appears in the abnormal event sequence.

[0029] Average the comprehensive abnormality degrees of the target components of all abnormal event sequences of each target data to obtain an overall abnormality degree; weight the overall abnormality degree by the failure degree to obtain a modified abnormality degree of each target data; and take the sum of the modified abnormality degrees of all target data of the target component as a failure decision degree of the target component.

[0030] Further, the method for selecting the reference cluster of the current log data from the log clusters comprises:

[0031] Take the average of the failure similarities between the current log data and all failure log data in the log cluster as a comprehensive similarity; and select the log cluster with the maximum comprehensive similarity as the reference cluster of the current log data.

[0032] Further, the method for obtaining the log records corresponding to the failure log data comprises:

[0033] Obtain all log records and failure time stamps within a historical analysis period of an operating system; failure log data is composed of log records between two adjacent failure time stamps; and all log records between the two failure time stamps corresponding to the failure log data are taken as the log records corresponding to the failure log data.

[0034] Further, the data fluctuation degree and the load trend value are positively correlated with the local abnormality degree.

[0035] Further, the method for clustering all failure log data is a K-means clustering algorithm.

[0036] Further, the method for structuring events in the log record analysis is a Drain algorithm.

[0037] The present application has the following beneficial effects:

[0038] First aspect: According to the similarity and correlation of the fault codes in the fault set of different failure log data, the present application measures the deep pattern similarity between the failure log data, determines the failure similarity, and clusters the failure log data, learns and induces the failure pattern from the historical failure data, without manually writing and maintaining a large number of rules, realizes adaptive perception and analysis of complex and variable failures, and overcomes the limitations of predefined rules.

[0039] The second aspect: based on the fluctuation degree and change trend of the resource load index of the service component in the target analysis period, the abnormal performance of the service group in the fault condition can be comprehensively and sensitively evaluated, and the comprehensive abnormality degree is obtained; based on the comprehensive abnormality degree of each analysis component of the log cluster in the corresponding abnormal event sequence of the fault log data, the degree of the analysis component in the decision-making of the severity of the corresponding fault condition of the log cluster is considered, the number of abnormal event sequences of the fault log data is introduced, the relative importance of the fault is presented, the noise interference is effectively filtered, the false alarm and the missed alarm are significantly reduced, and the root cause positioning result is more reliable.

[0040] The third aspect: the most relevant repair experience, i.e., the reference cluster, is selected from the log cluster through the similar fault mode, without relying on the rules written by experts on site, and the fault repair sequence of the reference cluster is used to repair the current log data, which means that the repair action is a highly reliable decision based on historical successful experience, and the accuracy and efficiency of the repair are significantly improved, and the problem that the pre-defined rules are difficult to adapt to the complexity and variability of the operating system fault is effectively solved. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0042] Figure 1 A step flow chart of a log automatic analysis method for operating system repair provided by an embodiment of the present application;

[0043] Figure 2 A computer device schematic diagram of a log automatic analysis device for operating system repair provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the specific embodiments, structures, features and effects of the log automatic analysis method for operating system repair according to the present application are described in detail as follows. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0046] Specifically, the application provides a specific scheme of a log automatic analysis method for operating system repair.

[0047] Embodiment 1

[0048] The application provides a log automatic analysis method for operating system repair. Figure 1 Fig. 1 shows a step flowchart of a log automatic analysis method for operating system repair according to an embodiment of the application, and the method comprises the following steps.

[0049] Step S1: obtaining current log data of an operating system and each fault log data and a fault set in a historical analysis period, wherein the fault set is composed of fault codes.

[0050] The specific operation of operating system repair of the server is as follows:

[0051] Data layer: deploying log collection agents such as Fluentd and Filebea on the server, which are used to collect operating system logs, application logs and middleware logs, etc., and the collected logs are first saved to a Kafka message queue for buffering to ensure that data is not lost during peak periods, and then stored in a database for subsequent analysis.

[0052] Analysis layer: real-time consumption of log streams in Kafka through a stream processing framework, data parsing and other operations, and application of rule engines or machine learning models to detect fault events, and once a fault is detected, the timestamp of the earliest log related to the fault is taken as a fault timestamp, so as to realize real-time discovery of system faults.

[0053] Decision and execution layer: according to the analysis result of the analysis layer, the best repair strategy is matched in the "repair knowledge base", such as restarting the service, expanding the instance and clearing the cache, and the analysis result is converted into actual operation.

[0054] Presentation and interaction layer: real-time display of system health status, abnormal events, alarms and execution of automatic repair through a visual dashboard.

[0055] The operating system may occur multiple times during actual operation, and the system records the timestamp of the fault occurrence in real time, which is recorded as a fault timestamp. The log records between the fault timestamp of the current fault and the fault timestamp of the previous fault are taken as current log data; the log records in the time window from the start of the historical analysis period to the first fault timestamp are taken as the first fault log data, and thereafter, the log records between each adjacent two fault timestamps are taken as the fault log data corresponding to the time window. Each log record is automatically marked with a timestamp generated at the time, which is recorded as a generation timestamp.

[0056] The failure of the operating system often presents a specific failure code, such as a Windows operating system environment operation 0x0010, a storage area shortage 0x0014, and the like, and is accompanied by a hardware failure code, such as a basic input / output system checksum error 1-1-4. For each failure log data, the analysis layer extracts all failure codes recorded in a corresponding time window, and a failure set is formed by the failure codes. For example, 0x0014 and 1-1-4 codes are detected in a certain time window, and the failure set is {0x0014, 1-1-4}.

[0057] In an implementation manner of the embodiment of the present application, a time period between a system startup timestamp and a last failure timestamp of a current failure is taken as a historical analysis time period.

[0058] Step S2: According to the similarity degree and the correlation degree of the failure codes in the failure sets of any two failure log data, a failure similarity of the two failure log data is obtained, and clustering is performed on all the failure log data to obtain log clusters.

[0059] The similarity degree of the failure codes in the failure sets of different failure log data directly presents the similarity of the failure modes; however, in the process of operating system running, there is often an inherent correlation between different failures, and the occurrence of one failure can be accompanied by the occurrence of another failure, for example, a hardware failure can cause multiple processes or system failures. Therefore, when analyzing the similarity of the failure modes between the failure log data, the correlation degree between different failures also needs to be considered, and the correlation relationship usually depends on external knowledge. The similarity degree and the correlation degree of the failure codes in the failure sets of different failure log data are combined to more accurately measure the deep pattern similarity between the failure log data, obtain the failure similarity, and perform clustering on the failure log data by using the failure similarity, so that each log cluster represents a failure mode with a common inherent rule, and the defect that a pre-defined rule cannot depict complex failure correlations is overcome, facilitating failure analysis and diagnosis.

[0060] In an implementation manner of the embodiment of the present application, a K-means clustering algorithm is selected to perform clustering on the failure log data, and the distance between two failure log data is measured by using data similarity, wherein the K value is determined by using a silhouette coefficient method. Each log cluster represents a similar failure mode.

[0061] Step S3: According to the abnormal situation of the structured events parsed from the log records corresponding to the failure log data, each abnormal event sequence and a service component of the failure log data are obtained; according to the fluctuation degree and the change trend of each type of resource load index of each service component of each abnormal event sequence of the failure log data obtained in a target analysis time period, a comprehensive abnormality degree is obtained.

[0062] The root cause of a failure ultimately traces back to a specific service component. However, log records contain a large amount of redundant information, making it impossible to directly quantify the abnormal state of a service component. By parsing and templatedizing log records, extracting core event types, and based on meaningful abnormal pattern fragments of structured event anomalies, an abnormal event sequence and its corresponding service component can be obtained, significantly reducing computational complexity and false positive rate.

[0063] The target analysis period is a critical time window for failures, and resource load indicators directly reflect the health of the operating system, capturing pattern changes before and after a failure. During normal operating system operation, the resource load of service components should be relatively stable with a relatively smooth trend. However, abnormal events such as configuration errors, resource contention, and external attacks can cause the system to deviate from its stable state. Therefore, analyzing the abnormal patterns of resource load based on the fluctuations and trends of service component resource load indicators during the target analysis period allows for a comprehensive and sensitive assessment of the abnormal performance of service groups under failure conditions, yielding a comprehensive anomaly score.

[0064] Step S4: Denote the service components of each abnormal event sequence of all fault log data within the log cluster as its analysis components; obtain the fault determination degree of each analysis component in the log cluster based on the comprehensive abnormality degree of the corresponding abnormal event sequence of the fault log data within the log cluster and the number of abnormal event sequences of the fault log data; arrange all analysis components in order based on the fault determination degree to obtain the fault repair sequence of the log cluster.

[0065] Within a fault cluster, multiple analytics components may exhibit anomalies simultaneously, but the root cause is usually only one or a few. This step aims to accurately identify the most probable root cause of the fault from the analytics components of the log cluster and determine the remediation priority.

[0066] The overall anomaly score measures the abnormal behavior of the analysis component in the corresponding fault situation of the target data, that is, the probability that the analysis component is the root cause of the fault in the log cluster, and presents the degree to which the analysis component plays a decisive role in the occurrence or severity of the fault situation in the log cluster. However, in reality, individual fault log data may contain fewer abnormal events due to noise or random factors, making its overall anomaly score unreliable. The number of abnormal event sequences in the fault log data shows its relative importance in the fault. Further analysis combining the number of abnormal event sequences in the fault log data determines the fault determinism. This greatly enhances the robustness of root cause localization, effectively filters noise interference, and the finally generated fault repair sequence provides a clear and reliable roadmap for automated repair actions.

[0067] The fault repair sequence defines the priority order of repair operations. In one implementation of this invention, the fault determination of the analysis components in the fault repair sequence of the log cluster is arranged in descending order from front to back. The analysis component with the smaller index in the fault repair sequence is more likely to be the root cause of the corresponding fault in the log cluster. The automated repair system should check and process faults according to the order of the analysis components in the fault repair sequence to locate and resolve the root cause most efficiently.

[0068] Step S5: Based on the fault similarity between the current log data and the fault log data in all log clusters, select a reference cluster for the current log data from the log clusters, and perform operating system repair based on the fault repair sequence of the reference cluster.

[0069] It is known that repair experiences for similar failure modes are similar. By measuring the similarity of failure modes between the current log data and the failure log data within the log cluster, the system can automatically and quickly match the most relevant repair experience, i.e., the reference cluster. Using the failure repair sequence of the reference cluster to repair the current log data means that the repair action is a highly reliable decision based on historical successful experience, significantly improving the accuracy and efficiency of the repair.

[0070] Preferably, in some possible implementations of the embodiments of the present invention, the method for obtaining fault similarity includes: obtaining a predefined association set, wherein each unordered pair in the association set consists of two fault codes; for any two fault log data, obtaining the union of the fault sets of the two fault log data, and counting the number of unordered pairs in the association set where all fault codes belong to the union, denoted as the association number; taking the ratio of the association number to the number of fault codes in the union as the fault association degree of the two fault log data; obtaining the Jaccard coefficient of the fault sets of the two fault log data, denoted as the fault set similarity; and obtaining the fault similarity of the two fault log data based on the fault association degree and the fault set similarity.

[0071] It should be noted that the method for obtaining the association set is as follows: whether there is an association between faults is predefined, usually based on historical data and expert experience. For example, if a bus fault can cause multiple control units to fail, then there is an association between the bus fault and the control unit fault. The unordered pair formed by the corresponding fault codes of the bus fault and the control unit fault can be used as an element of the association set. Based on all the predefined faults that have an association, the corresponding unordered pairs are determined, and the association set is formed by all the unordered pairs.

[0072] Fault correlation degree measures the common association patterns in two fault sets by counting the number of unordered pairs of associations belonging to the union. Fault set similarity represents the proportion of common fault codes in the fault sets of different fault log data, directly measuring the similarity of fault patterns between fault log data through the degree of overlap of fault codes. The greater the fault correlation degree and fault set similarity, the stronger the external fault correlation and the more similar the fault conditions between different fault log data, and thus the more similar the fault patterns between the fault log data. Therefore, both fault correlation degree and fault set similarity are positively correlated with fault similarity. In this embodiment of the invention, the product of the fault correlation degree and fault set similarity of any two fault log data is taken as the fault similarity.

[0073] It should be noted that the same method for obtaining fault similarity described above is used to calculate the fault similarity between pairs of fault log data.

[0074] Preferably, in some possible implementations of the embodiments of the present invention, the method for obtaining the abnormal event sequence and its service components includes: parsing all log records corresponding to the fault log data into structured events, wherein the attributes of the structured events include event timestamp, event type, and service component; arranging the structured events corresponding to all log records of the fault log data in order of event timestamp to obtain a structured event sequence; obtaining a predefined set of abnormal event types; each abnormal event sequence of the fault log data is a subsequence composed of consecutive structured events in the structured event sequence whose event type belongs to the set of abnormal event types; and the service component of each abnormal event sequence is the service component of all structured events within it.

[0075] It should be noted that the logs contain a large amount of normal operational information. Meaningful abnormal pattern fragments can be extracted from the structured event sequence using a predefined set of abnormal event types to obtain the abnormal event sequence, which can significantly reduce computational complexity and false alarm rate. This embodiment uses the Drain algorithm to parse the log records; the predefined set of abnormal event types is manually specified by domain experts based on prior knowledge, or it can be automatically learned by analyzing event types in historical fault log data.

[0076] Preferably, in some possible implementations of the embodiments of the present invention, the method for obtaining the comprehensive anomaly degree includes: arbitrarily selecting any service component from any abnormal event sequence of fault log data as an example component; the attributes of the log record include a generation timestamp; the time period between the generation timestamps of the log records corresponding to the two endpoint elements in the abnormal event sequence of the example component is designated as the target analysis period; obtaining different types of resource load indicators of the example component at each moment within the target analysis period; calculating the arithmetic mean difference of the same type of resource load indicators of the example component at all moments within the target analysis period, as the data volatility of the example component at each type of resource load indicator; performing linear fitting on the same type of resource load indicators of the example component at all moments within the target analysis period, and denoteing the slope of the obtained fitted line as the load trend value of the example component at each type of resource load indicator; obtaining the local anomaly degree of the example component at each type of resource load indicator based on the data volatility and the load trend value; and averaging the local anomalies of all types of resource load indicators to obtain the comprehensive anomaly degree of the example component.

[0077] It should be noted that failures are often accompanied by drastic fluctuations in resource usage. Greater data volatility indicates an unstable operating system, making it difficult for the example component to maintain stable processing capabilities, resulting in poorer service performance and a higher degree of anomaly. Load trend values ​​reflect the long-term trend of resource usage; when the load trend value is positive and larger, the example component may rapidly approach a resource bottleneck, i.e., increased system load, such as CPU saturation or memory exhaustion. The system may not be able to scale or optimize in time, leading to poorer service performance and a higher degree of anomaly. Therefore, both data volatility and load trend values ​​are positively correlated with local anomaly. In this embodiment, the load trend values ​​of each type of resource load indicator for the example component are normalized, and the product of the normalization result and data volatility is used as the local anomaly. The purpose of normalizing the load trend value is to convert it to a non-negative number. To avoid the bias of a single indicator evaluation, the local anomalies of all types of resource load indicators are combined to obtain a comprehensive anomaly.

[0078] In one implementation of this invention, a two-dimensional space is constructed with time as the horizontal axis and resource load index as the vertical axis. The resource load index of the example component at all times during the target analysis period is mapped to the two-dimensional space to obtain corresponding scattered points. The least squares method is used to perform linear fitting on all scattered points in the two-dimensional space.

[0079] In this embodiment of the invention, the Sigmoid function is used for normalization. However, other normalization methods such as function transformation or max-min normalization can also be used, and no limitation is made here.

[0080] In this embodiment of the invention, the types of resource load indicators include: Central Processing Unit (CPU) utilization and number of threads; the resource load indicators of the example component at each moment in the target analysis period are collected by monitoring software such as Prometheus. There should be at least two moments in the target analysis period, and the time interval between two adjacent moments is set to 15 seconds. The implementer can set it according to the specific situation.

[0081] It should be noted that the method for obtaining the overall anomaly score of each service component in each anomaly event sequence of the fault log data is the same as the method for obtaining the overall anomaly score of the example component.

[0082] Preferably, in some possible implementations of the embodiments of the present invention, the method for obtaining the fault determination degree includes: taking the ratio of the number of abnormal event sequences of each fault log data in the log cluster to the total number of abnormal event sequences of all fault log data as the fault degree of each fault log data in the log cluster; arbitrarily selecting an analysis component of the log cluster as the target component, selecting target data of the target component from the fault log data in the log cluster, wherein the target data appears in at least one service component of the abnormal event sequence; averaging the comprehensive abnormality degree of the target components of all abnormal event sequences of each target data to obtain the overall abnormality degree; weighting the overall abnormality degree using the fault degree to obtain the corrected abnormality degree of each target data; and taking the sum of the corrected abnormality degrees of all target data of the target component as the fault determination degree of the target component.

[0083] It's important to note that fault severity measures the relative importance of a single fault log data point within the overall fault of the log cluster. A higher fault severity indicates more anomalous events are contained in the fault log data, representing more complex or severe fault situations, thus making the fault log data more representative in fault analysis. Overall anomaly severity measures the overall anomalous performance of the target component within the fault situation corresponding to the target data. Weighting the overall anomaly severity with fault severity allows more critical fault situations to have a greater impact on fault determination, reducing the influence of noise or random fault situations. The more target data a target component has and the higher its corrected anomaly severity, the more significant the anomalous pattern of the target component, the greater the likelihood that the target component is the root cause of the fault in the log cluster, and the more severe the anomalous behavior of the target component, potentially directly exacerbating the fault impact. Therefore, the target component plays a more decisive role in the occurrence or severity of the fault situation in the log cluster, resulting in a higher fault determination.

[0084] It is important to note that the failure determination score of all analysis components in the log cluster is obtained using the same method as that of the target component. Abnormal event sequences in the analysis components that do not exist in the target component are not included in the overall anomaly score calculation.

[0085] Preferably, in some possible implementations of the embodiments of the present invention, the method for obtaining the reference cluster includes: taking the average of the fault similarity between the current log data and all fault log data in the log cluster as the comprehensive similarity; and selecting the log cluster with the maximum comprehensive similarity as the reference cluster of the current log data.

[0086] It should be noted that the greater the overall similarity, the more similar the fault modes of the current log data and the fault log data within the log cluster are. The repair experience of the log cluster with the highest overall similarity is most relevant to the repair scheme of the current log data. The repair scheme of the log cluster can be referred to to repair the operating system in the corresponding fault situation of the current log data.

[0087] This invention is now complete.

[0088] Example 2:

[0089] This invention also presents a schematic diagram of a computer device for automated log analysis of operating system repair; please refer to [link / reference]. Figure 2 The computer device includes a memory 601, a processor 602, and a computer program 603 stored in the memory 601 and running on the processor 602. When the processor 602 executes the computer program 603, the computer device can perform any of the aforementioned automated log analysis methods for operating system repair.

[0090] Furthermore, embodiments of this application also protect an apparatus that may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to perform an automated log analysis method for operating system repair provided in embodiments of this application.

[0091] This embodiment can divide the device into functional modules based on the above method example. For example, each module can correspond to a separate function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0092] When each module is divided according to its function, the device may also include a communication module, a signal analysis module, a complexity analysis module, and a positioning module. It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional descriptions of the corresponding functional modules, and will not be repeated here.

[0093] It should be understood that the apparatus provided in this embodiment is used to execute the above-described automated log analysis method for operating system repair, and therefore can achieve the same effect as the above-described implementation method.

[0094] When using integrated units, the device may include a processing module and a storage module. When applied to a workpiece, the processing module can be used to control and manage the workpiece's operations. The storage module can be used to support the execution of program code by the workpiece.

[0095] The processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits contained in conjunction with the disclosure of this application. The processor may also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc., and the storage module may be a memory.

[0096] Example 3:

[0097] This embodiment also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the aforementioned method steps to implement the automated log analysis method for operating system repair provided in the above embodiment.

[0098] Example 4:

[0099] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to realize the automated log analysis method for operating system repair provided in the above embodiment.

[0100] In this embodiment, the device, computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0101] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0102] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0103] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for log automated analysis for operating system repair, characterized in that, The method comprises: obtaining current log data of an operating system and each fault log data and a fault set in a historical analysis period, the fault set being composed of fault codes; obtaining fault similarity of the two fault log data according to the similarity and correlation degree of the fault codes in the fault sets of the two fault log data, and clustering all the fault log data to obtain a log cluster; obtaining each abnormal event sequence and a service component of the fault log data according to abnormal conditions of the structured events parsed from the log records corresponding to the fault log data; and obtaining comprehensive abnormality degree according to the fluctuation degree and change trend of each type of resource load index obtained by each service component in each abnormal event sequence of the fault log data in a target analysis period; regarding the service components of each abnormal event sequence of all the fault log data in the log cluster as analysis components; obtaining fault decision degree of each analysis component of the log cluster according to the comprehensive abnormality degree of the corresponding abnormal event sequence of the fault log data in the analysis component and the number of abnormal event sequences of the fault log data; and sequentially arranging all the analysis components based on the fault decision degree to obtain a fault repair sequence of the log cluster; selecting a reference cluster of the current log data from the log cluster according to the fault similarity of the current log data and all the fault log data in the log cluster, and performing operating system repair based on the fault repair sequence of the reference cluster.

2. The method for automated analysis of logs for operating system repair of claim 1, wherein, The method for obtaining the fault similarity comprises: obtaining a predefined correlation set, each unordered pair in the correlation set being composed of two fault codes; for any two fault log data, obtaining a union set of the fault sets of the two fault log data, counting the number of unordered pairs in the correlation set in which the fault codes belong to the union set, and regarding the number as a correlation number; and regarding the ratio of the correlation number to the number of fault codes in the union set as a fault correlation degree of the two fault log data; obtaining a Jaccard coefficient of the fault sets of the two fault log data, and regarding the Jaccard coefficient as a fault set similarity; obtaining the fault similarity of the two fault log data according to the fault correlation degree and the fault set similarity.

3. The method for log automated analysis for operating system repair of claim 1, wherein, The method for obtaining each abnormal event sequence and a service component of the fault log data comprises: parsing all the log records corresponding to the fault log data into structured events, the attributes of the structured events including event timestamp, event type and service component; and arranging the structured events of all the log records corresponding to the fault log data in order of event timestamp to obtain a structured event sequence; obtaining a predefined abnormal event type set; each abnormal event sequence of the fault log data being a subsequence composed of continuous structured events in the structured event sequence in which the event type belongs to the abnormal event type set; the service component of each abnormal event sequence being the service component of all the structured events in the abnormal event sequence.

4. The method for log automated analysis for operating system repair of claim 1, wherein, The method for obtaining the comprehensive abnormality degree comprises: regarding an arbitrary service component of an arbitrary abnormal event sequence of an optional fault log data as an example component; The attributes of the log records include generation timestamps; a time period between the generation timestamps of two end element corresponding log records in an example component corresponding abnormal event sequence, denoted as a target analysis time period; different types of resource load indicators of the example component at each time point in the target analysis time period are obtained; An arithmetic mean difference of the same type of resource load indicator of the example component at all time points in the target analysis time period is calculated as the data fluctuation degree of each type of resource load indicator of the example component; A straight line fitting is performed on the same type of resource load indicator of the example component at all time points in the target analysis time period, and a slope of the obtained fitting straight line is denoted as a load trend value of each type of resource load indicator of the example component; According to the data fluctuation degree and the load trend value, a local abnormality degree of each type of resource load indicator of the example component is obtained; and the local abnormality degrees of all types of resource load indicators are averaged to obtain a comprehensive abnormality degree of the example component.

5. The method for log automated analysis for operating system repair of claim 1, wherein, The method for obtaining the failure decision degree of each analysis component of the log cluster comprises: A ratio of the number of abnormal event sequences of each failure log data in the log cluster to the total number of abnormal event sequences of all failure log data is taken as a failure degree of each failure log data in the log cluster; An analysis component of the log cluster is optionally denoted as a target component, target data of the target component is selected from the failure log data in the log cluster, and the target data at least exist in a service component in which the target component appears in one abnormal event sequence; An overall abnormality degree is obtained by averaging the comprehensive abnormality degrees of the target component of all abnormal event sequences of each target data; a modified abnormality degree of each target data is obtained by weighting the overall abnormality degree by using the failure degree; and a sum value of the modified abnormality degrees of all target data of the target component is taken as the failure decision degree of the target component.

6. The method for log automated analysis for operating system repair of claim 1, wherein, The method for selecting a reference cluster of the current log data from the log cluster comprises: A comprehensive similarity is taken as an average value of the failure similarities of the current log data and all failure log data in the log cluster; and a log cluster with the maximum comprehensive similarity is selected as the reference cluster of the current log data.

7. The method of claim 1, wherein the method further comprises: The method for obtaining the log records corresponding to the failure log data comprises: All log records and failure timestamps of an operating system in a historical analysis time period are obtained; failure log data is formed by log records between adjacent two failure timestamps, and all log records between the two failure timestamps corresponding to the failure log data are taken as the log records corresponding to the failure log data.

8. The method of claim 4, wherein the method further comprises: The data fluctuation degree and the load trend value are positively correlated with the local abnormality degree.

9. The method for log automated analysis for operating system repair of claim 1, wherein, The method for clustering all failure log data is a K-means clustering algorithm.

10. The method of claim 1, wherein the method further comprises: The method for analyzing the structured events of the log records is a Drain algorithm.

Images