Intelligent operation and maintenance method and system

Abstract

The invention discloses an intelligent operation and maintenance method and system, and the method comprises the steps: obtaining a log text of a server node; processing the log text according to a pre-trained log emotion model to obtain a negative score of the log text; and determining a potential anomaly detection result of the log text according to the negative score, so as to determine whether the server node is abnormal within a future preset duration based on the potential anomaly detection result.

Description

Technical Field

[0001] This application relates to the field of operation and maintenance technology, and in particular to an intelligent operation and maintenance method and system. Background Technology

[0002] With the implementation of cloud-native transformation, the demand for operation and maintenance platformization is becoming increasingly urgent. In the field of computer automation tools, there are some mature solutions for event-driven automation. The core advantage of these solutions lies in event-driven automated orchestration, which is suitable for large-scale and complex operation and maintenance scenarios (such as cross-system linkage in large data centers). However, its complexity, learning cost, and ecosystem shortcomings make it less suitable for small and medium-sized teams, lightweight operation and maintenance scenarios, or environments that require "quick onboarding" and "low resource consumption".

[0003] In the realm of Infrastructure as Code (IaC), using open-source IaC tools (such as Terraform) can also securely and predictably create, change, and improve infrastructure. However, using Terraform for state management operations is complex and prone to inconsistencies, it is difficult to integrate with large existing resource stacks, it lacks a built-in control plane as a command-line tool and is difficult to track changes, and it requires learning a completely new programming language, resulting in high learning and environment configuration costs.

[0004] In the branch's daily operations and maintenance work, frequently used tools include those provided by the operations data center, such as automated operations and maintenance platforms, application monitoring platforms, and event integration platforms, as well as the bank's self-developed next-generation system transaction monitoring platform. However, for business-related errors, locating problems and business errors requires manual confirmation. Although log collection and keyword searches exist, they are inefficient and often lack experience to refer to, making the repair process lengthy and risky.

[0005] Specifically, existing automated operation and maintenance technologies mainly have the following problems:

[0006] On the one hand, the threshold setting is rigid and relies on fixed threshold alarms. For example, the CPU utilization rate of the central processing unit is fixed to alarm when it is greater than 90%. Ignoring the differences in business scenarios often leads to false alarms. Moreover, the pre-configured thresholds rely on personal experience and lack scientific data analysis support.

[0007] On the other hand, the coverage is limited: it is difficult to cover all metrics across the entire chain, such as network link quality, container internal resources, and API call chains. There is a lack of historical data accumulation; troubleshooting processes and solutions rely solely on documentation or personal memory, failing to form a reusable knowledge base. Summary of the Invention

[0008] This application discloses the following technical solution:

[0009] The first aspect of this application provides an intelligent operation and maintenance method, including:

[0010] Obtain the log text of the server node;

[0011] The log text is processed using a pre-trained log sentiment model to obtain a negative score for the log text;

[0012] The potential anomaly detection result of the log text is determined based on the negative score, and the server node is then used to determine whether an anomaly will occur within a preset time period in the future.

[0013] A second aspect of this application provides an intelligent operation and maintenance system, comprising:

[0014] The acquisition unit is used to obtain the log text of the server node;

[0015] The processing unit is used to process the log text according to a pre-trained log sentiment model to obtain a negative score for the log text.

[0016] The determining unit is used to determine the potential anomaly detection result of the log text based on the negative score, so as to determine whether the server node will experience an anomaly within a preset time period based on the potential anomaly detection result.

[0017] The beneficial effect of this solution is that it obtains negative scores for log text based on the log sentiment model, and predicts whether anomalies will occur on the corresponding server node within a preset time period based on the negative scores. This allows for the early detection of potential anomalies, making it easier for maintenance personnel to take timely countermeasures. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the logical architecture of an intelligent operation and maintenance system provided in an embodiment of this application;

[0020] Figure 2 This is a schematic diagram of the structure of an intelligent operation and maintenance system provided in an embodiment of this application;

[0021] Figure 3 This is a flowchart of an intelligent operation and maintenance method provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] As the number of new banking businesses continues to grow, service types become increasingly complex and diverse, and cluster sizes gradually expand, maintaining stable server and daily operations through expert systems with manually defined rules is becoming less efficient. Therefore, this application designs a lightweight intelligent operation and maintenance system and corresponding intelligent operation and maintenance methods based on Python and the Transformer model.

[0024] The main functional characteristics of this system are as follows:

[0025] 1. Targeted: It does not pursue "full-scenario intelligence", but focuses on the pain points of operation and maintenance that are "high-frequency, repetitive and easy to standardize".

[0026] 2. Low cost: It does not require excessive resource consumption and deployment costs. The data collection agent is lightweight and deployed on each node, only responsible for local data collection, and does not occupy the core resources of the cluster.

[0027] 3. Easy to standardize: Each service communicates through a network interface, such as an application programming interface (API) based on the HTTP protocol. The interface is standardized, making it easy to replace or upgrade individual components.

[0028] 4. Artificial Intelligence (AI) Driven: By combining real-time anomaly detection (in-process response) and proactive alarm prediction (prevention), the traditional "post-fault alarm" is upgraded to "pre-fault warning".

[0029] The intelligent operation and maintenance system described in this application comprises six core modules: a data acquisition agent, a central data storage, an anomaly detection and prediction service, an alarm management service, a self-healing service, and a cluster management service. Each service can be deployed, scaled, or updated independently to avoid single points of failure. The corresponding modules are primarily written in Python, using the Transform model to identify monitoring metrics that deviate from normal patterns to achieve anomaly detection and prediction. The system first collects initial data to train the model, then enters a monitoring loop. When an anomaly is detected, an alarm is triggered, and an attempt is made to automatically repair it.

[0030] The intelligent operation and maintenance system adopts a closed-loop design of "collection → storage → analysis → decision-making → execution" to realize intelligent operation and maintenance of server clusters. Each service is both independent and scalable, and works collaboratively through standardized interfaces to adapt to cluster environments of different sizes.

[0031] Please see Figure 1 The architecture of the intelligent operation and maintenance system can be logically divided into four layers, including the bottom node layer, data storage layer, core service layer, and external interaction layer.

[0032] The underlying node layer consists of multiple server nodes, all of which are equipped with lightweight data collection agents that are responsible for collecting local metrics and logs and serve as the data source for the entire system.

[0033] The data storage layer is where users deploy a central data storage service. This service receives and persists collected data from all nodes, categorizing it into different collections to provide a unified data query interface for upper-layer services. For example, it can be categorized and stored using collections of metric data, log data, server information, and service status.

[0034] The core service layer is used to deploy anomaly detection and prediction services, alarm management services, self-healing services, and cluster management services. Specifically, the anomaly detection and prediction service retrieves data from the storage layer to identify anomalies and predict future occurrences; the alarm management service receives anomaly signals, manages the alarm lifecycle, and triggers notifications; the self-healing service performs repair operations on alarms, relying on the cluster management service for cross-node scheduling; and the cluster management service maintains the cluster topology, supports service expansion and failover, and is the core of cross-node operations.

[0035] The external interaction layer is used to access notification channels and external integrations. Notification channels are used to push alarm information to operations and maintenance personnel via email, Slack, etc. External integrations can interface with third-party systems such as container platforms (Kubernetes) and monitoring systems (Prometheus). External integrations are optional.

[0036] When deploying the intelligent operation and maintenance system of this application, the necessary dependencies and databases for storing data (such as MongoDB database) can be installed first. Then, the above modules can be started one by one in the order of central data storage service, alarm management service, self-repair service, cluster management service, anomaly detection and prediction service and data acquisition agent. When starting the data acquisition agent, the data acquisition agent needs to be started on each server node.

[0037] Finally, the connection information for each service is configured through environment variables. For example, the data acquisition agent is configured to connect to the central data storage, and the alarm service is configured to connect to MongoDB, so that each module can connect according to... Figure 1 The architecture interacts with data, and after configuring the environment variables, the deployment of the intelligent operation and maintenance system is complete.

[0038] The anomaly detection and prediction service is responsible for identifying anomalies from the monitoring data reported by the data collection agents of each server node, and predicting possible future alarms based on an artificial intelligence model, that is, determining the predicted alarms. Specifically, when the anomaly detection and prediction service starts, it starts a background thread that periodically (e.g., hourly) updates the aforementioned isolated forest model for all servers to ensure the timeliness of the model.

[0039] The predictive function for alarm prediction is achieved through a combination of two methods: statistical prediction based on historical trends and log analysis prediction based on artificial intelligence models, outputting a first result and a second result. The first result is the result of statistical prediction, and the second result is the result of log analysis prediction.

[0040] The working principle of the anomaly detection and prediction service is as follows.

[0041] The first statistical model for predicting alarms is trained based on the Isolation Forest algorithm. The training process includes: 1. Obtaining the server node indicator data for the most recent 60 minutes from the database. The indicator data is collected by a data acquisition agent; 2. Extracting key monitoring indicator data from the indicator data using feature engineering techniques. Key monitoring indicator data includes, but is not limited to, any one or more of the following: CPU utilization, memory utilization, disk utilization, and total network traffic; 3. Standardizing the key monitoring indicator data to eliminate the influence of units and obtain standardized key monitoring indicator data; 4. Training the Isolation Forest model using the standardized key monitoring indicator data. Specific training methods can be found in existing technologies and will not be elaborated here; 5. Saving the trained Isolation Forest model and the normalizer. Specifically, they can be distinguished by the server node identifier, i.e., different server nodes correspond to different Isolation Forest models and normalizers.

[0042] The process of training the isolated forest model can be executed using the following code: `def train_model(self, server_id: str) -> bool`. `def train_model` represents the training function for the isolated forest model. `server_id: str` is one of the input parameters, representing a unique identifier for the server (e.g., "srv-10.0.1.5"), used to distinguish isolated forest models corresponding to different server nodes. `-> bool` indicates the return type of the training function; it returns `True` if training is successful, and `False` if training fails due to insufficient data or other reasons.

[0043] Metrics data for the server nodes over the last 60 minutes can be obtained using the code `metrics_df=self._fetch_recent_metrics(server_id, minutes=60)`. This code calls the class's internal method `_fetch_recent_metrics`, which queries the data stored in the central data storage service by `server_id` and the time range (last 60 minutes), returning a DataFrame format result, which contains the metric data for the server node corresponding to `server_id` over the last 60 minutes.

[0044] The DataFrame format is a predefined data format in this application, which can be represented in tabular form. Each row contains monitoring data at a specific point in time, including raw metrics such as timestamp, CPU usage, memory usage, disk usage, and total network traffic.

[0045] As an example, the DataFrame format returned by the above code can be represented by Table 1.

[0046] Table 1

[0047] Timestamp CPU utilization Memory usage Disk usage Total network traffic 2025-11-18T10:00:00 35.2 42.1 38.5 125000 2025-11-18T10:01:00 36.7 43.3 38.6 132000

[0048] The total network traffic can include the sum of the number of bytes sent and received by the server nodes within a certain period of time.

[0049] After obtaining the indicator data, the quantity of indicator data can be verified. If the quantity is less than a preset training quantity threshold, subsequent training steps can be skipped, and a training failure result can be returned. If the quantity is greater than or equal to the training threshold, subsequent training steps can continue. The training quantity threshold can be set based on experience. Too few data points can lead to model overfitting (inability to learn normal patterns). 50 data points can basically cover the normal load fluctuations of the server, so in this embodiment, the training quantity threshold can be set to 50. Verifying the quantity of indicator data can prevent the use of insufficient data to train an invalid model and ensure the basic quality of model training. The quantity of indicator data can be represented by the number of rows in the DataFrame format shown in Table 1. Each row is equivalent to a data point, and the total number of data points is the quantity of indicator data.

[0050] The process of extracting key monitoring indicator data may include: filtering core indicators, that is, retaining fields related to anomalies such as CPU utilization, memory utilization, disk utilization, and total network traffic; removing redundant fields, such as deleting timestamps and server node identifiers; format conversion to ensure that the output is a two-dimensional array, where rows correspond to data points and columns correspond to features; and finally outputting the extracted key monitoring indicator data, the specific format of which is not limited. For example, the extracted key monitoring indicator data can be represented by the feature structure shown in Table 2.

[0051] Table 2

[0052] CPU utilization Memory usage Disk usage Total network traffic 35.2 42.1 38.5 125000 36.7 43.3 38.6 132000

[0053] When standardizing the features of key monitoring indicator data, the standardization tool can be initialized first. The standardization tool can be any open-source standardization tool in the existing technology, such as the StandardScaler standardization tool in sklearn. The core logic is to convert the features into a standard normal distribution with "mean=0, variance=1".

[0054] After feature standardization, the mean and standard deviation of key monitoring indicator data can be saved so that the new data obtained in real-time detection can be standardized using the mean and standard deviation in subsequent real-time detection.

[0055] After initializing the standardization tool, it can be used to "fit and transform" the key monitoring indicator data to obtain standardized key monitoring indicator data. The specific process includes calculating the mean and standard deviation of each column in Table 2. For example, the mean of the CPU utilization column is 40, and the standard deviation is 15. Using the standardization formula, each feature value and the mean and standard deviation of its column are substituted into the standardization formula to convert it into standardized key monitoring indicator data with a standard normal distribution. For instance, if the original value of CPU utilization is 55, based on the aforementioned mean and standard deviation, after substituting into the standardization formula, the calculated standardized key monitoring indicator data is equal to (55-40) / 15, which is equal to 1.0. This method can eliminate differences in units, such as the percentage of CPU utilization and total network traffic, ensuring that the model assigns equal weight to each feature.

[0056] After processing by the standardization tool, the matrix formed by the standardized key monitoring indicator data corresponding to each key monitoring indicator data in Table 2 can be denoted as the standardized feature matrix scaled_features.

[0057] Before starting training, you can first execute the code `model=IsolationForest(**self.model_params)` to initialize the Isolation Forest model. `self.model_params` includes the global configuration parameters of the Isolation Forest model, such as the number of decision trees (100), the expected anomaly ratio (0.05), and the number of random seeds used for initialization (e.g., 42). The `model` obtained after executing the above code is the initialized, untrained Isolation Forest model instance, which only loads the parameters and has no training data.

[0058] After initializing the Isolation Forest model, it can be trained using the standardized feature matrix `scaled_features`. The core principle of Isolation Forest model training is as follows: randomly select a feature, randomly select a split value for that feature, and divide the data samples into two parts. Repeat this splitting process for each subset until each subset contains only one sample (forming a single-node tree). Repeat these steps to generate the target number of decision trees, for example, 100. The model learns the splitting paths (usually shorter) for "normal samples" and the splitting paths (usually longer) for "abnormal samples." Through this process, abnormal samples, because they deviate from the majority, are more easily and quickly isolated.

[0059] The input to the above training process includes a standardized feature matrix (containing all data points, including normal and potential anomalies), and the output includes a trained isolated forest model, which internally stores the splitting rules of the target number of decision trees. The above training process has the characteristic of being unsupervised, that is, no labels need to be passed in, and the model automatically learns the "normal pattern" of the data.

[0060] After training, the server node identifier can be used as the key, and the trained isolated forest model as the value, stored in a key-value pair model dictionary database. During subsequent real-time detection, the corresponding isolated forest model can be retrieved from the model dictionary database using the server node identifier and used directly for prediction. The trained isolated forest model is the first statistical model mentioned above used to determine predicted alarms.

[0061] After training is complete, the normalizer can be saved. Specifically, the mean and standard deviation of each column calculated by the normalizer during feature standardization can be saved. The normalizer can be used as the value and the server node identifier as the key, and the normalizer can be saved in the normalizer dictionary as a key-value pair.

[0062] For real-time collected monitoring metrics data, the anomaly detection and prediction service can determine whether it is an anomaly using a trained model.

[0063] Specifically, the anomaly detection and prediction service can first check if there is a first statistical model corresponding to the server node. If there is, it will execute the subsequent detection steps; otherwise, it will be trained according to the aforementioned method.

[0064] Following the aforementioned training method, key monitoring indicator data are extracted from the real-time collected monitoring indicator data, and the features are standardized using the mean and standard deviation from the previously saved standardizer to obtain real-time standardized monitoring data.

[0065] Then, based on the server node's identifier, the corresponding first statistical model is invoked to process the real-time standardized monitoring data, and the processing result output by the model is obtained. The processing result may include classification results, decision scores, or both classification results and decision scores. The processing result output by the first statistical model is equivalent to the aforementioned first result.

[0066] The classification result can be 1 or -1. If it is 1, it means that the real-time collected monitoring indicator data is normal. If it is -1, it means that the real-time collected monitoring indicator data is abnormal, which means that the corresponding server node is abnormal.

[0067] It should be noted that the first result, indicating abnormality or normality, refers to the current state of the corresponding server node. That is, a classification result of 1 indicates that the server node is currently normal, while a result of -1 indicates that an abnormality has occurred.

[0068] A decision score greater than or equal to 0 indicates that the real-time collected monitoring data is normal, while a decision score less than 0 indicates that the real-time collected monitoring data is abnormal, which means that the corresponding server node is abnormal.

[0069] Optionally, if the first statistical model outputs a decision score, an abnormal alarm can be output based on whether the decision score is lower than the alarm score threshold. The alarm score threshold can be set as needed, for example, to -0.5, and the alarm score threshold can be different for different server nodes.

[0070] If the classification results and / or decision scores indicate that the real-time collected monitoring metrics data is abnormal, an alarm message can be output to notify the operation and maintenance personnel to handle it in a timely manner. The output alarm message may include the identifier of the abnormal server node, the classification result, and the abnormal real-time collected monitoring metrics data. The alarm message can be output by calling the alarm management service, creating an alarm record, and triggering a notification via email or other means.

[0071] Optionally, regardless of whether the result is normal or abnormal, the processing results output by the first statistical model can be used to construct key-value pairs and store them in a processing result dictionary for use by callers (such as cluster management services or front-end interfaces). The key-value pairs can include server node identifiers, classification results, decision scores, and other user-readable prompts.

[0072] The data acquisition agent module is deployed independently on each server node. Its core function is to collect system metrics (i.e., the aforementioned monitoring metrics data) of the server nodes, such as CPU utilization, memory utilization, disk utilization, and total network traffic. In addition, it can also collect key log files of the server nodes, such as system and application logs under paths like / var / log as key log files, and periodically push the monitoring metrics data and key log files to the central data storage service for storage.

[0073] The workflow of the data acquisition agent is as follows:

[0074] The data acquisition agent is started, and the environment variable configuration is read. An agent instance is created, automatically obtaining the network address and hostname of the server node and discovering the server node's log file. A collection loop is started, performing a collection operation every N seconds. Each collection operation collects monitoring metrics such as CPU usage, memory usage, disk usage, and total network traffic, as well as the latest 10 lines of log data from the log file. The collected monitoring metrics and log data are sent to the central storage's data interface and log interface, respectively, for storage in the central data storage service. This loop continues until an interrupt signal is received. Upon receiving an interrupt signal, the loop stops, and the data acquisition agent program exits. The interval N seconds can be set as needed, such as 10 seconds, 20 seconds, etc., without limitation. The interrupt signal can be obtained when the user presses Ctrl+C.

[0075] Environment variable configuration is used to configure the output format, level, and storage method when the data acquisition agent outputs logs. It is also used to record the running status of the data acquisition agent, such as success or failure, and error messages when it fails.

[0076] Optionally, environment variable configuration can include specifying a log level. The data acquisition agent can collect log files at the specified log level and above to the central data storage service. For example, if the log level is specified as INFO, the data acquisition agent will output logs at the INFO level and above (such as WARNING, ERROR).

[0077] Before collecting data and logs from a server node, a data collection agent instance needs to be created for that server node. When creating the data collection agent instance, the server node's identifier and / or hostname, the server node's network address, the network address of the central data storage service, the instance's running status flag (False for stopped, True for running), the list of log file paths to be collected, the data storage address, and the interval for collecting data and logs (i.e., the aforementioned interval of every N seconds) can be passed as input parameters to the function used to create the data collection agent instance. This allows the data collection agent instance to collect the monitoring metric data and logs of the corresponding server node according to the above information.

[0078] The methods for obtaining the network address of a server node can be found in existing technologies. As examples, methods for obtaining the network address of a server node may include:

[0079] Create a UDP socket (no actual data transmission, only used to obtain the IP address);

[0080] Connecting to a public DNS server (e.g., 8.8.8.8, Google DNS) does not send data; it only binds the system to a local network address (IP).

[0081] Get the local IP address bound to the socket (i.e., the IP address used by the server for external communication).

[0082] For exception handling, if the acquisition fails (e.g., no network), log an error and return "unknown".

[0083] The list of log file paths to be collected can be obtained in the following ways:

[0084] Define common Linux system log paths to obtain a list of paths consisting of common Linux system log paths, such as syslog and messages under / var / log / , which can be expanded as needed;

[0085] Iterate through the path list, and check whether a log file exists under each log path in the path list and whether the existing file is a regular log file.

[0086] Collect the log paths that exist in regular log files, record these log paths in the list of valid log paths, and return the list of valid log paths. This list of valid log paths is the same as the list of log file paths that needed to be collected.

[0087] By using the above methods, common log files existing on the server can be automatically identified, avoiding errors caused by collecting non-existent files.

[0088] The data collection methods for the above monitoring indicators are as follows.

[0089] CPU utilization can be calculated by collecting the average CPU utilization over a 1-second period; memory utilization can be calculated by taking the total memory and available memory, and then dividing the available memory by the total memory; disk utilization can be calculated by taking the utilization of the root directory; and total network traffic can be calculated by taking the number of bytes and data packets sent and received by the server nodes from the network, and then taking the cumulative number of bytes or data packets sent and received as the total network traffic.

[0090] When collecting monitoring metrics data, if the collection is successful, a DEBUG log is recorded, and only the first 200 characters are output to avoid the log being too large. If the collection fails, an ERROR log is recorded, and None is returned.

[0091] Optionally, monitoring metrics may also include disk read / write counts, disk write bytes, and other metrics.

[0092] The method for collecting log files is as follows:

[0093] According to the list of log file paths that need to be collected, traverse the log files found under each path in the list, such as / var / log / syslog;

[0094] For each log file traversed, the log file is opened, and the last N lines (e.g., the last 10 lines) of the log file are retrieved. This avoids reading the entire large file and improves efficiency.

[0095] Each log entry in the N lines read generates corresponding structured log data, which may include the server node identifier, log source file, collection timestamp, and log content.

[0096] Collect log files from all paths in the list of log file paths to be collected and return the list; if a file fails to be read (e.g., insufficient permissions), record an ERROR log and skip the file.

[0097] Using the above method, the log files configured in the list of log file paths to be collected can be read, and the latest N lines can be collected to generate structured log data.

[0098] The collected logs (i.e., the aforementioned structured log data) and monitoring metric data can be sent to the central data storage service for storage in the following manner:

[0099] The network address and interface endpoint of the central data storage service (e.g., the data interface corresponding to monitoring indicator data, and the log interface corresponding to structured log data) are concatenated to form the request address (which can also be recorded as the request URL).

[0100] The data to be sent, namely monitoring metric data and structured log data, is converted into serialized data, such as JSON format, and the request header is set to Content-Type: application / json, so that the central data storage service can parse it.

[0101] The converted serialized data is sent to the request address, which is the interface endpoint of the central data storage service, and the response from the central data storage service is received.

[0102] Process the response. Specifically, if the received response is status code 200, it means the transmission was successful, return True, and log the DEBUG. If the received response is not status code 200, it means the transmission failed, log the ERROR (including the status code), and return False.

[0103] For exception handling, if errors such as network timeout or service unreachability occur, log the ERROR and return False.

[0104] Therefore, the data acquisition agent can send the collected monitoring indicator data and structured log data to the central data storage service via HTTP POST requests.

[0105] As explained above, the anomaly detection and prediction service can also perform log analysis and prediction based on artificial intelligence models to obtain a second result.

[0106] Log analysis and prediction based on artificial intelligence models are mainly performed by the artificial intelligence log analysis module in the anomaly detection and prediction service.

[0107] The core functions of the AI ​​Log Analysis Module include:

[0108] Log preprocessing: Filtering low-priority logs such as INFO / DEBUG, standardizing log format, extracting key fields and converting them to lowercase for easier unified processing;

[0109] Sentiment Analysis and Anomaly Detection: A pre-trained neural network model is used to analyze the negative sentiment tendency of log text. Anomaly scores are enhanced by weighting abnormal keywords, such as "fatal" and "out of memory". An anomaly score threshold (e.g., 0.85) is set to determine whether an anomaly is a potential anomaly based on the enhanced anomaly score.

[0110] The aforementioned pre-trained neural network model can be, for example, a Bidirectional Encoder Representations from Transformers (BERT) model, or a lightweight bidirectional encoder based on knowledge distillation (Distil) (DistilBERT) model.

[0111] Anomaly pattern detection uses the term frequency–inverse document frequency (TF-IDF) method to convert log text into vectors and identifies rare patterns (small clusters) using the density-based spatial clustering of applications with noise (DBSCAN) algorithm to mark possible new types of anomalies.

[0112] The core purpose of the new anomaly mode is to "fill the gap in the detection of unknown anomalies" and, through self-learning and correlation analysis, upgrade the system from "passively responding to known faults" to "actively discovering unknown faults and automatically adapting to new faults", ultimately reducing the "sudden fault handling cost" of operation and maintenance and improving system stability.

[0113] Future Problem Prediction: Based on anomaly scores, predict when a problem may occur; the higher the anomaly score, the earlier the expected occurrence.

[0114] Based on the severity level (e.g., high / medium), similar prediction results are deduplicated, and the most reliable prediction is retained.

[0115] The similarity prediction results may come from the following sources.

[0116] Firstly, multiple similar logs trigger duplicate predictions, with each log generating a separate result. This is the core mechanism. The AI ​​log analysis module iterates through each preprocessed log, and as long as the log meets the "potential anomaly" condition, it generates an independent prediction result. The "potential anomaly" condition could be, for example, an anomaly score greater than 0.85 or other anomaly score thresholds.

[0117] For example, the server node outputs the following three similar logs consecutively within one minute:

[0118] Log 1: "Connection timeout to DB: 10.0.2.3";

[0119] Log 2: "Connection timeout to DB: 10.0.2.3(retry 1)";

[0120] Log 3: "Connection timeout to DB: 10.0.2.3(retry 2)";

[0121] The anomaly scores of the three log entries are 0.91, 0.90, and 0.92, respectively, all greater than 0.85. Therefore, three prediction results are generated based on the three log entries. When deduplicating similar prediction results, the analysis found that the three log entries are highly similar in semantics, and will eventually generate three "predicted DB connection timeout" results. However, the core information (server ID, problem type, prediction time) is almost identical, which belongs to "duplicate prediction". Therefore, only one prediction result can be retained.

[0122] Secondly, multi-dimensional prediction of the same log, that is, generating multiple prediction results based on a single log.

[0123] If the prediction logic is subsequently expanded, for example, to predict both "CPU overload" and "DB connection failure" simultaneously, a single log entry may trigger multiple prediction dimensions, thereby generating multiple prediction results.

[0124] For example, a log entry "CPU usage 90%, DB connection timeout" contains two abnormal signals. The AI ​​log analysis module may generate two prediction results for it. Prediction result 1 is "CPU usage is expected to reach 95% in 10 minutes" with a high severity. Prediction result 2 is "DB connection timeout is expected to continue in 5 minutes" with a medium severity. When deduplicating similar prediction results, since multiple prediction results of different dimensions are generated from a single log entry, it is necessary to retain the prediction result with the highest severity in each dimension. Other prediction results with lower severity can be deleted. For example, the prediction result "DB connection timeout is expected to continue in 5 minutes" in this example should be deleted.

[0125] Thirdly, regarding the clustered small cluster logs, which are predictions corresponding to the same pattern from multiple logs. Combining this with the previous "anomaly pattern discovery" function, if a "rare small cluster" contains 5 similar anomaly logs after clustering, each log will generate a prediction, resulting in 5 prediction results with almost identical content. When deduplicating similar prediction results, these logs essentially belong to the "same anomaly pattern," so the prediction results are naturally duplicated and need to be deduplicated, which means deleting 4 of the above 5 prediction results.

[0126] The working principle of the AI ​​log analysis module is as follows.

[0127] First, import the dependency libraries required by the log analysis module, including but not limited to the following tool libraries:

[0128] re, a regular expression, is used for log filtering (such as ignoring INFO / DEBUG logs).

[0129] time / datetime: Calculate and analyze time consumption, process log timestamps, and predict future times;

[0130] logging: Records log data (outputs analysis status and progress);

[0131] numpy / pandas: Data processing (such as clustering result statistics);

[0132] transformers.pipeline: Loads a pre-trained text classification model (for log sentiment / anomaly analysis);

[0133] TfidfVectorizer: Converts log text into TF-IDF vectors (for clustering);

[0134] DBSCAN: Unsupervised clustering algorithm (used to discover new anomalous patterns);

[0135] typing: Type annotation (standardizes the input and output format of functions).

[0136] Next, configure the logging system. Specifically, you can configure the log output level (INFO and above) and configure the AI ​​log analysis module to record key information during the analysis process (such as the number of preprocessed logs and the analysis time) to facilitate troubleshooting.

[0137] An AI log analyzer class is defined, encapsulating all log analysis logic. By calling an instance of the AI ​​log analyzer class in the main method, logs stored in the central data storage service can be processed to obtain prediction results. These prediction results are equivalent to the second result obtained from the aforementioned log analysis prediction based on the artificial intelligence model.

[0138] The AI ​​Log Analyzer class includes an initialization method, which is used to load the model and configure parameters. Specifically, when an instance of the AI ​​Log Analyzer class is created, the initialization method is executed first. When the initialization method is executed, it initializes the core model, preprocessing rules, and exception keywords, preparing for subsequent analysis.

[0139] The initialization method can perform the following operations.

[0140] First, load a pre-trained log sentiment model. The log sentiment model can be, for example, distilbert-base-uncased-finetuned-sst-2-english (a lightweight BERT model specifically designed for English text sentiment classification); alternatively, the log sentiment model can also be other neural network models with relevant capabilities.

[0141] Configure `return_all_scores=True` so that instances of the AI ​​Log Analyzer class can return scores for all labels, including both positive and negative labels; configure `device=-1` so that instances of the AI ​​Log Analyzer class use the CPU to run the loaded pre-trained model, or you can configure `device=0` so that instances of the AI ​​Log Analyzer class use the GPU to run the loaded pre-trained model to improve speed.

[0142] Second, configure the relevant components used for clustering, such as TfidfVectorizer and DBSCAN. The former is used to convert log text into TF-IDF vectors (denoted as log vectors) based on the TF-IDF method to facilitate the measurement of word importance; specific conversion methods can be found in existing technologies. The latter is used to perform clustering based on the converted log vectors.

[0143] When configuring TfidfVectorizer, you can specifically configure stop_words="english" to filter English stop words (such as "the" and "and", which have no practical meaning); configure max_features=1000 to retain 1000 of the most valuable feature words; and configure ngram_range=(1,2) to consider both single words (unigrams) and phrases (bigrams, such as "connectionrefused").

[0144] When configuring DBSCAN, you can configure eps=0.5, where eps represents the cluster radius, meaning logs with a vector distance less than 0.5 are considered to be of the same type; configure min_samples=5, which means that the minimum number of samples (i.e., the number of logs) to form an effective cluster is 5. If a cluster has fewer than 5 samples, it is called a "rare small cluster".

[0145] Third, configure log filtering rules (ignored_patterns). In the log filtering rules, regular expressions are used to define which logs need to be ignored (e.g., ignore INFO / DEBUG level logs that have no abnormal value).

[0146] Fourth, anomaly keyword enhancement (anomaly_keywords) involves configuring anomaly keywords and their corresponding weights. Different anomaly keywords can have the same or different weights. For example, the anomaly keyword "out of memory" has a weight of 1.3, which has the greatest impact on the anomaly score and is used to enhance the accuracy of anomaly identification.

[0147] When an instance of the created AI Log Analyzer class is invoked, the instance can first execute the core method 1 defined in the AI ​​Log Analyzer class, namely the log preprocessing method.

[0148] The role of log preprocessing methods is to clean and standardize the raw logs obtained from the central data storage service, in order to filter out worthless logs, retain key information, and reduce noise for subsequent analysis.

[0149] The specific implementation method of the log preprocessing method is as follows.

[0150] First, logs without a content field are identified as empty logs and ignored.

[0151] Secondly, text standardization, which involves removing leading and trailing spaces from the log content and converting it to lowercase, is necessary to avoid analytical biases caused by differences in capitalization.

[0152] Then, filter irrelevant logs. Specifically, you can use regular expression matching to ignore INFO and DEBUG level logs and only keep ERROR and WARNING level logs that have anomaly potential. Anomaly potential means that the log can be used to predict whether the server node is abnormal.

[0153] Next, structured extraction is performed, which involves extracting core fields from the original logs. Core fields include, but are not limited to, log identifier, server node identifier, timestamp, log level, content, and process to which it belongs. Standardized dictionary entries are generated based on the extracted core fields.

[0154] Finally, the returned result is a list of preprocessed logs, which retains only the standardized log text of logs that are valuable for analysis.

[0155] After executing the log preprocessing method, the instance of the created AI log analyzer class can continue to execute the core method 2 defined in the AI ​​log analyzer class, namely the log sentiment and anomaly analysis method. The purpose of this method is to analyze the aforementioned standardized log text, quantify its "abnormality level", and determine whether each standardized log text is a potential anomaly.

[0156] The specific method for determining whether a standardized log text is a potential anomaly is as follows.

[0157] First, call the log sentiment model to analyze the sentiment of the standardized log text. Here, you can limit the processing to only the first 512 characters or other numbers of characters of the standardized log text to avoid overloading the model.

[0158] The model returns two label scores, including a positive score and a negative score. Abnormal logs usually correspond to high negative scores.

[0159] Extract the negative score. For example, a negative score of 0.82 means that there is an 82% probability that this standardized log text is a negative log, i.e., an abnormal log.

[0160] Then, the score is enhanced based on the keywords. Specifically, all the abnormal keywords configured in the fourth operation of the initialization method can be traversed to determine whether these abnormal keywords are contained in the standardized log text. If they are contained, the abnormal weight corresponding to the abnormal keyword is multiplied by the previously extracted negative score to obtain the enhanced abnormal score; for example, if the abnormal keyword "timeout" is contained, the negative score of 0.82 is multiplied by the abnormal weight of "timeout" of 1.05 to obtain an enhanced abnormal score of 0.861.

[0161] When boosting scores based on keywords, min(boosting anomaly score, 1) can be used to limit the boosting anomaly score to no more than 1.0 to avoid overflow;

[0162] If multiple abnormal keywords are included, when obtaining the enhanced abnormal score, you can multiply it by the abnormal weight corresponding to each abnormal keyword, or you can multiply it by only the largest abnormal weight.

[0163] Finally, anomaly is determined based on the enhanced anomaly score. An anomaly prediction threshold of 0.85 (or other values) is set. If the enhanced anomaly score is greater than the threshold, the normalized log text is marked as a potential anomaly. If it is less than or equal to the threshold, it is marked as not a potential anomaly. The enhanced anomaly score can be used as the confidence level of the normalized log text. After the judgment is completed, the result can be returned. The returned result can be dictionary data including the original negative score, enhanced anomaly score, whether it belongs to a potential anomaly, and confidence level.

[0164] In the above judgment methods, if a log entry does not contain abnormal keywords, it can be replaced with a negative score to determine whether it is abnormal, with the same judgment logic as above. Similarly, in other methods of this application that use enhanced abnormal scores, if the corresponding log entry does not contain abnormal keywords, then a negative score can be used instead, with the corresponding processing logic remaining unchanged.

[0165] An instance of the created AI Log Analyzer class can also execute the core method 3 defined in the AI ​​Log Analyzer class, namely the anomaly pattern detection method (detect_anomaly_patterns). The purpose of this method is to group semantically similar standardized log texts together using unsupervised clustering (such as DBSCAN), identify "rare small clusters" (new anomaly patterns), and solve the pain point of "unknown anomalies cannot be identified".

[0166] The implementation method for anomaly pattern detection is as follows.

[0167] The amount of data to be verified is the number of standardized log texts. If this number is less than 10 (or other preset thresholds), the abnormal pattern detection method will not be executed because the sample is insufficient and the result is meaningless. In this case, the clustering identifier of each log can be set to cluster=-1 to indicate no clustering, and the variable is_new_pattern representing rare small clusters of each log can be set to False to indicate that no new abnormal patterns have been found.

[0168] If the number of normalized log texts is greater than 10, log text vectorization can be performed.

[0169] Log text vectorization involves extracting the content of all logs and using the aforementioned TF-IDFVectorizer component to convert all log content into a TF-IDF vector matrix X. Each row corresponds to one log entry, and each column corresponds to the importance of a feature word in the log entry. Therefore, each row vector of X is equivalent to the log text vector converted from the corresponding log entry after log text vectorization. For methods of converting to a TF-IDF vector matrix, please refer to relevant existing technologies.

[0170] Unsupervised clustering can be achieved by using the DBSCAN component to cluster the log text vectors of all logs and return a cluster identifier for each log. The cluster identifier indicates which cluster each log belongs to. If two logs belong to the same cluster, it means that the two logs are semantically similar.

[0171] To identify rare clusters, count the number of samples in each cluster. For example, if a cluster with cluster identifier 0 has 50 logs, or if a cluster with 50 logs has a cluster identifier of 0, then the number of samples in the cluster with cluster identifier 0 is 50. Similarly, the number of samples in the cluster with cluster identifier 2 is 3. Clusters with a sample count less than the rare threshold of 5 are identified as rare clusters (rare_clusters). Logs from rare clusters are highly likely to represent new abnormal patterns. The rare threshold can also be set to other values, not limited to 5.

[0172] Finally, the clustering information for each log is returned. The clustering information for a log can include a cluster identifier (e.g., 0, 1, etc., indicating that it belongs to the cluster with cluster identifier 0 or the cluster with cluster identifier 1), and can also include a variable is_new_pattern that represents rare small clusters. If this variable is False, it means that the corresponding log does not belong to a rare small cluster, and if this variable is True, it means that the corresponding log belongs to a rare small cluster.

[0173] Once a rare small cluster is identified, all logs belonging to the rare small cluster can be output through the relevant operation and maintenance terminal for operation and maintenance personnel to view.

[0174] After executing core method 2, the instance of the created AI log analyzer class can continue to execute core method 4 defined in the AI ​​log analyzer class, namely the predict_future_issues method. The purpose of this method is to predict the time when future problems will occur based on the severity of the abnormal logs, providing a basis for early response for operations and maintenance.

[0175] The future problem prediction method can generate a corresponding prediction result for all logs other than INF and DEBUG levels that are potentially abnormal.

[0176] The implementation method of the future problem prediction method is as follows.

[0177] First, filter normal logs, that is, skip the INF and / DEBUG level logs and ignore logs that do not belong to potential exceptions, and only process the logs that belong to potential exceptions.

[0178] Then, check whether the enhanced anomaly score of logs marked as potential anomalies is greater than the corresponding threshold.

[0179] Next, the original timestamp (log_time) of logs belonging to potential anomalies is extracted, and the time offset (time_offset) of that log is calculated based on its enhanced anomaly score. This calculation can be expressed by the formula: time_offset = max(5, int(60 * (1 - enhanced score))), which defaults to predicting anomalies within the next 60 minutes. The original timestamp is then added to the time offset to obtain the predicted time: predicted time (predicted_time) = original timestamp + time offset. This predicted time indicates that the server node corresponding to the log may experience an anomaly at this time.

[0180] The principle behind the above formula is that the higher the enhancement anomaly score, the more serious the potential problem; the smaller the enhancement anomaly score (1 - enhancement anomaly score), the smaller the time offset, and the earlier the predicted occurrence time. For example, an enhancement anomaly score of 0.98 corresponds to a time offset of 5 minutes, and an enhancement anomaly score of 0.86 corresponds to a time offset of 8 minutes.

[0181] The severity is determined based on the confidence level (i.e., the enhanced anomaly score). If the enhanced anomaly score is greater than the high threshold, for example, greater than 0.95, the severity is determined to be high. If it is less than or equal to the high threshold, the severity is determined to be medium.

[0182] Finally, prediction results are generated. A prediction dictionary containing fields such as server node identifier, prediction time, source, message, severity, and confidence level is constructed and added to the prediction result list. The prediction result list can then be deduplicated, returning a deduplicated list. The source in the prediction results can include logs from the corresponding server node that belong to potential anomalies. Multiple prediction results in the returned list can be sorted in ascending order by prediction time, and the message in the prediction results can characterize the specific type of anomaly that occurred. The anomaly type can be determined based on the corresponding log; for example, for logs related to database connections, the message in the corresponding prediction result could indicate the database connection type.

[0183] For example, a prediction result indicates that server node A will experience an anomaly at the predicted time T1. The message of this prediction result may include logs from server node A that belong to potential anomalies.

[0184] Methods for deduplicating multiple predictions in the prediction results list can include: retaining the duplicate prediction with the highest confidence; defining the server node's identifier and the first 50 characters of the log as the deduplication key (similar issues on the same server are considered duplicates); if two predictions correspond to the same deduplication key, it means that they are duplicates, and the prediction with the higher confidence (or higher severity) is retained.

[0185] The above prediction results are equivalent to the second result of the aforementioned embodiment.

[0186] The AI ​​log analyzer class can include batch analysis (batch_analyze) as the entry method. By calling the entry method, all core methods are chained together to output structured final results, including analysis details, prediction lists, statistical summaries, etc.

[0187] When the entry method is called, the start time can be recorded (for calculating the time taken).

[0188] Call core method 1 to preprocess the raw log and output the preprocessed log;

[0189] Invoke the abnormal pattern discovery to perform clustering, and discover new abnormal patterns, namely rare small clusters;

[0190] Traverse the clustered logs, call core method 2 to calculate the enhanced anomaly score for each log entry to determine whether each log entry is a potential anomaly, and update the log dictionary accordingly;

[0191] Call core method 4 to generate a list of prediction results, which includes several secondary results;

[0192] Statistical key indicators, including but not limited to the total number of logs processed, the number of potentially abnormal logs, the number of new abnormal patterns, and the number of predictions;

[0193] Output the analysis completion log, which includes the total time taken to execute the above processes;

[0194] Returns a final results dictionary containing a detailed analysis of each log entry, a list of predicted results, and a statistical summary.

[0195] As an example, suppose we construct 3 sample log entries (including ERROR, WARNING, and INFO levels).

[0196] Initialize an instance of the AI ​​log analyzer class, analyzer;

[0197] Call the entry method to analyze logs in batches and predict issues for the next 60 minutes;

[0198] Iterate through the list of prediction results and print out the prediction time, severity, and message for each second result. The printed information may include the following:

[0199] INFO level logs are filtered, while ERROR and WARNING logs are identified as potential anomalies, generating two prediction results, such as "database connection timeout in 5 minutes" and "memory usage too high in 8 minutes".

[0200] It should be noted that not every log entry will generate a corresponding prediction result. Only log entries at a specified level, such as non-INFO and DEBUG levels, and with an enhanced anomaly score greater than a certain threshold (such as 0.85), may generate prediction results. Log entries that do not meet these conditions will not generate prediction results.

[0201] For example, if 100 logs are collected, 20 of which are ERROR and WARNING, and the enhanced anomaly score meets the standard, 20 prediction results will be generated, which may contain a large number of duplicates (e.g., 15 logs are the same type of database timeout logs).

[0202] The central data storage service MongoDB (a document-oriented database) stores different types of data through collections. The core table structure is as follows:

[0203] Server node information table (servers): Stores basic information and status of the servers;

[0204] Monitoring metrics data table: Stores the server's real-time monitoring metrics (by time series).

[0205] Server logs table (logs): Stores system and application logs;

[0206] Anomalies table: Stores anomaly records identified by the anomaly detection service;

[0207] Alerts log: Stores alerts generated by the alert management service;

[0208] Prediction results table: Stores future warning predictions generated by the AI ​​prediction service, i.e., the prediction results mentioned above;

[0209] The healing_operations table stores the operations performed by the self-healing service.

[0210] This application provides a complete implementation and design scheme for lightweight AIOps.

[0211] This method includes a dynamic training and adaptation method for personalized anomaly detection models for server nodes. This method can independently build anomaly detection models (such as isolated forests) for each server node in the cluster and automatically update the model parameters periodically based on the real-time load characteristics of the nodes (CPU, memory load characteristics). By comparing the historical baseline of the nodes with the real-time indicators, differentiated anomaly identification is achieved instead of a uniform threshold or a global model.

[0212] The solution also includes a hybrid alarm prediction mechanism that integrates indicator trends and log semantics, combining two prediction paths to form a collaborative logic, predicting future resource overrun times based on slope analysis of historical monitoring indicators, and extracting potential fault signals from error / warning information based on semantic parsing of server log text using a pre-trained Transformer model.

[0213] The solution also includes a tiered automatic repair and access control method based on fault severity. It matches progressive repair strategies according to the severity of the fault (high / medium / low), binds repair operations to role permissions, and requires secondary confirmation and audit log recording for high-risk operations. This strikes a balance between automation efficiency and system security, avoiding the risk of accidental misoperation.

[0214] The solution also implements a fault-tolerant and adaptive transmission mechanism for lightweight data acquisition agents. The acquisition agents deployed on the nodes have three main capabilities: temporarily storing monitoring data in the local cache when the network is interrupted and resuming transmission after the network is restored; dynamically adjusting the acquisition frequency based on the node load (reducing the frequency to reduce resource consumption when the load is high); and lightweight compression of the acquired data (transmitting only key features to reduce bandwidth consumption).

[0215] The solution adopts an intelligent operation and maintenance closed-loop architecture with distributed service collaboration. It defines standardized interfaces and triggering logic for "data collection → central storage → anomaly detection → alarm management → automatic repair → cluster scheduling". Each service achieves loosely coupled collaboration through API (e.g., the anomaly detection service automatically triggers alarms, the alarm service calls the repair service according to rules, and the repair service links with cluster management to achieve cross-node scheduling).

[0216] The operation and maintenance system proposed in this application optimizes the process and improves the efficiency of traditional operation and maintenance work from multiple dimensions such as technical architecture, intelligent capabilities, and operation and maintenance efficiency, and its specific advantages are as follows.

[0217] First, it adopts a distributed architecture design, which is suitable for large-scale clusters. The system's six core service components are decoupled and can be deployed independently. Each service can be deployed, scaled up, or updated independently, avoiding single points of failure and supporting cluster sizes from dozens to thousands of servers.

[0218] Second, AI-driven intelligent operation and maintenance capabilities enable real-time anomaly detection (in-process response) and the ability to identify metrics (CPU, memory, disk, etc.) that deviate from normal patterns without needing to label abnormal data. This is suitable for scenarios in server monitoring where "abnormal patterns are diverse and difficult to enumerate".

[0219] Third, the self-repair function is controllable. The self-repair service automatically performs repair operations based on the type of fault or anomaly. It is based on role-based access control, and the entire repair process is recorded, supporting auditing and backtracking.

[0220] The key to this application lies in integrating AI capabilities into traditional operation and maintenance processes, solving the pain points of server cluster operation and maintenance through a closed loop of "real-time detection - intelligent prediction - automatic repair". At the same time, the distributed architecture and modular design ensure the scalability of the system, enabling smooth expansion as the cluster size grows, making it suitable for diverse scenarios ranging from small and medium-sized data centers to large cloud service clusters.

[0221] Based on the above embodiments, this application also provides an intelligent operation and maintenance method, please refer to [link to relevant documentation]. Figure 3 The method may include the following steps.

[0222] S301, obtain the log text of the server node.

[0223] S302, Process log texts using a pre-trained log sentiment model to obtain negative scores for the log texts.

[0224] S303, determine the potential anomaly detection results of the log text based on the negative score, and determine whether the server node will experience an anomaly within a preset time period based on the potential anomaly detection results.

[0225] If the negative score or the enhanced anomaly score determined based on the negative score is greater than the preset anomaly prediction threshold (e.g., 0.85), it can be determined that the potential anomaly detection result of the log text belongs to the potential anomaly, and thus it can be determined that the server node corresponding to the log text will be abnormal within a preset time period in the future.

[0226] If the negative score or the enhanced anomaly score determined based on the negative score is less than or equal to the preset anomaly prediction threshold, it can be determined that the potential anomaly detection result of the log text does not belong to the potential anomaly, and thus it can be determined that the server node corresponding to the log text will not experience anomalies within the preset time period in the future.

[0227] For details on the specific implementation of the above methods, please refer to the process of determining whether each standardized log text is a potential anomaly in the log sentiment and anomaly analysis method of the foregoing embodiments, which will not be repeated here.

[0228] The preset duration for the future could be, for example, 60 minutes or other durations, without limitation. The above log text is equivalent to the standardized log text in the foregoing embodiments.

[0229] The beneficial effect of this solution is that it obtains negative scores for log text based on the log sentiment model, and predicts whether anomalies will occur on the corresponding server node within a preset time period based on the negative scores. This allows for the early detection of potential anomalies, making it easier for maintenance personnel to take timely countermeasures.

[0230] Optionally, the potential anomaly detection results of the log text can be determined based on the negative score, including:

[0231] Identify whether the log text contains preset abnormal keywords;

[0232] In the case of abnormal keywords, the negative score is multiplied by the preset abnormal weight corresponding to the abnormal keyword to obtain the enhanced abnormal score of the log text.

[0233] The negative score determines the potential anomaly detection results of the log text.

[0234] For details on the specific implementation of the above methods, please refer to the process of determining the enhanced anomalous score in the log sentiment and anomaly analysis method of the foregoing embodiments, which will not be repeated here.

[0235] Optional, also includes:

[0236] If a server node will experience an anomaly within a preset time period in the future, the time offset will be determined based on the enhanced anomaly score. The time offset and the enhanced anomaly score are negatively correlated.

[0237] The predicted time is determined based on the original timestamp and time offset corresponding to the log text.

[0238] Output the prediction results based on the prediction time.

[0239] For specific implementation methods of the above embodiments, please refer to the implementation method of the future problem prediction method in the foregoing embodiments, which will not be repeated here.

[0240] Optional, also includes:

[0241] Multiple log texts are clustered to identify and output log texts belonging to rare small clusters, where a rare small cluster refers to a cluster containing fewer than a preset rare threshold.

[0242] The implementation methods of the above embodiments can be found in the implementation methods of the abnormal pattern discovery method in the foregoing embodiments, and will not be repeated here.

[0243] Optional, also includes:

[0244] For each server node, an isolated forest model is built based on the monitoring metrics data of the server node, in order to detect whether the server node is currently experiencing any anomalies.

[0245] For a detailed implementation of the above embodiments, please refer to the process of obtaining the first result by the anomaly detection and prediction service in the foregoing embodiments. The first result obtained is the result that characterizes whether an anomaly has occurred in the server node.

[0246] This application also provides an intelligent operation and maintenance system; please refer to [link / reference]. Figure 2 The system may include the following units.

[0247] Unit 201 is used to obtain the log text of the server node;

[0248] Processing unit 202 is used to process log text according to a pre-trained log sentiment model to obtain a negative score for the log text;

[0249] The determining unit 203 is used to determine the potential anomaly detection result of the log text based on the negative score, so as to determine whether the server node will experience an anomaly within a preset time period in the future based on the potential anomaly detection result.

[0250] Optionally, the determining unit 203 determines the potential anomaly detection results of the log text based on the negative score, including:

[0251] Identify whether the log text contains preset abnormal keywords;

[0252] In the case of abnormal keywords, the negative score is multiplied by the preset abnormal weight corresponding to the abnormal keyword to obtain the enhanced abnormal score of the log text.

[0253] The negative score determines the potential anomaly detection results of the log text.

[0254] Optionally, the determining unit 203 is also used for:

[0255] If a server node will experience an anomaly within a preset time period in the future, the time offset will be determined based on the enhanced anomaly score. The time offset and the enhanced anomaly score are negatively correlated.

[0256] The predicted time is determined based on the original timestamp and time offset corresponding to the log text.

[0257] Output the prediction results based on the prediction time.

[0258] Optionally, the system also includes a discovery unit for:

[0259] Multiple log texts are clustered to identify and output log texts belonging to rare small clusters, where a rare small cluster refers to a cluster containing fewer than a preset rare threshold.

[0260] Optionally, the system also includes a detection unit for:

[0261] For each server node, an isolated forest model is built based on the monitoring metrics data of the server node, in order to detect whether the server node is currently experiencing any anomalies.

[0262] The working principle of the intelligent operation and maintenance system in this embodiment can be found in the foregoing embodiments, and will not be repeated here.

[0263] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0264] For ease of description, the above systems or devices are described separately as various modules or units based on their functions. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components.

[0265] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0266] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0267] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An intelligent operation and maintenance method, characterized in that, include: Obtain the log text of the server node; The log text is processed using a pre-trained log sentiment model to obtain a negative score for the log text; The potential anomaly detection result of the log text is determined based on the negative score, and the server node is then used to determine whether an anomaly will occur within a preset time period in the future.

2. The method according to claim 1, characterized in that, The step of determining the potential anomaly detection result of the log text based on the negative score includes: Identify whether the log text contains preset abnormal keywords; In the case of the abnormal keyword, the negative score and the preset abnormal weight corresponding to the abnormal keyword are multiplied to obtain the enhanced abnormal score of the log text; The potential anomaly detection results of the log text are determined based on the negative score.

3. The method according to claim 2, characterized in that, Also includes: If the server node will experience an anomaly within a preset time period in the future, a time offset is determined based on the enhanced anomaly score, and the time offset is negatively correlated with the enhanced anomaly score. The predicted time is determined based on the original timestamp corresponding to the log text and the time offset. Output the prediction result based on the prediction time.

4. The method according to claim 1, characterized in that, Also includes: Multiple log texts are clustered to identify and output log texts belonging to rare small clusters, where a rare small cluster refers to a cluster containing fewer than a preset rare threshold.

5. The method according to claim 1, characterized in that, Also includes: For each server node, an isolated forest model is established based on the monitoring metrics data of the server node, so as to detect whether the server node is currently experiencing anomalies.

6. An intelligent operation and maintenance system, characterized in that, include: The acquisition unit is used to obtain the log text of the server node; The processing unit is used to process the log text according to a pre-trained log sentiment model to obtain a negative score for the log text. The determining unit is used to determine the potential anomaly detection result of the log text based on the negative score, so as to determine whether the server node will experience an anomaly within a preset time period based on the potential anomaly detection result.

7. The system according to claim 6, characterized in that, The determining unit determines the potential anomaly detection result of the log text based on the negative score, including: Identify whether the log text contains preset abnormal keywords; In the case of the abnormal keyword, the negative score and the preset abnormal weight corresponding to the abnormal keyword are multiplied to obtain the enhanced abnormal score of the log text; The potential anomaly detection results of the log text are determined based on the negative score.

8. The system according to claim 7, characterized in that, The determining unit is further configured to: If the server node will experience an anomaly within a preset time period in the future, a time offset is determined based on the enhanced anomaly score, and the time offset is negatively correlated with the enhanced anomaly score. The predicted time is determined based on the original timestamp corresponding to the log text and the time offset. Output the prediction result based on the prediction time.

9. The system according to claim 6, characterized in that, The system also includes a detection unit for: Multiple log texts are clustered to identify and output log texts belonging to rare small clusters, where a rare small cluster refers to a cluster containing fewer than a preset rare threshold.

10. The system according to claim 6, characterized in that, The system also includes a detection unit for: For each server node, an isolated forest model is established based on the monitoring metrics data of the server node, so as to detect whether the server node is currently experiencing anomalies.

Images