Delimitation problem analysis method and system, electronic equipment and storage medium
By building a business topology diagram and a decision tree model using the C4.5 algorithm, the problem of insufficient automation delimitation capabilities in the existing technology is solved, and the automated analysis of delimitation problems of cross-cloud platform business applications is realized, which improves operation and maintenance efficiency and system stability.
Patent Information
- Application Number
- CN202510279126.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-11
- Publication Date
- 2025-04-08
- Estimated Expiration
- 2045-03-11
AI Technical Summary
When facing the complex and changeable cross-cloud platform business application delimitation problem, the existing technology relies on expert experience and traditional tools to have limitations such as insufficient automation delimitation capabilities, data silos problems and fixed location query restrictions.
By collecting data from multiple sources, building business topology diagrams, extracting exception nodes and features, and using the C4.5 algorithm to build a decision tree model to realize automated delimited problem analysis.
It realizes automated analysis of cross-cloud platform business application delimitation problems, improves operation and maintenance efficiency and system stability, and provides clear operation guidance and automation assistance delimitation for the operation and maintenance team.
Smart Images

Figure CN119788496A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of delimitation problem analysis, and in particular to a delimitation problem analysis method, system, electronic equipment and storage medium. Background Art
[0002] With the acceleration of enterprise digital transformation, many businesses are applied to multiple public and private cloud services. Monitoring and demarcating these complex and heterogeneous cloud environment business issues has become a severe challenge. The inherent differences between different cloud platforms not only easily lead to performance bottlenecks and data link dispersion problems, but also make it difficult to unify and interpret monitoring data. Faced with this complex situation, there is an urgent need to automatically analyze and demarcate various problems, and to provide standardized troubleshooting logic to assist the operation and maintenance team, so as to ensure the efficiency and consistency of demarcation problems of cross-cloud platform business applications.
[0003] Currently, problem demarcation mainly relies on the experience of experts and the support of a series of traditional tools. Use the Prometheus monitoring tool to monitor various performance indicators to ensure that abnormalities in the business application and machine indicator layers are captured. Troubleshoot and analyze log data, use log collection and analysis tools such as the EFK stack (Fluentd, Elasticsearch, Kibana), perform detailed data mining at fixed query locations, and preliminarily demarcate the fault. In addition, APM (application performance management) tools are widely used to track the request links of business applications, thereby accurately locating the fault link. For network-level problems, NPM (network performance monitoring) tools are usually used for troubleshooting.
[0004] The current fault demarcation process relies on a series of traditional tools and techniques, which, while adequate to a certain extent, also have several limitations.
[0005] Rely on expert experience Although the Prometheus monitoring tool can effectively monitor various performance indicators and capture abnormal situations at the business application and machine indicator layers, the final problem demarcation and solution still highly rely on the experience and judgment of experts. This not only increases labor costs, but also may not be flexible enough when facing complex and changing problems.
[0006] Fixed location query limit When using the EFK stack (Fluentd, Elasticsearch, Kibana) for log data mining, it is usually limited to fixed query locations. Although this approach helps with initial demarcation, it is inadequate when dealing with complex cross-system and cross-platform problems and it is difficult to provide a comprehensive perspective.
[0007] Lack of automated delimitation capabilities Although APM (application performance management) tools can track the request links of business applications and achieve accurate positioning, they still require manual intervention for further analysis of frequently occurring or complex related issues. Similarly, NPM (network performance monitoring) tools perform well in troubleshooting network problems, but their degree of automation is limited and cannot completely replace manual decision-making.
[0008] Data silo problem The data generated by different tools often exist in isolated environments and lack effective integration mechanisms, which leads to problems such as data dispersion, duplication of work, and information lag, affecting overall operation and maintenance efficiency. Summary of the invention
[0009] In view of the shortcomings existing in the above problems, the present invention provides a delimitation problem analysis method, system, electronic device and storage medium.
[0010] To achieve the above object, the present invention provides a delimitation problem analysis method, comprising: Collect data from different sources, including resource indicators, business application performance indicators, network status and network call indicators, and key events in log records; A service topology map is constructed based on the call indicators, the resource indicators are mapped to corresponding nodes of the service topology map, the service application performance indicators are superimposed on the service links of the service topology map, and the key time is marked on the relevant nodes of the service topology map; A threshold is set for each of the data, and an event exceeding the threshold is considered an abnormal event; Based on the abnormal event, extracting abnormal nodes from the service topology graph; Extracting abnormal features based on the abnormal nodes; Extracting numerical features from the abnormal features and supplementing the numerical features of missing values; Based on the numerical features, a decision tree model for anomaly demarcation is constructed using the C4.5 algorithm; After the training of the decision tree model is completed, the features to be input are input into the decision tree model to delimit abnormal events.
[0011] Preferably, the resource indicators include CPU utilization, memory utilization and disk IO indicator utilization; the business application performance indicators include response time and error rate; the network status and network call indicators include network traffic and delay.
[0012] Preferably, the label and type are set accordingly according to the root cause of the abnormal event.
[0013] Preferably, extracting abnormal nodes from the service topology graph includes: Nodes whose request volume, error rate or response time exceeds the threshold are monitored and set as abnormal nodes; Based on the label and the type, identifying the type of the abnormal node; All the abnormal nodes are extracted from the service topology graph and classified to form subsets.
[0014] Preferably, extracting abnormal features based on the abnormal node includes: Extract the data within a unified time period, calculate the mean and maximum values within the time period, and count the characteristics; Analyze the frequency of specific events over a period of time; Aggregate according to the service instance dimension and calculate summary statistics at the service instance dimension.
[0015] Preferably, constructing a decision tree model for outlier delimitation by using the C4.5 algorithm includes: The C4.5 algorithm uses the information gain ratio to select the optimal split attribute, and the formula is: ; Where: S is the current dataset; A is the attribute to be evaluated; Gain(S, A) represents information gain, which measures the importance of an attribute A in the classification task S; Intrinsic Value(A) represents the calculation of intrinsic value, which calculates the intrinsic value of attribute A and reflects the entropy of the distribution of different values in attribute A.
[0016] Preferably, the information gain is calculated as follows: ; ; Where: Values(A) represents all possible values of attribute A; Sv is the subset of attribute A in the dataset whose value is v; |S| and |Sv| represent the number of samples in dataset S and subset Sv respectively; Entropy(S) is the entropy of dataset S; P i is the probability of the i-th category; The calculation formula of the inherent value is: ; The present application also provides a delimited problem analysis system, comprising: A collection module is used to collect data from different sources, including resource indicators, business application performance indicators, network status and network call indicators, and key events in log records; A construction module, used to construct a business topology map based on the call indicators, the resource indicators are mapped to the corresponding nodes of the business topology map, the business application performance indicators are superimposed on the service links of the business topology map, and the key time is marked on the relevant nodes of the business topology map; A threshold is set for each of the data, and an event exceeding the threshold is considered an abnormal event; An extraction module, used for extracting abnormal nodes from the service topology diagram based on the abnormal event; A feature module, used for extracting abnormal features based on the abnormal nodes; A supplementing module, used to extract numerical features from the abnormal features and supplement the numerical features of missing values; A decision tree module, used to construct a decision tree model for anomaly demarcation based on the numerical features by using the C4.5 algorithm; The delimiting module is used to input the features to be input into the decision tree model to delimit abnormal events after the training of the decision tree model is completed.
[0017] The present invention also provides an electronic device, comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit executes the above method.
[0018] The present invention also provides a storage medium storing a computer program executable by an electronic device, and when the program runs on the electronic device, the electronic device executes the above method.
[0019] Compared with the prior art, the present invention has the following beneficial effects: The present invention integrates decision tree algorithm and multi-source data collection technology to automatically delimit problem analysis, and quickly delimits specific problems when anomalies are detected, such as application performance bottlenecks, network connection failures, or host resource overloads, thereby improving operation and maintenance efficiency and system stability, and providing clear operation guidance and automated assisted delimitation for the operation and maintenance team. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 It is a flow chart of the delimited problem analysis method of the present invention. DETAILED DESCRIPTION
[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0022] Reference Figure 1 The present invention provides a method for analyzing a delimited problem, comprising: Collect data from different sources, including resource indicators, business application performance indicators, network status and network call indicators, and key events in log records; Build a business topology map based on call indicators, map resource indicators to corresponding nodes in the business topology map, overlay business application performance indicators on the service links in the business topology map, and mark key times on relevant nodes in the business topology map; Set a threshold for each data, and events exceeding the threshold are considered abnormal events; Based on abnormal events, abnormal nodes are extracted from the business topology graph; Extract abnormal features based on abnormal nodes; Extract numerical features from abnormal features and supplement the numerical features of missing values; Based on numerical features, a decision tree model for anomaly demarcation is constructed using the C4.5 algorithm; After the decision tree model training is completed, the features to be input are input into the decision tree model to delimit abnormal events.
[0023] In this embodiment, resource indicators include CPU utilization, memory utilization and disk IO utilization; business application performance indicators include response time and error rate; network status and network call indicators include network traffic and delay.
[0024] Furthermore, each node in the business topology graph represents a specific resource or service instance, while the edges represent the interactions and dependencies between them. Based on the call topology graph, the multi-dimensional data is integrated: A real-time updated business topology map is generated based on the network call data collected from multiple dimensions to build a business topology map. For example, if business application A accesses business application B through the network within 1 hour, and then accesses business application C, a dynamic business topology map A -> B -> C is obtained, and all related services, components and their call relationships are covered through the topology map.
[0025] Map indicators (resource indicators) such as CPU usage, memory usage, and disk IO usage to corresponding nodes, and intuitively display the health status of resources through color or icon changes.
[0026] Indicators such as response time and error rate (business application performance indicators) are superimposed on the service link to identify performance bottlenecks. At the same time, indicators such as network traffic and latency (network status and network call indicators) are integrated into the link line.
[0027] After the log data is parsed, the key events in the log records are directly marked on the relevant nodes, which assists in the display of log information after the fault problem is demarcated.
[0028] After being collected and processed in a standardized manner, these data are mapped to the corresponding topology map nodes to form a multi-dimensional monitoring view.
[0029] In this embodiment, a baseline within the normal operating range is defined for each data indicator, and a reasonable threshold is set. Indicator events that exceed the threshold are considered abnormal events. For example, the threshold of the business node memory usage is 90%. Business nodes with memory usage exceeding 90% are considered abnormal events. After preprocessing the collected abnormal data, the root cause of each abnormal event should be marked as the target variable in the training set, i.e., the "label". The list of labels corresponding to the baseline thresholds of various indicators is set, see Table 1.
[0030] Table 1
[0031] Furthermore, when an anomaly is detected, the system automatically extracts all nodes marked as abnormal from the topology map to form an independent subset of sub-indicator types for in-depth analysis. Different types of abnormal nodes (such as application performance indicators, network indicator data, and resource indicator data) are classified and processed according to their data sources and labels, and the data of the same indicator is stored in a subset.
[0032] The following are the specific steps for generating abnormal node subsets for different types of data: Identify abnormal nodes Nodes whose request volume, error rate, or response time exceeds the set threshold are set as abnormal nodes.
[0033] Tag Matching According to the pre-defined types and labels (such as anomaly_request_volume, anomaly_error_rate, anomaly_response_time), the categories of these abnormal nodes are identified.
[0034] Extract Node Extract nodes for classification, extract all nodes marked as abnormal from the topology map, and then form subsets by classifying the same indicator.
[0035] In this embodiment, features useful for judging abnormalities are extracted, such as the average response time, peak CPU utilization, memory usage, and average inbound and outbound traffic indicators of the retained calculations. Different indicator types are extracted by performing feature value extraction in a specified manner, as follows: Method for extracting characteristic values of resource indicators Time series analysis: extract the data of CPU usage and memory usage (resource indicators) in the same time period, calculate the mean and maximum values in the time period, and count their features. For example, feature: average CPU usage of the current node within 1 hour.
[0036] Identify abnormal event frequency: Analyze the frequency of specific events within a period of time (such as the CPU usage index exceeding the average CPU usage index within 1 hour) and identify abnormal index information.
[0037] Aggregation and grouping: Aggregate by service instance dimension and calculate summary statistics under the service instance dimension. Example feature values: features such as average CPU usage, average memory usage, and average disk I / O utilization of abnormal nodes.
[0038] Method for extracting characteristic values of application performance indicators Time series analysis: extract the data of the same time period of indicators such as response time and error rate (business application performance indicators), calculate the mean and maximum values in the time period, and count their characteristics. For example, the feature is the average response time of the current node within 1 hour.
[0039] Identify abnormal event frequency: Analyze the frequency of specific events within a period of time (such as the indicator data of a single response time indicator exceeding the average response time within 1 hour) and identify abnormal indicator information.
[0040] Aggregation and grouping: Aggregate by service instance dimension and calculate summary statistics under the service instance dimension. Example feature value: average response time of the same instance.
[0041] Method for extracting characteristic values of network status indicators Time series analysis: extract the data of the average round-trip time and other indicators (network status indicators) in the same time period, calculate the mean and maximum values in the time period, and count their characteristics. For example, the feature is the average round-trip time indicator of the current node within 1 hour.
[0042] Identify abnormal event frequency: Analyze the frequency of specific events within a period of time (such as the indicator data of a single round-trip time exceeding the average round-trip time within 1 hour) and identify abnormal indicator information.
[0043] Aggregation and grouping: Aggregate by service instance dimension and calculate summary statistics under the service instance dimension. Example feature values: feature values such as average round-trip time usage of abnormal nodes and TCP connection success rate.
[0044] In this embodiment, a decision tree for delimiting abnormal indicators is constructed by the C4.5 algorithm, and the decision tree model is trained using training set data to ensure that the model can accurately delimit abnormal problems based on input features.
[0045] Algorithm implementation: Data preparation: Use the collected abnormal events to ensure that the dataset contains rich features, such as CPU utilization, memory usage, response time, etc.
[0046] Preprocessing: Fill in the missing feature values of abnormal events and extract numerical features. Filling in missing feature values is an important step in data preprocessing, especially when dealing with abnormal events. For extracting numerical features and filling in missing values, mean filling technology can be used.
[0047] The following are detailed strategies and methods for filling missing values: Get the raw sample data # Example data data = {'cpu_usage': [None, 0.8, 0.9, 0.6, 0.5], 'memory_usage': [0.7, 0.8, None, 0.5, 0.4]} Calculate the mean of the original example data cpu_avg: 0.7 memory_avg: 0.6 Fill in missing values in the original sample data # Completed sample data data = {'cpu_usage': [ 0.7, 0.8, 0.6, 0.9, 0.5], 'memory_usage': [0.7, 0.8, 0.6, 0.5, 0.4]} Select the best split attribute: The C4.5 algorithm uses the information gain ratio to select the best split attribute. The information gain ratio is the information gain divided by the intrinsic value of the attribute, which is used to penalize attributes with a large number of values, thereby avoiding the bias of selecting attributes with more possible values.
[0048] Calculate information gain: Information gain (IG) is a key concept used to measure the importance of an attribute in a classification task. It is based on the concept of entropy in information theory and is used to evaluate how much uncertainty can be reduced by introducing a certain attribute. Specifically, information gain reflects how much the purity of the category distribution is improved after using a certain attribute to divide the data set.
[0049] ; ; Where: Values(A) represents all possible values of attribute A; Sv is the subset of attribute A in the dataset whose value is v; |S| and |Sv| represent the number of samples in dataset S and subset Sv respectively; Entropy(S) is the entropy of dataset S; P i is the probability of the i-th category; Calculate Intrinsic Value: Calculate the intrinsic value of attribute A, which reflects the entropy of the distribution of different values in attribute A.
[0050] ; The C4.5 algorithm uses the information gain ratio to select the optimal split attribute. The formula is: ; Where: S is the current dataset; A is the attribute to be evaluated; Gain(S, A) represents information gain, which measures the importance of an attribute A in the classification task S; Intrinsic Value(A) represents the calculation of intrinsic value, which calculates the intrinsic value of attribute A and reflects the entropy of the distribution of different values in attribute A. Example
[0051] Suppose we have a data set containing several log entries, each of which records CPU usage and other related information, and is marked as abnormal ("normal" or "abnormal"). The information gain of the C4.5 algorithm will be used to evaluate the importance of CPU usage as a split attribute.
[0052] The example dataset is shown in Table 2: Table 2
[0053] Calculate the original entropy (Entropy of the original dataset) First, we need to calculate the entropy of the entire data set. Suppose there are n samples in the data set, of which p are "normal" and q are "abnormal".
[0054] ; For example, if there are 3 "normal" and 2 "anomaly" in the dataset: ; Select split points and calculate entropy of each subset For continuous value attributes such as CPU usage, we need to choose an optimal split point. Suppose we choose a CPU usage of 0.7 as the split point, dividing the data set into two parts: CPU usage ≤ 0.7: includes samples with CPU usage of 0.7, 0.6, and 0.5.
[0055] CPU usage > 0.7: includes samples with CPU usage of 0.8 and 0.9.
[0056] Subset 1 (CPU usage ≤ 0.7), see Table 3.
[0057] Table 3
[0058] There are 3 samples in this subset, all of which are "normal". Therefore, its entropy is 0 (completely pure).
[0059] Entropy(S≤0.7)=0 Subset 2 (CPU usage > 0.7), see Table 4.
[0060] Table 4
[0061] Entropy(S > 0.7)=0 Calculate weighted average entropy Calculate the weighted average entropy according to the proportion of each subset: ; Calculate information gain Finally, calculate the information gain: Gain(S,CPU usage)=Entropy(S)−Weighted Entropy Gain(S,CPU usage)=0.971−0=0.971 Calculating Intrinsic Value (Split Information) Based on the above split, we can calculate the intrinsic value SplitInfo(A): ; In this example, attribute A (i.e., CPU usage) is divided into two subsets with a total number of samples |S| = 5: S ≤ 0.7 includes 3 samples; S > 0.7 includes 2 samples; ; Calculate information gain ratio The information gain Gain(S,A) has been calculated to be 0.971, and the information gain ratio can be calculated: ; In this example, using CPU usage as the splitting attribute, the information gain is 0.971, which shows that partitioning by CPU usage significantly reduces uncertainty and makes the subsets purer. When using CPU usage as the splitting attribute, the intrinsic value SplitInfo(A) is about 0.971. This shows that although partitioning by CPU usage introduces a certain degree of complexity, because the information gain is also very high, the information gain ratio is close to 1, indicating that this is a very effective splitting attribute.
[0062] Processing continuous values: For continuous value attributes, C4.5 will try to find an optimal threshold to split the data. This process is called binary splitting. It traverses all possible threshold points and selects the one that maximizes the information gain as the split point. The binary splitting method is implemented in the following steps: Sorting: Continuous valued attributes are sorted first.
[0063] Traversal: Then traverse the sorted values to find the best split point. For each potential split point, calculate the information gain of the two subsets after the split.
[0064] Select the best split point: Select the split point with the largest information gain as the final split condition.
[0065] Building a decision tree Recursive splitting: Starting from the root node, select the best splitting attribute according to the above rules to create child nodes. For each child node, repeat this process until the stopping condition is met (such as all instances belong to the same class, there are no more attributes to choose from, etc.).
[0066] Pruning: To prevent overfitting, C4.5 introduces a post-pruning strategy. Pruning can simplify the tree structure by manually removing unnecessary branches while maintaining classification performance.
[0067] Model evaluation and optimization: Manual validation techniques were used to evaluate the performance of the model, and parameters were manually adjusted to optimize model performance, further enhancing the stability and accuracy of a single decision tree.
[0068] Apply the model for diagnosis: When a new anomaly is detected, the corresponding features are extracted and input into the trained decision tree model. The model will start from the root node along the tree structure, make judgments based on the feature values, and finally reach a leaf node. The label carried by the leaf node is the predicted root cause of the anomaly.
[0069] Continuous Improvement: As new data arrives, the training set is continuously updated and the model is retrained regularly to adapt to system changes and maintain the effectiveness of the model.
[0070] Automatic delimitation of decision tree models When a new abnormal event is detected, the corresponding features are extracted and input into the decision tree model. The model will start from the root node along the tree structure, make judgments based on the feature values, and finally reach a leaf node. The label carried by the leaf node is the predicted abnormal root cause. The automatic delimitation process of the decision tree model abnormal problem is as follows: Initial check of business application resources: The decision tree model starts by evaluating the key resource indicators of business applications, including CPU usage, memory usage, and disk I / O indicators. These resource indicators directly reflect the basic environment status when the application is running. If the test results show that these resources are all within the normal range, the problem caused by insufficient resources is ruled out and in-depth analysis is continued; conversely, if resource anomalies are found, they are immediately located as caused by resource bottlenecks.
[0071] Analyze network and application performance during slow request periods: Next, the decision tree model focuses on the network and application performance within 1 minute before and after the slow request occurs. Specifically, it checks whether there is a failure to establish a connection for backward requests (network indicators) or a continuous slow request (business application performance indicators). If significant network connection problems or performance degradation are found during this period, it will be demarcated as being caused by inefficient network or application internal processing; otherwise, continue to the next step of analysis.
[0072] Detecting garbage collection activity: The decision tree model further checks whether full GC (full garbage collection) occurs before and after the slow request period. Full GC is a memory management operation in Java applications that may cause the application to suspend execution, thereby affecting the response time. If a full GC event is confirmed, the user is prompted that the performance problem may be caused by improper memory management; otherwise, continue to explore other possibilities.
[0073] Evaluate the resource status of the host where the application is located: Finally, the decision tree model turns to checking the overall resources of the host hosting the application, and re-examines the use of CPU, memory, and disk I / O. If the host resources are abnormal, such as overload or saturation, it is attributed to the problem at the host level; if the host resources are normal, it is ultimately determined that the problem originates from the logic or configuration defects of the application itself.
[0074] Through the above hierarchical and progressive decision path, the model can not only accurately locate the cause of business application anomalies, but also provide clear operation guidance for the operation and maintenance team, thereby accelerating fault recovery and improving system stability and reliability. This approach ensures effective monitoring and rapid response to complex and changing application environments.
[0075] The present application also provides a delimited problem analysis system, comprising: The collection module is used to collect data from different sources, including resource indicators, business application performance indicators, network status and network call indicators, and key events in log records; A construction module is used to construct a business topology map based on call indicators. Resource indicators are mapped to corresponding nodes of the business topology map. Business application performance indicators are superimposed on the service links of the business topology map. Key times are marked on relevant nodes of the business topology map. Set a threshold for each data, and events exceeding the threshold are considered abnormal events; An extraction module, used to extract abnormal nodes from the business topology map based on abnormal events; Feature module, used to extract abnormal features based on abnormal nodes; The supplement module is used to extract numerical features from abnormal features and supplement the numerical features of missing values; The decision tree module is used to build a decision tree model for anomaly demarcation based on numerical features using the C4.5 algorithm; The delimitation module is used to input the features to be input into the decision tree model to delimit abnormal events after the decision tree model training is completed.
[0076] The present invention also provides an electronic device, comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit executes the above method.
[0077] The present invention also provides a storage medium storing a computer program executable by an electronic device. When the program runs on the electronic device, the electronic device executes the above method.
[0078] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for analyzing a delimited problem, characterized in that: include: Collect data from different sources, including resource indicators, business application performance indicators, network status and network call indicators, and key events in log records; A service topology map is constructed based on the call indicators, the resource indicators are mapped to corresponding nodes of the service topology map, the service application performance indicators are superimposed on the service links of the service topology map, and the key time is marked on the relevant nodes of the service topology map; A threshold is set for each of the data, and an event exceeding the threshold is considered an abnormal event; Based on the abnormal event, extracting abnormal nodes from the service topology graph; Extracting abnormal features based on the abnormal nodes; Extracting numerical features from the abnormal features and supplementing the numerical features of missing values; Based on the numerical features, a decision tree model for anomaly demarcation is constructed using the C4.5 algorithm; After the training of the decision tree model is completed, the features to be input are input into the decision tree model to delimit abnormal events.
2. The method for analyzing delimited problems according to claim 1, characterized in that: The resource indicators include CPU utilization, memory utilization and disk IO utilization; the business application performance indicators include response time and error rate; the network status and network call indicators include network traffic and delay.
3. The method for analyzing delimited problems according to claim 2, characterized in that: The label and type are set accordingly according to the root cause of the abnormal event.
4. The method for analyzing delimited problems according to claim 3, characterized in that: Extracting abnormal nodes from the service topology graph includes: Nodes whose request volume, error rate or response time exceeds the threshold are monitored and set as abnormal nodes; Based on the label and the type, identifying the type of the abnormal node; All the abnormal nodes are extracted from the service topology graph and classified to form subsets.
5. The method for analyzing delimited problems according to claim 4, characterized in that: Extracting abnormal features based on the abnormal node includes: Extract the data within a unified time period, calculate the mean and maximum values within the time period, and count the characteristics; Analyze the frequency of specific events over a period of time; Aggregate according to the service instance dimension and calculate summary statistics at the service instance dimension.
6. The method for analyzing delimited problems according to claim 5, characterized in that: The decision tree model for outlier demarcation constructed by the C4.5 algorithm includes: The C4.5 algorithm uses the information gain ratio to select the optimal split attribute, and the formula is: ; Where: S is the current dataset; A is the attribute to be evaluated; Gain(S, A) represents information gain, which measures the importance of an attribute A in the classification task S; Intrinsic Value(A) represents the calculation of intrinsic value, which calculates the intrinsic value of attribute A and reflects the entropy of the distribution of different values in attribute A.
7. The method for analyzing delimited problems according to claim 6, characterized in that: The calculation formula of the information gain is: ; ; Where: Values(A) represents all possible values of attribute A; Sv is the subset of attribute A in the dataset whose value is v; |S| and |Sv| represent the number of samples in dataset S and subset Sv respectively; Entropy(S) is the entropy of dataset S; P i is the probability of the i-th category; The calculation formula of the inherent value is: 。 8. A delimited problem analysis system, characterized in that: include: A collection module is used to collect data from different sources, including resource indicators, business application performance indicators, network status and network call indicators, and key events in log records; A construction module, used to construct a business topology map based on the call indicators, the resource indicators are mapped to the corresponding nodes of the business topology map, the business application performance indicators are superimposed on the service links of the business topology map, and the key time is marked on the relevant nodes of the business topology map; A threshold is set for each of the data, and an event exceeding the threshold is considered an abnormal event; An extraction module, used for extracting abnormal nodes from the service topology diagram based on the abnormal event; A feature module, used for extracting abnormal features based on the abnormal nodes; A supplementing module, used to extract numerical features from the abnormal features and supplement the numerical features of missing values; A decision tree module, used to construct a decision tree model for anomaly demarcation based on the numerical features by using the C4.5 algorithm; The delimiting module is used to input the features to be input into the decision tree model to delimit abnormal events after the training of the decision tree model is completed.
9. An electronic device, characterized in that: The method comprises at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit executes the method according to any one of claims 1 to 7.
10. A storage medium, characterized in that: It stores a computer program executable by an electronic device, and when the program runs on the electronic device, the electronic device executes the method described in any one of claims 1 to 7.
Citation Information
Patent Citations
Network anomaly positioning method and device
CN114765574A
Micro-service fault diagnosis method and system
CN115640159A
Decision analysis method and system for root cause analysis based on observation cloud
CN117271184A
Video perception analysis method and device, equipment and storage medium
CN117714782A
Fault positioning method and device for customer collection special line
CN118802498A