Batch abnormal node monitoring method, device, equipment, medium and product

Abstract

The invention relates to a batch abnormal node monitoring method and device, equipment, a medium and a product. The method comprises the following steps: acquiring node data of each operation node of a resource management system; obtaining a first global abnormal node set of the resource management system based on the first neural network model and the node data; obtaining a local abnormal node set corresponding to each operation node of the resource management system based on at least one second neural network model and each node data; determining an abnormal node set in the resource management system based on the first global abnormal node set and the local abnormal node sets corresponding to the second neural network models; wherein the first neural network model and the at least one second neural network model are constructed based on an isolated forest algorithm. Therefore, the probability of misjudgment of a single model can be reduced, so that the finally output abnormal node is ensured to conform to global abnormal characteristics and be confirmed as abnormal by at least one special dimension, and the accuracy of abnormal detection is further improved.

Description

Technical Field

[0001] This application relates to the technical field of financial data analysis, and in particular to a method, apparatus, equipment, medium, and product for batch anomaly node monitoring. Background Technology

[0002] With the development of economic resources, resource transformation can be managed through relevant resource management systems. However, as resource transformation structures become increasingly complex, batch anomalies may occur during the processing of bulk transactions by resource management systems. Therefore, accurate detection of batch anomalies becomes crucial to ensuring the secure operation of resource management systems.

[0003] In related technologies, the main method relies on regular manual inspections to detect batches of abnormal nodes in the resource management system. However, this method is inefficient and difficult to handle high-frequency, complex financial transaction scenarios. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for batch abnormal node monitoring to address the above-mentioned technical problems.

[0005] Firstly, this application provides a method for batch abnormal node monitoring, the method comprising:

[0006] Obtain node data from each running node of the resource management system;

[0007] Based on the first neural network model and the data of each node, the first global abnormal node set of the resource management system is obtained.

[0008] Based on at least one second neural network model and the data of each node, the set of local abnormal nodes corresponding to each running node of the resource management system is obtained.

[0009] Based on the first global set of abnormal nodes and the local set of abnormal nodes corresponding to each second neural network model, the set of abnormal nodes in the resource management system is determined; wherein, the first neural network model and at least one second neural network model are constructed based on the isolated forest algorithm.

[0010] In one embodiment, the set of abnormal nodes in the resource management system is determined based on a first global set of abnormal nodes and the set of local abnormal nodes corresponding to each second neural network model, including:

[0011] The sets of local abnormal nodes are merged to obtain the second global abnormal node set.

[0012] The set of abnormal nodes is determined based on the first set of global abnormal nodes and the second set of global abnormal nodes.

[0013] In one embodiment, determining the set of abnormal nodes in the resource management system based on a first global abnormal node set and a second global abnormal node set includes:

[0014] The set of global abnormal nodes that intersects with the first set of global abnormal nodes and the second set of global abnormal nodes is determined as the set of abnormal nodes.

[0015] In one embodiment, based on the first neural network model and the data of each node, a first global set of abnormal nodes in the resource management system is obtained, including:

[0016] Calculate the path length from each node's data to the leaf node in each decision tree of the first neural network model;

[0017] Calculate the average path length based on the path length of the decision tree in the first neural network model;

[0018] Based on the average path length and the preset anomaly calculation formula, determine the anomaly quantification value of each node's data.

[0019] Based on the abnormal metric values ​​of each node's data, determine the first set of global abnormal nodes.

[0020] In one embodiment, based on at least one second neural network model and node data, a set of local abnormal nodes corresponding to each running node of the resource management system is obtained, including:

[0021] Based on the feature information of each running node, feature processing is performed on the data of each node to obtain the feature data corresponding to each running node.

[0022] The feature data corresponding to each running node is input into the corresponding second neural network model to obtain a set of local abnormal nodes.

[0023] In one embodiment, the method further includes:

[0024] Based on the set of anomalies, generate prompt information;

[0025] The prompt message is output using the preset output method.

[0026] Secondly, this application also provides a batch abnormal node monitoring device, which includes:

[0027] The node data acquisition module is used to acquire node data of each running node in the resource management system.

[0028] The global abnormal node determination module is used to obtain the first global abnormal node set of the resource management system based on the first neural network model and the data of each node;

[0029] The local anomaly node determination module is used to obtain the set of local anomaly nodes corresponding to each running node of the resource management system based on at least one second neural network model and the data of each node.

[0030] The abnormal node set determination module is used to determine the abnormal node set in the resource management system based on the first global abnormal node set and the local abnormal node set corresponding to each second neural network model; wherein the first neural network model and at least one second neural network model are constructed based on the isolated forest algorithm.

[0031] Thirdly, this application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0032] Obtain node data from each running node of the resource management system;

[0033] Based on the first neural network model and the data of each node, the first global abnormal node set of the resource management system is obtained.

[0034] Based on at least one second neural network model and the data of each node, the set of local abnormal nodes corresponding to each running node of the resource management system is obtained.

[0035] Based on the first global set of abnormal nodes and the local set of abnormal nodes corresponding to each second neural network model, the set of abnormal nodes in the resource management system is determined; wherein, the first neural network model and at least one second neural network model are constructed based on the isolated forest algorithm.

[0036] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0037] Obtain node data from each running node of the resource management system;

[0038] Based on the first neural network model and the data of each node, the first global abnormal node set of the resource management system is obtained.

[0039] Based on at least one second neural network model and the data of each node, the set of local abnormal nodes corresponding to each running node of the resource management system is obtained.

[0040] Based on the first global set of abnormal nodes and the local set of abnormal nodes corresponding to each second neural network model, the set of abnormal nodes in the resource management system is determined; wherein, the first neural network model and at least one second neural network model are constructed based on the isolated forest algorithm.

[0041] Fifthly, this application also provides a computer program product comprising a computer program that, when executed by a processor, performs the following steps:

[0042] Obtain node data from each running node of the resource management system;

[0043] Based on the first neural network model and the data of each node, the first global abnormal node set of the resource management system is obtained.

[0044] Based on at least one second neural network model and the data of each node, the set of local abnormal nodes corresponding to each running node of the resource management system is obtained.

[0045] Based on the first global set of abnormal nodes and the local set of abnormal nodes corresponding to each second neural network model, the set of abnormal nodes in the resource management system is determined; wherein, the first neural network model and at least one second neural network model are constructed based on the isolated forest algorithm.

[0046] The aforementioned batch anomaly node monitoring method, device, equipment, medium, and product first acquire node data of each operating node in the resource management system; then, based on a first neural network model and the node data, a first global anomaly node set of the resource management system is obtained to identify anomaly nodes in the resource management system from a global feature perspective and capture common anomaly patterns; next, based on at least one second neural network model and the node data, a local anomaly node set corresponding to each operating node of the resource management system is obtained to specifically identify subdivided anomaly types from a specific dimension (specific feature) and find local anomaly nodes; finally, based on the first global anomaly node set and the local anomaly node sets corresponding to each second neural network model, the set of anomaly nodes in the resource management system is determined; wherein, both the first neural network model and at least one second neural network model are constructed based on the isolated forest algorithm. This reduces the probability of misjudgment by a single model, ensuring that the final output anomaly nodes not only conform to global anomaly features but are also confirmed as anomalies by at least one specific dimension, thereby improving the accuracy of anomaly detection. Attached Figure Description

[0047] Figure 1 Flowcharts of batch abnormal node monitoring methods provided in some embodiments of this application;

[0048] Figure 2 A flowchart for determining a set of abnormal nodes provided in some embodiments of this application;

[0049] Figure 3 A flowchart for determining a first set of global abnormal nodes provided in some embodiments of this application;

[0050] Figure 4 A flowchart illustrating the process of obtaining a set of local abnormal nodes provided in some embodiments of this application;

[0051] Figure 5 A flowchart illustrating the output prompt information provided in some embodiments of this application;

[0052] Figure 6 Structural block diagram of a batch abnormal node monitoring device provided in some embodiments of this application;

[0053] Figure 7 This is an internal structural diagram of a computer device provided in some embodiments of this application. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] The batch abnormal node monitoring method provided in this application embodiment can be applied to terminals equipped with resource management systems, including but not limited to computers and personal laptops. When the terminal executes the batch abnormal node monitoring method provided in this embodiment, it first acquires node data of each running node in the resource management system; then, based on a first neural network model and the node data, it obtains a first global abnormal node set of the resource management system to identify abnormal nodes in the resource management system from a global feature perspective and capture common abnormal patterns; next, based on at least one second neural network model and the node data, it obtains a local abnormal node set corresponding to each running node of the resource management system to specifically identify subdivided abnormal types from a specific dimension (specific feature) and find local abnormal nodes; finally, based on the first global abnormal node set and the local abnormal node sets corresponding to each second neural network model, it determines the set of abnormal nodes in the resource management system; wherein, both the first neural network model and at least one second neural network model are constructed based on the isolated forest algorithm. This reduces the probability of misjudgment by a single model, ensuring that the final output abnormal nodes not only conform to global abnormal features but are also confirmed as abnormal by at least one specific dimension, thereby improving the accuracy of abnormal detection.

[0056] In one embodiment, such as Figure 1 As shown, the method is illustrated using the terminal described above as an example. In this embodiment, the method includes the following steps:

[0057] Step 102: Obtain node data for each running node of the resource management system.

[0058] In this embodiment, the resource management system can be a financial transaction system, a core infrastructure supporting the stable operation of the financial market. It handles a large number of transaction processing tasks and is responsible for fund transfers and maintaining transaction order. In the context of digital transformation in the financial industry, this system faces challenges such as exponential expansion of transaction volume and complex transaction structures, requiring precise detection of batch anomalies to ensure its secure operation. A running node can be understood as a unit within the resource management system that executes specific transaction tasks. During batch business processing, each running node collaborates to complete the transaction process. For example, in financial transactions, different business modules and system components may be involved, and each module or component can be considered a running node.

[0059] Node data comprises multi-dimensional data related to the behavior of running nodes, including but not limited to basic characteristic data and specific characteristic data. Basic characteristic data includes start time, duration, supply quantity, file status, number of re-adjustments after liquidation failure, log information, whether batch skipping occurred, and whether the process was rerun. Specific characteristic data includes: start time, duration, file status, number of re-adjustments, whether batch skipping occurred, and whether the process was rerun for the reseller module; start time, duration, and number of re-adjustments after liquidation failure for the upstream system module; and start time, duration, and supply quantity for the external system supply.

[0060] Optionally, raw data related to the behavior of running nodes can be collected, and the raw data can be initially screened to remove data containing obvious errors; outliers can be processed, and values ​​that exceed the normal range can be replaced with the historical mean of the node; missing values ​​can be filled, and continuous features can be estimated using the nearest neighbor method, while discrete features can be filled with 0 (indicating no record), so as to provide an accurate and complete data foundation for subsequent model training and anomaly detection.

[0061] Step 104: Based on the first neural network model and the data of each node, obtain the first global abnormal node set of the resource management system.

[0062] The first neural network model can be a basic isolated forest model, which is an anomaly detection model trained on sample data with full features. It is used to capture common anomaly patterns in the resource management system and cover global features. The first global anomaly node set is the set of anomaly nodes marked and output by the first neural network model after calculating the anomaly score of the data. It represents the anomaly nodes judged from the perspective of global features.

[0063] Optionally, the acquired node data can be input into a pre-trained isolated forest model (first neural network model) based on the full set of feature sample data. The path length from the data sample to the leaf node in each decision tree is calculated. The average path length is calculated based on the path lengths of all decision trees. A predetermined anomaly score is then derived using the average path length, and the set of anomalous nodes is labeled according to the anomaly score. In this way, anomalous nodes in the resource management system are identified from a global feature perspective, capturing common anomaly patterns.

[0064] Alternatively, the autoencoder can be trained using node data from normally functioning nodes. During training, the autoencoder attempts to minimize the error between the input data and the reconstructed data, thereby learning the feature patterns of normal data. After training is complete, the node data from each running node is input into the trained autoencoder, and the reconstruction error of each node's data is calculated. A suitable reconstruction error threshold is set, and when the reconstruction error of a node's data exceeds this threshold, the node is marked as an anomalous node. Finally, all nodes marked as anomalous are collected to form the first global anomalous node set.

[0065] Step 106: Based on at least one second neural network model and the data of each node, obtain the set of local abnormal nodes corresponding to each running node of the resource management system.

[0066] The second neural network model can be a sub-isolated forest model, which is an anomaly detection model trained on sample data based on specific features. It focuses on a specific dimension and specifically identifies subdivided anomaly types. The local anomaly node set is the set of anomaly nodes marked and output by each second neural network model after calculating the anomaly score of the data. It represents the anomaly nodes judged from a specific dimension (specific feature).

[0067] Optionally, the node data can be input into at least one sub-isolated forest model (second neural network model) trained based on specific feature sample data. Similarly, the path length, average path length, and anomaly score of the data sample in each decision tree are calculated, and the sets of anomalous nodes output by each model are labeled. In this way, specific anomaly types can be identified and localized anomalous nodes can be found from a specific dimension (specific feature).

[0068] Additionally, subsets related to local features can be extracted from the data of each node and labeled. For example, based on historical experience or known anomalies, some node data can be labeled as normal or abnormal, and the data can be standardized to ensure that different features have the same scale, which helps improve the performance of the model. Then, the labeled local feature data is used to train the support vector machine. During the training process, the support vector machine attempts to find an optimal hyperplane to separate normal and abnormal data. Different kernel functions (such as linear kernels, radial basis kernels, etc.) can be selected for experimentation to find the kernel function that is most suitable for the current data. The local feature data of each running node is input into the trained support vector machine, and the model determines whether the node is an abnormal node based on the position of the hyperplane. For each second neural network model (i.e., the model corresponding to each local feature subset), the nodes labeled as abnormal are collected, thus forming the corresponding set of local abnormal nodes.

[0069] Step 108: Based on the first global abnormal node set and the local abnormal node set corresponding to each second neural network model, determine the abnormal node set in the resource management system.

[0070] In this system, both the first neural network model and at least one second neural network model are constructed based on the isolated forest algorithm. The set of anomalous nodes in the resource management system can be obtained by taking the intersection of the first global anomalous node set and the local anomalous node set corresponding to each second neural network model, based on the intersection strategy of "base model filtering + sub-model verification". That is, an anomalous node is a node that "meets the global anomalous characteristics and is confirmed as anomalous by at least one specific dimension".

[0071] Optionally, a "base model filtering + sub-model verification" intersection strategy can be adopted, taking the intersection of the first global anomaly node set and each local anomaly node set to obtain the final anomaly node set. This reduces false positives from a single model, ensuring that the final output anomaly nodes not only conform to global anomaly characteristics but are also confirmed as anomalies by at least one specific dimension, thus improving the accuracy of anomaly detection.

[0072] In addition, different weights can be assigned to the first global set of abnormal nodes and each local set of abnormal nodes. The weights can be adjusted according to the actual situation. For example, if the results of global anomaly detection are considered more reliable, a higher weight can be assigned to the first global set of abnormal nodes. For each running node, the number of times it is marked as abnormal in the first global set of abnormal nodes and each local set of abnormal nodes is counted and multiplied by the corresponding weight. A voting score threshold is set. When the voting score of a node exceeds the threshold, the node is identified as an abnormal node in the resource management system. Finally, all nodes identified as abnormal are collected to form the final set of abnormal nodes in the resource management system.

[0073] The aforementioned batch anomaly node monitoring method utilizes an automated neural network model for anomaly node detection, avoiding the inefficiency of manual periodic inspections. It can rapidly process large amounts of node data and adapt to high-frequency, complex financial transaction scenarios. Furthermore, it employs an isolated forest fusion framework combining a base model and specialized models. This involves using the first neural network model (base model) to capture common anomaly patterns and the second neural network model (specialized model) to specifically identify subdivided anomaly types. By using a multi-model result intersection strategy to output an anomaly node set, it reduces misjudgments by a single model, significantly improving the accuracy of anomaly detection and ensuring accurate identification of batch anomaly nodes in the resource management system. In addition, this solution can monitor the operational nodes of the resource management system in real time. Once an anomaly node is detected, it can be promptly identified and an alarm triggered, facilitating timely intervention and preventing the anomaly from causing greater impact on the system. This achieves real-time detection and rapid response to anomaly nodes.

[0074] In one embodiment, such as Figure 2 As shown, based on the first global set of abnormal nodes and the local set of abnormal nodes corresponding to each second neural network model, the set of abnormal nodes in the resource management system is determined, including:

[0075] Step 202: Merge the sets of local abnormal nodes to obtain the second set of global abnormal nodes.

[0076] Optionally, the local outlier node sets corresponding to each second neural network model can be simply merged. That is, the nodes in all local outlier node sets are summarized, duplicate nodes are removed, and a set containing all possible outlier nodes is obtained. For example, suppose there are three local outlier node sets A={1,2,3}, B={3,4,5}, and C={5,6,7}. By performing the union operation A∪B∪C={1,2,3,4,5,6,7}, this new set is the preliminary result of the merge.

[0077] Alternatively, a weighted merging method can be used. If different sets of local anomalous nodes have varying degrees of credibility, different weights can be assigned to each set. During merging, the anomalous probability of nodes is adjusted based on these weights. For example, if a set of local anomalous nodes has a higher weight, then nodes in that set are more likely to be considered anomalous in the merged set.

[0078] Step 204: Determine the set of abnormal nodes based on the first set of global abnormal nodes and the second set of global abnormal nodes.

[0079] Optionally, the intersection of the first and second global abnormal node sets can be taken, that is, the nodes that exist in both sets can be found. For example, if the first global abnormal node set G1={1,3,5} and the second global abnormal node set G2={3,5,7}, then their intersection G1∩G2={3,5} is the final determined abnormal node set.

[0080] In this embodiment, the sets of local anomalous nodes are merged to obtain a second global anomalous node set. This fully utilizes the anomalous information detected by multiple second neural network models from different specialized features and local situations, avoiding the situation where some anomalous nodes are missed due to relying on only a single local detection. This results in a wider coverage of anomaly detection and the discovery of more potential anomalous nodes. Furthermore, by comparing and filtering the first and second global anomalous node sets and taking their intersection, it is ensured that the finally identified anomalous nodes are verified as anomalous in both global and multiple local dimensions. This multi-dimensional verification method greatly reduces the possibility of misjudgment caused by single model or single-view detection, improving the accuracy and reliability of anomaly detection.

[0081] In one embodiment, determining the set of abnormal nodes in the resource management system based on a first set of global abnormal nodes and a second set of global abnormal nodes includes: determining the global abnormal nodes that intersect in the first set of global abnormal nodes and the second set of global abnormal nodes as the set of abnormal nodes.

[0082] Optionally, the intersection operation in set operations can be used to find nodes that exist simultaneously in both the first global abnormal node set and the second global abnormal node set, and the nodes in the intersection are determined as the abnormal node set in the resource management system. These nodes are judged as abnormal in the detection of global overall features and also exhibit abnormalities in the detection of multiple local specific features.

[0083] In this embodiment, by taking the intersection method, only those nodes that are detected as abnormal in both global and multiple local dimensions will be identified as the final abnormal nodes, which greatly reduces the possibility of false positives and improves the accuracy of anomaly detection.

[0084] In one embodiment, such as Figure 3 As shown, based on the first neural network model and the data of each node, the first global abnormal node set of the resource management system is obtained, including:

[0085] Step 302: Calculate the path length from each node's data to the leaf node in each decision tree of the first neural network model.

[0086] A decision tree is a tree-structured machine learning model consisting of nodes and edges. Each internal node represents a test on an attribute, each branch represents a test output, and each leaf node represents a category or value. In anomaly detection scenarios, a decision tree can partition the data based on the characteristics of the node data, progressively distributing the data to different branches until it reaches a leaf node, thereby determining whether the data is an anomaly.

[0087] A leaf node is a node in a decision tree that has no child nodes and is located at the end of the tree. In anomaly detection, a leaf node represents a classification result obtained after a series of attribute tests. For example, when determining whether node data is abnormal, a leaf node may indicate whether the data belongs to the normal category or the abnormal category.

[0088] For each node data point, in each decision tree of the first neural network model, starting from the root node, the path moves downwards along the branches according to the attribute testing rules of the decision tree until a leaf node is reached. The number of edges traversed is the path length of that node data point to the leaf node in that decision tree. The path length may differ between different node data points in the same decision tree, or between the same node data points in different decision trees.

[0089] Optionally, for each decision tree in the first neural network model, starting from the root node, the node data is sequentially input into the decision tree. Based on the attribute test conditions of the nodes within the decision tree, it is determined which branch the data should continue to move down along. The path length is incremented by 1 for each edge traversed until a leaf node is reached, at which point the path length is recorded. This process is repeated for each decision tree to calculate the path length of each node's data within each decision tree. Thus, by calculating the path length, the distribution of node data within each decision tree can be preliminarily assessed. A shorter path length indicates that the node data reached the leaf node earlier in the decision tree, potentially possessing characteristics different from other data, and is more likely to be outlier data.

[0090] Step 304: Calculate the average path length based on the path length of the decision tree in the first neural network model.

[0091] The average path length refers to the average path length of a given node to a leaf node across all decision trees in the first neural network model. By calculating the average path length, the performance of that node across multiple decision trees can be comprehensively considered, allowing for a more holistic evaluation of its characteristics.

[0092] Optionally, for each node data, the path lengths to the leaf nodes in all decision trees of the first neural network model are summed, and then divided by the total number of decision trees to obtain the average path length of that node data.

[0093] It is understandable that the path length of a single decision tree may have a certain degree of randomness. By calculating the average path length, information from multiple decision trees can be comprehensively considered, reflecting the characteristics of node data more stably and comprehensively, and reducing the error caused by a single decision tree.

[0094] Step 306: Determine the quantified value of the anomaly of each node's data based on the average path length and the preset anomaly calculation formula.

[0095] The preset anomaly calculation formula is a mathematical formula used to calculate the degree of anomaly in node data based on the average path length. Different anomaly detection algorithms may have different anomaly calculation formulas. Typically, the average path length is combined with some constants or other parameters to obtain a value that can quantify the degree of anomaly in node data.

[0096] Optionally, the preset anomaly calculation formula can be... ,in, It is an abnormal quantification value; It is the average path length; It is specific node data in the resource management system, and also various operational indicators of a certain node in the system at a specific moment, such as CPU utilization, memory usage, network traffic, etc. It is related to the number of samples Relevant constants.

[0097] Step 308: Determine the first global abnormal node set based on the abnormal metric values ​​of each node's data.

[0098] The anomaly metric is a value calculated from the average path length of the node data according to a preset anomaly calculation formula. This value measures the degree of anomaly in the node data; the larger the value, the more abnormal the node data is generally considered to be.

[0099] Optionally, an anomaly threshold can be set, and the anomaly metric value of each node's data can be compared with this threshold. If the anomaly metric value is greater than the threshold, the node corresponding to that node's data is marked as an anomaly node; all nodes marked as anomalies are collected to form the first global anomaly node set.

[0100] In this embodiment, by calculating the average path length and anomaly quantification, the anomalies of node data can be evaluated from multiple perspectives, avoiding the one-sidedness that may be caused by a single decision tree, and more comprehensively discovering abnormal nodes in the resource management system.

[0101] In one embodiment, such as Figure 4As shown, based on at least one second neural network model and the data of each node, a set of local abnormal nodes corresponding to each running node of the resource management system is obtained, including:

[0102] Step 402: Based on the feature information of each running node, perform feature processing on the data of each node to obtain the feature data corresponding to each running node.

[0103] Among them, the characteristic information of running nodes refers to the various information that each running node in the resource management system possesses, which can reflect its running status and characteristics, including system performance indicators (such as CPU utilization, memory usage, disk I / O rate, etc.), business-related data (such as processing time of business requests, success rate of business transactions, throughput of business data, etc.), and time-related characteristics (such as node runtime, whether it is in peak business period, etc.).

[0104] Optionally, data cleaning should be performed on the data at each node: check for missing values, outliers, etc. For missing values, imputation can be done using the mean, median, or by prediction based on other relevant data; for outliers, statistical methods (such as those based on standard deviation) can be used for identification and correction; then, normalization should be performed: use existing min-max normalization or Z-score normalization methods to normalize feature information of different ranges; finally, select the most valuable features for anomaly detection from the numerous feature information. Statistical methods (such as correlation analysis) or machine learning methods (such as feature importance ranking based on decision trees) can be used for feature selection to reduce data dimensionality and improve the efficiency and accuracy of the model.

[0105] Step 404: Input the feature data corresponding to each running node into the corresponding second neural network model to obtain a set of local abnormal nodes.

[0106] The feature data corresponding to the running node is the data obtained after feature processing of the feature information of the running node. The purpose of feature processing is to convert the original feature information into a format suitable for the input of the second neural network model, which usually involves data cleaning, normalization, feature selection, and other operations.

[0107] Optionally, before using the second neural network model, the model can be trained using known normal and abnormal node data to adjust its parameters and enable it to learn the feature patterns of normal and abnormal data. Then, the feature data corresponding to each running node, after feature processing, is input into the trained second neural network model. The trained model will then predict the data of each node based on the learned patterns, determining whether it is abnormal. For each running node, based on the model's prediction results, nodes judged as abnormal are collected to form a local abnormal node set.

[0108] In this embodiment, the feature processing steps clean, normalize, and select features for the original node data, which helps to improve the quality and usability of the data. High-quality data enables the second neural network model to better learn the features and patterns of the data, thereby improving the accuracy of anomaly detection.

[0109] In one embodiment, such as Figure 5 As shown, the method also includes:

[0110] Step 502: Generate a prompt message based on the set of anomalies.

[0111] The notification message is generated based on a set of anomaly points and is used to inform relevant personnel of anomalies in the resource management system. The notification message includes key information related to the anomaly node, such as the node's identifier (e.g., node number, name), the approximate type of anomaly (e.g., high CPU usage, memory leak), and the time the anomaly occurred or was detected, so that relevant personnel can quickly understand the basic situation of the anomaly.

[0112] Optionally, after obtaining the set of anomalies, you can first extract key information about the anomaly nodes from the set, such as node ID, anomaly characteristics (e.g., anomaly quantification value, specific indicator value when the anomaly occurred, etc.), and timestamp of the anomaly occurrence. Then, according to the template of the predefined prompt information, fill the extracted key information into the template. For example, the template could be "An anomaly was detected at [time] at node [node ID], with the anomaly indicator being [anomaly characteristic]". Then, fill the actual information into the corresponding positions.

[0113] Step 504: Output the prompt message using the preset output method.

[0114] Among them, the preset output method is a pre-set way or form to convey the prompt information to relevant personnel. There are many common preset output methods, such as displaying it through a pop-up window on the system interface, sending email notifications, pushing text messages, displaying it on a large monitoring screen, and triggering audible and visual alarms.

[0115] Optionally, a pop-up notification can be displayed on the resource management system's interface, showing a message. This can be triggered via front-end development when an anomaly is detected. Alternatively, an email service can be used to send the notification to the relevant personnel's inboxes. This requires configuring the email server and recipient information, and utilizing the email sending API to automate email delivery. Alternatively, an SMS platform can be used to send the notification to the relevant personnel's mobile phones. This requires cooperation with an SMS service provider to obtain the API interface and send the notification to the specified mobile phone number.

[0116] In this embodiment, by generating and outputting timely alerts, relevant personnel can be informed of any anomalies occurring in the resource management system immediately. This facilitates rapid response to abnormal events, reduces the impact of anomalies on the system, and prevents further escalation of problems.

[0117] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0118] Based on the same inventive concept, this application also provides a batch abnormal node monitoring device for implementing the batch abnormal node monitoring method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more batch abnormal node monitoring device embodiments provided below can be found in the limitations of the batch abnormal node monitoring method described above, and will not be repeated here.

[0119] In one embodiment, such as Figure 6 As shown, a batch abnormal node monitoring device is provided, including: a running node data acquisition module 602, a global abnormal node determination module 604, a local abnormal node determination module 606, and an abnormal node set determination module 608, wherein:

[0120] The node data acquisition module 602 is used to acquire node data of each running node in the resource management system.

[0121] The global abnormal node determination module 604 is used to obtain the first global abnormal node set of the resource management system based on the first neural network model and the data of each node.

[0122] The local abnormal node determination module 606 is used to obtain the set of local abnormal nodes corresponding to each running node of the resource management system based on at least one second neural network model and the data of each node.

[0123] The abnormal node set determination module 608 is used to determine the abnormal node set in the resource management system based on the first global abnormal node set and the local abnormal node set corresponding to each second neural network model; wherein the first neural network model and at least one second neural network model are both constructed based on the isolated forest algorithm.

[0124] In one embodiment, the abnormal node set determination module 608 is further configured to: merge the local abnormal node sets to obtain a second global abnormal node set; and determine the abnormal node set based on the first global abnormal node set and the second global abnormal node set.

[0125] In one embodiment, the abnormal node set determination module 608 is further configured to: determine the global abnormal nodes that intersect in the first global abnormal node set and the second global abnormal node set as the abnormal node set.

[0126] In one embodiment, the global abnormal node determination module 604 is further configured to: calculate the path length of each node data to the leaf node in each decision tree of the first neural network model; calculate the average path length based on the path length of the decision trees in the first neural network model; determine the abnormality quantification value of each node data according to the average path length and the preset abnormality calculation formula; and determine the first global abnormal node set based on the abnormality quantification value of each node data.

[0127] In one embodiment, the local abnormal node determination module 606 is further configured to: perform feature processing on the data of each node based on the feature information of each running node to obtain the feature data corresponding to each running node; and input the feature data corresponding to each running node into the corresponding second neural network model to obtain a set of local abnormal nodes.

[0128] In one embodiment, the device is further configured to: generate a prompt message based on a set of anomalies; and output the prompt message using a preset output method.

[0129] Each module in the aforementioned batch abnormal node monitoring device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0130] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for batch abnormal node monitoring. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0131] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0132] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0133] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0134] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0135] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0136] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0137] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for batch abnormal node monitoring, characterized in that, The method includes: Obtain node data from each running node of the resource management system; Based on the first neural network model and the data of each node, the first global abnormal node set of the resource management system is obtained; Based on at least one second neural network model and the node data, a set of local abnormal nodes corresponding to each running node of the resource management system is obtained. Based on the first global abnormal node set and the local abnormal node set corresponding to each of the second neural network models, the abnormal node set in the resource management system is determined; wherein, the first neural network model and at least one of the second neural network models are constructed based on the isolated forest algorithm.

2. The method according to claim 1, characterized in that, The step of determining the set of abnormal nodes in the resource management system based on the first global set of abnormal nodes and the set of local abnormal nodes corresponding to each of the second neural network models includes: The sets of local abnormal nodes are merged to obtain a second set of global abnormal nodes. The set of abnormal nodes is determined based on the first set of global abnormal nodes and the second set of global abnormal nodes.

3. The method according to claim 2, characterized in that, The step of determining the set of abnormal nodes in the resource management system based on the first set of global abnormal nodes and the second set of global abnormal nodes includes: The set of global abnormal nodes that intersects in the first set of global abnormal nodes and the second set of global abnormal nodes is determined as the set of abnormal nodes.

4. The method according to any one of claims 1 to 3, characterized in that, The first global abnormal node set of the resource management system, obtained based on the first neural network model and the node data, includes: Calculate the path length from each node data point to a leaf node in each decision tree of the first neural network model; Calculate the average path length based on the path length of the decision tree in the first neural network model; Based on the average path length and the preset anomaly calculation formula, the anomaly quantification value of each node data is determined; Based on the abnormal metric values ​​of the data of each node, a first global abnormal node set is determined.

5. The method according to any one of claims 1 to 3, characterized in that, The process of obtaining a set of locally abnormal nodes corresponding to each running node of the resource management system based on at least one second neural network model and the node data includes: Based on the feature information of each running node, feature processing is performed on the data of each node to obtain the feature data corresponding to each running node. The feature data corresponding to each running node is input into the corresponding second neural network model to obtain the set of local abnormal nodes.

6. The method according to claim 1, characterized in that, The method further includes: Based on the set of anomalies, a prompt message is generated; The prompt information is output using a preset output method.

7. A batch abnormal node monitoring device, characterized in that, The device includes: The node data acquisition module is used to acquire node data of each running node in the resource management system. The global abnormal node determination module is used to obtain the first global abnormal node set of the resource management system based on the first neural network model and the node data. The local abnormal node determination module is used to obtain a set of local abnormal nodes corresponding to each running node of the resource management system based on at least one second neural network model and the node data of each node. An abnormal node set determination module is used to determine the abnormal node set in the resource management system based on the first global abnormal node set and the local abnormal node set corresponding to each of the second neural network models; wherein the first neural network model and at least one of the second neural network models are constructed based on the isolated forest algorithm.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images