Cloud collaboration data anomaly analysis method and system based on job node information

Abstract

The application discloses a cloud cooperation data abnormality analysis method and system based on job node information, and the method comprises the following steps: job node data acquisition, node state cluster initialization, node correlation degree calculation, node quantitative evaluation, local preliminary judgment and cloud cooperation abnormality verification. The application belongs to the field of data analysis, and specifically refers to a cloud cooperation data abnormality analysis method and system based on job node information. According to the method, historical normal data is divided into clusters according to neighborhood aggregation degree, and neighborhood correlation degree is used to ensure that nodes running specific tasks for a long time are correctly identified as normal. Link correlation is introduced, the historical normality of associated nodes is indirectly used to assist in judging the state of a target node. A sampling weakening mechanism is introduced to calculate the cohesion degree of normal clusters of nodes, so that normal nodes are not misjudged as abnormal nodes due to load changes. The boundary definition of abnormal nodes and normal nodes is quantified by measuring the distinction between clusters, cloud cooperation verification is carried out, and isolated nodes are prevented from being misjudged. Therefore, the data abnormality analysis effect is improved.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, specifically to a cloud-based collaborative data anomaly analysis method and system based on job node information. Background Technology

[0002] Cloud-based collaborative data anomaly analysis methods rely on edge nodes to initially identify anomalies based on local data, while simultaneously uploading key information to the cloud for collaborative verification using global node correlation data. However, these methods typically suffer from several drawbacks: difficulty adapting to irregular distributions of job nodes, neglecting temporal dynamics and local correlations, high distortion rates, and ambiguous features of isolated nodes leading to misjudgments and low reliability of anomaly analysis. Furthermore, they often fail to adapt to dynamic workloads of job nodes, resulting in anomaly assessments that are detached from the context's credibility, high misjudgment rates, and difficulty handling node dependency issues, ultimately leading to poor data anomaly analysis results. Summary of the Invention

[0003] To address the above issues and overcome the shortcomings of existing technologies, this invention provides a cloud-based collaborative data anomaly analysis method and system based on job node information. Addressing the problems of general cloud-based collaborative data anomaly analysis methods, such as difficulty adapting to irregular job node distributions, neglecting temporal dynamics and local correlations, high distortion rates, and ambiguous features of isolated nodes leading to misjudgments and low reliability of anomaly analysis, this solution divides historical normal data into clusters based on neighborhood clustering, allowing normal states to exhibit multimodal irregular distributions. It solves the correlation lag problem caused by node state temporal drift through dynamic center correlation, and addresses the global center failure problem under irregular distributions through neighborhood correlation, ensuring that nodes running specific tasks for a long time are correctly identified as normal. Furthermore, it introduces link correlation, indirectly assisting in judgment through the historical normality of associated nodes. The target node status avoids the limitations of relying solely on the characteristics of isolated nodes, thereby improving the accuracy of data anomaly analysis. Addressing the issues of general cloud-based collaborative data anomaly analysis methods—such as inability to adapt to dynamic loads of job nodes, anomaly assessments being detached from the context and having high false positive rates, and difficulty in handling node link dependencies, leading to poor data anomaly analysis results—this solution introduces a sampling weakening mechanism to calculate the cohesion of normal clusters within nodes. This ensures consistent assessment of normal clusters, always reflecting the current true normal mode of job nodes and avoiding misjudging normal nodes as anomalies due to load changes. The consistency of current normal nodes is quantified by normal cluster cohesion, and the clarity of the boundary between anomalies and normal nodes is quantified by inter-cluster differentiation. Based on cloud-based collaborative verification and combined with global correlation, misjudgments of isolated nodes are avoided, thus improving the effectiveness of data anomaly analysis.

[0004] The technical solution adopted by this invention is as follows: The cloud-based collaborative data anomaly analysis method based on job node information provided by this invention includes the following steps:

[0005] Step S1: Data collection for job nodes;

[0006] Step S2: Initialize the node state cluster;

[0007] Step S3: Calculate node correlation degree;

[0008] Step S4: Node Quantitative Evaluation;

[0009] Step S5: Local initial judgment;

[0010] Step S6: Cloud-based collaborative anomaly verification.

[0011] Furthermore, in step S1, the data acquisition of the work nodes involves collecting raw data from each work node in real time and preprocessing it to obtain the standardized feature vector of the node to be tested.

[0012] Furthermore, in step S2, the node state cluster initialization selects the feature vector set of historically abnormal nodes, divides the clusters by neighborhood clustering degree, and temporarily classifies the feature vectors of the node to be tested into the abnormal cluster to be investigated.

[0013] Furthermore, in step S3, the node correlation calculation involves calculating the dynamic correlation between the node under test and the dynamic center of the normal cluster; calculating the dynamic correlation between the node under test and the nearest node within the normal cluster; and introducing a link correlation factor to construct a comprehensive correlation.

[0014] Furthermore, in step S4, the node quantification evaluation involves calculating the cohesion of normal clusters and introducing a sampling weakening mechanism; calculating the inter-cluster discrimination between normal clusters and abnormal clusters under investigation; and quantifying the boundary ambiguity of abnormal candidate nodes to normal clusters.

[0015] Furthermore, in step S5, the local initial judgment involves calculating the anomaly index of the node to be tested. And conduct tiered early warnings: when The assessment indicates a minor anomaly, and cloud-based collaborative anomaly verification is performed; when The corresponding performance continues to deteriorate, assessed as moderately abnormal, and an early warning is issued; when If the corresponding node does not respond, it is assessed as a serious anomaly, and an alert is issued and administrators are notified. , , and It is a tiered threshold.

[0016] Furthermore, in step S6, the cloud-based collaborative anomaly verification involves uploading nodes initially assessed locally as slightly abnormal to the cloud, calculating collaborative verification values, and setting verification thresholds. If a node's collaborative verification value is higher than the verification threshold, it is upgraded to a moderate anomaly; otherwise, it is determined to be an isolated fluctuation, and the node sampling frequency is increased.

[0017] The cloud-based collaborative data anomaly analysis system based on job node information provided by this invention includes a job node data acquisition module, a node status cluster initialization module, a node correlation calculation module, a node quantitative evaluation module, a local preliminary judgment module, and a cloud-based collaborative anomaly verification module.

[0018] The work node data acquisition module collects raw data from each work node in real time, and obtains the standardized feature vector of the node to be tested after preprocessing.

[0019] The node state cluster initialization module captures irregular distributions based on historical normal node data through neighborhood clustering, initializes normal state clusters, and temporarily classifies the nodes to be tested into abnormal clusters.

[0020] The node correlation calculation module introduces a link correlation factor to calculate the comprehensive correlation between the node under test and the normal cluster.

[0021] The node quantification evaluation module is based on a sampling weakening mechanism and calculates the cohesion of normal clusters and the inter-cluster discrimination between normal clusters and abnormal clusters based on the correlation degree.

[0022] The local preliminary judgment module calculates the anomaly index of the node to be tested and realizes hierarchical anomaly early warning through multi-level anomaly thresholds;

[0023] The cloud-based collaborative anomaly verification module performs cloud-based collaborative anomaly verification on nodes that are initially judged to be slightly abnormal locally.

[0024] The beneficial effects achieved by adopting the above solution are as follows:

[0025] (1) In view of the problems that general cloud-based collaborative data anomaly analysis methods have, such as difficulty in adapting to the irregular distribution of job nodes, ignoring time-series dynamics and local correlations, high distortion rate, and ambiguous features of isolated nodes, which are easy to misjudge and thus lead to low reliability of anomaly analysis, this solution divides historical normal data into clusters by neighborhood clustering degree, allowing normal states to present multimodal irregular distribution; solves the problem of correlation degree lag caused by the time-series drift of node states by dynamic center correlation degree, and solves the problem of global center failure under irregular distribution by neighborhood correlation degree, ensuring that nodes that have been running specific tasks for a long time are correctly identified as normal; and introduces link correlation, which indirectly assists in judging the state of target nodes by the historical normality of related nodes, avoiding the limitation of relying solely on the characteristics of isolated nodes; thereby improving the accuracy of data anomaly analysis.

[0026] (2) To address the problems of general cloud-based collaborative data anomaly analysis methods, such as inability to adapt to dynamic loads of job nodes, anomaly assessment being detached from the scenario's credibility, high misjudgment rate, and difficulty in handling node link dependencies, which lead to poor data anomaly analysis results, this solution introduces a sampling weakening mechanism to calculate the cohesion of normal clusters of nodes, ensuring consistent assessment of normal clusters of nodes, always reflecting the current true normal mode of job nodes, and avoiding misjudging normal as anomalies due to load changes; by measuring the cohesion of normal clusters, the consistency of current normal nodes is quantified, and by measuring the distinction between clusters, the clarity of the boundary between anomalies and normal nodes is quantified, based on cloud-based collaborative verification, combined with global association to avoid misjudging isolated nodes; thereby improving the data anomaly analysis effect. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the cloud-based collaborative data anomaly analysis method based on job node information provided by the present invention.

[0028] Figure 2 This is a schematic diagram of the cloud-based collaborative data anomaly analysis system based on job node information provided by the present invention.

[0029] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. Detailed Implementation

[0030] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0031] In the description of this invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the system or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0032] Example 1, see Figure 1 The present invention provides a cloud-based collaborative data anomaly analysis method based on job node information, which includes the following steps:

[0033] Step S1: Data collection at work nodes. Raw data is collected in real time from each work node and preprocessed to obtain the standardized feature vector of the node to be tested.

[0034] Step S2: Initialize the node state cluster. Based on historical normal node data, capture irregular distributions by dividing the neighborhood clustering degree, initialize the normal state cluster, and temporarily classify the node to be tested into the abnormal cluster to be investigated.

[0035] Step S3: Node correlation calculation. Introduce the link correlation factor to calculate the comprehensive correlation between the node under test and the normal cluster.

[0036] Step S4: Node quantification evaluation, based on the sampling weakening mechanism, calculate the cohesion of normal clusters and the inter-cluster discrimination between normal clusters and abnormal clusters under investigation by comprehensively considering the correlation degree;

[0037] Step S5: Local initial judgment, calculate the anomaly index of the node to be tested, and realize hierarchical anomaly early warning through multi-level anomaly thresholds;

[0038] Step S6: Cloud-based collaborative anomaly verification. Perform cloud-based collaborative anomaly verification on nodes that are initially judged to be slightly abnormal locally.

[0039] Example 2, see Figure 1 This embodiment is based on the above embodiment. In step S1, the work node data acquisition is to collect raw data in real time from each work node, including hardware resource indicators, system operation indicators, network communication indicators and work task indicators, and perform preprocessing. Noise is eliminated by cleaning and standardization to obtain the standardized feature vector of the node to be tested.

[0040] The specific operations are as follows: Distributed data collection, using edge agents to collect node data in real time, and using timestamp alignment to ensure the consistency of time sequence between cloud and node data; Cleaning rules, removing missing values ​​caused by communication interruptions, and filling with the average of the first 3 times for the same node to filter out obvious noise; Standardization, mapping indicators of different magnitudes to the [0,1] interval to eliminate the influence of units; Let X be the standardized feature vector of the i-th node to be tested. i .

[0041] Example 3, see Figure 1 This embodiment is based on the above embodiment. In step S2, the node state cluster initialization is based on the historical normal node data to initialize the normal state cluster. The data of the node to be tested is temporarily classified as the abnormal cluster to be investigated. Through dynamic center adjustment and adaptation to irregular distribution, the cluster center shift caused by the non-ellipsoidal distribution of the normal state is avoided.

[0042] The specific operation is as follows: Normal cluster initialization: Select the feature vector set S of historically no abnormal nodes. nl Clusters are defined by neighborhood clustering to ensure accurate capture of irregular distributions, including those with low CPU and high memory, and those with high CPU and low memory. The feature vectors of the nodes to be tested are temporarily classified into anomaly clusters C. al ; The j-th cluster in the normal state Node X satisfies: ;in, It is a node that is distinct from X; It is the clustering threshold;

[0043] To avoid misjudgment caused by cluster center shift due to non-ellipsoidal distribution in normal state, traditional methods are used, including situations where some nodes are in a high memory + low CPU state for a long time due to running specific tasks, which is still considered normal.

[0044] Example 4, see Figure 1 This embodiment is based on the above embodiment. In step S3, the node correlation degree calculation is based on the dynamic center correlation degree and the neighborhood correlation degree. The correlation degree of the node to be tested to the normal cluster is calculated, which reflects both the deviation from the center of the normal cluster and the correlation with the nearest node in the normal cluster, thereby improving the robustness of anomaly identification.

[0045] The specific operation is as follows: Calculate the dynamic correlation between the node under test and the dynamic center of the normal cluster. The center is updated in real time with historical data in the cloud to adapt to the temporal changes in the node's state, and is represented as: ;in, It is the dynamic correlation degree between the node i under test and the normal cluster j; and Here, represents the centers of normal cluster j and normal cluster l, respectively; m is the correlation coefficient; k is the total number of normal clusters; and l is the index of a normal cluster. The dynamic correlation degree between the node under test and its nearest node within a normal cluster is calculated to adapt to irregular distributions and avoid the influence of center offset. This is expressed as: ;in, It is the degree of neighborhood correlation; and These are the node samples that belong to normal cluster j and normal cluster l and are closest to the node i to be tested, respectively. It is the l-th normal cluster; a link correlation factor is introduced. Build comprehensive correlation By indirectly assisting in judging the state of the target node by associating the normality of the nodes under test, the misjudgment caused by the fuzzy features of isolated nodes is resolved, as expressed as: ; ; , and It is a comprehensive weight; It represents the overall correlation degree of node X at the previous time step; It is the set of associated nodes of the node to be tested i, taking the first 3 nodes and the last 3 downstream nodes in the data transmission link;

[0046] By dynamically adjusting the temporal drift of node states through the center, and by using a neighborhood mechanism to resolve the distortion of correlation under irregular distribution, including the fact that some normal nodes are far from the center due to task characteristics but are still considered normal.

[0047] By performing the above operations, this solution addresses the problems of general cloud-based collaborative data anomaly analysis methods, such as difficulty adapting to irregular distributions of job nodes, neglecting temporal dynamics and local correlations, high distortion rates, and ambiguous features of isolated nodes leading to misjudgments and low reliability of anomaly analysis. This solution divides historical normal data into clusters based on neighborhood clustering, allowing normal states to exhibit multimodal irregular distributions. It solves the problem of correlation lag caused by temporal drift in node states through dynamic central correlation, and addresses the problem of global central failure under irregular distributions through neighborhood correlation, ensuring that nodes running specific tasks for a long time are correctly identified as normal. Furthermore, it introduces link correlation, indirectly assisting in judging the target node's state through the historical normality of associated nodes, avoiding the limitations of relying solely on the characteristics of isolated nodes. This improves the accuracy of data anomaly analysis.

[0048] Example 5, see Figure 1 This embodiment is based on the above embodiment. In step S4, the node quantitative evaluation is based on the comprehensive correlation degree, calculating the cohesion of normal clusters to reflect the consistency of normal nodes, and the inter-cluster differentiation between normal clusters and abnormal clusters to be investigated to reflect the differentiation between abnormal nodes and normal nodes, thereby realizing the node quantitative evaluation.

[0049] The specific steps are as follows: Calculate the cohesion of normal clusters. This method quantifies the correlation strength between nodes within a normal cluster and their own cluster. A higher value indicates stronger consistency among normal nodes. A sampling weakening mechanism is introduced, which dynamically weights the correlation between nodes within a cluster according to the sampling interval. As the workload of working nodes surges, the frequency of state changes increases significantly. At this point, the early sampled normal state data can no longer reflect the current normal pattern. Through the sampling weakening mechanism, the weight of older data is specifically weakened, as shown below: ; This is the current sampling round; yes The sampling rounds; T is the sampling round threshold; calculate the inter-cluster discrimination between normal clusters and abnormal clusters under investigation. This quantifies the boundary ambiguity of anomalous candidate nodes relative to normal clusters. A lower value indicates a clearer distinction between anomalous and normal clusters, expressed as: ; It is X i The overall correlation degree belonging to the l-th normal cluster; the inner summation is the node that is considered the most likely normal cluster among the abnormal candidate nodes.

[0050] Example 6, see Figure 1 This embodiment is based on the above embodiment. In step S5, the local preliminary judgment is based on the calculation of the abnormal index of the node to be tested based on the comprehensive correlation, cohesion and distinguishability. The local node is first judged, and the hierarchical abnormality warning is realized through multi-level abnormality thresholds.

[0051] The specific operation is as follows: Calculate the anomaly index of the node to be tested. , is represented as: The lower the maximum correlation between the node under test and the normal cluster, the higher the probability of an anomaly. The larger the value, the clearer the distinction between normal and abnormal, and the more reliable the abnormality index; graded early warning: when The assessment indicates a minor anomaly, and cloud-based collaborative anomaly verification is performed; when The corresponding performance continues to deteriorate, assessed as moderately abnormal, and an early warning is issued; when If the corresponding node does not respond, it is assessed as a serious anomaly, and an alert is issued and administrators are notified. , , and It is a tiered threshold.

[0052] Example 7, see Figure 1 This embodiment is based on the above embodiment. In step S6, the cloud collaborative anomaly verification is to upload the nodes that are initially assessed as slightly abnormal locally to the cloud. The cloud combines the multi-node association information to perform collaborative verification. Through global association analysis and cross-cluster consistency verification, the misjudgment of isolated nodes is avoided.

[0053] The specific steps are as follows: Calculate the co-verification value. A higher score indicates a more credible anomaly, expressed as: ;in, and It is the linkage coefficient; It is the comprehensive correlation degree of node X with normal cluster j; a verification threshold is set. If the collaborative verification value of a node is higher than the verification threshold, it is upgraded to a moderate anomaly; otherwise, it is judged as an isolated fluctuation and the node sampling frequency is increased; all parameters are optimized through particle swarm search algorithm.

[0054] By performing the above operations, this solution addresses the problems of general cloud-based collaborative data anomaly analysis methods, such as inability to adapt to dynamic loads of job nodes, anomaly assessments being detached from the context and lacking credibility, high false positive rates, and difficulty in handling node link dependencies, leading to poor data anomaly analysis results. This solution introduces a sampling weakening mechanism to calculate the cohesion of normal clusters within nodes, ensuring consistent assessment of normal clusters and always reflecting the current true normal mode of job nodes, avoiding misjudging normal nodes as anomalies due to load changes. It quantifies the consistency of current normal nodes by measuring the cohesion within normal clusters, and quantifies the clarity of the boundary between anomalies and normal nodes by distinguishing between clusters. Based on cloud-based collaborative verification and combined with global correlation, it avoids misjudging isolated nodes, thereby improving the effectiveness of data anomaly analysis.

[0055] Example 8, see Figure 2Based on the above embodiments, the cloud-based collaborative data anomaly analysis system based on job node information provided by the present invention includes a job node data acquisition module, a node status cluster initialization module, a node correlation calculation module, a node quantitative evaluation module, a local preliminary judgment module, and a cloud-based collaborative anomaly verification module.

[0056] The work node data acquisition module collects raw data from each work node in real time, and obtains the standardized feature vector of the node to be tested after preprocessing.

[0057] The node state cluster initialization module captures irregular distributions based on historical normal node data through neighborhood clustering, initializes normal state clusters, and temporarily classifies the nodes to be tested into abnormal clusters.

[0058] The node correlation calculation module introduces a link correlation factor to calculate the comprehensive correlation between the node under test and the normal cluster.

[0059] The node quantification evaluation module is based on a sampling weakening mechanism and calculates the cohesion of normal clusters and the inter-cluster discrimination between normal clusters and abnormal clusters based on the correlation degree.

[0060] The local preliminary judgment module calculates the anomaly index of the node to be tested and realizes hierarchical anomaly early warning through multi-level anomaly thresholds;

[0061] The cloud-based collaborative anomaly verification module performs cloud-based collaborative anomaly verification on nodes that are initially judged to be slightly abnormal locally.

[0062] It should be noted that, in this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0063] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention.

[0064] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A cloud-based collaborative data anomaly analysis method based on job node information, characterized in that: The method includes the following steps: Step S1: Data collection at work nodes. Raw data is collected in real time from each work node and preprocessed to obtain the standardized feature vector of the node to be tested. Step S2: Initialize the node state cluster. Based on historical normal node data, capture irregular distributions by dividing the neighborhood clustering degree, initialize the normal state cluster, and temporarily classify the node to be tested into the abnormal cluster to be investigated. Step S3: Node correlation calculation. Introduce the link correlation factor to calculate the comprehensive correlation between the node under test and the normal cluster. Step S4: Node quantification evaluation, based on the sampling weakening mechanism, calculate the cohesion of normal clusters and the inter-cluster discrimination between normal clusters and abnormal clusters under investigation by comprehensively considering the correlation degree; Step S5: Local initial judgment, calculate the anomaly index of the node to be tested, and realize hierarchical anomaly early warning through multi-level anomaly thresholds; Step S6: Cloud-based collaborative anomaly verification. Perform cloud-based collaborative anomaly verification on nodes that are initially identified as having minor anomalies locally. In step S3, the node correlation calculation is to calculate the dynamic correlation between the node under test and the normal cluster dynamic center, expressed as: ;in, X is the dynamic correlation degree between the node i under test and the normal cluster j; i It is the standardized feature vector of the i-th node to be tested; and , respectively, are the centers of normal cluster j and normal cluster l; m is the correlation coefficient; k is the total number of normal clusters, and l is the index of a normal cluster; the dynamic correlation degree between the node under test and the nearest node in a normal cluster is calculated and expressed as: ;in, It is the degree of neighborhood correlation; and These are the node samples that belong to normal cluster j and normal cluster l and are closest to the node i to be tested, respectively. It is the l-th normal cluster; It is the j-th normal cluster; a link correlation factor is introduced. Build comprehensive correlation The state of the target node is indirectly determined by associating the normality of the nodes under test, as shown below: ; ; , and It is a comprehensive weight; It represents the overall correlation degree of node X at the previous time step; It is the set of associated nodes of the node to be tested i, taking the first 3 nodes and the last 3 downstream nodes in the data transmission link; In step S4, the node quantization evaluation is to calculate the normal cluster cohesion. Furthermore, a sampling weakening mechanism is introduced, the core of which is to dynamically weight the correlation between nodes within a cluster according to the sampling round interval, expressed as: ; This is the current sampling round; yes The sampling rounds; T is the sampling round threshold; calculate the inter-cluster discrimination between normal clusters and abnormal clusters under investigation. The fuzziness of the boundary between abnormal candidate nodes and normal clusters is quantified as follows: ; It is X i The overall correlation degree belonging to the l-th normal cluster; C al It is an abnormal cluster to be investigated.

2. The cloud-based collaborative data anomaly analysis method based on job node information according to claim 1, characterized in that: In step S2, the node state cluster initialization selects the feature vector set of historically abnormal nodes, divides the clusters by neighborhood clustering degree, and temporarily classifies the feature vectors of the node to be tested into the abnormal cluster to be investigated.

3. The cloud-based collaborative data anomaly analysis method based on job node information according to claim 2, characterized in that: In step S5, the local initial judgment involves calculating the anomaly index of the node to be tested. And conduct tiered early warnings: when The assessment indicates a minor anomaly, and cloud-based collaborative anomaly verification is performed; when The corresponding performance continues to deteriorate, assessed as moderately abnormal, and an early warning is issued; when If the corresponding node does not respond, it is assessed as a serious anomaly, and an alert is issued and administrators are notified. , , and It is a tiered threshold.

4. The cloud-based collaborative data anomaly analysis method based on job node information according to claim 3, characterized in that: In step S6, the cloud-based collaborative anomaly verification involves uploading nodes initially assessed locally as slightly abnormal to the cloud, calculating collaborative verification values, and setting verification thresholds. If a node's collaborative verification value is higher than the verification threshold, it is upgraded to a moderate anomaly; otherwise, it is determined to be an isolated fluctuation, and the node sampling frequency is increased.

5. A cloud-based collaborative data anomaly analysis system based on job node information, used to implement the cloud-based collaborative data anomaly analysis method based on job node information as described in any one of claims 1-4, characterized in that: It includes a job node data acquisition module, a node status cluster initialization module, a node correlation calculation module, a node quantitative evaluation module, a local preliminary judgment module, and a cloud-based collaborative anomaly verification module; The work node data acquisition module collects raw data from each work node in real time, and obtains the standardized feature vector of the node to be tested after preprocessing. The node state cluster initialization module captures irregular distributions based on historical normal node data through neighborhood clustering, initializes normal state clusters, and temporarily classifies the nodes to be tested into abnormal clusters. The node correlation calculation module introduces a link correlation factor to calculate the comprehensive correlation between the node under test and the normal cluster. The node quantification evaluation module is based on a sampling weakening mechanism and calculates the cohesion of normal clusters and the inter-cluster discrimination between normal clusters and abnormal clusters based on the correlation degree. The local preliminary judgment module calculates the anomaly index of the node to be tested and realizes hierarchical anomaly early warning through multi-level anomaly thresholds; The cloud-based collaborative anomaly verification module performs cloud-based collaborative anomaly verification on nodes that are initially judged to be slightly abnormal locally.

Images