Network resource abnormal allocation scheduling processing method and system and storage medium

Abstract

The invention discloses a network resource abnormal allocation scheduling processing method and system and a storage medium, and the method comprises the following operation steps: collecting transmission data in a dynamic network environment in real time, and carrying out the preprocessing of the transmission data in the dynamic network environment, and obtaining the data of each network node; analyzing behavior data of the network node data, and clustering the network node data with the same behavior data by using a clustering algorithm to obtain abnormal network nodes; analyzing the abnormal network node according to a pre-collected historical database, and calculating a load score of the abnormal network node; and performing node data resource allocation on each node according to the load score of each network node data. The load condition of the network nodes is identified through clustering or combination of roughness, time, similarity and final confliction on the network nodes through various types of collected data in transmission, so that the abnormal distribution of the network nodes is adjusted.

Description

Technical Field

[0001] This invention relates to the field of networking, and in particular to a method, system, and storage medium for handling abnormal allocation and scheduling of network resources. Background Technology

[0002] With the rapid development of information technology, networks have become one of the fundamental infrastructures of modern society, widely used in various industries. The increasing number of devices, systems, and applications within networks has led to a growing demand for network resources. However, as network scale expands and complexity increases, the allocation, scheduling, and management of network resources face increasing challenges. Especially in dynamically changing network environments, how to efficiently and accurately allocate and schedule network resources, avoiding performance degradation due to improper resource allocation, has become a hot research topic.

[0003] Traditional network resource management methods often rely on static resource allocation strategies, which are ineffective in addressing changes in network load and the challenges of dynamic environments. These methods are typically based on preset rules or algorithms, lacking real-time data analysis and intelligent decision support, leading to resource overload or idleness in some nodes and impacting overall network performance.

[0004] With the continuous advancement of network technology, intelligent network resource management has gradually become an effective way to solve this problem. By collecting real-time traffic data and node status data in the network, and combining data analysis and machine learning algorithms, dynamic and intelligent scheduling of network resources can be achieved, and potential performance bottlenecks and resource abuse can be detected in a timely manner. In particular, using clustering algorithms to analyze network nodes can effectively identify nodes with abnormal loads or behaviors, further optimizing resource allocation and scheduling strategies.

[0005] However, existing intelligent network resource allocation methods still have some problems in practical applications. For example, how to accurately identify abnormal nodes, how to flexibly adjust resource allocation according to the load of network nodes, how to reduce false alarms and false negatives, and how to improve the accuracy of anomaly detection are still pressing issues in the field of network resource management. Summary of the Invention

[0006] The purpose of this invention is to provide a method, system, and storage medium for handling abnormal allocation and scheduling of network resources, which solves the above-mentioned technical problems pointed out in the prior art.

[0007] This invention provides a method for handling abnormal allocation and scheduling of network resources, comprising the following steps: Real-time acquisition of transmission data in a dynamic network environment; preprocessing of the transmission data in the dynamic network environment to obtain data of each network node.

[0008] The behavioral data of each network node is analyzed, and a clustering algorithm is used to cluster the network node data with the same behavioral data to obtain abnormal network nodes; the abnormal network nodes are analyzed based on the pre-collected historical database to calculate the load score of the abnormal network nodes.

[0009] Based on the load score of each network node's data, node data resources are allocated to each node.

[0010] Accordingly, this invention also proposes a network resource abnormal allocation and scheduling processing system, comprising: a data acquisition module; an analysis module; and a result module;

[0011] The acquisition module is used to acquire transmission data in a dynamic network environment in real time, preprocess the transmission data in the dynamic network environment, and obtain data of each network node.

[0012] The analysis module is used to analyze the behavioral data of each network node, and to use a clustering algorithm to cluster the network node data with the same behavioral data to obtain abnormal network nodes; and to analyze the abnormal network nodes based on the pre-collected historical database to calculate the load score of the abnormal network nodes.

[0013] The result module is used to allocate node data resources to each node based on the load score of the data of each network node.

[0014] Accordingly, the present invention also proposes a storage medium storing a computer program, which, when executed by a processor, implements the steps of the eye image recognition and segmentation method based on a convolutional neural network as described above.

[0015] Compared with the prior art, the embodiments of the present invention have at least the following technical advantages: Analysis of the above-mentioned network resource abnormal allocation and scheduling processing method, system and storage medium provided by the present invention shows that, in specific applications, the system can accurately capture the traffic characteristics of each network node, such as the number of data packets, total number of bytes, transmission interval, etc., through a pre-designed feature extraction model. These features comprehensively reflect the communication behavior and load patterns of nodes, helping to build behavioral data records for each node and laying the foundation for subsequent anomaly detection. By comparing with historical behavior records and calculating significant deviation comparison values, the system can automatically identify nodes that are significantly different from historical patterns, thereby marking potential abnormal activities. After using clustering algorithms to analyze the behavioral patterns of network nodes, the system optimizes the feature extraction model, making anomaly detection more accurate. By training the anomaly detection model with support vector machines, the node behavioral data is further classified in a secondary manner to identify time-related abnormal nodes, ensuring that the system can discover load anomalies and potential risk sources. Principal component analysis (PCA) further reduces the data dimensionality, retains the most representative behavioral features, and improves analysis efficiency. Finally, by calculating the deviation between the low-dimensional feature set and historical normal behavior patterns, the system can identify high-risk nodes and adjust or isolate them in a timely manner. This multi-level analysis method not only enhances the accuracy of network anomaly detection but also improves the system's ability to respond to potential threats, ensuring the stability and security of the network.

[0016] Furthermore, through high-precision data acquisition and standardized time series generation, the system can accurately capture the traffic fluctuation patterns of network nodes; automated algorithms analyze traffic attribute information to identify the maximum and minimum traffic values, helping to detect sudden traffic changes and abnormal fluctuations; when traffic exceeds a preset threshold, the system accurately reflects the trend of drastic traffic changes through the rate of change and the duration of the sudden event; based on this, the system re-screens time-related abnormal network nodes to identify more similar abnormal nodes; through continuous screening, it ensures accurate identification of load abnormal nodes, reduces false alarms, enhances the responsiveness of network management, and thus improves the stability and load balancing capabilities of the system.

[0017] Furthermore, by collecting and analyzing various resource metrics of network nodes (such as CPU utilization, memory usage, and network bandwidth), a resource utilization distribution matrix is ​​constructed to comprehensively understand the load status of each network node. By setting standardized thresholds, abnormal loads can be automatically identified, conflict events can be marked, and conflict frequencies can be calculated to help identify node regions that frequently experience overload. Next, based on node feature vectors, clustering algorithms are used to identify conflict-prone network nodes, forming high-incidence conflict areas. By calculating resource allocation similarity, resource allocation is optimized to avoid resource waste. Furthermore, the system optimizes load and schedules resources by calculating node remaining capacity and load scores to ensure stable system operation. The load score combines resource utilization, conflict frequency, resource similarity, and load trends, providing a precise basis for subsequent resource scheduling, helping to avoid overload and improve the overall performance and resource utilization efficiency of the system. Attached Figure Description

[0018] Figure 1 This is a flowchart of a network resource abnormal allocation and scheduling processing method according to Embodiment 1;

[0019] Figure 2 This is a flowchart illustrating the secondary evaluation load score of abnormal network nodes in a network resource abnormal allocation and scheduling processing method according to Embodiment 1.

[0020] Figure 3 This is a flowchart illustrating the process of obtaining load scores for abnormal network nodes in three steps according to a network resource abnormal allocation and scheduling processing method in Embodiment 1.

[0021] Figure 4 This is a schematic diagram illustrating the three steps of obtaining abnormal network nodes in a network resource abnormal allocation and scheduling processing method according to Embodiment 1.

[0022] Figure 5 This is a flowchart illustrating the four steps of obtaining load scores for abnormal network nodes in a network resource abnormal allocation and scheduling processing method according to Embodiment 1.

[0023] Figure 6 This is a schematic diagram illustrating the four steps of obtaining abnormal network nodes in a network resource abnormal allocation and scheduling processing method according to Embodiment 1.

[0024] Figure 7 This is a flowchart of a network resource abnormal allocation and scheduling processing system according to Embodiment 2;

[0025] Figure 8 This is a schematic diagram of a storage medium according to Embodiment 3;

[0026] Labels: Acquisition module 10; Analysis module 20; Result module 30; Processor 1110; Communication interface 1120; Memory 1130; Computer storage medium 1140. Detailed Implementation

[0027] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings.

[0029] Example 1 like Figure 1 As shown, this embodiment of the invention provides a method for handling abnormal allocation and scheduling of network resources, including the following steps:

[0030] S1: Real-time acquisition of transmission data in the dynamic network environment, preprocessing of the transmission data in the dynamic network environment to obtain data of each network node;

[0031] S2: Analyze the behavioral data of each network node, and use a clustering algorithm to cluster the network node data with the same behavioral data to obtain abnormal network nodes; analyze the abnormal network nodes according to the pre-collected historical database, and calculate the load score of the abnormal network nodes.

[0032] It should be noted that clustering algorithms can categorize similar nodes based on their data behavior and characteristics, helping to identify nodes exhibiting abnormal behavior. These abnormal nodes may be manifestations of problems such as excessive load, bandwidth exceeding limits, or frequent disconnections. Through comparative analysis of historical databases, load scores are applied to the identified abnormal network nodes (i.e., abnormal network nodes include the following: coarse clustering abnormal network nodes, time-related abnormal network nodes, similarity-related abnormal network nodes, and conflict-related abnormal network nodes) to quantitatively assess the degree of abnormality of each node. Load scores can accurately measure the severity of the anomaly, facilitating subsequent decision-making.

[0033] Cluster analysis can efficiently group nodes with similar behaviors, thereby identifying nodes with abnormal load behavior. This can effectively reduce false alarms and false negatives and improve the accuracy of anomaly detection. By comparing historical data, the system can identify long-term and potential performance bottlenecks or resource abuse, accurately identify abnormal nodes, and evaluate them based on historical behavior. This increases the intelligence of decision-making, helps the system identify abnormal nodes, anticipate potential performance problems, and thus take measures in advance to reduce system load and improve system stability.

[0034] The introduction of load scoring helps the system accurately allocate resources to each node; if a node's load score is abnormally high, the system can take action based on the score, such as allocating more resources or offloading tasks, to prevent node overload from affecting the overall network performance.

[0035] S3: Allocate node data resources to each node based on the load score of each network node's data.

[0036] It's important to note that based on load scoring, the system can intelligently allocate resources. When a node has a high load score, the system can alleviate the pressure by transferring its load to other nodes or providing additional resources. Through optimized resource allocation, the system can reduce the risk of node overload and improve the overall network efficiency and reliability. The goal is to avoid overloading a single node by allocating resources rationally, ensuring balanced resource utilization across all nodes. By reducing the pressure on nodes with high loads, the overall stability and response speed of the system can be improved.

[0037] Based on changes in node load scores, the system can dynamically adjust resource allocation to meet different load demands. Changes in load scores reflect the resource consumption status of each node. Reasonable resource allocation can effectively optimize performance, improve user experience, and realize the system's adaptive capability. It can dynamically adjust resources based on real-time data and load conditions, reduce the risk of overloaded nodes, and ensure the efficient and stable operation of the system.

[0038] Specifically, such as Figure 2 As shown, in step S2, behavioral data of each network node is analyzed, and clustering algorithms are used to cluster network node data with the same behavioral data to obtain abnormal network nodes; the abnormal network nodes are then analyzed based on the pre-collected historical database to calculate the load score of the abnormal network nodes. The specific operation steps are as follows:

[0039] S21: Analyze the traffic statistics characteristics (such as the number of data packets, total number of bytes, transmission interval, peak traffic, average traffic, etc.) of the data of each network node using a pre-designed feature extraction model, and extract typical behavioral characteristics (such as the communication frequency of the node in a specific time period, periodic activity patterns, frequency of connection establishment and disconnection, etc.).

[0040] The typical behavioral characteristics are stored as structured data to form behavioral data records for each network node;

[0041] It should be noted that by designing a pre-designed feature extraction method, key statistical features are extracted from the data stream of network nodes. These features include the number of data packets, total number of bytes, peak traffic, etc., which can comprehensively reflect the activity patterns of network nodes. The purpose is to establish a record of the behavioral characteristics of each network node to provide basic data for subsequent analysis.

[0042] By accurately and comprehensively recording the behavioral characteristics of nodes, high-quality input data is provided for subsequent anomaly detection, helping to detect potential abnormal activities;

[0043] Feature extraction and behavior pattern recognition can accurately describe the behavior of network nodes under different load conditions and capture key features that affect load changes (such as the number of data packets, transmission intervals, etc.). These features can effectively help the system determine whether the load exceeds the normal working range and react quickly.

[0044] Feature extraction helps capture the behavioral characteristics of network nodes, including information such as traffic and latency; these characteristics can help identify abnormal fluctuations in network load; when the traffic or other behavioral characteristics of certain nodes deviate significantly from the normal pattern, they can be marked as abnormal in a timely manner, thereby effectively allocating network load and avoiding the situation where resources are over-concentrated on abnormal nodes;

[0045] S22: Collect historical database; extract historical behavior data records corresponding to the current behavior data records and extract historical normal behavior patterns from the historical database;

[0046] The current behavioral data record is compared with the historical behavioral data record by a significant deviation comparison value (that is, the significant deviation comparison value can be calculated and compared by using statistical methods to calculate the mean, variance, extreme values ​​and other indicators between the current behavioral data record and the historical behavioral data record, so as to reflect whether there is a deviation in the current behavioral data).

[0047] A preset deviation threshold q is set; it is then determined whether the significant deviation comparison value is greater than the deviation threshold q.

[0048] If so, then anomaly labeling is performed on the network nodes corresponding to the current behavior data records that exceed the deviation threshold q, resulting in coarse clustering of abnormal network nodes;

[0049] If not, then the remaining network nodes are labeled normally to obtain normal network nodes;

[0050] It should be noted that by comparing with historical behavior records, significantly deviating values ​​are calculated to automatically identify network nodes exhibiting abnormal behavior. This comparison quickly identifies nodes with anomalous behavior. Simultaneously, the extracted historical normal behavior pattern refers to each node's resource requirements and usage (such as CPU, memory, network bandwidth, etc.) at different time periods. The historical normal behavior pattern reveals resource usage fluctuations during normal node operation, helping the system determine which resource usages are common and normal. Nodes may experience different load levels at different times. The historical normal behavior pattern can identify which load fluctuations are within the normal range and which exceed it by statistically analyzing past load fluctuations. These fluctuations vary across different time periods or events. The historical normal behavior pattern considers these temporal factors, helping the system understand the behavioral characteristics of nodes within specific time periods, thereby identifying which periods of resource demand and load fluctuations are normal.

[0051] This comparison helps the system automatically identify nodes that are significantly different from historical behavior. These nodes usually represent potential sources of attack or failure. Early detection of abnormal behavior helps to take timely measures and reduce potential network risks.

[0052] By comparing historical behavior records, the system can identify which nodes' load or behavior deviates significantly from the normal pattern. For these nodes with large deviations, the system can reallocate the load in the later stages to avoid assigning too many requests or tasks to these potentially abnormal nodes, thereby reducing the system's burden and preventing resource overload or performance degradation.

[0053] The purpose of coarse clustering of abnormal network nodes is to perform preliminary anomaly detection and isolation; to initially identify nodes in the system that exhibit obvious load anomalies and separate them from normal nodes; and to initially screen network nodes with abnormal load performance by setting standardized resource thresholds and marking conflict events. These nodes may exhibit high resource consumption, exceeding normal working load. It can quickly find "hot spots" nodes with abnormal loads in the system, but these nodes may only exhibit extreme load conditions and do not necessarily represent persistent resource needs. They represent the initial load imbalance and potential resource bottlenecks in the system. It can quickly identify nodes that need attention, but it is not deep enough in understanding the causes of these nodes' anomalies and subsequent processing.

[0054] S23: Using a clustering algorithm, the coarsely clustered abnormal network nodes and normal network nodes are clustered separately based on typical behavioral characteristics (i.e., clustering statistics based on similar behavioral patterns). Based on the clustering results, the designed feature extraction model is optimized (e.g., adjusting weights, thresholds, window sizes, etc.). The traffic statistical characteristics of each network node data are re-analyzed to obtain an optimized node behavior dataset (i.e., a dataset formed by re-extracting new behavioral data records for each network node, reflecting the behavioral patterns of the network nodes).

[0055] It should be noted that clustering methods are used to group abnormal and normal nodes, clustering based on similar behavioral patterns, and optimizing feature extraction methods based on the clustering results. By grouping network nodes according to similar behavioral patterns through clustering, more behavioral patterns can be identified, thereby improving the accuracy of subsequent analysis.

[0056] Clustering classifies nodes based on similar behavioral patterns, improving the effectiveness of anomaly detection. By optimizing feature extraction methods, it is possible to more accurately identify the behavior of network nodes.

[0057] By clustering, the system can classify network nodes according to behavioral patterns, thus identifying which nodes exhibit abnormal load handling. In later stages, for example, some nodes may still exhibit normal behavior under high load, while other nodes may exhibit abnormal behavior when the load increases. By optimizing the behavioral dataset, the load distribution strategy can be adjusted more accurately to distribute the load reasonably to nodes that are performing normally, ensuring stable system operation.

[0058] S24: Use a support vector machine to train a model on the historical behavior data records to obtain an anomaly detection model;

[0059] The anomaly detection model is used to perform secondary classification and labeling of the node behavior dataset to obtain time-related anomaly network nodes.

[0060] A preset time window (e.g., the past few minutes or hours) is used to extract typical behavioral features of the time-abnormal network nodes within that preset time window (i.e., mainly to extract behavioral features within a specific time period).

[0061] It should be noted that by analyzing and optimizing the node behavior data, each node is classified to further identify potential abnormal network nodes. Through further classification, it is ensured that the network node behavior data can more accurately reflect whether there are anomalies, and that each node can be classified more accurately, thereby improving the accuracy of anomaly detection and reducing the possibility of misjudgment.

[0062] The categorized data helps to further and more accurately identify the load capacity of each node; if some nodes are classified as high-load-capacity nodes, the system can allocate more tasks to these nodes, while marking nodes with lower load as low priority, thus avoiding abnormal load distribution that could lead to system overload or response delays.

[0063] The purpose of clustering time-related anomaly network nodes is to aggregate anomalies in the time dimension; to analyze and aggregate anomaly load nodes after a period of time, and further analyze whether these nodes are the "hardest hit areas" of long-term or frequent overload of the system; based on the first clustering (i.e., rough clustering of anomaly network nodes), these nodes are analyzed in the time dimension to identify which nodes have multiple load anomalies within a certain period of time, or whose anomalies last for a long time, which can be determined by characteristics such as event frequency and load volatility.

[0064] The second clustering no longer relies solely on instantaneous resource consumption but focuses more on the persistence and frequency of load, thus preventing occasional load anomalies from being incorrectly identified as long-term problems. The second clustering demonstrates the persistent identification of load anomaly issues; through aggregation along the time dimension, it can identify the true sources of load anomalies. These nodes may have a long-term impact on system stability, therefore requiring in-depth analysis and processing.

[0065] S25: Construct a covariance matrix by standardizing the typical behavioral characteristics within the preset time window using principal component analysis;

[0066] The covariance matrix is ​​subjected to eigenvalue decomposition to extract the principal components and their corresponding variance contribution rates. The calculation formula is as follows: ;

[0067] In the formula, The number of network nodes exhibiting temporal anomalies;

[0068] The number of typical behavioral features of the i-th temporally anomalous network node (or the number of principal components retained by PCA) reflects the complexity and diversity of node behavior patterns in this temporally anomalous network node.

[0069] Indicates the first The first of the time-anomaly network nodes The variance (i.e., eigenvalue) of a typical behavioral characteristic is used to measure the contribution of that characteristic to overall behavioral performance.

[0070] It is a normalization constant used to ensure the consistency of the dimensions of the exponential weights;

[0071] Represented as a weighting correction factor;

[0072] Represented as the first The first of the time-anomaly network nodes The variance (i.e., eigenvalue) of a typical behavioral characteristic;

[0073] Represented as the first The sum of the variances of all typical behavioral characteristics of a temporally anomalous network node;

[0074] It should be noted that PCA is used to standardize high-dimensional typical behavioral features through the covariance matrix and extract principal components. This dimensionality reduction method can effectively simplify the complexity of data and retain key behavioral patterns. It reduces high-dimensional data to low-dimensional data, which is convenient for subsequent analysis and processing. Dimensionality reduction reduces redundant information, making the analysis more efficient and concise. It is crucial for the processing and understanding of high-dimensional data and helps to improve the efficiency of the overall analysis.

[0075] After dimensionality reduction, Principal Component Analysis (PCA) can remove irrelevant or redundant data, allowing the system to focus on the most important features. In this way, when allocating load, the system can more accurately judge the load capacity of nodes based on their main behavioral characteristics, thereby avoiding incorrect load allocation caused by too many irrelevant features.

[0076] S26: By using the variance contribution rate, the top K typical behavioral features of each network node are retained, and finally a low-dimensional behavioral feature set of time-abnormal network nodes is formed.

[0077] It should be noted that, based on the results of principal component analysis, the most representative behavioral features of each network node are retained to form a low-dimensional behavioral feature set. This low-dimensional behavioral feature set improves the efficiency of subsequent analysis and reduces computational overhead. The low-dimensional feature set helps reduce computational burden and improve the system's response speed, especially when processing large-scale data, where it can significantly improve performance.

[0078] Low-dimensional feature sets can simplify node behavior analysis and focus on the most representative load characteristics. This enables the system to identify load anomalies more quickly and adjust load distribution strategies in real time. For example, when the system detects that the load behavior characteristics of some nodes are abnormal in the low-dimensional data space, it can take immediate measures to redistribute the load and prevent abnormal load from affecting the overall system performance.

[0079] S27: Calculate the deviation between the historical normal behavior pattern in the historical database and the low-dimensional behavior feature set of each time-abnormal network node (i.e., the deviation can be judged by calculating the average value and standard of the features or patterns of each node, and then the deviation between the two is obtained by calculating the Euclidean distance between the low-dimensional behavior feature set and the historical normal behavior pattern based on the historical normal behavior pattern).

[0080] Set a preset deviation threshold e; determine whether the calculated deviation is greater than the deviation threshold e;

[0081] If so, the temporal abnormal network nodes corresponding to the low-dimensional behavioral feature set are determined to be high-risk nodes, and a list of high-risk nodes is generated.

[0082] It should be noted that by calculating the deviation between the low-dimensional behavioral characteristics of each abnormal node and its historical normal behavior pattern, high-risk nodes can be further identified. By calculating the deviation, it is determined whether a node deviates from the normal behavior pattern and high-risk nodes are marked. This helps to accurately distinguish high-risk nodes, which may represent potential threats in the system, such as attacks and virus propagation. Timely detection and marking of high-risk nodes helps to strengthen network security protection.

[0083] By calculating node deviations and identifying high-risk nodes, the system can adjust its load distribution strategy in a timely manner when abnormal loads are detected. For nodes exhibiting abnormal deviations, load distribution can be reduced or they can be isolated directly to ensure that these nodes do not slow down the performance of the entire system. In this way, the system can distribute load more intelligently and prevent the load of abnormal nodes from affecting the performance of other normal nodes.

[0084] S28: Collect traffic attribute information for the time-abnormal network nodes, calculate the rate of change of traffic based on the adjacent time points corresponding to the traffic attribute information; analyze the traffic status change trend of the time-abnormal network nodes based on the rate of change, cluster the time-abnormal network nodes based on the traffic status change trend to obtain similar abnormal network nodes; evaluate the indicators of various resources of the similar abnormal network nodes, and calculate the load score based on the indicators of various resources.

[0085] It should be noted that by collecting traffic attribute information of network nodes with temporal anomalies and calculating the rate of traffic change between adjacent time points, the rate of traffic change of that node can be obtained. This allows for real-time tracking and measurement of traffic fluctuation trends, providing dynamic real-time data support for subsequent analysis. The purpose of calculating the rate of traffic change is to understand the fluctuation of node traffic, especially when a node has already become abnormal. Observing the speed and direction of its traffic changes can more accurately assess the trend of node load and provide early warnings of possible overload or anomalies. The rate of traffic change reflects the changes in network load status. By tracking and analyzing these rates, potential network bottlenecks or node failures can be detected earlier, which helps to make timely adjustments and avoid system crashes or performance degradation.

[0086] Clustering time-related anomaly network nodes based on traffic status change trends yields similar anomaly network nodes. Grouping similar traffic change patterns together allows for better identification and classification of different types of anomalies, helping the system accurately segment different types of load anomalies. The purpose of clustering is to further subdivide different types of anomalies within anomaly nodes. By observing traffic status change trends, the system can identify which nodes exhibit short-term fluctuations and which show long-term unstable trends, thus providing more precise load balancing solutions. By clustering traffic trends, the system can classify and manage different types of load anomalies, improving the efficiency of anomaly management and making load allocation and resource scheduling more targeted. Further clustering can help avoid over-intervention, preventing the need to react the same way to every anomaly node, thereby optimizing resource utilization.

[0087] This system analyzes various resource metrics of network nodes exhibiting asymmetry in similarity and calculates load scores based on these metrics. By comprehensively evaluating a node's resource usage (such as CPU, memory, and bandwidth), the system arrives at its load score. Through load score calculation, the system can comprehensively assess the load status of each node and allocate appropriate resources accordingly. If a node has a high load score, indicating severe abnormal load, the system will prioritize adjusting its resource configuration for load balancing. The introduction of load scores allows for quantitative analysis of each node's load, reducing the subjectivity of human intervention and automating resource allocation and optimization. By comprehensively analyzing multiple resource metrics, the system ensures that node resource needs are fully considered, avoiding uneven resource allocation and ensuring network stability.

[0088] Specifically, such as Figure 3As shown, in step S28, traffic attribute information is collected from the time-abnormal network nodes, and the rate of change of traffic is calculated based on the adjacent time points corresponding to the traffic attribute information. The traffic status change trend of the time-abnormal network nodes is analyzed based on the rate of change, and the time-abnormal network nodes are clustered based on the traffic status change trend to obtain similar abnormal network nodes. Indicators of various resources for the similar abnormal network nodes are then calculated, and load scores are calculated using these indicators. The specific operation steps are as follows:

[0089] S281: Collect traffic attribute information (i.e., traffic attribute information includes transmission rate, number of data packets, number of bytes, etc. within each time window, to ensure the continuity and temporality of data) for the time-abnormal network nodes in the high-risk node list, and arrange the traffic attribute information of each time-abnormal network node in the order of the collection time interval to form a standard time series (i.e., arranged in the time order of the collected traffic attribute information).

[0090] It should be noted that the system first collects traffic attribute information from network nodes in the high-risk node list that exhibit time-related anomalies. The collected traffic attribute information includes important information such as transmission rate, number of data packets, and number of bytes within each time window. The key is to ensure the continuity and temporal sequence of the data, ensuring that the data at each point in time is completely recorded. The collected traffic attribute information is sorted by time interval to form a standard time series. The standard time series represents the order in which the traffic attribute information changes at different points in time.

[0091] Ensuring the timeliness and continuity of data, and ensuring the order of data, is crucial, because subsequent anomaly detection requires judging the pattern of traffic fluctuations based on the time sequence.

[0092] Through high-precision data acquisition and standardization, the load status of network nodes can be obtained in real time, ensuring the continuity and timeliness of the data, providing a solid data foundation for anomaly detection and load analysis; the raw traffic attribute information collected is transformed into a standardized time series, reducing noise in the data and thus improving data comparability.

[0093] S282: Use an automated algorithm to determine the maximum flow value of the flow attribute information within a time period of the standard time series (that is, to determine which time period in the standard time series has the maximum flow), and record the occurrence time and duration (that is, the length of the time period) of the time period corresponding to the maximum flow value.

[0094] And determine the minimum flow value of the flow attribute information within the time period, and record the occurrence time and duration of the time period corresponding to the minimum flow value;

[0095] It should be noted that the automated algorithm is used to analyze the flow attribute information in the standard time series. First, the maximum and minimum flow values ​​within a time period are determined. The time period corresponding to the maximum flow value, as well as the time and duration of its occurrence, are recorded. Similarly, the time period corresponding to the minimum flow value, as well as the time and duration of its occurrence, are recorded.

[0096] By identifying the maximum and minimum traffic values ​​in a time series, extreme values ​​of traffic fluctuations can be determined. These extreme values ​​can help identify patterns in network traffic fluctuations, especially the occurrence of abnormal fluctuations. The maximum traffic value may indicate a sudden increase in traffic, while the minimum traffic value may indicate a decrease in network performance or an abnormal decrease in traffic.

[0097] By calculating the maximum and minimum traffic values, extreme load conditions of network nodes within a certain time period can be identified. These extreme load conditions (such as sudden surges or extreme drops in traffic) may indicate the onset of load anomalies, providing early warning signals for timely adjustments to load distribution. By identifying the maximum traffic value of the load, load concentration can be detected in a timely manner, providing network administrators with signals of overload, thereby enabling them to take traffic scheduling measures in advance.

[0098] S283: Preset peak flow threshold r; Determine whether the maximum flow value of the flow attribute information is greater than the peak flow threshold r;

[0099] If so, the maximum flow value of that flow attribute information is determined as an abnormal fluctuation;

[0100] The occurrence time and duration of the traffic attribute information are used as the duration of the sudden abnormal fluctuation.

[0101] The number of abnormal fluctuations in the traffic attribute information of the corresponding time-sensitive network node is statistically analyzed within a specified period based on the duration of the sudden event.

[0102] For each abnormal fluctuation, the flow attribute information is calculated to determine the flow difference between adjacent time points before and after the fluctuation, which is used as the rate of change (that is, the rate of change between the flow attribute information of normal fluctuation and abnormal fluctuation, which is the acceleration from normal fluctuation to abnormal fluctuation, reflecting the severity of flow change in the entire time series).

[0103] It should be noted that a peak flow threshold (r) is preset, and it is determined whether the maximum flow value collected exceeds the threshold; if the maximum flow value is greater than the threshold, the flow attribute information is determined to be an abnormal fluctuation; the sudden time of the flow attribute information is statistically analyzed, including the time of occurrence and the duration.

[0104] Each abnormal fluctuation calculates the difference in traffic between adjacent time points. This difference is called the rate of change, which reflects the severity of the traffic change. Abnormal fluctuations are usually accompanied by a large rate of change, indicating that the traffic of network nodes has changed rapidly and drastically.

[0105] By setting a peak traffic threshold, sudden traffic fluctuations can be effectively filtered out. By calculating the rate of change, the severity of the fluctuations can be further analyzed. This allows for the accurate identification of abnormal fluctuations and an assessment of their potential impact on network stability. Calculating the rate of change helps the system better understand drastic traffic fluctuations, thereby providing valuable diagnostic information for network administrators.

[0106] By setting peak traffic thresholds and detecting sudden fluctuations, abnormal load fluctuation events can be identified. For example, when network node traffic fluctuates significantly, it may indicate traffic conflicts or system failures. In this case, network administrators can take emergency measures to adjust the load and prevent network crashes. Detecting sudden fluctuations can help identify load imbalances. By analyzing the severity and duration of the fluctuations, administrators can perform load balancing on abnormal nodes, thereby preventing a node from bearing excessive load for an extended period.

[0107] S284: Analyze the trend of flow status changes based on the rate of change and the duration of the sudden event (i.e., the trend of flow status changes refers to the severity of flow fluctuations and the duration of the fluctuations; the rate of change reflects the severity of flow fluctuations, a large rate of change indicates a high degree of severity, and a small rate of change indicates a low degree of severity; and combined with the duration of the sudden event, it reflects whether the trend of flow change is large or small).

[0108] The network nodes with temporal anomalies are re-screened based on the trend of traffic status changes to obtain similar anomaly network nodes (that is, the screening of similar anomaly network nodes is based on the amplitude of abnormal fluctuations in traffic; that is, the greater the rate of change and the longer the duration, the more likely the network node is anomaly. Therefore, by re-screening the network nodes with temporal anomalies based on the trend of traffic status changes, a threshold can be set to judge the rate and duration of the trend of traffic status changes and screen similar anomaly network nodes).

[0109] It should be noted that the analysis is based on the rate of change and the duration of the sudden event; the trend of traffic status change describes the severity and duration of traffic fluctuations; a large rate of change indicates severe fluctuations, while a small rate of change means relatively gentle fluctuations; the trend of traffic status change combined with the duration of the sudden event can reflect the overall trend of traffic changes, and thus assess whether the load of network nodes is normal or whether there are potential problems.

[0110] Analyzing traffic status change trends can help distinguish between normal and abnormal fluctuations; combining the rate of change and the duration of the sudden change can provide a clearer understanding of the traffic change trend; the purpose is to re-screen nodes based on the traffic change trend, identify which nodes may experience more severe abnormal fluctuations, and perform more precise screening by setting thresholds.

[0111] By analyzing traffic status change trends, we can understand the regularity of load changes. This analysis not only focuses on traffic extremes, but also delves into the amplitude and persistence of traffic fluctuations. Based on this information, we can more accurately determine whether a node is in an abnormal load state. Analyzing traffic status change trends helps administrators determine whether certain nodes have persistent abnormal load problems, thereby enabling load balancing scheduling and avoiding the occurrence of system bottlenecks.

[0112] The purpose of filtering out network nodes with anomalous similarity is to perform deep clustering based on resource similarity, such as... Figure 4 As shown; based on the trend of traffic status changes, the time-related abnormal network nodes are re-screened to obtain the third abnormal nodes (i.e., similar abnormal network nodes); by analyzing the amplitude (rate of change) and duration (burst time) of traffic fluctuations, more serious or potential abnormal nodes are screened out; reasonable thresholds are set to judge the rate and duration of traffic status change trends, and nodes that may be abnormal are identified based on these thresholds.

[0113] By filtering nodes with large and prolonged traffic fluctuations, truly abnormal nodes can be identified more accurately. Unlike the first (coarse clustering of abnormal network nodes) and the second (temporal abnormal network nodes) filtering, this round of filtering focuses on the severity and persistence of fluctuations to ensure that nodes that may cause network problems are detected earlier. The purpose of this process is to reduce false positives and false negatives, and to further improve the accuracy of abnormal node identification through refined analysis. After three rounds of filtering, nodes with abnormal loads can be identified, ensuring that abnormal nodes are prioritized and their loads are redistributed to prevent them from continuously bearing overload pressure, thereby improving the overall system stability.

[0114] S285: Collect various resource indicators for each of the similarity anomaly network nodes, calculate the conflict frequency exceeding resource occupancy for each resource indicator; cluster the similarity anomaly network nodes according to the conflict frequency to obtain conflict anomaly network nodes; calculate the remaining capacity in the conflict anomaly network nodes according to the various resource indicators; calculate the load score of the conflict anomaly network nodes based on the remaining capacity.

[0115] It should be noted that, firstly, by collecting various resource metrics (such as CPU, memory, bandwidth, etc.) from network nodes with abnormal similarity and calculating the frequency of conflicts exceeding resource usage, the frequency of resource conflicts can be identified, and it can be determined in which aspects nodes frequently experience resource contention. The purpose of calculating the conflict frequency is to gain a deeper understanding of resource usage conflicts and bottlenecks, especially since frequent conflicts may indicate insufficient or unreasonable node resource allocation. By capturing this conflict information, resource bottlenecks can be identified, thereby helping to optimize resource management strategies. Conflict frequency is an important indicator for measuring resource contention; frequent conflicts often lead to network performance degradation or resource imbalance. By calculating the conflict frequency, nodes in a high resource contention state can be better identified, providing support for subsequent resource scheduling and load management.

[0116] Based on the frequency of conflicts, similarity-related abnormal network nodes are clustered to obtain conflict-prone abnormal network nodes. These nodes are then classified according to the characteristics of resource conflicts, allowing for more accurate identification of which nodes have similar resource problems. The purpose of cluster analysis is to refine the classification of abnormal network nodes, enabling different types of nodes to be grouped according to their resource conflict characteristics, facilitating further targeted optimization measures. For example, some nodes may require more resources, while others may need to optimize their resource allocation strategies. Through clustering, nodes with more severe resource conflicts can be grouped together, achieving more targeted resource optimization. This helps the system reduce the overall frequency of resource conflicts, improves the network's load balancing capabilities, and ensures the fairness and efficiency of resource allocation.

[0117] Based on various resource indicators, the remaining capacity of conflict-prone network nodes is calculated. Remaining capacity refers to the amount of additional resources a node can still support under the current resource configuration. By calculating remaining capacity, the system can clearly understand the available space for each node in terms of resource demand and allocation. The purpose of calculating remaining capacity is to understand the resource utilization rate and remaining resources of each node, thereby making more effective use of remaining capacity during resource scheduling. It helps determine whether a node has the capacity to further support higher loads or whether it needs to expand resources to cope with the current load. Remaining capacity reflects the resource health status of network nodes. By monitoring remaining capacity, it is possible to identify which nodes are about to reach their resource limits in advance, avoiding overloading and system failures, and providing a scientific basis for dynamic resource scheduling and load prediction.

[0118] Based on remaining capacity, load scores are calculated for network nodes exhibiting conflict anomalies. Load scores reflect the load health status of nodes by comprehensively evaluating their remaining capacity and resource usage. The scores help the system quantify node load status and determine whether resource optimization or adjustment is necessary. The purpose of load scoring is to quantify node load status, facilitating performance evaluation. Through load scoring, it's possible to quickly identify which nodes are carrying excessive loads and which are relatively idle, enabling resource reallocation or optimization. Load scoring provides an intuitive load status assessment in a quantitative manner, helping to avoid system crashes or delays due to overload. It also provides an objective basis for subsequent resource allocation, ensuring timely adjustments under high load conditions.

[0119] Specifically, such as Figure 5 As shown, in step S285, various resource indicators of each of the similarity anomaly network nodes are collected, and the frequency of conflicts exceeding resource occupancy is calculated for each resource indicator; the similarity anomaly network nodes are clustered according to the conflict frequency to obtain conflict-prone anomaly network nodes; the remaining capacity of the conflict-prone network nodes is calculated according to the various resource indicators; and the load score of the conflict-prone network nodes is calculated based on the remaining capacity. The specific operation steps are as follows:

[0120] S2851: Collect various resource indicators within the similarity anomaly network nodes;

[0121] A resource occupancy distribution matrix is ​​constructed for the similarity anomaly network nodes and the corresponding resource indicators within the similarity anomaly network nodes;

[0122] The rows of the resource occupancy distribution matrix represent each similarity anomaly network node, and the columns represent the various resource indicators corresponding to each similarity anomaly network node.

[0123] It should be noted that each network node consumes various system or network resources during operation. These resources typically include, but are not limited to: CPU utilization: the percentage of processor time used by a node within a certain period; memory utilization: the actual physical memory size and utilization rate of a node; network bandwidth utilization: the bandwidth consumption of a node during data transmission; IO read / write load: the frequency and speed of read / write operations on disks or storage devices; and the number of connections or sessions: recording the number of connections or sessions processed simultaneously by a node (particularly important for network nodes). This resource information causes the resource consumption of various similarly anomalous network nodes. Integrating this resource consumption information yields various resource metrics, providing a comprehensive understanding of the load of each network node. Integrating this resource information into a resource consumption distribution matrix effectively tracks the resource consumption of each node across different resources.

[0124] The matrix-style distribution of resource usage facilitates subsequent analysis and comparison, making the resource usage of different nodes clear at a glance. This helps identify nodes with excessive resource consumption, thereby predicting and addressing potential system overload or performance bottlenecks. The resource usage matrix structures the resource usage of each node, making it easier for subsequent analysis, clustering, and optimization.

[0125] By collecting various resource metrics (such as CPU utilization, memory usage, bandwidth, etc.), we can gain a comprehensive understanding of the load status of each network node. This provides a benchmark for subsequent load anomaly allocation and can accurately determine which nodes have exceeded the normal load range. Through the resource utilization distribution matrix, the system can display the resource consumption patterns of each node, which is crucial for detecting load anomalies. Matrix processing facilitates subsequent analysis of which nodes are overloaded or have abnormal resource consumption, thereby triggering resource scheduling.

[0126] S2852: Preset a standardized threshold z for resource occupancy for each type of resource indicator; determine whether the corresponding resource indicators within the similarity anomaly network node are greater than the standardized threshold z (i.e., the standardized threshold is used to judge each resource indicator, and a relatively average standard indicator is used to judge the resource occupancy and overload of the entire similarity anomaly network node).

[0127] When any of the resource indicators exceeds the standardized resource occupancy threshold z, the time corresponding to the resource indicator that exceeds the standardized resource occupancy threshold z is marked as a conflict event; the conflict frequency of all conflict events (that is, the frequency of the node being in a resource overload or abnormal state) is calculated.

[0128] It should be noted that by setting standardized thresholds for each resource metric, conflict event marking can be automatically triggered when resource usage is abnormal. By marking the time of conflict, the state of a node when resource usage exceeds limits can be clearly identified; by calculating the frequency of conflicts, the degree of overload of a node can be quantified, providing data support for subsequent optimization decisions.

[0129] By setting thresholds, abnormal loads can be quickly identified and alarms can be issued in a timely manner, preventing network nodes from crashing or experiencing significant performance degradation due to resource overload. Recording the frequency of conflicts helps analyze the severity of load problems and reflects the load fluctuations of nodes over a period of time, thereby determining whether resource scheduling is necessary.

[0130] By setting standardized thresholds for each resource metric, a conflict event can be automatically marked when the node load exceeds the normal range. The marking of a conflict event directly reflects the abnormal load of a node at a certain point in time. By calculating the frequency of conflicts, the system can identify which nodes frequently experience resource overload events, thereby helping to identify and locate node areas with high load anomalies. These nodes will become the key targets for subsequent load scheduling and optimization.

[0131] S2853: Construct node feature vectors for similarity-abnormal network nodes using the aforementioned resource indicators and conflict frequencies;

[0132] Clustering of the similarity anomaly network nodes based on the conflict frequency of the clustering algorithm according to the node feature vectors yields conflict anomaly network nodes, which are then identified as high-incidence conflict areas.

[0133] It should be noted that combining resource indicators with conflict frequency to construct node feature vectors provides data support for subsequent clustering analysis; clustering algorithms are used to group nodes and identify "high-conflict areas" (i.e., conflict-prone network nodes), which are node areas where resource overload occurs frequently.

[0134] Cluster analysis can identify nodes with high frequency of resource conflicts, facilitating centralized resource optimization. The clustered node feature vectors provide different load patterns for different nodes, enabling more targeted resource scheduling.

[0135] Clustering algorithms can group nodes with similar resource loads together and mark them as "high-incidence conflict areas". These areas have a high frequency of abnormal node loads, and centralized processing of these nodes can help the system optimize resource scheduling. The clustered node feature vectors help the system locate areas where resource load problems are concentrated, thereby enabling targeted load allocation or adjustment.

[0136] The purpose of the above-mentioned clustering of conflicting and abnormal network nodes is to assess remaining capacity and optimize load distribution. Taking into account factors such as the remaining capacity and load score of each node, the final load optimization adjustment is carried out to determine which nodes can bear more load and which nodes should be offloaded.

[0137] Building upon previous clustering iterations, factors such as remaining capacity and load score are incorporated to assess the processing capacity and resource allocation priority of each node. Then, based on these assessment results, a final clustering adjustment is performed to ensure optimal load distribution. This clustering not only relies on the node's load history and current resource usage but also considers the node's remaining capacity, taking into account the needs for load offloading and reallocation. It represents the first clustering iteration to optimize the load across the entire system. The fourth clustering iteration reflects the system's resource scheduling capabilities. Through remaining capacity assessment, the load can be precisely allocated to the most suitable nodes, thereby avoiding overload and improving system resource utilization and overall performance.

[0138] S2854: Associate all conflicting and anomalous network nodes with each other to form a node association network;

[0139] Calculate the resource allocation similarity for various resource indicators of each conflicting and abnormal network node in the node association network.

[0140] A preset similarity threshold c is set; it is then determined whether the resource allocation similarity is greater than the similarity threshold c.

[0141] If so, then determine to construct a resource scheduling distribution map for the corresponding conflicting abnormal network nodes whose resource allocation similarity is greater than the similarity threshold c;

[0142] It should be noted that the resource allocation similarity and the constructed resource scheduling distribution map reflect the resource occupancy ratio among various network nodes, which can better allocate resources and adjust the load of resource occupancy.

[0143] By analyzing the relationships between nodes, we can discover which nodes have similar resource usage patterns; grouping these similar nodes together helps optimize resource allocation; calculating the similarity of resource allocation helps determine which nodes can share resources or optimize resource allocation, thereby improving resource utilization efficiency.

[0144] By associating nodes, resources can be allocated more efficiently, reducing resource waste or bottlenecks and improving overall system performance; similarity calculation and resource scheduling distribution maps can help formulate refined resource scheduling strategies and improve the system's resource utilization.

[0145] By analyzing the relationships between nodes, we can identify which nodes have similar resource usage patterns. This provides a basis for resource sharing and adjustment; nodes with high similarity can share resources or perform load balancing. Through resource allocation similarity, the system can more rationally adjust resource quotas among nodes with similar load patterns, achieving load balancing and preventing overload of individual nodes. Figure 6 As shown;

[0146] S2855: Calculate the remaining capacity of each conflicting network node using the aforementioned resource indicators.

[0147] The remaining capacity of each conflict-prone network node is used to calculate a load score for that node. The calculation formula is as follows: ;

[0148] in, Represented as the first The adjusted comprehensive load score of each conflicting network node is used to reflect the overall load status of the conflicting network node, thereby providing a quantitative basis for subsequent resource scheduling or offloading decisions.

[0149] Represented as the first The basic load value of a conflicting network node (i.e., a comprehensive indicator obtained by standardizing resource indicators such as CPU utilization, memory usage, network bandwidth utilization, IO read / write load, and number of connections collected in real time).

[0150] This represents the scaling factor for adjusting the correction term, which aims to balance the relative weight of the load data of conflicting network nodes themselves and the neighborhood information (contributions from other nodes);

[0151] This represents the number of conflicting anomalous network nodes participating in the calculation, typically referring to the number of nodes located in areas with high conflict rates (e.g., conflicting anomalous network nodes).

[0152] Represented as the total number of conflicting anomalous network nodes used for normalization, which in most cases can be made This ensures that the accumulated value is not affected by changes in the number of nodes;

[0153] It is represented as a weighting factor based on load pattern distance, used to attenuate the influence of distant nodes on the target node; the greater the distance, the lower the weight.

[0154] Represented as the first The and the first The load pattern distance between nodes in a conflict-prone network (typically using Euclidean distance, cosine distance, or other similarity metrics, and normalized) indicates that the two nodes are more similar in terms of load patterns, and their weighted influence (i.e., ...) is also higher. (The larger)

[0155] Represented as a comprehensive characteristic factor (i.e. It consists of three key factors The specific meanings are explained as follows: This reflects the frequency of conflict events between or within nodes, representing past or current instances of source theft anomalies. : indicates the first The and the first The degree of similarity between conflicting network nodes in terms of resource consumption or allocation patterns; the higher the similarity, the more consistent their load states. : Captures the trend of node load changes (e.g., the rate of load increase or decrease), predicting possible dynamic load changes; simultaneously, the three feature keywords are assigned weights. , and And satisfy: );

[0156] It should be noted that the above process, by calculating the remaining capacity of each node, can identify which nodes have more resources available for scheduling and which nodes are already overloaded; by using a comprehensive load scoring formula, the overall load of each node can be quantified, thereby determining which nodes need load offloading or resource adjustments.

[0157] Load scoring provides a quantitative basis for dynamic load adjustment, enabling resource scheduling to rely not only on static indicators but also on real-time load data. Steps S2851-S2855, through load scoring, achieve finer-grained load balancing, allocating resources from overloaded nodes to less loaded nodes, thus preventing system performance degradation. After calculating the remaining capacity of each node, the system can understand which nodes have sufficient resources to handle more load and which nodes are nearing their resource limits, helping to determine which nodes need load offloading. Through load scoring calculation, a comprehensive load evaluation can be generated for each node. The score quantifies the node's load status, facilitating further analysis of node resource requirements and guiding resource scheduling, such as resource scheduling or load offloading, to ensure network stability.

[0158] Example 2 like Figure 7As shown, the present invention also provides a network resource abnormal allocation and scheduling processing system, including: a data acquisition module 10; an analysis module 20; and a result module 30.

[0159] The acquisition module 10 is used to acquire transmission data in a dynamic network environment in real time, preprocess the transmission data in the dynamic network environment, and obtain data of each network node.

[0160] The analysis module 20 is used to analyze the behavioral data of each network node, and to use a clustering algorithm to cluster the network node data with the same behavioral data to obtain abnormal network nodes; and to analyze the abnormal network nodes according to the pre-collected historical database to calculate the load score of the abnormal network nodes.

[0161] The result module 30 is used to allocate node data resources to each node based on the load score of each network node data.

[0162] Example 3 On the other hand, such as Figure 8 As shown, this third embodiment, based on the network resource abnormal allocation and scheduling processing method provided in the first embodiment of the invention, also provides a computer storage medium 1140 (hereinafter referred to as the storage medium). Figure 8 The diagram shown is a schematic of a computer storage medium structure framework provided in Embodiment 3 of the present invention, which includes:

[0163] Memory 1130 is used to store computer programs;

[0164] Communication interface 1120 is used to connect memory 1130 and processor 1110;

[0165] Processor 1110 is configured to execute a computer program to implement a network resource abnormal allocation and scheduling processing method disclosed in an embodiment of any combination of the above-described embodiments.

[0166] In summary, the network resource abnormal allocation and scheduling processing method and system proposed in this invention continuously filters network nodes in the collected network transmission data to obtain coarse clustering abnormal network nodes, time-related abnormal network nodes, similarity-related abnormal network nodes, and conflict-related abnormal network nodes. By continuously filtering abnormal network nodes, network nodes with severe loads are identified, thereby improving the accuracy of load calculation and adjusting and balancing the load.

[0167] Coarse-grained clustering of abnormal network nodes is identified through cluster analysis. In this process, the system clusters nodes based on characteristics such as resource consumption and conflict frequency, thereby identifying a group of similar nodes. The behavior or load patterns of coarse-grained clustering of abnormal network nodes show consistency in some aspects, and thus they are classified into one category. Compared with other types of nodes, coarse-grained clustering of abnormal network nodes helps the system locate areas with similar load patterns, which facilitates more centralized and efficient resource scheduling and load optimization.

[0168] Further, the abnormal behavior of time-related network nodes manifests as fluctuations or overloads in resource usage within a specific time period, which may be caused by factors such as peak hours, specific events, or seasonal loads. Compared with other types of nodes, time-related abnormal nodes often overlap with similar abnormal nodes, especially when the load patterns of certain nodes generate high-frequency conflicts within a specific time period. In such cases, the system may simultaneously identify these nodes as both time-related and conflict-related abnormal nodes. The occurrence of time-related anomalies may also exacerbate the frequent occurrence of conflict-related abnormal network nodes.

[0169] Further similarity anomaly network nodes exhibit similarities in resource usage or allocation patterns, possibly due to factors such as the same node type, similar load requirements, or similar operating environments. By calculating resource allocation similarity, the system can identify these types of nodes. Compared to other types of nodes, similarity anomaly nodes are the core objects in clustering conflicting anomaly nodes because they have a certain consistency in resource usage. Therefore, similar nodes often experience resource conflicts or overload problems within the same time period, thus forming conflicting anomaly network nodes.

[0170] The final conflict-prone anomaly nodes exhibited resource overload or exceeded standardized thresholds within a certain period, manifesting as frequent conflict events. These conflicts typically stemmed from nodes consuming excessive resources at a particular moment, leading to system performance degradation. Compared to other node types, conflict-prone anomaly nodes usually rely on the analysis results of other node types. First, through cluster analysis of similarity-based anomaly nodes, the system can identify which nodes share similar resource usage patterns, thereby predicting which nodes are likely to experience conflicts simultaneously. Second, the identification of temporal anomaly nodes helps predict which time periods have a higher frequency of conflict events, providing temporal support for the identification of conflict-prone anomaly nodes.

[0171] Meanwhile, the rough clustering of abnormal network nodes provides a grouping basis for subsequent system analysis, helping to centrally manage similar abnormal nodes and thus contributing to resource scheduling optimization. Temporal abnormal network nodes and similar abnormal network nodes are intertwined. The appearance of temporal abnormal nodes may lead to nodes with similar resource allocation exhibiting a higher frequency of conflicts, thereby increasing the frequency of conflicting abnormal network nodes. Similar abnormal network nodes provide a basis for identifying conflicting abnormal network nodes through the similarity of resource allocation patterns, and the appearance of these conflicting abnormal nodes will affect the system's resource allocation strategy and load optimization scheme.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; those skilled in the art can modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A network resource abnormal allocation scheduling processing method, characterized in that, The method comprises the following operation steps: Real-time collection of transmission data in a dynamic network environment, preprocessing of the transmission data in the dynamic network environment, and obtaining of network node data; Analysis of behavior data of the network node data, clustering of network node data with the same behavior data by using a clustering algorithm, and obtaining of abnormal network nodes; Analysis of the abnormal network nodes based on a pre-collected historical database, and calculation of a load score of the abnormal network nodes; 2. The network resource abnormal allocation and scheduling processing method according to claim 1, characterized in that, Resource allocation of node data to each node based on the load score of the network node data. The specific operation steps of analyzing the abnormal network nodes based on the pre-collected historical database and calculating the load score of the abnormal network nodes are as follows: Analysis of traffic statistical features of the network node data by using a pre-designed feature extraction model, and extraction of typical behavior features; Storage of the typical behavior features as structured data to form a behavior data record of each network node; Collection of a historical database; Extraction of a historical behavior data record corresponding to the current behavior data record and extraction of a historical normal behavior mode from the historical database; Comparison of the current behavior data record and the historical behavior data record for a significant deviation value; Pre-setting of a deviation threshold q, and judging whether the significant deviation value is greater than the deviation threshold q; If yes, the corresponding network node of the current behavior data record exceeding the deviation threshold q is marked as abnormal to obtain a rough clustering abnormal network node; If no, the remaining network nodes are marked as normal to obtain normal network nodes; Reanalysis of traffic statistical features of the network node data by using a clustering algorithm to obtain an optimized node behavior data set; Training of a model by using a support vector machine to obtain an abnormal detection model; Secondary classification and labeling of abnormal network nodes from the node behavior data set by using the abnormal detection model to obtain time-based abnormal network nodes; 3. The network resource abnormal allocation and scheduling processing method according to claim 2, characterized in that, Pre-setting of a time window, and extraction of typical behavior features within the preset time window from the time-based abnormal network nodes. The specific operation steps of analyzing the abnormal network nodes based on the pre-collected historical database and calculating the load score of the abnormal network nodes are as follows: Standardization of the typical behavior features within the preset time window by using a principal component analysis method to construct a covariance matrix; Eigenvalue decomposition of the covariance matrix, extraction of principal components and their corresponding variance contribution rates, reservation of the first K number of typical behavior features of the typical behavior features of each network node by using the variance contribution rates, and finally formation of a low-dimensional behavior feature set of the time-based abnormal network nodes; Calculation of a deviation between a historical normal behavior mode in the historical database and the low-dimensional behavior feature set of each time-based abnormal network node; Pre-setting of a deviation threshold e, and judging whether the calculated deviation is greater than the deviation threshold e; If yes, it is determined that the time anomaly network node corresponding to the low-dimensional behavior feature set is a high-risk node, and a high-risk node list is generated; The traffic attribute information of the time anomaly network node is collected, the change rate of the traffic is calculated according to the adjacent time points corresponding to the collected traffic attribute information, the traffic state change trend of the time anomaly network node is analyzed according to the change rate, the time anomaly network node is clustered based on the traffic state change trend, and a similarity anomaly network node is obtained; The indexes of various resources of the similarity anomaly network node are collected, and the load score is calculated based on the indexes of various resources.

4. The network resource abnormal allocation and scheduling processing method according to claim 3, characterized in that, The traffic attribute information of the time anomaly network node is collected, the change rate of the traffic is calculated according to the adjacent time points corresponding to the collected traffic attribute information, and the specific operation steps are as follows: The traffic attribute information of the time anomaly network node in the high-risk node list is collected, the traffic attribute information of each time anomaly network node is arranged in order of the interval of the collection time, and a standard time sequence is formed; The maximum traffic value of the traffic attribute information in a time period is determined by using an automatic algorithm, and the occurrence time and the duration of the time period corresponding to the maximum traffic value are recorded; A peak traffic threshold r is preset; It is judged whether the maximum traffic value of the traffic attribute information is greater than the peak traffic threshold r; If yes, it is determined that the maximum traffic value of the traffic attribute information is an abnormal fluctuation; The occurrence time and the duration of the traffic attribute information are taken as the burst time length of the abnormal fluctuation; The number of abnormal fluctuations in the traffic attribute information of the corresponding time anomaly network node in a specified period is counted. Based on the traffic state change trend, the time anomaly network node is clustered to obtain a similarity anomaly network node, and the indexes of various resources of the similarity anomaly network node are collected, and the load score is calculated based on the indexes of various resources. The specific operation steps are as follows:

5. The network resource abnormal allocation and scheduling processing method according to claim 4, characterized in that, The traffic state change trend is analyzed according to the change rate and the burst event length; The time anomaly network node is re-screened based on the traffic state change trend, and a similarity anomaly network node is obtained; The indexes of various resources of each node of the similarity anomaly network node are collected, and the conflict frequency of exceeding resource occupation is calculated based on the indexes of various resources; The similarity anomaly network node is clustered according to the conflict frequency, and a conflict anomaly network node is obtained; The residual capacity in the conflict anomaly network node is calculated according to the indexes of various resources; The load score of the conflict anomaly network node is calculated based on the residual capacity. The indexes of various resources of each node of the similarity anomaly network node are collected, and the conflict frequency of exceeding resource occupation is calculated based on the indexes of various resources. The specific operation steps are as follows:

6. The network resource abnormal allocation and scheduling processing method according to claim 5, characterized in that, The indexes of various resources of each node of the similarity anomaly network node are collected, and the conflict frequency of exceeding resource occupation is calculated based on the indexes of various resources. The specific operation steps are as follows: The indexes of various resources of each node of the similarity anomaly network node are collected, and the conflict frequency of exceeding resource occupation is calculated based on the indexes of various resources. The specific operation steps are as follows: The resource occupation distribution matrix is constructed for the similarity anomaly network node and corresponding resource indicators in the similarity anomaly network node; The rows of the resource occupation distribution matrix represent the similarity anomaly network nodes, and the columns represent corresponding resource indicators in the similarity anomaly network node; A resource occupation standardization threshold z is preset for the resource indicators, and it is determined whether the corresponding resource indicators in the similarity anomaly network node are greater than the resource occupation standardization threshold z; When the resource indicator of one of the resource indicators exceeds the resource occupation standardization threshold z, the time corresponding to the resource indicator exceeding the resource occupation standardization threshold z is marked as a conflict event, and the conflict frequency of all conflict events is calculated.

7. The network resource abnormal allocation and scheduling processing method according to claim 6, characterized in that, The similarity anomaly network node is clustered according to the conflict frequency, and a conflict anomaly network node is obtained; the residual capacity in the conflict anomaly network node is calculated according to the resource indicators; and the load score of the conflict anomaly network node is calculated based on the residual capacity, and the specific operation steps are as follows: The node feature vector of the similarity anomaly network node is constructed using the resource indicators and the conflict frequency; The conflict anomaly network node is obtained by clustering the conflict frequency of the similarity anomaly network node according to the node feature vector, as a conflict-prone area; The node association network is formed by correlating all conflict anomaly network nodes with each other; The resource allocation similarity is calculated for the resource indicators in each conflict anomaly network node in the node association network; A similarity threshold c is preset, and it is determined whether the resource allocation similarity is greater than the similarity threshold c; If yes, the corresponding conflict anomaly network node with the resource allocation similarity greater than the similarity threshold c is constructed into a resource scheduling distribution diagram; The residual capacity of each conflict anomaly network node is calculated using the resource indicators; The load score of each conflict anomaly network node is calculated using the residual capacity of each conflict anomaly network node.

8. A network resource abnormal allocation scheduling processing system, characterized by, It includes: a collection module; an analysis module; a result module; The collection module is used to collect transmission data in a dynamic network environment in real time, pre-process the transmission data in the dynamic network environment, and obtain network node data; The analysis module is used to analyze behavior data of the network node data, cluster the network node data with the same behavior data using a clustering algorithm, obtain an anomaly network node, analyze the anomaly network node according to a pre-collected historical database, and calculate a load score of the anomaly network node; The result module is used to allocate node data resources to each node according to the load score of each network node data.

9. A storage medium, characterized by The storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the network resource anomaly allocation and scheduling processing method in any one of claims 1-7.

Images